Warning

This document is for an in-development version of Galaxy. You can alternatively view this page in the latest release if it exists or view the top of the latest release's documentation.

galaxy.tool_util.verify package¶

Module of utilities for verifying test results.

galaxy.tool_util.verify.files_contains(file1, file2, attributes=None)[source]¶: Check the contents of file2 for substrings found in file1, on a per-line basis.

galaxy.tool_util.verify.files_delta(file1, file2, attributes=None)[source]¶: Check the contents of 2 files for size differences.

galaxy.tool_util.verify.files_diff(file1, file2, attributes=None)[source]¶: Check the contents of 2 files for differences.

galaxy.tool_util.verify.files_re_match(file1, file2, attributes=None)[source]¶: Check the contents of 2 files for differences using re.match.

galaxy.tool_util.verify.files_re_match_multiline(file1, file2, attributes=None)[source]¶: Check the contents of 2 files for differences using re.match in multiline mode.

galaxy.tool_util.verify.get_compressed_formats(attributes)[source]¶

galaxy.tool_util.verify.make_temp_fname(fname=None)[source]¶: Safe temp name - preserve the file extension for tools that interpret it.

galaxy.tool_util.verify.verify(item_label, output_content, attributes, filename=None, get_filecontent=None, get_filename=None, keep_outputs_dir=None, verify_extra_files=None, mode='file')[source]¶

Verify the content of a test output using test definitions described by attributes.

Throw an informative assertion error if any of these tests fail.

Subpackages¶

galaxy.tool_util.verify.asserts package

Submodules¶

galaxy.tool_util.verify.interactor module¶

class galaxy.tool_util.verify.interactor.DictClientTestConfig(tools)[source]¶

Bases: object

__init__(tools)[source]¶

get_test_config(job_data)[source]¶

class galaxy.tool_util.verify.interactor.GalaxyInteractorApi(**kwds)[source]¶

Bases: object

__init__(**kwds)[source]¶

api_key_header(key, admin, anon, headers)[source]¶

delete_history(history)[source]¶

ensure_user_with_email(email, password=None)[source]¶

format_for_summary(blob, empty_message, prefix='| ')[source]¶

get_api_url(path: str) → str[source]¶

get_history(history_name='test_history')[source]¶

get_job_stdio(job_id)[source]¶

get_tests_summary()[source]¶

get_tool_tests(tool_id, tool_version=None)[source]¶

new_history(history_name='test_history', publish_history=False)[source]¶

output_hid(output_data)[source]¶

publish_history(history_id)[source]¶

run_tool(testdef, history_id, resource_parameters=None)[source]¶

stage_data_async(test_data, history_id, tool_id, force_path_paste=False, maxseconds=86400, tool_version=None)[source]¶

supports_test_data_download¶

target_galaxy_version¶

test_data_download(tool_id, filename, mode='file', is_output=True, tool_version=None)[source]¶

test_data_path(tool_id, filename, tool_version=None)[source]¶

verify_output(history_id, jobs, output_data, output_testdef, tool_id, maxseconds, tool_version=None)[source]¶

verify_output_collection(output_collection_def, output_collection_id, history, tool_id, tool_version=None)[source]¶

verify_output_dataset(history_id, hda_id, outfile, attributes, tool_id, tool_version=None)[source]¶

wait_for(func, what='tool test run', **kwd)[source]¶

wait_for_job(job_id, history_id=None, maxseconds=86400)[source]¶

wait_for_jobs(history_id, jobs, maxseconds)[source]¶

exception galaxy.tool_util.verify.interactor.JobOutputsError(output_exceptions, job_stdio)[source]¶

Bases: AssertionError

__init__(output_exceptions, job_stdio)[source]¶

class galaxy.tool_util.verify.interactor.NullClientTestConfig[source]¶

Bases: object

get_test_config(job_data)[source]¶

class galaxy.tool_util.verify.interactor.OutputsDict[source]¶

Bases: dict

Ordered dict that can also be accessed by index.

>>> out = OutputsDict()
>>> out['item1'] = 1
>>> out['item2'] = 2
>>> out[1] == 2 == out['item2']
True

exception galaxy.tool_util.verify.interactor.RunToolException(message, inputs=None, dynamic_param_error=False)[source]¶

Bases: Exception

__init__(message, inputs=None, dynamic_param_error=False)[source]¶

class galaxy.tool_util.verify.interactor.ToolTestDescription(processed_test_dict)[source]¶

Bases: object

Encapsulates information about a tool test, and allows creation of a dynamic TestCase class (the unittest framework is very class oriented, doing dynamic tests in this way allows better integration)

__init__(processed_test_dict)[source]¶

test_data()[source]¶: Iterator over metadata representing the required files for upload.

to_dict()[source]¶

galaxy.tool_util.verify.interactor.ensure_tool_run_response_okay(submit_response_object, request_desc, inputs=None)[source]¶

galaxy.tool_util.verify.interactor.stage_data_in_history(galaxy_interactor, tool_id, all_test_data, history=None, force_path_paste=False, maxseconds=86400, tool_version=None)[source]¶

galaxy.tool_util.verify.interactor.test_data_iter(required_files)[source]¶

galaxy.tool_util.verify.interactor.verify_collection(output_collection_def, data_collection, verify_dataset)[source]¶

galaxy.tool_util.verify.interactor.verify_hid(filename, hda_id, attributes, test_data_downloader, hid='', dataset_fetcher=None, keep_outputs_dir=False)[source]¶

galaxy.tool_util.verify.interactor.verify_tool(tool_id, galaxy_interactor, resource_parameters=None, register_job_data=None, test_index=0, tool_version=None, quiet=False, test_history=None, no_history_cleanup=False, publish_history=False, force_path_paste=False, maxseconds=86400, tool_test_dicts=None, client_test_config=None, skip_with_reference_data=False, skip_on_dynamic_param_errors=False)[source]¶

galaxy.tool_util.verify.script module¶

class galaxy.tool_util.verify.script.Results(default_suitename, test_json, append=False, galaxy_url=None)[source]¶

Bases: object

__init__(default_suitename, test_json, append=False, galaxy_url=None)[source]¶

already_executed(test_reference)[source]¶

already_successful(test_reference)[source]¶

error_count¶

failure_count¶

info_message()[source]¶

register_exception(test_exception)[source]¶

register_result(result)[source]¶

skip_count¶

success_count¶

write()[source]¶

class galaxy.tool_util.verify.script.TestException(tool_id, exception, was_recorded)¶

Bases: tuple

exception¶: Alias for field number 1

tool_id¶: Alias for field number 0

was_recorded¶: Alias for field number 2

class galaxy.tool_util.verify.script.TestReference(tool_id, tool_version, test_index)¶

Bases: tuple

test_index¶: Alias for field number 2

tool_id¶: Alias for field number 0

tool_version¶: Alias for field number 1

galaxy.tool_util.verify.script.arg_parser()[source]¶

galaxy.tool_util.verify.script.build_case_references(galaxy_interactor, tool_id='*', tool_version=None, test_index=-1, page_size=0, page_number=0, test_filters=None, log=None)[source]¶

galaxy.tool_util.verify.script.main(argv=None)[source]¶

galaxy.tool_util.verify.script.run_tests(args, test_filters=None, log=None)[source]¶

galaxy.tool_util.verify.script.setup_global_logger(name, log_file=None, verbose=False)[source]¶

galaxy.tool_util.verify.script.test_tools(galaxy_interactor, test_references, results, log=None, parallel_tests=1, history_per_test_case=False, history_name=None, no_history_reuse=False, no_history_cleanup=False, publish_history=False, retries=0, verify_kwds=None)[source]¶

Run through tool tests and write report.

Refactor this into Galaxy in 21.01.

galaxy.tool_util.verify.test_data module¶

class galaxy.tool_util.verify.test_data.FileDataResolver(file_dir)[source]¶

Bases: object

__init__(file_dir)[source]¶

exists(filename)[source]¶

path(filename)[source]¶

class galaxy.tool_util.verify.test_data.GitDataResolver(repository, environ)[source]¶

Bases: galaxy.tool_util.verify.test_data.FileDataResolver

__init__(repository, environ)[source]¶

execute(cmd)[source]¶

exists(filename)[source]¶

update_repository()[source]¶

class galaxy.tool_util.verify.test_data.TestDataResolver(file_dirs=None, env_var='GALAXY_TEST_FILE_DIR', environ=environ({'JENKINS_SERVER_COOKIE': 'f442d5996560f01b', 'VIRTUAL_ENV': '/data/jenkins/workspace/galaxy-sphinx-by-branch/PYTHON/System-CPython-2.7/TARGET_GIT_BRANCH/master/.venv', 'ROOT_BUILD_CAUSE_TIMERTRIGGER': 'true', 'PYTHON': 'System-CPython-2.7', 'GIT_URL': 'https://github.com/galaxyproject/galaxy/', 'JOB_NAME': 'galaxy-sphinx-by-branch/PYTHON=System-CPython-2.7, TARGET_GIT_BRANCH=master', 'JOB_DISPLAY_URL': 'https://jenkins.galaxyproject.org/job/galaxy-sphinx-by-branch/PYTHON=System-CPython-2.7, TARGET_GIT_BRANCH=master/display/redirect', 'JOB_URL': 'https://jenkins.galaxyproject.org/job/galaxy-sphinx-by-branch/PYTHON=System-CPython-2.7, TARGET_GIT_BRANCH=master/', 'PWD': '/data/jenkins/workspace/galaxy-sphinx-by-branch/PYTHON/System-CPython-2.7/TARGET_GIT_BRANCH/master', 'PS1': '(.venv) (.venv) $ ', 'RUN_CHANGES_DISPLAY_URL': 'https://jenkins.galaxyproject.org/job/galaxy-sphinx-by-branch/PYTHON=System-CPython-2.7, TARGET_GIT_BRANCH=master/4089/display/redirect?page=changes', 'CI': 'true', 'LANG': 'C.UTF-8', 'MAKEFLAGS': 'w', 'BUILD_TAG': 'jenkins-galaxy-sphinx-by-branch-PYTHON=System-CPython-2.7, TARGET_GIT_BRANCH=master-4089', 'GIT_PREVIOUS_SUCCESSFUL_COMMIT': '0a8eb4d711e973cb95120c44c1964d2a7fe0a958', 'JOB_BASE_NAME': 'PYTHON=System-CPython-2.7, TARGET_GIT_BRANCH=master', 'BUILD_NUMBER': '4089', 'NODE_NAME': 'jenkins0.galaxyproject.org', 'MFLAGS': '-w', 'NPM_CONFIG_PREFIX': '/data/jenkins/workspace/galaxy-sphinx-by-branch/PYTHON/System-CPython-2.7/TARGET_GIT_BRANCH/master/.venv', 'XDG_SESSION_TYPE': 'tty', 'GIT_BRANCH': 'origin/dev', 'WORKSPACE': '/data/jenkins/workspace/galaxy-sphinx-by-branch/PYTHON/System-CPython-2.7/TARGET_GIT_BRANCH/master', 'RUN_TESTS_DISPLAY_URL': 'https://jenkins.galaxyproject.org/job/galaxy-sphinx-by-branch/PYTHON=System-CPython-2.7, TARGET_GIT_BRANCH=master/4089/display/redirect?page=tests', 'RUN_ARTIFACTS_DISPLAY_URL': 'https://jenkins.galaxyproject.org/job/galaxy-sphinx-by-branch/PYTHON=System-CPython-2.7, TARGET_GIT_BRANCH=master/4089/display/redirect?page=artifacts', 'DBUS_SESSION_BUS_ADDRESS': 'unix:path=/run/user/1000/bus', 'HUDSON_URL': 'https://jenkins.galaxyproject.org/', 'HUDSON_COOKIE': '37f082e1-4c6a-4b72-9ba4-4060ec3507f5', 'SHELL': '/bin/sh', 'npm_config_prefix': '/data/jenkins/workspace/galaxy-sphinx-by-branch/PYTHON/System-CPython-2.7/TARGET_GIT_BRANCH/master/.venv', 'HUDSON_HOME': '/data/jenkins', 'PATH': '/data/jenkins/workspace/galaxy-sphinx-by-branch/PYTHON/System-CPython-2.7/TARGET_GIT_BRANCH/master/.venv/lib/node_modules/.bin:/data/jenkins/workspace/galaxy-sphinx-by-branch/PYTHON/System-CPython-2.7/TARGET_GIT_BRANCH/master/.venv/bin:/data/jenkins/workspace/galaxy-sphinx-by-branch/PYTHON/System-CPython-2.7/TARGET_GIT_BRANCH/master/.venv/bin:/data/jenkins/workspace/galaxy-sphinx-by-branch/PYTHON/System-CPython-2.7/TARGET_GIT_BRANCH/master/.venv/lib/node_modules/.bin:/data/jenkins/workspace/galaxy-sphinx-by-branch/PYTHON/System-CPython-2.7/TARGET_GIT_BRANCH/master/.venv/bin:/data/jenkins/workspace/galaxy-sphinx-by-branch/PYTHON/System-CPython-2.7/TARGET_GIT_BRANCH/master/.venv/bin:/usr/local/bin:/usr/bin:/bin:/usr/games', 'AWS_SECRET_ACCESS_KEY': 'L+6N5EfuLyZB/84Hv0vAun6h9DXcn0pLcePGuSV9', 'MOTD_SHOWN': 'pam', 'JENKINS_HOME': '/data/jenkins', 'SSH_CLIENT': '172.18.2.23 46110 22', 'AWS_ACCESS_KEY_ID': 'AKIAYX7HSEMC7DKPIMU7', 'LOGNAME': 'jenkins', 'GIT_PREVIOUS_COMMIT': '0a8eb4d711e973cb95120c44c1964d2a7fe0a958', 'BUILD_CAUSE_UPSTREAMTRIGGER': 'true', 'BUILD_URL': 'https://jenkins.galaxyproject.org/job/galaxy-sphinx-by-branch/PYTHON=System-CPython-2.7, TARGET_GIT_BRANCH=master/4089/', 'USER': 'jenkins', 'NODE_VIRTUAL_ENV': '/data/jenkins/workspace/galaxy-sphinx-by-branch/PYTHON/System-CPython-2.7/TARGET_GIT_BRANCH/master/.venv', 'GIT_COMMIT': '1a7934dee46774642a5fbdf94ac4bda688744eff', 'RUN_DISPLAY_URL': 'https://jenkins.galaxyproject.org/job/galaxy-sphinx-by-branch/PYTHON=System-CPython-2.7, TARGET_GIT_BRANCH=master/4089/display/redirect', 'NODE_LABELS': 'doc_builder jenkins0.galaxyproject.org psu_node', 'XDG_RUNTIME_DIR': '/run/user/1000', 'BUILD_DISPLAY_NAME': '#4089', 'NODE_PATH': '/data/jenkins/workspace/galaxy-sphinx-by-branch/PYTHON/System-CPython-2.7/TARGET_GIT_BRANCH/master/.venv/lib/node_modules', 'HUDSON_SERVER_COOKIE': 'f442d5996560f01b', 'XDG_SESSION_CLASS': 'user', 'HOME': '/home/jenkins', 'BUILD_ID': '4089', 'XDG_SESSION_ID': '6541', 'EXECUTOR_NUMBER': '1', 'JENKINS_URL': 'https://jenkins.galaxyproject.org/', 'WORKSPACE_TMP': '/data/jenkins/workspace/galaxy-sphinx-by-branch/PYTHON/System-CPython-2.7/TARGET_GIT_BRANCH/master@tmp', 'ROOT_BUILD_CAUSE': 'TIMERTRIGGER', 'TARGET_GIT_BRANCH': 'master', 'GRAVITY_STATE_DIR': '/data/jenkins/workspace/galaxy-sphinx-by-branch/PYTHON/System-CPython-2.7/TARGET_GIT_BRANCH/master/database/gravity', 'BUILD_CAUSE': 'UPSTREAMTRIGGER', 'SSH_CONNECTION': '172.18.2.23 46110 172.18.2.30 22', 'MAKELEVEL': '2'}))[source]¶

Bases: object

__init__(file_dirs=None, env_var='GALAXY_TEST_FILE_DIR', environ=environ({'JENKINS_SERVER_COOKIE': 'f442d5996560f01b', 'VIRTUAL_ENV': '/data/jenkins/workspace/galaxy-sphinx-by-branch/PYTHON/System-CPython-2.7/TARGET_GIT_BRANCH/master/.venv', 'ROOT_BUILD_CAUSE_TIMERTRIGGER': 'true', 'PYTHON': 'System-CPython-2.7', 'GIT_URL': 'https://github.com/galaxyproject/galaxy/', 'JOB_NAME': 'galaxy-sphinx-by-branch/PYTHON=System-CPython-2.7, TARGET_GIT_BRANCH=master', 'JOB_DISPLAY_URL': 'https://jenkins.galaxyproject.org/job/galaxy-sphinx-by-branch/PYTHON=System-CPython-2.7, TARGET_GIT_BRANCH=master/display/redirect', 'JOB_URL': 'https://jenkins.galaxyproject.org/job/galaxy-sphinx-by-branch/PYTHON=System-CPython-2.7, TARGET_GIT_BRANCH=master/', 'PWD': '/data/jenkins/workspace/galaxy-sphinx-by-branch/PYTHON/System-CPython-2.7/TARGET_GIT_BRANCH/master', 'PS1': '(.venv) (.venv) $ ', 'RUN_CHANGES_DISPLAY_URL': 'https://jenkins.galaxyproject.org/job/galaxy-sphinx-by-branch/PYTHON=System-CPython-2.7, TARGET_GIT_BRANCH=master/4089/display/redirect?page=changes', 'CI': 'true', 'LANG': 'C.UTF-8', 'MAKEFLAGS': 'w', 'BUILD_TAG': 'jenkins-galaxy-sphinx-by-branch-PYTHON=System-CPython-2.7, TARGET_GIT_BRANCH=master-4089', 'GIT_PREVIOUS_SUCCESSFUL_COMMIT': '0a8eb4d711e973cb95120c44c1964d2a7fe0a958', 'JOB_BASE_NAME': 'PYTHON=System-CPython-2.7, TARGET_GIT_BRANCH=master', 'BUILD_NUMBER': '4089', 'NODE_NAME': 'jenkins0.galaxyproject.org', 'MFLAGS': '-w', 'NPM_CONFIG_PREFIX': '/data/jenkins/workspace/galaxy-sphinx-by-branch/PYTHON/System-CPython-2.7/TARGET_GIT_BRANCH/master/.venv', 'XDG_SESSION_TYPE': 'tty', 'GIT_BRANCH': 'origin/dev', 'WORKSPACE': '/data/jenkins/workspace/galaxy-sphinx-by-branch/PYTHON/System-CPython-2.7/TARGET_GIT_BRANCH/master', 'RUN_TESTS_DISPLAY_URL': 'https://jenkins.galaxyproject.org/job/galaxy-sphinx-by-branch/PYTHON=System-CPython-2.7, TARGET_GIT_BRANCH=master/4089/display/redirect?page=tests', 'RUN_ARTIFACTS_DISPLAY_URL': 'https://jenkins.galaxyproject.org/job/galaxy-sphinx-by-branch/PYTHON=System-CPython-2.7, TARGET_GIT_BRANCH=master/4089/display/redirect?page=artifacts', 'DBUS_SESSION_BUS_ADDRESS': 'unix:path=/run/user/1000/bus', 'HUDSON_URL': 'https://jenkins.galaxyproject.org/', 'HUDSON_COOKIE': '37f082e1-4c6a-4b72-9ba4-4060ec3507f5', 'SHELL': '/bin/sh', 'npm_config_prefix': '/data/jenkins/workspace/galaxy-sphinx-by-branch/PYTHON/System-CPython-2.7/TARGET_GIT_BRANCH/master/.venv', 'HUDSON_HOME': '/data/jenkins', 'PATH': '/data/jenkins/workspace/galaxy-sphinx-by-branch/PYTHON/System-CPython-2.7/TARGET_GIT_BRANCH/master/.venv/lib/node_modules/.bin:/data/jenkins/workspace/galaxy-sphinx-by-branch/PYTHON/System-CPython-2.7/TARGET_GIT_BRANCH/master/.venv/bin:/data/jenkins/workspace/galaxy-sphinx-by-branch/PYTHON/System-CPython-2.7/TARGET_GIT_BRANCH/master/.venv/bin:/data/jenkins/workspace/galaxy-sphinx-by-branch/PYTHON/System-CPython-2.7/TARGET_GIT_BRANCH/master/.venv/lib/node_modules/.bin:/data/jenkins/workspace/galaxy-sphinx-by-branch/PYTHON/System-CPython-2.7/TARGET_GIT_BRANCH/master/.venv/bin:/data/jenkins/workspace/galaxy-sphinx-by-branch/PYTHON/System-CPython-2.7/TARGET_GIT_BRANCH/master/.venv/bin:/usr/local/bin:/usr/bin:/bin:/usr/games', 'AWS_SECRET_ACCESS_KEY': 'L+6N5EfuLyZB/84Hv0vAun6h9DXcn0pLcePGuSV9', 'MOTD_SHOWN': 'pam', 'JENKINS_HOME': '/data/jenkins', 'SSH_CLIENT': '172.18.2.23 46110 22', 'AWS_ACCESS_KEY_ID': 'AKIAYX7HSEMC7DKPIMU7', 'LOGNAME': 'jenkins', 'GIT_PREVIOUS_COMMIT': '0a8eb4d711e973cb95120c44c1964d2a7fe0a958', 'BUILD_CAUSE_UPSTREAMTRIGGER': 'true', 'BUILD_URL': 'https://jenkins.galaxyproject.org/job/galaxy-sphinx-by-branch/PYTHON=System-CPython-2.7, TARGET_GIT_BRANCH=master/4089/', 'USER': 'jenkins', 'NODE_VIRTUAL_ENV': '/data/jenkins/workspace/galaxy-sphinx-by-branch/PYTHON/System-CPython-2.7/TARGET_GIT_BRANCH/master/.venv', 'GIT_COMMIT': '1a7934dee46774642a5fbdf94ac4bda688744eff', 'RUN_DISPLAY_URL': 'https://jenkins.galaxyproject.org/job/galaxy-sphinx-by-branch/PYTHON=System-CPython-2.7, TARGET_GIT_BRANCH=master/4089/display/redirect', 'NODE_LABELS': 'doc_builder jenkins0.galaxyproject.org psu_node', 'XDG_RUNTIME_DIR': '/run/user/1000', 'BUILD_DISPLAY_NAME': '#4089', 'NODE_PATH': '/data/jenkins/workspace/galaxy-sphinx-by-branch/PYTHON/System-CPython-2.7/TARGET_GIT_BRANCH/master/.venv/lib/node_modules', 'HUDSON_SERVER_COOKIE': 'f442d5996560f01b', 'XDG_SESSION_CLASS': 'user', 'HOME': '/home/jenkins', 'BUILD_ID': '4089', 'XDG_SESSION_ID': '6541', 'EXECUTOR_NUMBER': '1', 'JENKINS_URL': 'https://jenkins.galaxyproject.org/', 'WORKSPACE_TMP': '/data/jenkins/workspace/galaxy-sphinx-by-branch/PYTHON/System-CPython-2.7/TARGET_GIT_BRANCH/master@tmp', 'ROOT_BUILD_CAUSE': 'TIMERTRIGGER', 'TARGET_GIT_BRANCH': 'master', 'GRAVITY_STATE_DIR': '/data/jenkins/workspace/galaxy-sphinx-by-branch/PYTHON/System-CPython-2.7/TARGET_GIT_BRANCH/master/database/gravity', 'BUILD_CAUSE': 'UPSTREAMTRIGGER', 'SSH_CONNECTION': '172.18.2.23 46110 172.18.2.30 22', 'MAKELEVEL': '2'}))[source]¶

get_directory(name)[source]¶

get_filecontent(name)[source]¶

get_filename(name)[source]¶

galaxy.tool_util.verify.test_data.build_resolver(uri, environ)[source]¶

galaxy.tool_util.verify.wait module¶

Abstraction for waiting on API conditions to become true.

galaxy.tool_util.verify.wait.wait_on(function: typing.Callable, desc: str, timeout: typing.Union[int, float], delta: typing.Union[int, float] = 0.25, polling_backoff: typing.Union[int, float] = 0, sleep_: typing.Optional[typing.Callable] = None)[source]¶

Wait for function to return non-None value.

Grow the polling interval (initially delta defaulting to 0.25 seconds) incrementally by the supplied polling_backoff (defaulting to 0).

Throw a TimeoutAssertionError if the supplied timeout is reached without supplied function ever returning a non-None value.

exception galaxy.tool_util.verify.wait.TimeoutAssertionError(message)[source]¶

Bases: AssertionError

Derivative of AssertionError indicating wait_on exceeded max time.

__init__(message)[source]¶