Warning

This document is for an old release of Galaxy. You can alternatively view this page in the latest release if it exists or view the top of the latest release's documentation.

galaxy.tool_util.verify package

Module of utilities for verifying test results.

galaxy.tool_util.verify.verify(item_label, output_content, attributes, filename=None, get_filecontent=None, get_filename=None, keep_outputs_dir=None, verify_extra_files=None, mode='file')[source]

Verify the content of a test output using test definitions described by attributes.

Throw an informative assertion error if any of these tests fail.

galaxy.tool_util.verify.make_temp_fname(fname=None)[source]

Safe temp name - preserve the file extension for tools that interpret it.

galaxy.tool_util.verify.files_delta(file1, file2, attributes=None)[source]

Check the contents of 2 files for size differences.

galaxy.tool_util.verify.get_compressed_formats(attributes)[source]
galaxy.tool_util.verify.files_diff(file1, file2, attributes=None)[source]

Check the contents of 2 files for differences.

galaxy.tool_util.verify.files_re_match(file1, file2, attributes=None)[source]

Check the contents of 2 files for differences using re.match.

galaxy.tool_util.verify.files_re_match_multiline(file1, file2, attributes=None)[source]

Check the contents of 2 files for differences using re.match in multiline mode.

galaxy.tool_util.verify.files_contains(file1, file2, attributes=None)[source]

Check the contents of file2 for substrings found in file1, on a per-line basis.

Submodules

galaxy.tool_util.verify.interactor module

class galaxy.tool_util.verify.interactor.OutputsDict[source]

Bases: dict

Ordered dict that can also be accessed by index.

>>> out = OutputsDict()
>>> out['item1'] = 1
>>> out['item2'] = 2
>>> out[1] == 2 == out['item2']
True
galaxy.tool_util.verify.interactor.stage_data_in_history(galaxy_interactor, tool_id, all_test_data, history=None, force_path_paste=False, maxseconds=86400)[source]
class galaxy.tool_util.verify.interactor.GalaxyInteractorApi(**kwds)[source]

Bases: object

__init__(**kwds)[source]

Initialize self. See help(type(self)) for accurate signature.

property target_galaxy_version
property supports_test_data_download
get_tests_summary()[source]
get_tool_tests(tool_id, tool_version=None)[source]
verify_output_collection(output_collection_def, output_collection_id, history, tool_id)[source]
verify_output(history_id, jobs, output_data, output_testdef, tool_id, maxseconds)[source]
wait_for_jobs(history_id, jobs, maxseconds)[source]
verify_output_dataset(history_id, hda_id, outfile, attributes, tool_id)[source]
wait_for_job(job_id, history_id=None, maxseconds=86400)[source]
wait_for(func, what='tool test run', **kwd)[source]
get_job_stdio(job_id)[source]
new_history(history_name='test_history', publish_history=False)[source]
publish_history(history_id)[source]
test_data_path(tool_id, filename)[source]
test_data_download(tool_id, filename, mode='file', is_output=True)[source]
stage_data_async(test_data, history_id, tool_id, force_path_paste=False, maxseconds=86400)[source]
run_tool(testdef, history_id, resource_parameters=None)[source]
output_hid(output_data)[source]
delete_history(history)[source]
format_for_summary(blob, empty_message, prefix='| ')[source]
ensure_user_with_email(email, password=None)[source]
api_key_header(key, admin, anon)[source]
galaxy.tool_util.verify.interactor.ensure_tool_run_response_okay(submit_response_object, request_desc, inputs=None)[source]
exception galaxy.tool_util.verify.interactor.RunToolException(message, inputs=None, dynamic_param_error=False)[source]

Bases: Exception

__init__(message, inputs=None, dynamic_param_error=False)[source]

Initialize self. See help(type(self)) for accurate signature.

galaxy.tool_util.verify.interactor.verify_hid(filename, hda_id, attributes, test_data_downloader, hid='', dataset_fetcher=None, keep_outputs_dir=False)[source]
galaxy.tool_util.verify.interactor.verify_collection(output_collection_def, data_collection, verify_dataset)[source]
class galaxy.tool_util.verify.interactor.NullClientTestConfig[source]

Bases: object

get_test_config(job_data)[source]
class galaxy.tool_util.verify.interactor.DictClientTestConfig(tools)[source]

Bases: object

__init__(tools)[source]

Initialize self. See help(type(self)) for accurate signature.

get_test_config(job_data)[source]
galaxy.tool_util.verify.interactor.verify_tool(tool_id, galaxy_interactor, resource_parameters=None, register_job_data=None, test_index=0, tool_version=None, quiet=False, test_history=None, no_history_cleanup=False, publish_history=False, force_path_paste=False, maxseconds=86400, tool_test_dicts=None, client_test_config=None, skip_with_reference_data=False, skip_on_dynamic_param_errors=False)[source]
exception galaxy.tool_util.verify.interactor.JobOutputsError(output_exceptions, job_stdio)[source]

Bases: AssertionError

__init__(output_exceptions, job_stdio)[source]

Initialize self. See help(type(self)) for accurate signature.

class galaxy.tool_util.verify.interactor.ToolTestDescription(processed_test_dict)[source]

Bases: object

Encapsulates information about a tool test, and allows creation of a dynamic TestCase class (the unittest framework is very class oriented, doing dynamic tests in this way allows better integration)

__init__(processed_test_dict)[source]

Initialize self. See help(type(self)) for accurate signature.

test_data()[source]

Iterator over metadata representing the required files for upload.

to_dict()[source]
galaxy.tool_util.verify.interactor.test_data_iter(required_files)[source]
galaxy.tool_util.verify.interactor.galaxy_requests_post(url, data=None, files=None, as_json=False, params=None, headers=None)[source]

Handle some Galaxy conventions and work around requests issues.

This is admittedly kind of hacky, so the interface may change frequently - be careful on reuse.

If as_json is True, use post payload using request’s json parameter instead of the data parameter (i.e. assume the contents is a json-ified blob instead of form parameters with individual parameters json-ified if needed). requests doesn’t allow files to be specified with the json parameter - so rewrite the parameters to handle that if as_json is True with specified files.

galaxy.tool_util.verify.script module

class galaxy.tool_util.verify.script.TestReference(tool_id, tool_version, test_index)

Bases: tuple

property test_index

Alias for field number 2

property tool_id

Alias for field number 0

property tool_version

Alias for field number 1

class galaxy.tool_util.verify.script.TestException(tool_id, exception, was_recorded)

Bases: tuple

property exception

Alias for field number 1

property tool_id

Alias for field number 0

property was_recorded

Alias for field number 2

class galaxy.tool_util.verify.script.Results(default_suitename, test_json, append=False, galaxy_url=None)[source]

Bases: object

__init__(default_suitename, test_json, append=False, galaxy_url=None)[source]

Initialize self. See help(type(self)) for accurate signature.

register_result(result)[source]
register_exception(test_exception)[source]
already_successful(test_reference)[source]
already_executed(test_reference)[source]
write()[source]
info_message()[source]
property success_count
property skip_count
property error_count
property failure_count
galaxy.tool_util.verify.script.test_tools(galaxy_interactor, test_references, results, log=None, parallel_tests=1, history_per_test_case=False, no_history_cleanup=False, publish_history=False, retries=0, verify_kwds=None)[source]

Run through tool tests and write report.

Refactor this into Galaxy in 21.01.

galaxy.tool_util.verify.script.build_case_references(galaxy_interactor, tool_id='*', tool_version=None, test_index=- 1, page_size=0, page_number=0, test_filters=None, log=None)[source]
galaxy.tool_util.verify.script.main(argv=None)[source]
galaxy.tool_util.verify.script.run_tests(args, test_filters=None, log=None)[source]
galaxy.tool_util.verify.script.setup_global_logger(name, log_file=None, verbose=False)[source]
galaxy.tool_util.verify.script.arg_parser()[source]

galaxy.tool_util.verify.test_data module

class galaxy.tool_util.verify.test_data.TestDataResolver(file_dirs=None, env_var='GALAXY_TEST_FILE_DIR', environ=environ({'POWERSHELL_DISTRIBUTION_CHANNEL': 'GitHub-Actions-ubuntu20', 'ANDROID_NDK_ROOT': '/usr/local/lib/android/sdk/ndk-bundle', 'GITHUB_EVENT_NAME': 'push', 'GITHUB_ACTION': '__run_8', 'GITHUB_REPOSITORY': 'galaxyproject/galaxy', 'GRADLE_HOME': '/usr/share/gradle-7.2', 'GITHUB_WORKFLOW': 'Build docs', 'PERFLOG_LOCATION_SETTING': 'RUNNER_PERFLOG', 'JOURNAL_STREAM': '8:22305', 'LEIN_HOME': '/usr/local/lib/lein', 'GITHUB_GRAPHQL_URL': 'https://api.github.com/graphql', 'GOROOT_1_16_X64': '/opt/hostedtoolcache/go/1.16.9/x64', 'GITHUB_BASE_REF': '', 'GITHUB_ENV': '/home/runner/work/_temp/_runner_file_commands/set_env_af35710e-307c-42a6-8b32-01f13bb5d91a', 'STATS_KEEPALIVE': 'false', 'GOROOT_1_17_X64': '/opt/hostedtoolcache/go/1.17.2/x64', '_': '/usr/bin/make', 'ANDROID_NDK_LATEST_HOME': '/usr/local/lib/android/sdk/ndk/23.0.7599858', 'RUNNER_PERFLOG': '/home/runner/perflog', 'RUNNER_TOOL_CACHE': '/opt/hostedtoolcache', 'DOTNET_MULTILEVEL_LOOKUP': '0', 'DEPLOYMENT_BASEPATH': '/opt/runner', 'DOTNET_SKIP_FIRST_TIME_EXPERIENCE': '1', 'PATH': '/opt/hostedtoolcache/Python/3.7.12/x64/bin:/opt/hostedtoolcache/Python/3.7.12/x64:/home/linuxbrew/.linuxbrew/bin:/home/linuxbrew/.linuxbrew/sbin:/home/runner/.local/bin:/opt/pipx_bin:/usr/share/rust/.cargo/bin:/home/runner/.config/composer/vendor/bin:/usr/local/.ghcup/bin:/home/runner/.dotnet/tools:/snap/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin', 'TARGET_BRANCH': 'release_21.01', 'ImageOS': 'ubuntu20', 'GITHUB_ACTION_REF': '', 'GRAALVM_11_ROOT': '/usr/local/graalvm/graalvm-ce-java11-21.2.0', 'GITHUB_SERVER_URL': 'https://github.com', 'PIPX_BIN_DIR': '/opt/pipx_bin', 'PIPX_HOME': '/opt/pipx', 'GITHUB_RUN_NUMBER': '6516', 'INVOCATION_ID': '99d55cf0e43b424eaa6ac95068f8f304', 'ANDROID_HOME': '/usr/local/lib/android/sdk', 'GITHUB_ACTOR': 'mvdbeek', 'LEIN_JAR': '/usr/local/lib/lein/self-installs/leiningen-2.9.7-standalone.jar', 'AZURE_EXTENSION_DIR': '/opt/az/azcliextensions', 'CONDA': '/usr/share/miniconda', 'JAVA_HOME_8_X64': '/usr/lib/jvm/adoptopenjdk-8-hotspot-amd64', 'HOMEBREW_CELLAR': '/home/linuxbrew/.linuxbrew/Cellar', 'ANT_HOME': '/usr/share/ant', 'ImageVersion': '20211011.1', 'GITHUB_REPOSITORY_OWNER': 'galaxyproject', 'HOME': '/home/runner', 'LD_LIBRARY_PATH': '/opt/hostedtoolcache/Python/3.7.12/x64/lib', 'SGX_AESM_ADDR': '1', 'GECKOWEBDRIVER': '/usr/local/share/gecko_driver', 'RUNNER_TEMP': '/home/runner/work/_temp', 'VCPKG_INSTALLATION_ROOT': '/usr/local/share/vcpkg', 'GITHUB_REF': 'refs/heads/release_21.01', 'GITHUB_PATH': '/home/runner/work/_temp/_runner_file_commands/add_path_af35710e-307c-42a6-8b32-01f13bb5d91a', 'ANDROID_NDK_HOME': '/usr/local/lib/android/sdk/ndk-bundle', 'CI': 'true', 'SHLVL': '1', 'HOMEBREW_CLEANUP_PERIODIC_FULL_DAYS': '3650', 'GOROOT_1_15_X64': '/opt/hostedtoolcache/go/1.15.15/x64', 'GITHUB_JOB': 'build', 'JAVA_HOME': '/usr/lib/jvm/adoptopenjdk-11-hotspot-amd64', 'USER': 'runner', 'GITHUB_ACTION_REPOSITORY': '', 'BOOTSTRAP_HASKELL_NONINTERACTIVE': '1', 'GITHUB_ACTIONS': 'true', 'GITHUB_HEAD_REF': '', 'GITHUB_RUN_ATTEMPT': '1', 'RUNNER_TRACKING_ID': 'github_7c17e3f0-8b59-44f9-959a-c5afe9e5ef4a', 'MAKEFLAGS': 'w', 'MFLAGS': '-w', 'NVM_DIR': '/home/runner/.nvm', 'HOMEBREW_SHELLENV_PREFIX': '/home/linuxbrew/.linuxbrew', 'HOMEBREW_PREFIX': '/home/linuxbrew/.linuxbrew', 'GITHUB_EVENT_PATH': '/home/runner/work/_temp/_github_workflow/event.json', 'AGENT_TOOLSDIRECTORY': '/opt/hostedtoolcache', 'HOMEBREW_REPOSITORY': '/home/linuxbrew/.linuxbrew/Homebrew', 'ACCEPT_EULA': 'Y', 'HOMEBREW_NO_AUTO_UPDATE': '1', 'GOROOT_1_14_X64': '/opt/hostedtoolcache/go/1.14.15/x64', 'XDG_CONFIG_HOME': '/home/runner/.config', 'RUNNER_USER': 'runner', 'GITHUB_API_URL': 'https://api.github.com', 'ANDROID_SDK_ROOT': '/usr/local/lib/android/sdk', 'PWD': '/home/runner/work/galaxy/galaxy', 'GITHUB_RUN_ID': '1353402645', 'SWIFT_PATH': '/usr/share/swift/usr/bin', 'DEBIAN_FRONTEND': 'noninteractive', 'CHROME_BIN': '/usr/bin/google-chrome', 'RUNNER_NAME': 'GitHub Actions 28', 'GITHUB_WORKSPACE': '/home/runner/work/galaxy/galaxy', 'GITHUB_SHA': '14b6579eb10eb6ae097e27cbd25966701aef3cfe', 'pythonLocation': '/opt/hostedtoolcache/Python/3.7.12/x64', 'RUNNER_WORKSPACE': '/home/runner/work/galaxy', 'CHROMEWEBDRIVER': '/usr/local/share/chrome_driver', 'DOTNET_NOLOGO': '1', 'JAVA_HOME_11_X64': '/usr/lib/jvm/adoptopenjdk-11-hotspot-amd64', 'RUNNER_OS': 'Linux', 'GITHUB_RETENTION_DAYS': '90', 'SELENIUM_JAR_PATH': '/usr/share/java/selenium-server-standalone.jar', 'LANG': 'C.UTF-8', 'MAKELEVEL': '2', 'DOCUTILSCONFIG': '/home/runner/work/galaxy/galaxy/doc/source/docutils.conf'}))[source]

Bases: object

__init__(file_dirs=None, env_var='GALAXY_TEST_FILE_DIR', environ=environ({'POWERSHELL_DISTRIBUTION_CHANNEL': 'GitHub-Actions-ubuntu20', 'ANDROID_NDK_ROOT': '/usr/local/lib/android/sdk/ndk-bundle', 'GITHUB_EVENT_NAME': 'push', 'GITHUB_ACTION': '__run_8', 'GITHUB_REPOSITORY': 'galaxyproject/galaxy', 'GRADLE_HOME': '/usr/share/gradle-7.2', 'GITHUB_WORKFLOW': 'Build docs', 'PERFLOG_LOCATION_SETTING': 'RUNNER_PERFLOG', 'JOURNAL_STREAM': '8:22305', 'LEIN_HOME': '/usr/local/lib/lein', 'GITHUB_GRAPHQL_URL': 'https://api.github.com/graphql', 'GOROOT_1_16_X64': '/opt/hostedtoolcache/go/1.16.9/x64', 'GITHUB_BASE_REF': '', 'GITHUB_ENV': '/home/runner/work/_temp/_runner_file_commands/set_env_af35710e-307c-42a6-8b32-01f13bb5d91a', 'STATS_KEEPALIVE': 'false', 'GOROOT_1_17_X64': '/opt/hostedtoolcache/go/1.17.2/x64', '_': '/usr/bin/make', 'ANDROID_NDK_LATEST_HOME': '/usr/local/lib/android/sdk/ndk/23.0.7599858', 'RUNNER_PERFLOG': '/home/runner/perflog', 'RUNNER_TOOL_CACHE': '/opt/hostedtoolcache', 'DOTNET_MULTILEVEL_LOOKUP': '0', 'DEPLOYMENT_BASEPATH': '/opt/runner', 'DOTNET_SKIP_FIRST_TIME_EXPERIENCE': '1', 'PATH': '/opt/hostedtoolcache/Python/3.7.12/x64/bin:/opt/hostedtoolcache/Python/3.7.12/x64:/home/linuxbrew/.linuxbrew/bin:/home/linuxbrew/.linuxbrew/sbin:/home/runner/.local/bin:/opt/pipx_bin:/usr/share/rust/.cargo/bin:/home/runner/.config/composer/vendor/bin:/usr/local/.ghcup/bin:/home/runner/.dotnet/tools:/snap/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin', 'TARGET_BRANCH': 'release_21.01', 'ImageOS': 'ubuntu20', 'GITHUB_ACTION_REF': '', 'GRAALVM_11_ROOT': '/usr/local/graalvm/graalvm-ce-java11-21.2.0', 'GITHUB_SERVER_URL': 'https://github.com', 'PIPX_BIN_DIR': '/opt/pipx_bin', 'PIPX_HOME': '/opt/pipx', 'GITHUB_RUN_NUMBER': '6516', 'INVOCATION_ID': '99d55cf0e43b424eaa6ac95068f8f304', 'ANDROID_HOME': '/usr/local/lib/android/sdk', 'GITHUB_ACTOR': 'mvdbeek', 'LEIN_JAR': '/usr/local/lib/lein/self-installs/leiningen-2.9.7-standalone.jar', 'AZURE_EXTENSION_DIR': '/opt/az/azcliextensions', 'CONDA': '/usr/share/miniconda', 'JAVA_HOME_8_X64': '/usr/lib/jvm/adoptopenjdk-8-hotspot-amd64', 'HOMEBREW_CELLAR': '/home/linuxbrew/.linuxbrew/Cellar', 'ANT_HOME': '/usr/share/ant', 'ImageVersion': '20211011.1', 'GITHUB_REPOSITORY_OWNER': 'galaxyproject', 'HOME': '/home/runner', 'LD_LIBRARY_PATH': '/opt/hostedtoolcache/Python/3.7.12/x64/lib', 'SGX_AESM_ADDR': '1', 'GECKOWEBDRIVER': '/usr/local/share/gecko_driver', 'RUNNER_TEMP': '/home/runner/work/_temp', 'VCPKG_INSTALLATION_ROOT': '/usr/local/share/vcpkg', 'GITHUB_REF': 'refs/heads/release_21.01', 'GITHUB_PATH': '/home/runner/work/_temp/_runner_file_commands/add_path_af35710e-307c-42a6-8b32-01f13bb5d91a', 'ANDROID_NDK_HOME': '/usr/local/lib/android/sdk/ndk-bundle', 'CI': 'true', 'SHLVL': '1', 'HOMEBREW_CLEANUP_PERIODIC_FULL_DAYS': '3650', 'GOROOT_1_15_X64': '/opt/hostedtoolcache/go/1.15.15/x64', 'GITHUB_JOB': 'build', 'JAVA_HOME': '/usr/lib/jvm/adoptopenjdk-11-hotspot-amd64', 'USER': 'runner', 'GITHUB_ACTION_REPOSITORY': '', 'BOOTSTRAP_HASKELL_NONINTERACTIVE': '1', 'GITHUB_ACTIONS': 'true', 'GITHUB_HEAD_REF': '', 'GITHUB_RUN_ATTEMPT': '1', 'RUNNER_TRACKING_ID': 'github_7c17e3f0-8b59-44f9-959a-c5afe9e5ef4a', 'MAKEFLAGS': 'w', 'MFLAGS': '-w', 'NVM_DIR': '/home/runner/.nvm', 'HOMEBREW_SHELLENV_PREFIX': '/home/linuxbrew/.linuxbrew', 'HOMEBREW_PREFIX': '/home/linuxbrew/.linuxbrew', 'GITHUB_EVENT_PATH': '/home/runner/work/_temp/_github_workflow/event.json', 'AGENT_TOOLSDIRECTORY': '/opt/hostedtoolcache', 'HOMEBREW_REPOSITORY': '/home/linuxbrew/.linuxbrew/Homebrew', 'ACCEPT_EULA': 'Y', 'HOMEBREW_NO_AUTO_UPDATE': '1', 'GOROOT_1_14_X64': '/opt/hostedtoolcache/go/1.14.15/x64', 'XDG_CONFIG_HOME': '/home/runner/.config', 'RUNNER_USER': 'runner', 'GITHUB_API_URL': 'https://api.github.com', 'ANDROID_SDK_ROOT': '/usr/local/lib/android/sdk', 'PWD': '/home/runner/work/galaxy/galaxy', 'GITHUB_RUN_ID': '1353402645', 'SWIFT_PATH': '/usr/share/swift/usr/bin', 'DEBIAN_FRONTEND': 'noninteractive', 'CHROME_BIN': '/usr/bin/google-chrome', 'RUNNER_NAME': 'GitHub Actions 28', 'GITHUB_WORKSPACE': '/home/runner/work/galaxy/galaxy', 'GITHUB_SHA': '14b6579eb10eb6ae097e27cbd25966701aef3cfe', 'pythonLocation': '/opt/hostedtoolcache/Python/3.7.12/x64', 'RUNNER_WORKSPACE': '/home/runner/work/galaxy', 'CHROMEWEBDRIVER': '/usr/local/share/chrome_driver', 'DOTNET_NOLOGO': '1', 'JAVA_HOME_11_X64': '/usr/lib/jvm/adoptopenjdk-11-hotspot-amd64', 'RUNNER_OS': 'Linux', 'GITHUB_RETENTION_DAYS': '90', 'SELENIUM_JAR_PATH': '/usr/share/java/selenium-server-standalone.jar', 'LANG': 'C.UTF-8', 'MAKELEVEL': '2', 'DOCUTILSCONFIG': '/home/runner/work/galaxy/galaxy/doc/source/docutils.conf'}))[source]

Initialize self. See help(type(self)) for accurate signature.

get_filename(name)[source]
get_filecontent(name)[source]
get_directory(name)[source]
galaxy.tool_util.verify.test_data.build_resolver(uri, environ)[source]
class galaxy.tool_util.verify.test_data.FileDataResolver(file_dir)[source]

Bases: object

__init__(file_dir)[source]

Initialize self. See help(type(self)) for accurate signature.

exists(filename)[source]
path(filename)[source]
class galaxy.tool_util.verify.test_data.GitDataResolver(repository, environ)[source]

Bases: galaxy.tool_util.verify.test_data.FileDataResolver

__init__(repository, environ)[source]

Initialize self. See help(type(self)) for accurate signature.

exists(filename)[source]
update_repository()[source]
execute(cmd)[source]

galaxy.tool_util.verify.wait module

Abstraction for waiting on API conditions to become true.

galaxy.tool_util.verify.wait.wait_on(function, desc, timeout, delta=0.25, polling_backoff=0, sleep_=None)[source]

Wait for function to return non-None value.

Grow the polling interval (initially delta defaulting to 0.25 seconds) incrementally by the supplied polling_backoff (defaulting to 0).

Throw a TimeoutAssertionError if the supplied timeout is reached without supplied function ever returning a non-None value.

exception galaxy.tool_util.verify.wait.TimeoutAssertionError(message)[source]

Bases: AssertionError

Derivative of AssertionError indicating wait_on exceeded max time.

__init__(message)[source]

Initialize self. See help(type(self)) for accurate signature.