Warning
This document is for an in-development version of Galaxy. You can alternatively view this page in the latest release if it exists or view the top of the latest release's documentation.
galaxy.tool_util package¶
Subpackages¶
- galaxy.tool_util.client package
- galaxy.tool_util.cwl package
- galaxy.tool_util.deps package
- Subpackages
- galaxy.tool_util.deps.container_resolvers package
- galaxy.tool_util.deps.mulled package
- Submodules
- galaxy.tool_util.deps.mulled.get_tests module
- galaxy.tool_util.deps.mulled.mulled_build module
- galaxy.tool_util.deps.mulled.mulled_build_channel module
- galaxy.tool_util.deps.mulled.mulled_build_files module
- galaxy.tool_util.deps.mulled.mulled_build_tool module
- galaxy.tool_util.deps.mulled.mulled_list module
- galaxy.tool_util.deps.mulled.mulled_search module
- galaxy.tool_util.deps.mulled.mulled_update_singularity_containers module
- galaxy.tool_util.deps.mulled.util module
- galaxy.tool_util.deps.resolvers package
- Submodules
- galaxy.tool_util.deps.resolvers.brewed_tool_shed_packages module
- galaxy.tool_util.deps.resolvers.conda module
- galaxy.tool_util.deps.resolvers.galaxy_packages module
- galaxy.tool_util.deps.resolvers.homebrew module
- galaxy.tool_util.deps.resolvers.lmod module
- galaxy.tool_util.deps.resolvers.modules module
- galaxy.tool_util.deps.resolvers.resolver_mixins module
- galaxy.tool_util.deps.resolvers.tool_shed_packages module
- galaxy.tool_util.deps.resolvers.unlinked_tool_shed_packages module
- Submodules
- galaxy.tool_util.deps.brew_exts module
- galaxy.tool_util.deps.brew_util module
- galaxy.tool_util.deps.commands module
- galaxy.tool_util.deps.conda_compat module
- galaxy.tool_util.deps.conda_util module
- galaxy.tool_util.deps.container_classes module
- galaxy.tool_util.deps.containers module
- galaxy.tool_util.deps.dependencies module
- galaxy.tool_util.deps.docker_util module
- galaxy.tool_util.deps.dockerfiles module
- galaxy.tool_util.deps.installable module
- galaxy.tool_util.deps.requirements module
- galaxy.tool_util.deps.singularity_util module
- galaxy.tool_util.deps.views module
- Subpackages
- galaxy.tool_util.linters package
- Submodules
- galaxy.tool_util.linters.citations module
- galaxy.tool_util.linters.command module
- galaxy.tool_util.linters.cwl module
- galaxy.tool_util.linters.general module
- galaxy.tool_util.linters.help module
- galaxy.tool_util.linters.inputs module
- galaxy.tool_util.linters.outputs module
- galaxy.tool_util.linters.stdio module
- galaxy.tool_util.linters.tests module
- galaxy.tool_util.linters.xml_order module
- galaxy.tool_util.locations package
- galaxy.tool_util.parser package
- Submodules
- galaxy.tool_util.parser.cwl module
- galaxy.tool_util.parser.factory module
- galaxy.tool_util.parser.interface module
- galaxy.tool_util.parser.output_actions module
- galaxy.tool_util.parser.output_collection_def module
- galaxy.tool_util.parser.output_objects module
- galaxy.tool_util.parser.stdio module
- galaxy.tool_util.parser.util module
- galaxy.tool_util.parser.xml module
- galaxy.tool_util.parser.yaml module
- galaxy.tool_util.verify package
Submodules¶
galaxy.tool_util.fetcher module¶
galaxy.tool_util.lint module¶
This modules contains the functions that drive the tool linting framework.
LintContext: a container for LintMessages LintMessage: the actual message and level
The idea is to define a LintContext and to apply a linting function foo on a target. The level (defined by LintLevel) determines which linting messages are shown.
`
lint_ctx = LintContext(level) # level is the reporting level
lint_ctx.lint(..., lint_func = foo, lint_target = target, ...)
`
The lint function essentially calls foo(target, self). Hence the function foo must have two parameters
the object to lint
the lint context
In foo the lint context can be used to add LintMessages to the lint context by using its valid, info, warn, and error functions:
``` lint_foo(target, lint_ctx):
lint_ctx.error(“target is screwed”)
Calling lint prints out the messages emmited by the linter function immediately. Which messages are shown can be determined with the level argument of the LintContext constructor. If set to SILENT, no messages will be printed.
For special lint targets it might be useful to have additional information in the lint messages. This can be achieved by sublassing LintMessage. See for instance XMLLintMessageLine which has an additional agrument node in its constructor which is used to determine the line and filename in an XML document that caused the message.
In order to use this.
the lint context needs to be initialized with the additional parameter lint_message_class=XMLLintMessageLine
the additional parameter needs to be added as well to calls adding messages to the lint context, e.g. lint_ctx.error(“some message”, node=X). Note that the additional properties must be given as keyword arguments.
- class galaxy.tool_util.lint.LintLevel(value)[source]¶
Bases:
IntEnum
An enumeration.
- SILENT = 5¶
- ERROR = 4¶
- WARN = 3¶
- INFO = 2¶
- VALID = 1¶
- ALL = 0¶
- class galaxy.tool_util.lint.LintMessage(level: str, message: str, **kwargs)[source]¶
Bases:
object
a message from the linter
- class galaxy.tool_util.lint.XMLLintMessageLine(level: str, message: str, node: Optional[Element] = None)[source]¶
Bases:
LintMessage
- class galaxy.tool_util.lint.XMLLintMessageXPath(level: str, message: str, node: Optional[Element] = None)[source]¶
Bases:
LintMessage
- class galaxy.tool_util.lint.LintContext(level: ~typing.Union[~galaxy.tool_util.lint.LintLevel, str], lint_message_class: ~typing.Type[~galaxy.tool_util.lint.LintMessage] = <class 'galaxy.tool_util.lint.LintMessage'>, skip_types: ~typing.Optional[~typing.List[str]] = None, object_name: ~typing.Optional[str] = None)[source]¶
Bases:
object
- __init__(level: ~typing.Union[~galaxy.tool_util.lint.LintLevel, str], lint_message_class: ~typing.Type[~galaxy.tool_util.lint.LintMessage] = <class 'galaxy.tool_util.lint.LintMessage'>, skip_types: ~typing.Optional[~typing.List[str]] = None, object_name: ~typing.Optional[str] = None)[source]¶
- lint_message_class: Type[LintMessage]¶
- message_list: List[LintMessage]¶
- lint(name: str, lint_func: Callable[[LintTargetType, LintContext], None], lint_target: LintTargetType)[source]¶
- property valid_messages: List[LintMessage]¶
- property info_messages: List[LintMessage]¶
- property warn_messages: List[LintMessage]¶
- property error_messages: List[LintMessage]¶
- galaxy.tool_util.lint.lint_tool_source(tool_source, level=LintLevel.ALL, fail_level=LintLevel.WARN, extra_modules=None, skip_types=None, name=None) bool [source]¶
apply all (applicable) linters from the linters submodule and the ones in extramodules
immediately print linter messages (wrt level) and return if linting failed (wrt fail_level)
- galaxy.tool_util.lint.get_lint_context_for_tool_source(tool_source, extra_modules=None, skip_types=None, name=None) LintContext [source]¶
this is the silent variant of lint_tool_source it returns the LintContext from which all linter messages and the status can be obtained
- galaxy.tool_util.lint.lint_xml(tool_xml, level=LintLevel.ALL, fail_level=LintLevel.WARN, lint_message_class=<class 'galaxy.tool_util.lint.LintMessage'>, extra_modules=None, skip_types=None, name=None) bool [source]¶
lint an xml tool
- galaxy.tool_util.lint.lint_tool_source_with(lint_context, tool_source, extra_modules=None) LintContext [source]¶
- galaxy.tool_util.lint.lint_xml_with(lint_context, tool_xml, extra_modules=None) LintContext [source]¶
galaxy.tool_util.loader module¶
- galaxy.tool_util.loader.load_tool(path)¶
- galaxy.tool_util.loader.load_tool_with_refereces(path)¶
Load XML documentation from file system and preprocesses XML macros.
Return the XML representation of the expanded tree and paths to referenced files that were imported (macros).
- galaxy.tool_util.loader.raw_tool_xml_tree(path)¶
Load raw (no macro expansion) tree representation of XML represented at the specified path.
galaxy.tool_util.loader_directory module¶
Utilities for loading and reasoning about unparsed tools in directories.
- galaxy.tool_util.loader_directory.find_possible_tools_from_path(path, recursive=False, enable_beta_formats=False)[source]¶
Walk a directory and find potential tool files.
- galaxy.tool_util.loader_directory.is_a_yaml_with_class(path, classes)[source]¶
Determine if a file is a valid YAML with a supplied
class
entry.
- galaxy.tool_util.loader_directory.is_tool_load_error(obj)[source]¶
Predicate to determine if object loaded for tool is a tool error.
- galaxy.tool_util.loader_directory.load_tool_elements_from_path(path, load_exception_handler=<function load_exception_handler>, recursive=False, register_load_errors=False)[source]¶
Walk a directory and load tool XML elements.
- galaxy.tool_util.loader_directory.load_tool_sources_from_path(path, load_exception_handler=<function load_exception_handler>, recursive=False, register_load_errors=False)[source]¶
Walk a directory and ToolSource objects.
- galaxy.tool_util.loader_directory.looks_like_a_cwl_artifact(path, classes=None)[source]¶
Quick check to see if a file looks like it may be a CWL artifact.
- galaxy.tool_util.loader_directory.looks_like_a_tool_cwl(path)[source]¶
Quick check to see if a file looks like it may be a CWL tool.
galaxy.tool_util.output_checker module¶
- class galaxy.tool_util.output_checker.DETECTED_JOB_STATE(value)[source]¶
-
An enumeration.
- OK = 'ok'¶
- OUT_OF_MEMORY_ERROR = 'oom_error'¶
- GENERIC_ERROR = 'generic_error'¶
- galaxy.tool_util.output_checker.check_output_regex(job_id_tag, regex, stream, stream_name, job_messages, max_error_level)[source]¶
check a single regex against a stream
regex the regex to check stream the stream to search in job_messages a list where the descriptions of the detected regexes can be appended max_error_level the maximum error level that has been detected so far returns the max of the error_level of the regex and the given max_error_level
- galaxy.tool_util.output_checker.check_output(stdio_regexes, stdio_exit_codes, stdout, stderr, tool_exit_code, job_id_tag)[source]¶
Check the output of a tool - given the stdout, stderr, and the tool’s exit code, return DETECTED_JOB_STATE.OK if the tool exited succesfully or error type otherwise. No exceptions should be thrown. If this code encounters an exception, it returns OK so that the workflow can continue; otherwise, a bug in this code could halt workflow progress.
Note that, if the tool did not define any exit code handling or any stdio/stderr handling, then it reverts back to previous behavior: if stderr contains anything, then False is returned.
galaxy.tool_util.provided_metadata module¶
- galaxy.tool_util.provided_metadata.parse_tool_provided_metadata(meta_file, provided_metadata_style=None, job_wrapper=None)[source]¶
Return a ToolProvidedMetadata object for specified file path.
If meta_file is absent, return a NullToolProvidedMetadata. If provided_metadata_style is None attempt to guess tool provided metadata type.
- class galaxy.tool_util.provided_metadata.BaseToolProvidedMetadata[source]¶
Bases:
object
- get_new_datasets(output_name)[source]¶
Find new datasets for dataset discovery for specified output.
Return a list of such datasets.
Called only in the context of discovering datasets when discover_via=”tool_provided_metadata” is defined in the tool.
- has_failed_outputs()[source]¶
Determine if generation of any of the outputs failed.
If True, this method should also log information about at least the first such failed output.
- get_new_dataset_meta_by_basename(output_name, basename)[source]¶
For a discovered dataset, get the corresponding metadata entry.
The discovery may have been from explicit listing in this file (returned from get_new_datasets) or via file regex, either way the basename of the file is used to index the fetching of the metadata entry.
- get_unnamed_outputs()[source]¶
Return unnamed outputs dataset introduced for upload 2.0.
Needs more formal specification but see output_collect for how destinations, types, elements, etc… are consumed.
- get_dataset_meta(output_name, dataset_id, dataset_uuid)[source]¶
Return primary dataset metadata for specified output.
- class galaxy.tool_util.provided_metadata.NullToolProvidedMetadata[source]¶
Bases:
BaseToolProvidedMetadata
- class galaxy.tool_util.provided_metadata.LegacyToolProvidedMetadata(meta_file, job_wrapper=None)[source]¶
Bases:
BaseToolProvidedMetadata
- get_dataset_meta(output_name, dataset_id, dataset_uuid)[source]¶
Return primary dataset metadata for specified output.
- get_new_dataset_meta_by_basename(output_name, basename)[source]¶
For a discovered dataset, get the corresponding metadata entry.
The discovery may have been from explicit listing in this file (returned from get_new_datasets) or via file regex, either way the basename of the file is used to index the fetching of the metadata entry.
- get_new_datasets(output_name)[source]¶
Find new datasets for dataset discovery for specified output.
Return a list of such datasets.
Called only in the context of discovering datasets when discover_via=”tool_provided_metadata” is defined in the tool.
- has_failed_outputs()[source]¶
Determine if generation of any of the outputs failed.
If True, this method should also log information about at least the first such failed output.
- get_unnamed_outputs()[source]¶
Return unnamed outputs dataset introduced for upload 2.0.
Needs more formal specification but see output_collect for how destinations, types, elements, etc… are consumed.
- class galaxy.tool_util.provided_metadata.ToolProvidedMetadata(meta_file)[source]¶
Bases:
BaseToolProvidedMetadata
- get_dataset_meta(output_name, dataset_id, dataset_uuid)[source]¶
Return primary dataset metadata for specified output.
- get_new_dataset_meta_by_basename(output_name, basename)[source]¶
For a discovered dataset, get the corresponding metadata entry.
The discovery may have been from explicit listing in this file (returned from get_new_datasets) or via file regex, either way the basename of the file is used to index the fetching of the metadata entry.
- get_new_datasets(output_name)[source]¶
Find new datasets for dataset discovery for specified output.
Return a list of such datasets.
Called only in the context of discovering datasets when discover_via=”tool_provided_metadata” is defined in the tool.
- has_failed_outputs()[source]¶
Determine if generation of any of the outputs failed.
If True, this method should also log information about at least the first such failed output.