Warning
This document is for an old release of Galaxy. You can alternatively view this page in the latest release if it exists or view the top of the latest release's documentation.
galaxy.tool_util.deps package¶
Dependency management for tools.
-
galaxy.tool_util.deps.
build_dependency_manager
(app_config_dict=None, resolution_config_dict=None, conf_file=None, default_tool_dependency_dir=None)[source]¶ Build a DependencyManager object from app and/or resolution config.
If app_config_dict is specified, it should be application configuration information and configuration options are generally named to identify the context of dependency management (e.g. conda_prefix not prefix or use_cached_dependency_manager not cache). resolution_config_dict if specified is assumed to be the to_dict() version of a DependencyManager and should only contain dependency configuration options.
-
class
galaxy.tool_util.deps.
DependencyManager
(default_base_path, conf_file=None, app_config=None)[source]¶ Bases:
object
A DependencyManager attempts to resolve named and versioned dependencies by searching for them under a list of directories. Directories should be of the form:
$BASE/name/version/…
and should each contain a file ‘env.sh’ which can be sourced to make the dependency available in the current shell environment.
-
cached
= False¶
-
__init__
(default_base_path, conf_file=None, app_config=None)[source]¶ Create a new dependency manager looking for packages under the paths listed in base_paths. The default base path is app.config.tool_dependency_dir.
-
set_enabled_container_types
(container_types_to_destinations)[source]¶ Set the union of all enabled container types.
-
property
enabled_container_types
¶ Returns the union of enabled container types.
-
get_resolver_option
(resolver, key, explicit_resolver_options=None)[source]¶ Look in resolver-specific settings for option and then fallback to global settings.
-
property
precache
¶
-
-
class
galaxy.tool_util.deps.
CachedDependencyManager
(default_base_path, **kwd)[source]¶ Bases:
galaxy.tool_util.deps.DependencyManager
-
cached
= True¶
-
__init__
(default_base_path, **kwd)[source]¶ Create a new dependency manager looking for packages under the paths listed in base_paths. The default base path is app.config.tool_dependency_dir.
-
dependency_shell_commands
(requirements, **kwds)[source]¶ Runs a set of requirements through the dependency resolvers and returns a list of commands required to activate the dependencies. If dependencies are cacheable and the cache does not exist, will try to create it. If cached environment exists or is successfully created, will generate commands to activate it.
-
-
class
galaxy.tool_util.deps.
NullDependencyManager
(default_base_path=None, conf_file=None, app_config=None)[source]¶ Bases:
galaxy.tool_util.deps.DependencyManager
-
cached
= False¶
-
Subpackages¶
- galaxy.tool_util.deps.container_resolvers package
- galaxy.tool_util.deps.mulled package
- Submodules
- galaxy.tool_util.deps.mulled.get_tests module
- galaxy.tool_util.deps.mulled.mulled_build module
- galaxy.tool_util.deps.mulled.mulled_build_channel module
- galaxy.tool_util.deps.mulled.mulled_build_files module
- galaxy.tool_util.deps.mulled.mulled_build_tool module
- galaxy.tool_util.deps.mulled.mulled_list module
- galaxy.tool_util.deps.mulled.mulled_search module
- galaxy.tool_util.deps.mulled.mulled_update_singularity_containers module
- galaxy.tool_util.deps.mulled.util module
- galaxy.tool_util.deps.resolvers package
- Submodules
- galaxy.tool_util.deps.resolvers.brewed_tool_shed_packages module
- galaxy.tool_util.deps.resolvers.conda module
- galaxy.tool_util.deps.resolvers.galaxy_packages module
- galaxy.tool_util.deps.resolvers.homebrew module
- galaxy.tool_util.deps.resolvers.lmod module
- galaxy.tool_util.deps.resolvers.modules module
- galaxy.tool_util.deps.resolvers.resolver_mixins module
- galaxy.tool_util.deps.resolvers.tool_shed_packages module
- galaxy.tool_util.deps.resolvers.unlinked_tool_shed_packages module
Submodules¶
galaxy.tool_util.deps.brew_exts module¶
-
class
galaxy.tool_util.deps.brew_exts.
RecipeContext
(recipe, version, brew_context=None)[source]¶ Bases:
object
-
__init__
(recipe, version, brew_context=None)[source]¶ Initialize self. See help(type(self)) for accurate signature.
-
property
cellar_path
¶
-
property
tap_path
¶
-
-
exception
galaxy.tool_util.deps.brew_exts.
CommandLineException
(command, stdout, stderr)[source]¶ Bases:
Exception
-
galaxy.tool_util.deps.brew_exts.
versioned_install
(recipe_context, package=None, version=None, installed_deps=None)[source]¶
-
galaxy.tool_util.deps.brew_exts.
build_env_statements_from_recipe_context
(recipe_context, **kwds)[source]¶
-
galaxy.tool_util.deps.brew_exts.
build_env_statements
(cellar_root, cellar_path, relaxed=None, custom_only=False)[source]¶
-
galaxy.tool_util.deps.brew_exts.
build_env_actions
(deps, cellar_root, cellar_path, relaxed=None, custom_only=False)[source]¶
galaxy.tool_util.deps.brew_util module¶
brew_exts defines generic extensions to Homebrew this file builds on those abstraction and provides Galaxy specific functionality not useful to the brew external commands.
galaxy.tool_util.deps.commands module¶
galaxy.tool_util.deps.conda_compat module¶
Compat. layer with conda_build/verify if Galaxy/galaxy-lib not installed through conda.
In general there are utilities available for Conda building and parsing that are high-quality and should be utilized when available but that are only available in conda channels and not in PyPI. This module serves as a PyPI capable interface to these utilities.
-
galaxy.tool_util.deps.conda_compat.
MetaData
¶ alias of
galaxy.tool_util.deps.conda_compat._MetaData
galaxy.tool_util.deps.conda_util module¶
-
class
galaxy.tool_util.deps.conda_util.
CondaContext
(conda_prefix=None, conda_exec=None, shell_exec=None, debug=False, ensure_channels='', condarc_override=None, use_path_exec=False, copy_dependencies=False, use_local=False)[source]¶ Bases:
galaxy.tool_util.deps.installable.InstallableContext
-
installable_description
= 'Conda'¶
-
__init__
(conda_prefix=None, conda_exec=None, shell_exec=None, debug=False, ensure_channels='', condarc_override=None, use_path_exec=False, copy_dependencies=False, use_local=False)[source]¶ Initialize self. See help(type(self)) for accurate signature.
-
property
conda_version
¶
-
property
conda_build_available
¶
-
can_install_conda
()[source]¶ If conda_exec is set to a path outside of conda_prefix, there is no use installing conda into conda_prefix, since it can’t be used by galaxy. If conda_exec equals conda_prefix/bin/conda, we can install conda if either conda_prefix does not exist or is empty.
-
exec_command
(operation, args, stdout_path=None)[source]¶ Execute the requested command.
Return the process exit code (i.e. 0 in case of success).
-
exec_create
(args, allow_local=True, stdout_path=None)[source]¶ Return the process exit code (i.e. 0 in case of success).
-
exec_remove
(args)[source]¶ Remove a conda environment using conda env remove -y –name args.
Return the process exit code (i.e. 0 in case of success).
-
exec_install
(args, allow_local=True, stdout_path=None)[source]¶ Return the process exit code (i.e. 0 in case of success).
-
exec_clean
(args=None, quiet=False)[source]¶ Clean up after conda installation.
Return the process exit code (i.e. 0 in case of success).
-
property
envs_path
¶
-
property
deactivate
¶
-
property
activate
¶
-
property
parent_path
¶ Return parent path of the location the installable will be created within.
-
-
class
galaxy.tool_util.deps.conda_util.
CondaTarget
(package, version=None, channel=None)[source]¶ Bases:
object
-
__init__
(package, version=None, channel=None)[source]¶ Initialize self. See help(type(self)) for accurate signature.
-
property
package_specifier
¶ Return a package specifier as consumed by conda install/create.
-
property
install_environment
¶ The dependency resolution and installation frameworks will expect each target to be installed it its own environment with a fixed and predictable name given package and version.
-
galaxy.tool_util.deps.container_classes module¶
-
class
galaxy.tool_util.deps.container_classes.
Container
(container_id, app_info, tool_info, destination_info, job_info, container_description, container_name=None)[source]¶ Bases:
object
-
__init__
(container_id, app_info, tool_info, destination_info, job_info, container_description, container_name=None)[source]¶ Initialize self. See help(type(self)) for accurate signature.
-
property
resolve_dependencies
¶
-
property
shell
¶
-
property
source_environment
¶
-
-
galaxy.tool_util.deps.container_classes.
preprocess_volumes
(volumes_raw_str, container_type)[source]¶ Process Galaxy volume specification string to either Docker or Singularity specification.
Galaxy allows the mount try “default_ro” which translates to ro for Docker and ro for Singularity iff no subdirectories are rw (Singularity does not allow ro parent directories with rw subdirectories).
>>> preprocess_volumes("/a/b", DOCKER_CONTAINER_TYPE) ['/a/b:rw'] >>> preprocess_volumes("/a/b:ro,/a/b/c:rw", DOCKER_CONTAINER_TYPE) ['/a/b:ro', '/a/b/c:rw'] >>> preprocess_volumes("/a/b:/a:ro,/a/b/c:/a/b:rw", DOCKER_CONTAINER_TYPE) ['/a/b:/a:ro', '/a/b/c:/a/b:rw'] >>> preprocess_volumes("/a/b:default_ro,/a/b/c:rw", DOCKER_CONTAINER_TYPE) ['/a/b:ro', '/a/b/c:rw'] >>> preprocess_volumes("/a/b:default_ro,/a/b/c:ro", SINGULARITY_CONTAINER_TYPE) ['/a/b:ro', '/a/b/c:ro'] >>> preprocess_volumes("/a/b:default_ro,/a/b/c:rw", SINGULARITY_CONTAINER_TYPE) ['/a/b', '/a/b/c']
-
class
galaxy.tool_util.deps.container_classes.
HasDockerLikeVolumes
[source]¶ Bases:
object
Mixin to share functionality related to Docker volume handling.
Singularity seems to have a fairly compatible syntax for volume handling.
-
class
galaxy.tool_util.deps.container_classes.
DockerContainer
(container_id, app_info, tool_info, destination_info, job_info, container_description, container_name=None)[source]¶ Bases:
galaxy.tool_util.deps.container_classes.Container
,galaxy.tool_util.deps.container_classes.HasDockerLikeVolumes
-
container_type
= 'docker'¶
-
property
docker_host_props
¶
-
property
connection_configuration
¶
-
-
class
galaxy.tool_util.deps.container_classes.
SingularityContainer
(container_id, app_info, tool_info, destination_info, job_info, container_description, container_name=None)[source]¶ Bases:
galaxy.tool_util.deps.container_classes.Container
,galaxy.tool_util.deps.container_classes.HasDockerLikeVolumes
-
container_type
= 'singularity'¶
-
property
connection_configuration
¶
-
galaxy.tool_util.deps.containers module¶
-
class
galaxy.tool_util.deps.containers.
ResolvedContainerDescription
(container_resolver, container_description)¶ Bases:
tuple
-
property
container_description
¶ Alias for field number 1
-
property
container_resolver
¶ Alias for field number 0
-
property
-
class
galaxy.tool_util.deps.containers.
ContainerFinder
(app_info, mulled_resolution_cache=None)[source]¶ Bases:
object
-
__init__
(app_info, mulled_resolution_cache=None)[source]¶ Initialize self. See help(type(self)) for accurate signature.
-
find_best_container_description
(enabled_container_types, tool_info, **kwds)[source]¶ Regardless of destination properties - find best container for tool.
Given container types and container.ToolInfo description of the tool.
-
-
class
galaxy.tool_util.deps.containers.
ContainerRegistry
(app_info, mulled_resolution_cache=None)[source]¶ Bases:
object
Loop through enabled ContainerResolver plugins and find first match.
-
__init__
(app_info, mulled_resolution_cache=None)[source]¶ Initialize self. See help(type(self)) for accurate signature.
-
galaxy.tool_util.deps.dependencies module¶
-
class
galaxy.tool_util.deps.dependencies.
AppInfo
(galaxy_root_dir=None, default_file_path=None, tool_data_path=None, shed_tool_data_path=None, outputs_to_working_directory=False, container_image_cache_path=None, library_import_dir=None, enable_mulled_containers=False, containers_resolvers_config_file=None, involucro_path=None, involucro_auto_init=True, mulled_channels=['conda-forge', 'bioconda'])[source]¶ Bases:
object
-
__init__
(galaxy_root_dir=None, default_file_path=None, tool_data_path=None, shed_tool_data_path=None, outputs_to_working_directory=False, container_image_cache_path=None, library_import_dir=None, enable_mulled_containers=False, containers_resolvers_config_file=None, involucro_path=None, involucro_auto_init=True, mulled_channels=['conda-forge', 'bioconda'])[source]¶ Initialize self. See help(type(self)) for accurate signature.
-
-
class
galaxy.tool_util.deps.dependencies.
ToolInfo
(container_descriptions=None, requirements=None, requires_galaxy_python_environment=False, env_pass_through=None, guest_ports=None, tool_id=None, tool_version=None, profile=- 1)[source]¶ Bases:
object
-
class
galaxy.tool_util.deps.dependencies.
JobInfo
(working_directory, tool_directory, job_directory, tmp_directory, home_directory, job_directory_type)[source]¶ Bases:
object
-
class
galaxy.tool_util.deps.dependencies.
DependenciesDescription
(requirements=None, installed_tool_dependencies=None)[source]¶ Bases:
object
Capture (in a readily serializable way) context related a tool dependencies - both the tool’s listed requirements and the tool shed related context required to resolve dependencies via the ToolShedPackageDependencyResolver.
This is meant to enable remote resolution of dependencies, by the Pulsar or other potential remote execution mechanisms.
galaxy.tool_util.deps.docker_util module¶
Utilities for building up Docker commands…
…using common defaults and configuration mechanisms.
-
galaxy.tool_util.deps.docker_util.
build_docker_simple_command
(command, docker_cmd='docker', sudo=False, sudo_cmd='sudo', container_name=None, **kwd)[source]¶
-
galaxy.tool_util.deps.docker_util.
build_docker_run_command
(container_command, image, interactive=False, terminal=False, tag=None, volumes=None, volumes_from=None, memory=None, env_directives=None, working_directory=None, name=None, net=None, run_extra_arguments=None, docker_cmd='docker', sudo=False, sudo_cmd='sudo', auto_rm=True, set_user='$UID', host=None, guest_ports=False, container_name=None)[source]¶
-
galaxy.tool_util.deps.docker_util.
command_list
(command, command_args=None, **kwds)[source]¶ Return Docker command as an argv list.
galaxy.tool_util.deps.dockerfiles module¶
galaxy.tool_util.deps.installable module¶
Abstractions for installing local software managed and required by Galaxy/galaxy-lib.
-
class
galaxy.tool_util.deps.installable.
InstallableContext
[source]¶ Bases:
object
Represent a directory/configuration of something that can be installed.
-
abstract property
installable_description
¶ Short description of thing being installed for log statements.
-
abstract property
parent_path
¶ Return parent path of the location the installable will be created within.
-
abstract property
galaxy.tool_util.deps.requirements module¶
-
class
galaxy.tool_util.deps.requirements.
ToolRequirement
(name=None, type=None, version=None, specs=None)[source]¶ Bases:
object
Represents an external requirement that must be available for the tool to run (for example, a program, package, or library). Requirements can optionally assert a specific version.
-
class
galaxy.tool_util.deps.requirements.
RequirementSpecification
(uri, version=None)[source]¶ Bases:
object
Refine a requirement using a URI.
-
property
specifies_version
¶
-
property
short_name
¶
-
property
-
class
galaxy.tool_util.deps.requirements.
ToolRequirements
(tool_requirements=None)[source]¶ Bases:
object
Represents all requirements (packages, env vars) needed to run a tool.
-
__init__
(tool_requirements=None)[source]¶ Initialize self. See help(type(self)) for accurate signature.
-
property
resolvable
¶
-
property
packages
¶
-
-
class
galaxy.tool_util.deps.requirements.
ContainerDescription
(identifier=None, type='docker', resolve_dependencies=False, shell='/bin/sh')[source]¶ Bases:
object
-
galaxy.tool_util.deps.requirements.
parse_requirements_from_xml
(xml_root)[source]¶ >>> from galaxy.util import parse_xml_string >>> def load_requirements(contents): ... contents_document = '''<tool><requirements>%s</requirements></tool>''' ... root = parse_xml_string(contents_document % contents) ... return parse_requirements_from_xml(root) >>> reqs, containers = load_requirements('''<requirement>bwa</requirement>''') >>> reqs[0].name 'bwa' >>> reqs[0].version is None True >>> reqs[0].type 'package' >>> reqs, containers = load_requirements('''<requirement type="binary" version="1.3.3">cufflinks</requirement>''') >>> reqs[0].name 'cufflinks' >>> reqs[0].version '1.3.3' >>> reqs[0].type 'binary'
galaxy.tool_util.deps.singularity_util module¶
galaxy.tool_util.deps.views module¶
-
class
galaxy.tool_util.deps.views.
DependencyResolversView
(app)[source]¶ Bases:
object
Provide a RESTfulish/JSONy interface to a galaxy.tool_util.deps.DependencyResolver object. This can be adapted by the Galaxy web framework or other web apps.
-
show_dependencies
(tool_requirements_d, installed_tool_dependencies=None, **kwd)[source]¶ Resolves dependencies to build a requirements status in the admin panel/API
-
uninstall_dependencies
(index=None, resolver_type=None, container_type=None, **payload)[source]¶ Attempt to uninstall requirements. Returns 0 if successfull, else None.
-
property
unused_dependency_paths
¶ List dependencies that are not currently installed.
-
remove_unused_dependency_paths
(envs)[source]¶ Remove dependencies that are not currently used.
Returns a list of all environments that have been successfully removed.
-
install_dependency
(index=None, **payload)[source]¶ Installs dependency using highest priority resolver that supports dependency installation (Currently only the conda resolver supports this). If index is given, attempt installation directly using the corresponding resolver. Returns True on success, False on failure. payload is dictionary that must container name, version and type, e.g. {‘name’: ‘numpy’, version=’1.9.1’, type=’package’}
-
property
installable_resolvers
¶ List index for all active resolvers that have the ‘install_dependency’ attribute.
-
property
uninstallable_resolvers
¶ List index for all active resolvers that can uninstall dependencies that have been installed through this resolver.
-
property
tool_ids_by_requirements
¶ Dictionary with requirements as keys, and tool_ids as values.
-
property
toolbox_requirements_status
¶
-