Warning

This document is for an in-development version of Galaxy. You can alternatively view this page in the latest release if it exists or view the top of the latest release's documentation.

galaxy.tool_util.deps.mulled package

Submodules

galaxy.tool_util.deps.mulled.get_tests module

searches for tests for packages in the bioconda-recipes repo and on Anaconda, looking in different file locations. If no test can be found for the specified version, it will look for tests for other versions of the same package.

A shallow search (default for singularity and conda generation scripts) just checks once on Anaconda for the specified version.

galaxy.tool_util.deps.mulled.get_tests.get_commands_from_yaml(yaml_content: bytes) Dict[str, Any] | None[source]

Parse tests from Conda’s meta.yaml file contents

galaxy.tool_util.deps.mulled.get_tests.get_run_test(file: str) Dict[str, Any][source]

Get tests from a run_test.sh file

galaxy.tool_util.deps.mulled.get_tests.get_anaconda_url(container, anaconda_channel='bioconda')[source]

Download tarball from anaconda for test

galaxy.tool_util.deps.mulled.get_tests.prepend_anaconda_url(url)[source]

Take a partial url and prepend ‘https://anaconda.org

galaxy.tool_util.deps.mulled.get_tests.get_test_from_anaconda(url: str) Dict[str, Any] | None[source]

Given the URL of an anaconda tarball, return tests

galaxy.tool_util.deps.mulled.get_tests.find_anaconda_versions(name, anaconda_channel='bioconda')[source]

Find a list of available anaconda versions for a given container name

galaxy.tool_util.deps.mulled.get_tests.open_recipe_file(file, recipes_path=None, github_repo='bioconda/bioconda-recipes')[source]

Open a file at a particular location and return contents as string

galaxy.tool_util.deps.mulled.get_tests.get_alternative_versions(filepath, filename, recipes_path=None, github_repo='bioconda/bioconda-recipes')[source]

Return files that match filepath/*/filename in the bioconda-recipes repository

galaxy.tool_util.deps.mulled.get_tests.try_a_func(func1, func2, param, container)[source]

Try to perform a function (or actually a combination of two functions: first getting the file and then processing it)

Look in bioconda-recipes repo as well as anaconda for the tests, checking in multiple possible locations. If no test is found for the specified version, search if other package versions have a test available.

Download tarball from anaconda for test

galaxy.tool_util.deps.mulled.get_tests.import_test_to_command_list(import_lang: str, import_: str) List[str][source]

Get test for hashed containers

galaxy.tool_util.deps.mulled.mulled_build module

Build a mulled image for specified conda targets.

Examples

Build a mulled image with:

mulled-build build ‘samtools=1.3.1–4,bedtools=2.22’

galaxy.tool_util.deps.mulled.mulled_build.main(argv=None)[source]

Main entry-point for the CLI tool.

galaxy.tool_util.deps.mulled.mulled_build.build_target(package_name: str, version: str | None = None, build: str | None = None, tag: str | None = None) CondaTarget[source]

Use supplied arguments to build a CondaTarget object.

galaxy.tool_util.deps.mulled.mulled_build_channel module

Build a mulled images for all recent conda recipe updates that don’t have existing images.

Examples:

Build mulled images for recent bioconda changes with:

mulled-build-channel build

Build, test, and publish images with the follow command:

mulled-build-channel all

See recent changes that would be built with:

mulled-build-channel list

galaxy.tool_util.deps.mulled.mulled_build_channel.main(argv=None)[source]

Main entry-point for the CLI tool.

galaxy.tool_util.deps.mulled.mulled_build_files module

Build all composite mulled recipes discovered in TSV files.

Use mulled-build-channel to build images for single recipes for a whole conda channel. This script instead builds images for combinations of recipes. This script can be given a single TSV file or a directory of TSV files to process.

Examples:

Build all recipes discovered in tsv files in a single directory.

mulled-build-files build

galaxy.tool_util.deps.mulled.mulled_build_files.main(argv=None)[source]

Main entry-point for the CLI tool.

galaxy.tool_util.deps.mulled.mulled_build_tool module

Build a mulled images for a tool source (Galaxy or CWL tool).

Examples:

Build mulled images for requirements defined in a tool:

mulled-build-tool build path/to/tool_file.xml

galaxy.tool_util.deps.mulled.mulled_build_tool.main(argv=None) None[source]

Main entry-point for the CLI tool.

galaxy.tool_util.deps.mulled.mulled_build_tool.requirements_to_mulled_targets(requirements) List[CondaTarget][source]

Convert Galaxy’s representation of requirements into a list of CondaTarget objects.

Only package requirements are retained.

galaxy.tool_util.deps.mulled.mulled_hash module

Produce a mulled hash for specified conda targets.

Examples

Produce a mulled hash with:

mulled-hash samtools=1.3.1,bedtools=2.22

galaxy.tool_util.deps.mulled.mulled_hash.main(argv=None)[source]

Main entry-point for the CLI tool.

galaxy.tool_util.deps.mulled.mulled_list module

galaxy.tool_util.deps.mulled.mulled_list.get_quay_containers(repository='biocontainers')[source]

Get all quay containers in the biocontainers repo

galaxy.tool_util.deps.mulled.mulled_list.get_singularity_containers()[source]

Get all existing singularity containers from “https://depot.galaxyproject.org/singularity/

galaxy.tool_util.deps.mulled.mulled_list.get_conda_envs(filepath)[source]

Get list of already existing envs

galaxy.tool_util.deps.mulled.mulled_list.get_missing_containers(quay_list, singularity_list, blocklist_file=None)[source]

Return list of quay containers that do not exist as singularity containers. Files stored in a blocklist will be ignored

galaxy.tool_util.deps.mulled.mulled_list.get_missing_envs(quay_list, conda_list, blocklist_file=None)[source]

Compares list of conda envs and docker containers and returns missing conda envs

galaxy.tool_util.deps.mulled.mulled_list.main()[source]

galaxy.tool_util.deps.mulled.mulled_update_singularity_containers module

galaxy.tool_util.deps.mulled.mulled_update_singularity_containers.get_list_from_file(filename)[source]

Returns a list of containers stored in a file (one on each line)

galaxy.tool_util.deps.mulled.mulled_update_singularity_containers.docker_to_singularity(container, installation, filepath, no_sudo=False)[source]

Convert docker to singularity container.

galaxy.tool_util.deps.mulled.mulled_update_singularity_containers.singularity_container_test(tests: Dict[str, Dict[str, Any]], installation: str, filepath: str | PathLike) Dict[str, List][source]

Run tests, record if they pass or fail

galaxy.tool_util.deps.mulled.mulled_update_singularity_containers.main()[source]
galaxy.tool_util.deps.mulled.mulled_update_singularity_containers.container_testing(args=None)[source]

galaxy.tool_util.deps.mulled.util module

Utilities for working with mulled abstractions outside the mulled package.

galaxy.tool_util.deps.mulled.util.build_target(package_name: str, version: str | None = None, build: str | None = None, tag: str | None = None) CondaTarget[source]

Use supplied arguments to build a CondaTarget object.

galaxy.tool_util.deps.mulled.util.conda_build_target_str(target: CondaTarget) str[source]
galaxy.tool_util.deps.mulled.util.get_files_from_conda_package(url: str, filepaths: Iterable[str]) Dict[str, bytes][source]

Get content of specified files in a conda package. The url can be a path to a local file or an url. The filepaths is an iterable of paths to extract from the conda package, if found in it. Return a dictionary mapping each found filepath to the corresponding content (as bytes).

>>> content_dict = get_files_from_conda_package("https://anaconda.org/conda-forge/chopin2/1.0.6/download/noarch/chopin2-1.0.6-pyhd8ed1ab_0.tar.bz2", ["info/recipe/meta.yaml"])
>>> assert "info/recipe/meta.yaml" in content_dict, content_dict
>>> assert isinstance(content_dict["info/recipe/meta.yaml"], bytes)
>>> content_dict = get_files_from_conda_package("https://anaconda.org/conda-forge/chopin2/1.0.7/download/noarch/chopin2-1.0.7-pyhd8ed1ab_1.conda", ["info/about.json", "info/recipe/meta.yaml", "foo/bar"])
>>> assert sorted(content_dict.keys()) == ["info/about.json", "info/recipe/meta.yaml"], content_dict
galaxy.tool_util.deps.mulled.util.image_name(targets: Iterable[CondaTarget], image_build: str | None = None, name_override: str | None = None) str

Generate mulled hash version 1 container identifier for supplied arguments.

If a single target is specified, simply use the supplied name and version as the repository name and tag respectively. If multiple targets are supplied, hash the package names and versions together as the repository name. For mulled version 1 containers the image build is the repository tag (if supplied).

>>> single_targets = [build_target("samtools", version="1.3.1")]
>>> v1_image_name(single_targets)
'samtools:1.3.1'
>>> multi_targets = [build_target("samtools", version="1.3.1"), build_target("bwa", version="0.7.13")]
>>> v1_image_name(multi_targets)
'mulled-v1-b06ecbd9141f0dbbc0c287375fc0813adfcbdfbd'
>>> multi_targets_on_versionless = [build_target("samtools", version="1.3.1"), build_target("bwa")]
>>> v1_image_name(multi_targets_on_versionless)
'mulled-v1-bda945976caa5734347fbf7f35066d9f58519e0c'
>>> multi_targets_versionless = [build_target("samtools"), build_target("bwa")]
>>> v1_image_name(multi_targets_versionless)
'mulled-v1-fe8faa35dbf6dc65a0f7f5d4ea12e31a79f73e40'
galaxy.tool_util.deps.mulled.util.mulled_tags_for(namespace: str, image: str, tag_prefix: str | None = None, resolution_cache: ResolutionCache | None = None, session: Session | None = None, expire: float = 300) List[str][source]

Fetch remote tags available for supplied image name.

The result will be sorted so newest tags are first.

galaxy.tool_util.deps.mulled.util.quay_versions(namespace: str, pkg_name: str, session: Session | None = None) List[str][source]

Get all version tags for a Docker image stored on quay.io for supplied package name.

galaxy.tool_util.deps.mulled.util.split_container_name(name: str) List[str][source]

Takes a container name (e.g. samtools:1.7–1) and returns a list (e.g. [‘samtools’, ‘1.7’, ‘1’]) >>> split_container_name(‘samtools:1.7–1’) [‘samtools’, ‘1.7’, ‘1’]

galaxy.tool_util.deps.mulled.util.split_tag(tag: str) List[str][source]

Split mulled image tag into conda version and conda build.

galaxy.tool_util.deps.mulled.util.v1_image_name(targets: Iterable[CondaTarget], image_build: str | None = None, name_override: str | None = None) str[source]

Generate mulled hash version 1 container identifier for supplied arguments.

If a single target is specified, simply use the supplied name and version as the repository name and tag respectively. If multiple targets are supplied, hash the package names and versions together as the repository name. For mulled version 1 containers the image build is the repository tag (if supplied).

>>> single_targets = [build_target("samtools", version="1.3.1")]
>>> v1_image_name(single_targets)
'samtools:1.3.1'
>>> multi_targets = [build_target("samtools", version="1.3.1"), build_target("bwa", version="0.7.13")]
>>> v1_image_name(multi_targets)
'mulled-v1-b06ecbd9141f0dbbc0c287375fc0813adfcbdfbd'
>>> multi_targets_on_versionless = [build_target("samtools", version="1.3.1"), build_target("bwa")]
>>> v1_image_name(multi_targets_on_versionless)
'mulled-v1-bda945976caa5734347fbf7f35066d9f58519e0c'
>>> multi_targets_versionless = [build_target("samtools"), build_target("bwa")]
>>> v1_image_name(multi_targets_versionless)
'mulled-v1-fe8faa35dbf6dc65a0f7f5d4ea12e31a79f73e40'
galaxy.tool_util.deps.mulled.util.v2_image_name(targets: Iterable[CondaTarget], image_build: str | None = None, name_override: str | None = None) str[source]

Generate mulled hash version 2 container identifier for supplied arguments.

If a single target is specified, simply use the supplied name and version as the repository name and tag respectively. If multiple targets are supplied, hash the package names as the repository name and hash the package versions (if set) as the tag.

>>> single_targets = [build_target("samtools", version="1.3.1")]
>>> v2_image_name(single_targets)
'samtools:1.3.1'
>>> single_targets = [build_target("samtools", version="1.3.1", build="py_1")]
>>> v2_image_name(single_targets)
'samtools:1.3.1--py_1'
>>> single_targets = [build_target("samtools", version="1.3.1")]
>>> v2_image_name(single_targets, image_build="0")
'samtools:1.3.1'
>>> single_targets = [build_target("samtools", version="1.3.1", build="py_1")]
>>> v2_image_name(single_targets, image_build="0")
'samtools:1.3.1--py_1'
>>> multi_targets = [build_target("samtools", version="1.3.1"), build_target("bwa", version="0.7.13")]
>>> v2_image_name(multi_targets)
'mulled-v2-fe8faa35dbf6dc65a0f7f5d4ea12e31a79f73e40:4d0535c94ef45be8459f429561f0894c3fe0ebcf'
>>> multi_targets_on_versionless = [build_target("samtools", version="1.3.1"), build_target("bwa")]
>>> v2_image_name(multi_targets_on_versionless)
'mulled-v2-fe8faa35dbf6dc65a0f7f5d4ea12e31a79f73e40:b0c847e4fb89c343b04036e33b2daa19c4152cf5'
>>> multi_targets_versionless = [build_target("samtools"), build_target("bwa")]
>>> v2_image_name(multi_targets_versionless)
'mulled-v2-fe8faa35dbf6dc65a0f7f5d4ea12e31a79f73e40'
galaxy.tool_util.deps.mulled.util.version_sorted(elements: Iterable[str]) List[str][source]

Sort iterable based on loose description of “version” from newest to oldest.