Warning

This document is for an old release of Galaxy. You can alternatively view this page in the latest release if it exists or view the top of the latest release's documentation.

tool_shed.util package

Submodules

tool_shed.util.basic_util module

tool_shed.util.basic_util.evaluate_template(text, install_environment)

Substitute variables defined in XML blocks from dependencies file. The value of the received repository_install_dir is the root installation directory of the repository that contains the tool dependency. The value of the received install_dir is the root installation directory of the tool_dependency.

tool_shed.util.basic_util.get_env_var_values(install_environment)

Return a dictionary of values, some of which enable substitution of reserved words for the values. The received install_enviroment object has 3 important attributes for reserved word substitution: install_environment.tool_shed_repository_install_dir is the root installation directory of the repository that contains the tool dependency being installed, install_environment.install_dir is the root installation directory of the tool dependency, and install_environment.tmp_work_dir is the temporary directory where the tool dependency compilation/installation is being processed.

tool_shed.util.basic_util.get_file_type_str(changeset_revision, file_type)

tool_shed.util.basic_util.move_file(current_dir, source, destination, rename_to=None)

tool_shed.util.basic_util.remove_dir(dir)

Attempt to remove a directory from disk.

tool_shed.util.basic_util.size_string(raw_text, size=32768)

Return a subset of a string (up to MAX_DISPLAY_SIZE) translated to a safe string for display in a browser.

tool_shed.util.basic_util.stringify(list)

tool_shed.util.basic_util.strip_path(fpath)

Attempt to strip the path from a file name.

tool_shed.util.basic_util.to_html_string(text)

Translates the characters in text to an html string

tool_shed.util.commit_util module

tool_shed.util.commit_util.check_archive(repository, archive)

tool_shed.util.commit_util.check_file_contents_for_email_alerts(app)

See if any admin users have chosen to receive email alerts when a repository is updated. If so, the file contents of the update must be checked for inappropriate content.

tool_shed.util.commit_util.check_file_content_for_html_and_images(file_path)

tool_shed.util.commit_util.get_change_lines_in_file_for_tag(tag, change_dict)

The received change_dict is the jsonified version of the changes to a file in a changeset being pushed to the Tool Shed from the command line. This method cleans and returns appropriate lines for inspection.

tool_shed.util.commit_util.get_upload_point(repository, **kwd)

tool_shed.util.commit_util.handle_bz2(repository, uploaded_file_name)

tool_shed.util.commit_util.handle_directory_changes(app, host, username, repository, full_path, filenames_in_archive, remove_repo_files_not_in_tar, new_repo_alert, commit_message, undesirable_dirs_removed, undesirable_files_removed)

tool_shed.util.commit_util.handle_gzip(repository, uploaded_file_name)

tool_shed.util.commit_util.uncompress(repository, uploaded_file_name, uploaded_file_filename, isgzip=False, isbz2=False)

tool_shed.util.common_util module

tool_shed.util.common_util.accumulate_tool_dependencies(tool_shed_accessible, tool_dependencies, all_tool_dependencies)

tool_shed.util.common_util.check_for_missing_tools(app, tool_panel_configs, latest_tool_migration_script_number)

tool_shed.util.common_util.check_tool_tag_set(elem, migrated_tool_configs_dict, missing_tool_configs_dict)

tool_shed.util.common_util.generate_clone_url_for_installed_repository(app, repository)

Generate the URL for cloning a repository that has been installed into a Galaxy instance.

tool_shed.util.common_util.generate_clone_url_for_repository_in_tool_shed(user, repository)

Generate the URL for cloning a repository that is in the tool shed.

tool_shed.util.common_util.generate_clone_url_from_repo_info_tup(app, repo_info_tup)

Generate the URL for cloning a repository given a tuple of toolshed, name, owner, changeset_revision.

tool_shed.util.common_util.get_non_shed_tool_panel_configs(app)

Get the non-shed related tool panel configs - there can be more than one, and the default is tool_conf.xml.

tool_shed.util.common_util.get_repository_dependencies(app, tool_shed_url, repository_name, repository_owner, changeset_revision)

tool_shed.util.common_util.get_protocol_from_tool_shed_url(tool_shed_url)

Return the protocol from the received tool_shed_url if it exists.

tool_shed.util.common_util.get_tool_dependencies(app, tool_shed_url, repository_name, repository_owner, changeset_revision)

tool_shed.util.common_util.get_tool_shed_repository_ids(as_string=False, **kwd)

tool_shed.util.common_util.get_tool_shed_url_from_tool_shed_registry(app, tool_shed)

The value of tool_shed is something like: toolshed.g2.bx.psu.edu. We need the URL to this tool shed, which is something like: http://toolshed.g2.bx.psu.edu/

tool_shed.util.common_util.get_user_by_username(app, username)

Get a user from the database by username.

tool_shed.util.common_util.handle_galaxy_url(trans, **kwd)

tool_shed.util.common_util.handle_tool_shed_url_protocol(app, shed_url)

Handle secure and insecure HTTP protocol since they may change over time.

tool_shed.util.common_util.parse_repository_dependency_tuple(repository_dependency_tuple, contains_error=False)

tool_shed.util.common_util.remove_port_from_tool_shed_url(tool_shed_url)

Return a partial Tool Shed URL, eliminating the port if it exists.

tool_shed.util.common_util.remove_protocol_and_port_from_tool_shed_url(tool_shed_url)

Return a partial Tool Shed URL, eliminating the protocol and/or port if either exists.

tool_shed.util.common_util.remove_protocol_and_user_from_clone_url(repository_clone_url)

Return a URL that can be used to clone a repository, eliminating the protocol and user if either exists.

tool_shed.util.common_util.remove_protocol_from_tool_shed_url(tool_shed_url)

Return a partial Tool Shed URL, eliminating the protocol if it exists.

tool_shed.util.container_util module

tool_shed.util.container_util.generate_repository_dependencies_key_for_repository(toolshed_base_url, repository_name, repository_owner, changeset_revision, prior_installation_required, only_if_compiling_contained_td)

Assumes tool shed is current tool shed since repository dependencies across tool sheds is not yet supported.

tool_shed.util.container_util.get_components_from_key(key)

Assumes tool shed is current tool shed since repository dependencies across tool sheds is not yet supported.

tool_shed.util.container_util.print_folders(pad, folder)

tool_shed.util.encoding_util module

tool_shed.util.encoding_util.tool_shed_decode(value)

tool_shed.util.encoding_util.tool_shed_encode(val)

tool_shed.util.hg_util module

tool_shed.util.hg_util.add_changeset(repo_ui, repo, path_to_filename_in_archive)

tool_shed.util.hg_util.archive_repository_revision(app, repository, archive_dir, changeset_revision)

Create an un-versioned archive of a repository.

tool_shed.util.hg_util.bundle_to_json(fh)

Convert the received HG10xx data stream (a mercurial 1.0 bundle created using hg push from the command line) to a json object.

tool_shed.util.hg_util.clone_repository(repository_clone_url, repository_file_dir, ctx_rev)

Clone the repository up to the specified changeset_revision. No subsequent revisions will be present in the cloned repository.

tool_shed.util.hg_util.commit_changeset(repo_ui, repo, full_path_to_changeset, username, message)

tool_shed.util.hg_util.copy_file_from_manifest(repo, ctx, filename, dir)

Copy the latest version of the file named filename from the repository manifest to the directory to which dir refers.

tool_shed.util.hg_util.create_hgrc_file(app, repository)

tool_shed.util.hg_util.get_changectx_for_changeset(repo, changeset_revision, **kwd)

Retrieve a specified changectx from a repository.

tool_shed.util.hg_util.get_config(config_file, repo, ctx, dir)

Return the latest version of config_filename from the repository manifest.

tool_shed.util.hg_util.get_config_from_disk(config_file, relative_install_dir)

tool_shed.util.hg_util.get_configured_ui()

Configure any desired ui settings.

tool_shed.util.hg_util.get_ctx_file_path_from_manifest(filename, repo, changeset_revision)

Get the ctx file path for the latest revision of filename from the repository manifest up to the value of changeset_revision.

tool_shed.util.hg_util.get_file_context_from_ctx(ctx, filename)

Return the mercurial file context for a specified file.

tool_shed.util.hg_util.get_mercurial_default_options_dict(command, command_table=None, **kwd)

Borrowed from repoman - get default parameters for a mercurial command.

tool_shed.util.hg_util.get_named_tmpfile_from_ctx(ctx, filename, dir)

Return a named temporary file created from a specified file with a given name included in a repository changeset revision.

tool_shed.util.hg_util.get_readable_ctx_date(ctx)

Convert the date of the changeset (the received ctx) to a human-readable date.

tool_shed.util.hg_util.get_repo_for_repository(app, repository=None, repo_path=None, create=False)

tool_shed.util.hg_util.get_repository_heads(repo)

Return current repository heads, which are changesets with no child changesets.

tool_shed.util.hg_util.get_reversed_changelog_changesets(repo)

Return a list of changesets in reverse order from that provided by the repository manifest.

tool_shed.util.hg_util.get_revision_label(app, repository, changeset_revision, include_date=True, include_hash=True)

Return a string consisting of the human read-able changeset rev and the changeset revision string which includes the revision date if the receive include_date is True.

tool_shed.util.hg_util.get_rev_label_changeset_revision_from_repository_metadata(app, repository_metadata, repository=None, include_date=True, include_hash=True)

tool_shed.util.hg_util.get_revision_label_from_ctx(ctx, include_date=True, include_hash=True)

tool_shed.util.hg_util.get_rev_label_from_changeset_revision(repo, changeset_revision, include_date=True, include_hash=True)

Given a changeset revision hash, return two strings, the changeset rev and the changeset revision hash which includes the revision date if the receive include_date is True.

tool_shed.util.hg_util.pull_repository(repo, repository_clone_url, ctx_rev)

Pull changes from a remote repository to a local one.

tool_shed.util.hg_util.remove_file(repo_ui, repo, selected_file, force=True)

tool_shed.util.hg_util.reversed_lower_upper_bounded_changelog(repo, excluded_lower_bounds_changeset_revision, included_upper_bounds_changeset_revision)

Return a reversed list of changesets in the repository changelog after the excluded_lower_bounds_changeset_revision, but up to and including the included_upper_bounds_changeset_revision. The value of excluded_lower_bounds_changeset_revision will be the value of INITIAL_CHANGELOG_HASH if no valid changesets exist before included_upper_bounds_changeset_revision.

tool_shed.util.hg_util.reversed_upper_bounded_changelog(repo, included_upper_bounds_changeset_revision)

Return a reversed list of changesets in the repository changelog up to and including the included_upper_bounds_changeset_revision.

tool_shed.util.hg_util.unpack_chunks(hg_unbundle10_obj)

This method provides a generator of parsed chunks of a “group” in a mercurial unbundle10 object which is created when a changeset that is pushed to a Tool Shed repository using hg push from the command line is read using readbundle.

tool_shed.util.hg_util.unpack_groups(hg_unbundle10_obj)

This method provides a generator of parsed groups from a mercurial unbundle10 object which is created when a changeset that is pushed to a Tool Shed repository using hg push from the command line is read using readbundle.

tool_shed.util.hg_util.unpack_patches(hg_unbundle10_obj, remaining)

This method provides a generator of patches from the data field in a chunk. As there is no delimiter for this data field, a length argument is required.

tool_shed.util.hg_util.update_repository(repo, ctx_rev=None)

Update the cloned repository to changeset_revision. It is critical that the installed repository is updated to the desired changeset_revision before metadata is set because the process for setting metadata uses the repository files on disk.

tool_shed.util.metadata_util module

tool_shed.util.metadata_util.get_all_dependencies(app, metadata_entry, processed_dependency_links=[])

tool_shed.util.metadata_util.get_current_repository_metadata_for_changeset_revision(app, repository, changeset_revision)

tool_shed.util.metadata_util.get_dependencies_for_metadata_revision(app, metadata)

tool_shed.util.metadata_util.get_latest_changeset_revision(app, repository, repo)

tool_shed.util.metadata_util.get_latest_downloadable_changeset_revision(app, repository, repo=None)

tool_shed.util.metadata_util.get_latest_repository_metadata(app, decoded_repository_id, downloadable=False)

Get last metadata defined for a specified repository from the database.

tool_shed.util.metadata_util.get_metadata_by_id(app, metadata_id)

tool_shed.util.metadata_util.get_metadata_changeset_revisions(repository, repo)

Return an unordered list of changeset_revisions and changeset numbers that are defined as installable.

tool_shed.util.metadata_util.get_metadata_revisions(repository, repo, sort_revisions=True, reverse=False, downloadable=True)

Return a list of changesets for the provided repository.

tool_shed.util.metadata_util.get_next_downloadable_changeset_revision(repository, repo, after_changeset_revision)

Return the installable changeset_revision in the repository changelog after the changeset to which after_changeset_revision refers. If there isn’t one, return None. If there is only one installable changeset, and that matches the requested revision, return it.

tool_shed.util.metadata_util.get_previous_metadata_changeset_revision(repository, repo, before_changeset_revision, downloadable=True)

Return the changeset_revision in the repository changelog that has associated metadata prior to the changeset to which before_changeset_revision refers. If there isn’t one, return the hash value of an empty repository changelog, hg_util.INITIAL_CHANGELOG_HASH.

tool_shed.util.metadata_util.get_repository_dependencies(app, metadata_id)

Return a list of RepositoryDependency objects that specify the provided repository metadata record as the parent.

tool_shed.util.metadata_util.get_repository_dependency_tups_from_repository_metadata(app, repository_metadata, deprecated_only=False)

Return a list of of tuples defining repository objects required by the received repository. The returned list defines the entire repository dependency tree. This method is called only from the Tool Shed.

tool_shed.util.metadata_util.get_repository_metadata_by_changeset_revision(app, id, changeset_revision)

Get metadata for a specified repository change set from the database.

tool_shed.util.metadata_util.get_repository_metadata_by_id(app, id)

Get repository metadata from the database

tool_shed.util.metadata_util.get_repository_metadata_by_repository_id_changeset_revision(app, id, changeset_revision, metadata_only=False)

Get a specified metadata record for a specified repository in the tool shed.

tool_shed.util.metadata_util.get_repository_metadata_revisions_for_review(repository, reviewed=True)

tool_shed.util.metadata_util.get_updated_changeset_revisions(app, name, owner, changeset_revision)

Return a string of comma-separated changeset revision hashes for all available updates to the received changeset revision for the repository defined by the received name and owner.

tool_shed.util.metadata_util.get_updated_changeset_revisions_from_tool_shed(app, tool_shed_url, name, owner, changeset_revision)

Get all appropriate newer changeset revisions for the repository defined by the received tool_shed_url / name / owner combination.

tool_shed.util.metadata_util.is_downloadable(metadata_dict)

tool_shed.util.metadata_util.is_malicious(app, id, changeset_revision, **kwd)

Check the malicious flag in repository metadata for a specified change set revision.

tool_shed.util.readme_util module

tool_shed.util.readme_util.build_readme_files_dict(app, repository, changeset_revision, metadata, tool_path=None)

Return a dictionary of valid readme file name <-> readme file content pairs for all readme files defined in the received metadata. Since the received changeset_revision (which is associated with the received metadata) may not be the latest installable changeset revision, the README file contents may not be available on disk. This method is used by both Galaxy and the Tool Shed.

tool_shed.util.readme_util.get_readme_files_dict_for_display(app, tool_shed_url, repo_info_dict)

Return a dictionary of README files contained in the single repository being installed so they can be displayed on the tool panel section selection page.

tool_shed.util.readme_util.get_readme_file_names(repository_name)

Return a list of file names that will be categorized as README files for the received repository_name.

tool_shed.util.repository_content_util module

tool_shed.util.repository_content_util.upload_tar(trans, rdah, tdah, repository, tar, uploaded_file, upload_point, remove_repo_files_not_in_tar, commit_message, new_repo_alert)

tool_shed.util.repository_util module

tool_shed.util.repository_util.change_repository_name_in_hgrc_file(hgrc_file, new_name)

tool_shed.util.repository_util.check_for_updates(app, model, repository_id=None)

tool_shed.util.repository_util.check_or_update_tool_shed_status_for_installed_repository(app, repository)

tool_shed.util.repository_util.create_or_update_tool_shed_repository(app, name, description, installed_changeset_revision, ctx_rev, repository_clone_url, metadata_dict, status, current_changeset_revision=None, owner='', dist_to_shed=False)

Update a tool shed repository record in the Galaxy database with the new information received. If a record defined by the received tool shed, repository name and owner does not exist, create a new record with the received information.

tool_shed.util.repository_util.create_repo_info_dict(app, repository_clone_url, changeset_revision, ctx_rev, repository_owner, repository_name=None, repository=None, repository_metadata=None, tool_dependencies=None, repository_dependencies=None)

Return a dictionary that includes all of the information needed to install a repository into a local Galaxy instance. The dictionary will also contain the recursive list of repository dependencies defined for the repository, as well as the defined tool dependencies.

This method is called from Galaxy under four scenarios: 1. During the tool shed repository installation process via the tool shed’s get_repository_information() method. In this case both the received repository and repository_metadata will be objects, but tool_dependencies and repository_dependencies will be None. 2. When getting updates for an installed repository where the updates include newly defined repository dependency definitions. This scenario is similar to 1. above. The tool shed’s get_repository_information() method is the caller, and both the received repository and repository_metadata will be objects, but tool_dependencies and repository_dependencies will be None. 3. When a tool shed repository that was uninstalled from a Galaxy instance is being reinstalled with no updates available. In this case, both repository and repository_metadata will be None, but tool_dependencies and repository_dependencies will be objects previously retrieved from the tool shed if the repository includes definitions for them. 4. When a tool shed repository that was uninstalled from a Galaxy instance is being reinstalled with updates available. In this case, this method is reached via the tool shed’s get_updated_repository_information() method, and both repository and repository_metadata will be objects but tool_dependencies and repository_dependencies will be None.

tool_shed.util.repository_util.create_repository_admin_role(app, repository)

Create a new role with name-spaced name based on the repository name and its owner’s public user name. This will ensure that the tole name is unique.

tool_shed.util.repository_util.create_repository(app, name, type, description, long_description, user_id, category_ids=[], remote_repository_url=None, homepage_url=None)

Create a new ToolShed repository

tool_shed.util.repository_util.extract_components_from_tuple(repository_components_tuple)

Extract the repository components from the provided tuple in a backward-compatible manner.

tool_shed.util.repository_util.generate_sharable_link_for_repository_in_tool_shed(repository, changeset_revision=None)

Generate the URL for sharing a repository that is in the tool shed.

tool_shed.util.repository_util.generate_tool_shed_repository_install_dir(repository_clone_url, changeset_revision)

Generate a repository installation directory that guarantees repositories with the same name will always be installed in different directories. The tool path will be of the form: <tool shed url>/repos/<repository owner>/<repository name>/<installed changeset revision>

tool_shed.util.repository_util.get_absolute_path_to_file_in_repository(repo_files_dir, file_name)

Return the absolute path to a specified disk file contained in a repository.

tool_shed.util.repository_util.get_ids_of_tool_shed_repositories_being_installed(app, as_string=False)

tool_shed.util.repository_util.get_installed_repository(app, tool_shed=None, name=None, owner=None, changeset_revision=None, installed_changeset_revision=None, repository_id=None, refresh=False)

Return a tool shed repository database record defined by the combination of a toolshed, repository name, repository owner and either current or originally installed changeset_revision.

tool_shed.util.repository_util.get_installed_tool_shed_repository(app, id)

Get a tool shed repository record from the Galaxy database defined by the id.

tool_shed.util.repository_util.get_prior_import_or_install_required_dict(app, tsr_ids, repo_info_dicts)

This method is used in the Tool Shed when exporting a repository and its dependencies, and in Galaxy when a repository and its dependencies are being installed. Return a dictionary whose keys are the received tsr_ids and whose values are a list of tsr_ids, each of which is contained in the received list of tsr_ids and whose associated repository must be imported or installed prior to the repository associated with the tsr_id key.

tool_shed.util.repository_util.get_repo_info_dict(app, user, repository_id, changeset_revision)

tool_shed.util.repository_util.get_repo_info_tuple_contents(repo_info_tuple)

Take care in handling the repo_info_tuple as it evolves over time as new tool shed features are introduced.

tool_shed.util.repository_util.get_repositories_by_category(app, category_id, installable=False)

tool_shed.util.repository_util.get_repository_admin_role_name(repository_name, repository_owner)

tool_shed.util.repository_util.get_repository_and_repository_dependencies_from_repo_info_dict(app, repo_info_dict)

Return a tool_shed_repository or repository record defined by the information in the received repo_info_dict.

tool_shed.util.repository_util.get_repository_by_id(app, id)

Get a repository from the database via id.

tool_shed.util.repository_util.get_repository_by_name_and_owner(app, name, owner)

Get a repository from the database via name and owner

tool_shed.util.repository_util.get_repository_by_name(app, name)

Get a repository from the database via name.

tool_shed.util.repository_util.get_repository_dependency_types(repository_dependencies)

Inspect the received list of repository_dependencies tuples and return boolean values for has_repository_dependencies and has_repository_dependencies_only_if_compiling_contained_td.

tool_shed.util.repository_util.get_repository_for_dependency_relationship(app, tool_shed, name, owner, changeset_revision)

Return an installed tool_shed_repository database record that is defined by either the current changeset revision or the installed_changeset_revision.

tool_shed.util.repository_util.get_repository_ids_requiring_prior_import_or_install(app, tsr_ids, repository_dependencies)

This method is used in the Tool Shed when exporting a repository and its dependencies, and in Galaxy when a repository and its dependencies are being installed. Inspect the received repository_dependencies and determine if the encoded id of each required repository is in the received tsr_ids. If so, then determine whether that required repository should be imported / installed prior to its dependent repository. Return a list of encoded repository ids, each of which is contained in the received list of tsr_ids, and whose associated repositories must be imported / installed prior to the dependent repository associated with the received repository_dependencies.

tool_shed.util.repository_util.get_repository_in_tool_shed(app, id)

Get a repository on the tool shed side from the database via id.

tool_shed.util.repository_util.get_repository_owner(cleaned_repository_url)

Gvien a “cleaned” repository clone URL, return the owner of the repository.

tool_shed.util.repository_util.get_repository_owner_from_clone_url(repository_clone_url)

Given a repository clone URL, return the owner of the repository.

tool_shed.util.repository_util.get_repository_query(app)

tool_shed.util.repository_util.get_role_by_id(app, role_id)

Get a Role from the database by id.

tool_shed.util.repository_util.get_tool_shed_from_clone_url(repository_clone_url)

tool_shed.util.repository_util.get_tool_shed_repository_by_id(app, repository_id)

Return a tool shed repository database record defined by the id.

tool_shed.util.repository_util.get_tool_shed_repository_status_label(app, tool_shed_repository=None, name=None, owner=None, changeset_revision=None, repository_clone_url=None)

Return a color-coded label for the status of the received tool-shed_repository installed into Galaxy.

tool_shed.util.repository_util.get_tool_shed_status_for_installed_repository(app, repository)

Send a request to the tool shed to retrieve information about newer installable repository revisions, current revision updates, whether the repository revision is the latest downloadable revision, and whether the repository has been deprecated in the tool shed. The received repository is a ToolShedRepository object from Galaxy.

tool_shed.util.repository_util.handle_role_associations(app, role, repository, **kwd)

tool_shed.util.repository_util.is_tool_shed_client(app)

The tool shed and clients to the tool (i.e. Galaxy) require a lot of similar functionality in this file but with small differences. This method should determine if the app performing the action is the tool shed or a client of the tool shed.

tool_shed.util.repository_util.repository_was_previously_installed(app, tool_shed_url, repository_name, repo_info_tuple, from_tip=False)

Find out if a repository is already installed into Galaxy - there are several scenarios where this is necessary. For example, this method will handle the case where the repository was previously installed using an older changeset_revsion, but later the repository was updated in the tool shed and now we’re trying to install the latest changeset revision of the same repository instead of updating the one that was previously installed. We’ll look in the database instead of on disk since the repository may be currently uninstalled.

tool_shed.util.repository_util.set_repository_attributes(app, repository, status, error_message, deleted, uninstalled, remove_from_disk=False)

tool_shed.util.repository_util.update_repository(app, trans, id, **kwds)

Update an existing ToolShed repository

tool_shed.util.repository_util.validate_repository_name(app, name, user)

Validate whether the given name qualifies as a new TS repo name. Repository names must be unique for each user, must be at least two characters in length and must contain only lower-case letters, numbers, and the ‘_’ character.

tool_shed.util.review_util module

tool_shed.util.review_util.can_browse_repository_reviews(app, user, repository)

Determine if there are any reviews of the received repository for which the current user has permission to browse any component reviews.

tool_shed.util.review_util.changeset_revision_reviewed_by_user(user, repository, changeset_revision)

Determine if the current changeset revision has been reviewed by the current user.

tool_shed.util.review_util.get_component(app, id)

Get a component from the database.

tool_shed.util.review_util.get_component_review(app, id)

Get a component_review from the database

tool_shed.util.review_util.get_component_by_name(app, name)

Get a component from the database via a name.

tool_shed.util.review_util.get_component_review_by_repository_review_id_component_id(app, repository_review_id, component_id)

Get a component_review from the database via repository_review_id and component_id.

tool_shed.util.review_util.get_components(app)

tool_shed.util.review_util.get_previous_repository_reviews(app, repository, changeset_revision)

Return an ordered dictionary of repository reviews up to and including the received changeset revision.

tool_shed.util.review_util.get_review(app, id)

Get a repository_review from the database via id.

tool_shed.util.review_util.get_review_by_repository_id_changeset_revision_user_id(app, repository_id, changeset_revision, user_id)

Get a repository_review from the database via repository id, changeset_revision and user_id.

tool_shed.util.review_util.get_reviews_by_repository_id_changeset_revision(app, repository_id, changeset_revision)

Get all repository_reviews from the database via repository id and changeset_revision.

tool_shed.util.review_util.has_previous_repository_reviews(app, repository, changeset_revision)

Determine if a repository has a changeset revision review prior to the received changeset revision.

tool_shed.util.search_util module

tool_shed.util.search_util.in_tool_dict(tool_dict, exact_matches_checked, tool_id=None, tool_name=None, tool_version=None)

tool_shed.util.search_util.in_workflow_dict(workflow_dict, exact_matches_checked, workflow_name)

tool_shed.util.search_util.make_same_length(list1, list2)

tool_shed.util.search_util.search_ids_names(tool_dict, exact_matches_checked, match_tuples, repository_metadata, tool_ids, tool_names)

tool_shed.util.search_util.search_ids_versions(tool_dict, exact_matches_checked, match_tuples, repository_metadata, tool_ids, tool_versions)

tool_shed.util.search_util.search_names_versions(tool_dict, exact_matches_checked, match_tuples, repository_metadata, tool_names, tool_versions)

tool_shed.util.search_util.search_repository_metadata(app, exact_matches_checked, tool_ids='', tool_names='', tool_versions='', workflow_names='', all_workflows=False)

tool_shed.util.shed_util_common module

tool_shed.util.shed_util_common.can_eliminate_repository_dependency(metadata_dict, tool_shed_url, name, owner)

Determine if the relationship between a repository_dependency record associated with a tool_shed_repository record on the Galaxy side can be eliminated.

tool_shed.util.shed_util_common.can_eliminate_tool_dependency(metadata_dict, name, dependency_type, version)

Determine if the relationship between a tool_dependency record associated with a tool_shed_repository record on the Galaxy side can be eliminated.

tool_shed.util.shed_util_common.clean_dependency_relationships(trans, metadata_dict, tool_shed_repository, tool_shed_url)

Repositories of type tool_dependency_definition allow for defining a package dependency at some point in the change log and then removing the dependency later in the change log. This function keeps the dependency relationships on the Galaxy side current by deleting database records that defined the now-broken relationships.

tool_shed.util.shed_util_common.generate_tool_guid(repository_clone_url, tool)

Generate a guid for the installed tool. It is critical that this guid matches the guid for the tool in the Galaxy tool shed from which it is being installed. The form of the guid is <tool shed host>/repos/<repository owner>/<repository name>/<tool id>/<tool version>

tool_shed.util.shed_util_common.get_categories(app)

Get all categories from the database.

tool_shed.util.shed_util_common.get_category(app, id)

Get a category from the database.

tool_shed.util.shed_util_common.get_category_by_name(app, name)

Get a category from the database via name.

tool_shed.util.shed_util_common.get_tool_shed_repo_requirements(app, tool_shed_url, repositories=None, repo_info_dicts=None)

Contact tool_shed_url for a list of requirements for a repository or a list of repositories. Returns a list of requirements, where each requirement is a dictionary with name and version as keys.

tool_shed.util.shed_util_common.get_requirements_from_tools(tools)

tool_shed.util.shed_util_common.get_requirements_from_repository(repository)

tool_shed.util.shed_util_common.get_ctx_rev(app, tool_shed_url, name, owner, changeset_revision)

Send a request to the tool shed to retrieve the ctx_rev for a repository defined by the combination of a name, owner and changeset revision.

tool_shed.util.shed_util_common.get_next_prior_import_or_install_required_dict_entry(prior_required_dict, processed_tsr_ids)

This method is used in the Tool Shed when exporting a repository and its dependencies, and in Galaxy when a repository and its dependencies are being installed. The order in which the prior_required_dict is processed is critical in order to ensure that the ultimate repository import or installation order is correctly defined. This method determines the next key / value pair from the received prior_required_dict that should be processed.

tool_shed.util.shed_util_common.get_repository_categories(app, id)

Get categories of a repository on the tool shed side from the database via id

tool_shed.util.shed_util_common.get_repository_file_contents(app, file_path, repository_id, is_admin=False)

Return the display-safe contents of a repository file for display in a browser.

tool_shed.util.shed_util_common.get_repository_files(folder_path)

Return the file hierarchy of a tool shed repository.

tool_shed.util.shed_util_common.get_repository_from_refresh_on_change(app, **kwd)

tool_shed.util.shed_util_common.get_repository_type_from_tool_shed(app, tool_shed_url, name, owner)

Send a request to the tool shed to retrieve the type for a repository defined by the combination of a name and owner.

tool_shed.util.shed_util_common.get_tool_dependency_definition_metadata_from_tool_shed(app, tool_shed_url, name, owner)

Send a request to the tool shed to retrieve the current metadata for a repository of type tool_dependency_definition defined by the combination of a name and owner.

tool_shed.util.shed_util_common.get_tool_panel_config_tool_path_install_dir(app, repository)

Return shed-related tool panel config, the tool_path configured in it, and the relative path to the directory where the repository is installed. This method assumes all repository tools are defined in a single shed-related tool panel config.

tool_shed.util.shed_util_common.get_tool_path_by_shed_tool_conf_filename(app, shed_tool_conf)

Return the tool_path config setting for the received shed_tool_conf file by searching the tool box’s in-memory list of shed_tool_confs for the dictionary whose config_filename key has a value matching the received shed_tool_conf.

tool_shed.util.shed_util_common.get_user(app, id)

Get a user from the database by id.

tool_shed.util.shed_util_common.handle_email_alerts(app, host, repository, content_alert_str='', new_repo_alert=False, admin_only=False)

There are 2 complementary features that enable a tool shed user to receive email notification: 1. Within User Preferences, they can elect to receive email when the first (or first valid)

change set is produced for a new repository.

2. When viewing or managing a repository, they can check the box labeled “Receive email alerts” which caused them to receive email alerts when updates to the repository occur. This same feature is available on a per-repository basis on the repository grid within the tool shed.

There are currently 4 scenarios for sending email notification when a change is made to a repository: 1. An admin user elects to receive email when the first change set is produced for a new repository

from User Preferences. The change set does not have to include any valid content. This allows for the capture of inappropriate content being uploaded to new repositories.

2. A regular user elects to receive email when the first valid change set is produced for a new repository from User Preferences. This differs from 1 above in that the user will not receive email until a change set tha tincludes valid content is produced.
3. An admin user checks the “Receive email alerts” check box on the manage repository page. Since the user is an admin user, the email will include information about both HTML and image content that was included in the change set.
4. A regular user checks the “Receive email alerts” check box on the manage repository page. Since the user is not an admin user, the email will not include any information about both HTML and image content that was included in the change set.

tool_shed.util.shed_util_common.have_shed_tool_conf_for_install(app)

tool_shed.util.shed_util_common.is_path_browsable(app, path, repository_id, is_admin=False)

Detects whether the given path is browsable i.e. is within the allowed repository folders. Admins can additionaly browse folders with tool dependencies.

tool_shed.util.shed_util_common.is_path_within_dependency_dir(app, path)

Detect whether the given path is within the tool_dependency_dir folder on the disk. (Specified by the config option). Use to filter malicious symlinks targeting outside paths.

tool_shed.util.shed_util_common.is_path_within_repo(app, path, repository_id)

Detect whether the given path is within the repository folder on the disk. Use to filter malicious symlinks targeting outside paths.

tool_shed.util.shed_util_common.open_repository_files_folder(app, folder_path, repository_id, is_admin=False)

Return a list of dictionaries, each of which contains information for a file or directory contained within a directory in a repository file hierarchy.

tool_shed.util.shed_util_common.set_image_paths(app, encoded_repository_id, text)

Handle tool help image display for tools that are contained in repositories in the tool shed or installed into Galaxy as well as image display in repository README files. This method will determine the location of the image file and return the path to it that will enable the caller to open the file.

tool_shed.util.shed_util_common.tool_shed_is_this_tool_shed(toolshed_base_url)

Determine if a tool shed is the current tool shed.

tool_shed.util.tool_dependency_util module

tool_shed.util.tool_dependency_util.build_tool_dependencies_select_field(app, tool_shed_repository, name, multiple=True, display='checkboxes', uninstalled_only=False)

Generate a SelectField consisting of the current list of tool dependency ids for an installed tool shed repository.

tool_shed.util.tool_dependency_util.create_or_update_tool_dependency(app, tool_shed_repository, name, version, type, status, set_status=True)

Create or update a tool_dependency record in the Galaxy database.

tool_shed.util.tool_dependency_util.create_tool_dependency_objects(app, tool_shed_repository, relative_install_dir, set_status=True)

Create or update a ToolDependency for each entry in tool_dependencies_config. This method is called when installing a new tool_shed_repository.

tool_shed.util.tool_dependency_util.get_download_url_for_platform(url_templates, platform_info_dict)

Compare the dict returned by get_platform_info() with the values specified in the url_template element. Return true if and only if all defined attributes match the corresponding dict entries. If an entry is not defined in the url_template element, it is assumed to be irrelevant at this stage. For example, <url_template os=”darwin”>http://hgdownload.cse.ucsc.edu/admin/exe/macOSX.${architecture}/faToTwoBit</url_template> where the OS must be ‘darwin’, but the architecture is filled in later using string.Template.

tool_shed.util.tool_dependency_util.get_platform_info_dict()

Return a dict with information about the current platform.

tool_shed.util.tool_dependency_util.get_tool_dependency(app, id)

Get a tool_dependency from the database via id

tool_shed.util.tool_dependency_util.get_tool_dependency_by_name_type_repository(app, repository, name, type)

tool_shed.util.tool_dependency_util.get_tool_dependency_by_name_version_type(app, name, version, type)

tool_shed.util.tool_dependency_util.get_tool_dependency_by_name_version_type_repository(app, repository, name, version, type)

tool_shed.util.tool_dependency_util.get_tool_dependency_ids(as_string=False, **kwd)

tool_shed.util.tool_dependency_util.get_tool_dependency_install_dir(app, repository_name, repository_owner, repository_changeset_revision, tool_dependency_type, tool_dependency_name, tool_dependency_version)

tool_shed.util.tool_dependency_util.parse_package_elem(package_elem, platform_info_dict=None, include_after_install_actions=True)

Parse a <package> element within a tool dependency definition and return a list of action tuples. This method is called when setting metadata on a repository that includes a tool_dependencies.xml file or when installing a repository that includes a tool_dependencies.xml file. If installing, platform_info_dict must be a valid dictionary and include_after_install_actions must be True.

tool_shed.util.tool_dependency_util.remove_tool_dependency(app, tool_dependency)

The received tool_dependency must be in an error state.

tool_shed.util.tool_dependency_util.remove_tool_dependency_installation_directory(dependency_install_dir)

tool_shed.util.tool_dependency_util.set_tool_dependency_attributes(app, tool_dependency, status, error_message=None)

tool_shed.util.tool_util module

tool_shed.util.tool_util.build_shed_tool_conf_select_field(app)

Build a SelectField whose options are the keys in app.toolbox.shed_tool_confs.

tool_shed.util.tool_util.build_tool_panel_section_select_field(app)

Build a SelectField whose options are the sections of the current in-memory toolbox.

tool_shed.util.tool_util.copy_sample_file(app, filename, dest_path=None)

Copies a sample file at filename to the dest_path directory and strips the ‘.sample’ extensions from filename. Returns the path to the copied file (with the .sample extension).

tool_shed.util.tool_util.copy_sample_files(app, sample_files, tool_path=None, sample_files_copied=None, dest_path=None)

Copy all appropriate files to dest_path in the local Galaxy environment that have not already been copied. Those that have been copied are contained in sample_files_copied. The default value for dest_path is ~/tool-data. We need to be careful to copy only appropriate files here because tool shed repositories can contain files ending in .sample that should not be copied to the ~/tool-data directory.

tool_shed.util.tool_util.generate_message_for_invalid_tools(app, invalid_file_tups, repository, metadata_dict, as_html=True, displaying_invalid_tool=False)

tool_shed.util.tool_util.get_tool_path_install_dir(partial_install_dir, shed_tool_conf_dict, tool_dict, config_elems)

tool_shed.util.tool_util.handle_missing_index_file(app, tool_path, sample_files, repository_tools_tups, sample_files_copied)

Inspect each tool to see if it has any input parameters that are dynamically generated select lists that depend on a .loc file. This method is not called from the tool shed, but from Galaxy when a repository is being installed.

tool_shed.util.tool_util.is_data_index_sample_file(file_path)

Attempt to determine if a .sample file is appropriate for copying to ~/tool-data when a tool shed repository is being installed into a Galaxy instance.

tool_shed.util.tool_util.new_state(trans, tool, invalid=False)

Create a new DefaultToolState for the received tool. Only inputs on the first page will be initialized.

tool_shed.util.tool_util.panel_entry_per_tool(tool_section_dict)

tool_shed.util.web_util module

tool_shed.util.web_util.escape(string)

A tool shed variant of markupsafe.escape that allows a select few HTML elements that are repeatedly used in messages created deep in the toolshed components. Ideally abstract things would be produced in these components and messages in the views or client side - this is what should be worked toward - but for now - we have this hack.

>>> escape("A <b>repo</b>")
u'A <b>repo</b>'

tool_shed.util.workflow_util module

Tool shed helper methods for dealing with workflows - only two methods are utilized outside of this modules - generate_workflow_image and import_workflow.

class tool_shed.util.workflow_util.RepoToolModule(trans, repository_id, changeset_revision, tools_metadata, tool_id)

Bases: galaxy.workflow.modules.ToolModule

type = 'tool'

__init__(trans, repository_id, changeset_revision, tools_metadata, tool_id)

classmethod from_dict(Class, trans, step_dict, repository_id, changeset_revision, tools_metadata, secure=True)

classmethod from_workflow_step(Class, trans, step, repository_id, changeset_revision, tools_metadata)

get_data_inputs()

get_data_outputs()

class tool_shed.util.workflow_util.RepoWorkflowModuleFactory(module_types)

Bases: galaxy.workflow.modules.WorkflowModuleFactory

__init__(module_types)

from_dict(trans, repository_id, changeset_revision, step_dict, tools_metadata, **kwd)

Return module initialized from the data in dictionary step_dict.

from_workflow_step(trans, repository_id, changeset_revision, tools_metadata, step)

Return module initialized from the WorkflowStep object step.

tool_shed.util.workflow_util.generate_workflow_image(trans, workflow_name, repository_metadata_id=None, repository_id=None)

Return an svg image representation of a workflow dictionary created when the workflow was exported. This method is called from both Galaxy and the tool shed. When called from the tool shed, repository_metadata_id will have a value and repository_id will be None. When called from Galaxy, repository_metadata_id will be None and repository_id will have a value.

tool_shed.util.workflow_util.get_workflow_data_inputs(step, module)

tool_shed.util.workflow_util.get_workflow_data_outputs(step, module, steps)

tool_shed.util.workflow_util.get_workflow_from_dict(trans, workflow_dict, tools_metadata, repository_id, changeset_revision)

Return an in-memory Workflow object from the dictionary object created when it was exported. This method is called from both Galaxy and the tool shed to retrieve a Workflow object that can be displayed as an SVG image. This method is also called from Galaxy to retrieve a Workflow object that can be used for saving to the Galaxy database.

tool_shed.util.workflow_util.get_workflow_module_name(module, missing_tool_tups)

tool_shed.util.workflow_util.import_workflow(trans, repository, workflow_name)

Import a workflow contained in an installed tool shed repository into Galaxy (this method is called only from Galaxy).

tool_shed.util.workflow_util.save_workflow(trans, workflow, workflow_dict=None)

Use the received in-memory Workflow object for saving to the Galaxy database.

tool_shed.util.xml_util module

class tool_shed.util.xml_util.Py26CommentedTreeBuilder(html=0, target=None)

Bases: xml.etree.ElementTree.XMLParser

__init__(html=0, target=None)

handle_comment(data)

class tool_shed.util.xml_util.Py27CommentedTreeBuilder(element_factory=None)

Bases: xml.etree.ElementTree.TreeBuilder

comment(data)

tool_shed.util.xml_util.create_and_write_tmp_file(elems, use_indent=False)

tool_shed.util.xml_util.create_element(tag, attributes=None, sub_elements=None)

Create a new element whose tag is the value of the received tag, and whose attributes are all key / value pairs in the received attributes and sub_elements.

tool_shed.util.xml_util.indent(elem, level=0)

Prints an XML tree with each node indented according to its depth. This method is used to print the shed tool config (e.g., shed_tool_conf.xml from the in-memory list of config_elems because each config_elem in the list may be a hierarchical structure that was not created using the parse_xml() method below, and so will not be properly written with xml.etree.ElementTree.tostring() without manually indenting the tree first.

tool_shed.util.xml_util.parse_xml(file_name)

Returns a parsed xml tree with comments intact.

tool_shed.util.xml_util.xml_to_string(elem, encoding='utf-8', use_indent=False, level=0)