Warning
This document is for an in-development version of Galaxy. You can alternatively view this page in the latest release if it exists or view the top of the latest release's documentation.
galaxy.job_execution package¶
Submodules¶
galaxy.job_execution.datasets module¶
Utility classes allowing Job interface to reason about datasets.
- class galaxy.job_execution.datasets.DatasetPath(dataset_id, real_path, false_path=None, false_extra_files_path=None, false_metadata_path=None, mutable=True, dataset_uuid=None, object_store_id=None)[source]¶
Bases:
object
- class galaxy.job_execution.datasets.DatasetPathRewriter[source]¶
Bases:
object
Used by runner to rewrite paths.
- class galaxy.job_execution.datasets.NullDatasetPathRewriter[source]¶
Bases:
DatasetPathRewriter
Used by default for jobwrapper, do not rewrite anything.
- class galaxy.job_execution.datasets.OutputsToWorkingDirectoryPathRewriter(working_directory, outputs_directory_name)[source]¶
Bases:
DatasetPathRewriter
Rewrites all paths to place them in the specified working directory for normal jobs when Galaxy is configured with app.config.outputs_to_working_directory. Job runner base class is responsible for copying these out after job is complete.
- class galaxy.job_execution.datasets.TaskPathRewriter(working_directory, job_dataset_path_rewriter)[source]¶
Bases:
DatasetPathRewriter
Rewrites all paths to place them in the specified working directory for TaskWrapper. TaskWrapper is responsible for putting them there and pulling them out.
- galaxy.job_execution.datasets.get_path_rewriter(outputs_to_working_directory, working_directory, outputs_directory, is_task) DatasetPathRewriter [source]¶
galaxy.job_execution.output_collect module¶
Code allowing tools to define extra files associated with an output datset.
- class galaxy.job_execution.output_collect.PermissionProvider(inp_data, security_agent, job)[source]¶
Bases:
PermissionProvider
- property permissions¶
- class galaxy.job_execution.output_collect.MetadataSourceProvider(inp_data)[source]¶
Bases:
MetadataSourceProvider
- galaxy.job_execution.output_collect.collect_dynamic_outputs(job_context, output_collections)[source]¶
- class galaxy.job_execution.output_collect.BaseJobContext[source]¶
Bases:
ModelPersistenceContext
- tool_provided_metadata: BaseToolProvidedMetadata¶
- class galaxy.job_execution.output_collect.JobContext(tool, tool_provided_metadata: BaseToolProvidedMetadata, job, job_working_directory, permission_provider, metadata_source_provider, input_dbkey, object_store, final_job_state, max_discovered_files: Optional[int], flush_per_n_datasets=None)[source]¶
Bases:
BaseJobContext
- __init__(tool, tool_provided_metadata: BaseToolProvidedMetadata, job, job_working_directory, permission_provider, metadata_source_provider, input_dbkey, object_store, final_job_state, max_discovered_files: Optional[int], flush_per_n_datasets=None)[source]¶
- tool_provided_metadata: BaseToolProvidedMetadata¶
- property tag_handler¶
Return a galaxy.model.tags.TagHandler-like object for persisting tags.
- property work_context¶
- property sa_session: scoped_session¶
If bound to a database, return the SQL Alchemy session.
Return None otherwise.
- property permission_provider: PermissionProvider¶
If bound to a database, return the SQL Alchemy session.
Return None otherwise.
- property metadata_source_provider: MetadataSourceProvider¶
Return associated MetadataSourceProvider object.
- property job: Job¶
Return associated job object if bound to a job finish context connected to a database.
- property object_store: ObjectStore¶
Return object store to use for populating discovered dataset contents.
- property user¶
If bound to a database, return the user the datasets should be created for.
Return None otherwise.
- create_library_folder(parent_folder, name, description)[source]¶
Create a library folder ready from supplied attributes for supplied parent.
- add_output_dataset_association(name, dataset)[source]¶
If discovering outputs for a job, persist output dataset association.
- add_library_dataset_to_folder(library_folder, ld)[source]¶
Add library dataset to persisted library folder.
- class galaxy.job_execution.output_collect.SessionlessJobContext(metadata_params, tool_provided_metadata: BaseToolProvidedMetadata, object_store, export_store, import_store, working_directory, final_job_state, max_discovered_files: Optional[int])[source]¶
Bases:
SessionlessModelPersistenceContext
,BaseJobContext
- __init__(metadata_params, tool_provided_metadata: BaseToolProvidedMetadata, object_store, export_store, import_store, working_directory, final_job_state, max_discovered_files: Optional[int])[source]¶
- tool_provided_metadata: BaseToolProvidedMetadata¶
- galaxy.job_execution.output_collect.collect_primary_datasets(job_context: Union[JobContext, SessionlessJobContext], output, input_ext)[source]¶
- galaxy.job_execution.output_collect.discover_files(output_name, tool_provided_metadata, extra_file_collectors, job_working_directory, matchable)[source]¶
- galaxy.job_execution.output_collect.walk_over_file_collectors(extra_file_collectors, job_working_directory, matchable)[source]¶
- galaxy.job_execution.output_collect.walk_over_extra_files(target_dir, extra_file_collector, job_working_directory, matchable, parent_paths=None)[source]¶
Walks through all files in a given directory, and returns all files that match the given collector’s match criteria. If the collector has the recurse flag enabled, will also recursively descend into child folders.
- class galaxy.job_execution.output_collect.ToolMetadataDatasetCollector(dataset_collection_description)[source]¶
Bases:
object
- class galaxy.job_execution.output_collect.DatasetCollector(dataset_collection_description)[source]¶
Bases:
object
- galaxy.job_execution.output_collect.read_exit_code_from(exit_code_file, id_tag)[source]¶
Read exit code reported for a Galaxy job.
galaxy.job_execution.setup module¶
Utilities to help job and tool code setup jobs.
- class galaxy.job_execution.setup.JobIO(sa_session, job: Job, working_directory: str, outputs_directory: str, outputs_to_working_directory: bool, galaxy_url: str, version_path: str, tool_directory: str, home_directory: str, tmp_directory: str, tool_data_path: str, galaxy_data_manager_data_path: str, new_file_path: str, len_file_path: str, builds_file_path: str, check_job_script_integrity: bool, check_job_script_integrity_count: int, check_job_script_integrity_sleep: float, file_sources_dict: Dict[str, Any], user_context: Union[ProvidesUserFileSourcesUserContext, Dict[str, Any]], tool_source: Optional[str] = None, tool_source_class: Optional[str] = 'XmlToolSource', tool_dir: Optional[str] = None, is_task: bool = False)[source]¶
Bases:
Dictifiable
- dict_collection_visible_keys = ('job_id', 'working_directory', 'outputs_directory', 'outputs_to_working_directory', 'galaxy_url', 'version_path', 'tool_directory', 'home_directory', 'tmp_directory', 'tool_data_path', 'galaxy_data_manager_data_path', 'new_file_path', 'len_file_path', 'builds_file_path', 'file_sources_dict', 'check_job_script_integrity', 'check_job_script_integrity_count', 'check_job_script_integrity_sleep', 'tool_source', 'tool_source_class', 'tool_dir', 'is_task')¶
- __init__(sa_session, job: Job, working_directory: str, outputs_directory: str, outputs_to_working_directory: bool, galaxy_url: str, version_path: str, tool_directory: str, home_directory: str, tmp_directory: str, tool_data_path: str, galaxy_data_manager_data_path: str, new_file_path: str, len_file_path: str, builds_file_path: str, check_job_script_integrity: bool, check_job_script_integrity_count: int, check_job_script_integrity_sleep: float, file_sources_dict: Dict[str, Any], user_context: Union[ProvidesUserFileSourcesUserContext, Dict[str, Any]], tool_source: Optional[str] = None, tool_source_class: Optional[str] = 'XmlToolSource', tool_dir: Optional[str] = None, is_task: bool = False)[source]¶
- property file_sources: ConfiguredFileSources¶
- property dataset_path_rewriter: DatasetPathRewriter¶
- property output_paths: List[DatasetPath]¶
- property output_hdas_and_paths: Dict[str, Tuple[DatasetInstance, DatasetPath]]¶
- get_input_dataset_fnames(ds: DatasetInstance) List[str] [source]¶
- get_input_paths() List[DatasetPath] [source]¶
- get_input_path(dataset: DatasetInstance) DatasetPath [source]¶
- get_output_fnames() List[DatasetPath] [source]¶
- get_output_hdas_and_fnames() Dict[str, Tuple[DatasetInstance, DatasetPath]] [source]¶