galaxy.job_execution package
Subpackages
- galaxy.job_execution.actions package
- galaxy.job_execution.ports package
Submodules
galaxy.job_execution.compute_environment module
- class galaxy.job_execution.compute_environment.ComputeEnvironment[source]
Bases:
object
Definition of the job as it will be run on the (potentially) remote compute server.
- abstract input_extra_files_rewrite(dataset)[source]
Input extra files path rewrite for specified dataset.
- abstract output_extra_files_rewrite(dataset)[source]
Output extra files path rewrite for specified dataset.
- abstract input_metadata_rewrite(dataset, metadata_value)[source]
Input metadata path rewrite for specified dataset.
Bases:
SimpleComputeEnvironment
,ComputeEnvironment
Default ComputeEnvironment for job and task wrapper to pass to ToolEvaluator - valid when Galaxy and compute share all the relevant file systems.
Return file sources dict for current user.
Output unqualified filenames defined by job.
Input path for specified dataset.
Output path for specified dataset.
Input extra files path rewrite for specified dataset.
Output extra files path rewrite for specified dataset.
Input metadata path rewrite for specified dataset.
Rewrite loc file paths, etc..
Job working directory (potentially remote)
Working directory (possibly as environment variable evaluation).
Absolute path to dump new files for this job on compute server.
Location of the version file for the underlying tool.
Absolute path to tool files for this job on compute server.
Home directory of target job - none if HOME should not be set.
Temp directory of target job - none if HOME should not be set.
URL to access Galaxy API from for this compute environment.
galaxy.job_execution.datasets module
Utility classes allowing Job interface to reason about datasets.
- class galaxy.job_execution.datasets.DatasetPath(dataset_id, real_path, false_path=None, false_extra_files_path=None, false_metadata_path=None, mutable=True, dataset_uuid=None, object_store_id=None)[source]
Bases:
object
- class galaxy.job_execution.datasets.DatasetPathRewriter[source]
Bases:
object
Used by runner to rewrite paths.
- class galaxy.job_execution.datasets.NullDatasetPathRewriter[source]
Bases:
DatasetPathRewriter
Used by default for jobwrapper, do not rewrite anything.
- class galaxy.job_execution.datasets.OutputsToWorkingDirectoryPathRewriter(working_directory, outputs_directory_name)[source]
Bases:
DatasetPathRewriter
Rewrites all paths to place them in the specified working directory for normal jobs when Galaxy is configured with app.config.outputs_to_working_directory. Job runner base class is responsible for copying these out after job is complete.
- class galaxy.job_execution.datasets.TaskPathRewriter(working_directory, job_dataset_path_rewriter)[source]
Bases:
DatasetPathRewriter
Rewrites all paths to place them in the specified working directory for TaskWrapper. TaskWrapper is responsible for putting them there and pulling them out.
- galaxy.job_execution.datasets.get_path_rewriter(outputs_to_working_directory, working_directory, outputs_directory, is_task) DatasetPathRewriter [source]
galaxy.job_execution.output_collect module
Code allowing tools to define extra files associated with an output datset.
- class galaxy.job_execution.output_collect.PermissionProvider(inp_data, security_agent, job)[source]
Bases:
PermissionProvider
- property permissions
- class galaxy.job_execution.output_collect.MetadataSourceProvider(inp_data)[source]
Bases:
MetadataSourceProvider
- galaxy.job_execution.output_collect.collect_dynamic_outputs(job_context, output_collections)[source]
- class galaxy.job_execution.output_collect.BaseJobContext[source]
Bases:
ModelPersistenceContext
- tool_provided_metadata: BaseToolProvidedMetadata
- class galaxy.job_execution.output_collect.JobContext(tool, tool_provided_metadata: BaseToolProvidedMetadata, job, job_working_directory, permission_provider, metadata_source_provider, input_dbkey, object_store, final_job_state, max_discovered_files: int | None, flush_per_n_datasets=None)[source]
Bases:
BaseJobContext
- __init__(tool, tool_provided_metadata: BaseToolProvidedMetadata, job, job_working_directory, permission_provider, metadata_source_provider, input_dbkey, object_store, final_job_state, max_discovered_files: int | None, flush_per_n_datasets=None)[source]
- tool_provided_metadata: BaseToolProvidedMetadata
- property change_datatype_actions
- property tag_handler
Return a galaxy.model.tags.TagHandler-like object for persisting tags.
- property work_context
- property sa_session: scoped_session
If bound to a database, return the SQL Alchemy session.
Return None otherwise.
- property permission_provider: PermissionProvider
If bound to a database, return the SQL Alchemy session.
Return None otherwise.
- property metadata_source_provider: MetadataSourceProvider
Return associated MetadataSourceProvider object.
- property job: Job
Return associated job object if bound to a job finish context connected to a database.
- property object_store: ObjectStore
Return object store to use for populating discovered dataset contents.
- property user
If bound to a database, return the user the datasets should be created for.
Return None otherwise.
- create_library_folder(parent_folder, name, description)[source]
Create a library folder ready from supplied attributes for supplied parent.
- add_output_dataset_association(name, dataset)[source]
If discovering outputs for a job, persist output dataset association.
- add_library_dataset_to_folder(library_folder, ld)[source]
Add library dataset to persisted library folder.
- class galaxy.job_execution.output_collect.SessionlessJobContext(metadata_params, tool_provided_metadata: BaseToolProvidedMetadata, object_store, export_store, import_store, working_directory, final_job_state, max_discovered_files: int | None)[source]
Bases:
SessionlessModelPersistenceContext
,BaseJobContext
- __init__(metadata_params, tool_provided_metadata: BaseToolProvidedMetadata, object_store, export_store, import_store, working_directory, final_job_state, max_discovered_files: int | None)[source]
- tool_provided_metadata: BaseToolProvidedMetadata
- property change_datatype_actions
- galaxy.job_execution.output_collect.collect_primary_datasets(job_context: JobContext | SessionlessJobContext, output, input_ext)[source]
- galaxy.job_execution.output_collect.discover_files(output_name, tool_provided_metadata, extra_file_collectors, job_working_directory, matchable)[source]
- galaxy.job_execution.output_collect.walk_over_file_collectors(extra_file_collectors, job_working_directory, matchable)[source]
- galaxy.job_execution.output_collect.walk_over_extra_files(target_dir, extra_file_collector, job_working_directory, matchable, parent_paths=None)[source]
Walks through all files in a given directory, and returns all files that match the given collector’s match criteria. If the collector has the recurse flag enabled, will also recursively descend into child folders.
- class galaxy.job_execution.output_collect.ToolMetadataDatasetCollector(dataset_collection_description)[source]
Bases:
object
- class galaxy.job_execution.output_collect.DatasetCollector(dataset_collection_description)[source]
Bases:
object
- galaxy.job_execution.output_collect.read_exit_code_from(exit_code_file, id_tag)[source]
Read exit code reported for a Galaxy job.
- galaxy.job_execution.output_collect.collect_extra_files(object_store: ObjectStore, dataset: DatasetInstance, job_working_directory: str, outputs_to_working_directory: bool = False)[source]
galaxy.job_execution.setup module
Utilities to help job and tool code setup jobs.
- class galaxy.job_execution.setup.JobOutput(output_name, dataset, dataset_path)[source]
Bases:
tuple
- dataset: DatasetInstance
Alias for field number 1
- dataset_path: DatasetPath
Alias for field number 2
- class galaxy.job_execution.setup.JobIO(sa_session, job: Job, working_directory: str, outputs_directory: str, outputs_to_working_directory: bool, galaxy_url: str, version_path: str, tool_directory: str, home_directory: str, tmp_directory: str, tool_data_path: str, galaxy_data_manager_data_path: str, new_file_path: str, len_file_path: str, builds_file_path: str, check_job_script_integrity: bool, check_job_script_integrity_count: int, check_job_script_integrity_sleep: float, file_sources_dict: Dict[str, Any], user_context: FileSourcesUserContext | Dict[str, Any], tool_source: str | None = None, tool_source_class: str | None = 'XmlToolSource', tool_dir: str | None = None, is_task: bool = False)[source]
Bases:
Dictifiable
- dict_collection_visible_keys = ('job_id', 'working_directory', 'outputs_directory', 'outputs_to_working_directory', 'galaxy_url', 'version_path', 'tool_directory', 'home_directory', 'tmp_directory', 'tool_data_path', 'galaxy_data_manager_data_path', 'new_file_path', 'len_file_path', 'builds_file_path', 'file_sources_dict', 'check_job_script_integrity', 'check_job_script_integrity_count', 'check_job_script_integrity_sleep', 'tool_source', 'tool_source_class', 'tool_dir', 'is_task')
- __init__(sa_session, job: Job, working_directory: str, outputs_directory: str, outputs_to_working_directory: bool, galaxy_url: str, version_path: str, tool_directory: str, home_directory: str, tmp_directory: str, tool_data_path: str, galaxy_data_manager_data_path: str, new_file_path: str, len_file_path: str, builds_file_path: str, check_job_script_integrity: bool, check_job_script_integrity_count: int, check_job_script_integrity_sleep: float, file_sources_dict: Dict[str, Any], user_context: FileSourcesUserContext | Dict[str, Any], tool_source: str | None = None, tool_source_class: str | None = 'XmlToolSource', tool_dir: str | None = None, is_task: bool = False)[source]
- property job
- property file_sources: ConfiguredFileSources
- property dataset_path_rewriter: DatasetPathRewriter
- property output_paths: List[DatasetPath]
- property output_hdas_and_paths: Dict[str, Tuple[DatasetInstance, DatasetPath]]
- get_input_dataset_fnames(ds: DatasetInstance) List[str] [source]
- get_input_datasets() List[DatasetInstance] [source]
- get_input_paths() List[DatasetPath] [source]
- get_input_path(dataset: DatasetInstance) DatasetPath [source]
- get_output_fnames() List[DatasetPath] [source]
- get_output_hdas_and_fnames() Dict[str, Tuple[DatasetInstance, DatasetPath]] [source]