galaxy.job_metrics.instrumenters package

This module describes the abstract interface for InstrumentPlugin.

These are responsible for collecting and formatting a coherent set of metrics.

class galaxy.job_metrics.instrumenters.InstrumentPlugin

Bases: object

Describes how to instrument job scripts and retrieve collected metrics.

formatter = <galaxy.job_metrics.formatting.JobMetricFormatter object>

default_safety = 4

abstract property plugin_type

Short string providing labelling this plugin

pre_execute_instrument(job_directory: str) → Optional[Union[str, List[str]]]

Optionally return one or more commands to instrument job. These commands will be executed on the compute server prior to the job running.

post_execute_instrument(job_directory: str) → Optional[Union[str, List[str]]]

Optionally return one or more commands to instrument job. These commands will be executed on the compute server after the tool defined command is ran.

abstract job_properties(job_id, job_directory: str) → Dict[str, Any]

Collect properties for this plugin from specified job directory. This method will run on the Galaxy server and can assume files created in job_directory with pre_execute_instrument and post_execute_instrument are available.

safety(metric_name: str) → Safety

Return safety level of metric.

Submodules

galaxy.job_metrics.instrumenters.cgroup module

The module describes the cgroup job metrics plugin.

class galaxy.job_metrics.instrumenters.cgroup.CgroupPlugin(**kwargs)

Bases: InstrumentPlugin

Plugin that collects memory and cpu utilization from within a cgroup.

plugin_type = 'cgroup'

formatter = <galaxy.job_metrics.instrumenters.cgroup.CgroupPluginFormatter object>

__init__(**kwargs)

post_execute_instrument(job_directory: str) → List[str]

Optionally return one or more commands to instrument job. These commands will be executed on the compute server after the tool defined command is ran.

job_properties(job_id, job_directory: str) → Dict[str, Any]

Collect properties for this plugin from specified job directory. This method will run on the Galaxy server and can assume files created in job_directory with pre_execute_instrument and post_execute_instrument are available.

galaxy.job_metrics.instrumenters.core module

The module describes the core job metrics plugin.

class galaxy.job_metrics.instrumenters.core.CorePlugin(**kwargs)

Bases: InstrumentPlugin

Simple plugin that collects data without external dependencies. In particular it currently collects value set for Galaxy slots.

plugin_type = 'core'

formatter = <galaxy.job_metrics.instrumenters.core.CorePluginFormatter object>

default_safety = 6

__init__(**kwargs)

pre_execute_instrument(job_directory: str) → List[str]

Optionally return one or more commands to instrument job. These commands will be executed on the compute server prior to the job running.

post_execute_instrument(job_directory: str) → List[str]

Optionally return one or more commands to instrument job. These commands will be executed on the compute server after the tool defined command is ran.

job_properties(job_id, job_directory: str) → Dict[str, Any]

Collect properties for this plugin from specified job directory. This method will run on the Galaxy server and can assume files created in job_directory with pre_execute_instrument and post_execute_instrument are available.

galaxy.job_metrics.instrumenters.cpuinfo module

The module describes the cpuinfo job metrics plugin.

class galaxy.job_metrics.instrumenters.cpuinfo.CpuInfoPlugin(**kwargs)

Bases: InstrumentPlugin

Gather information about processor configuration from /proc/cpuinfo. Linux only.

plugin_type = 'cpuinfo'

formatter = <galaxy.job_metrics.instrumenters.cpuinfo.CpuInfoFormatter object>

__init__(**kwargs)

pre_execute_instrument(job_directory)

Optionally return one or more commands to instrument job. These commands will be executed on the compute server prior to the job running.

job_properties(job_id, job_directory)

Collect properties for this plugin from specified job directory. This method will run on the Galaxy server and can assume files created in job_directory with pre_execute_instrument and post_execute_instrument are available.

galaxy.job_metrics.instrumenters.env module

The module describes the env job metrics plugin.

class galaxy.job_metrics.instrumenters.env.EnvPlugin(**kwargs)

Bases: InstrumentPlugin

Instrumentation plugin capable of recording all or specific environment variables for a job at runtime.

plugin_type = 'env'

formatter = <galaxy.job_metrics.instrumenters.env.EnvFormatter object>

default_safety = 2

__init__(**kwargs)

variables: Optional[List[str]]

pre_execute_instrument(job_directory: str)

Use env to dump all environment variables to a file.

post_execute_instrument(job_directory)

Optionally return one or more commands to instrument job. These commands will be executed on the compute server after the tool defined command is ran.

job_properties(job_id, job_directory)

Recover environment variables dumped out on compute server and filter out specific variables if needed.

galaxy.job_metrics.instrumenters.hostname module

The module describes the hostname job metrics plugin.

class galaxy.job_metrics.instrumenters.hostname.HostnamePlugin(**kwargs)

Bases: InstrumentPlugin

Gather hostname

plugin_type = 'hostname'

formatter = <galaxy.job_metrics.instrumenters.hostname.HostnameFormatter object>

__init__(**kwargs)

pre_execute_instrument(job_directory)

Optionally return one or more commands to instrument job. These commands will be executed on the compute server prior to the job running.

job_properties(job_id, job_directory)

Collect properties for this plugin from specified job directory. This method will run on the Galaxy server and can assume files created in job_directory with pre_execute_instrument and post_execute_instrument are available.

galaxy.job_metrics.instrumenters.meminfo module

The module describes the meminfo job metrics plugin.

class galaxy.job_metrics.instrumenters.meminfo.MemInfoPlugin(**kwargs)

Bases: InstrumentPlugin

Gather information about processor configuration from /proc/cpuinfo. Linux only.

plugin_type = 'meminfo'

formatter = <galaxy.job_metrics.instrumenters.meminfo.MemInfoFormatter object>

default_safety = 6

__init__(**kwargs)

pre_execute_instrument(job_directory)

Optionally return one or more commands to instrument job. These commands will be executed on the compute server prior to the job running.

job_properties(job_id, job_directory)

Collect properties for this plugin from specified job directory. This method will run on the Galaxy server and can assume files created in job_directory with pre_execute_instrument and post_execute_instrument are available.

galaxy.job_metrics.instrumenters.uname module

The module describes the uname job metrics plugin.

class galaxy.job_metrics.instrumenters.uname.UnamePlugin(**kwargs)

Bases: InstrumentPlugin

Use uname to gather operating system information about remote system job is running on. Linux only.

plugin_type = 'uname'

formatter = <galaxy.job_metrics.instrumenters.uname.UnameFormatter object>

__init__(**kwargs)

pre_execute_instrument(job_directory)

Optionally return one or more commands to instrument job. These commands will be executed on the compute server prior to the job running.

job_properties(job_id, job_directory)

Collect properties for this plugin from specified job directory. This method will run on the Galaxy server and can assume files created in job_directory with pre_execute_instrument and post_execute_instrument are available.