Warning

This document is for an old release of Galaxy. You can alternatively view this page in the latest release if it exists or view the top of the latest release's documentation.

galaxy.job_metrics.collectl package

Helper functions and data structures for interacting with collectl & data.

More information on collectl can be found at: http://collectl.sourceforge.net/.

Submodules

galaxy.job_metrics.collectl.cli module

This module describes CollectlCli - an abstraction for building collectl command lines.

class galaxy.job_metrics.collectl.cli.CollectlCli(**kwargs)[source]

Bases: object

Abstraction over (some of) the command-line arguments of collectl. Ideally this will be useful for building up command line arguments for remote execution as well as runnning directly on local host.

This is meant to be a fairly generic utility - for interfacing with collectl CLI - logic more directly related to the Galaxy job metric plugin plugin should be placed in other modules.

Keyword Arguments:

collectl_path

Path to collectl executable (defaults to collectl - i.e. search the PATH).

playback_path (defaults to None)

If this is None, collectl will run in record mode, else it will playback specified file.

Playback Mode Options:

sep

Separator used in playback mode (set to 9 to produce tsv) (defaults to None).

Record Mode Options (some of these may work in playback mode also)

destination_path

Location of path files to write to (defaults to None and collectl will just use cwd). Really this is just to prefix - collectl will append hostname and datetime to file.

interval

Setup polling interval (secs) for most subsystems (defaults to None and when unspecified collectl will use default of 1 second).

interval2

Setup polling interval (secs) for process information (defaults to None and when unspecified collectl will use default to 60 seconds).

interval3

Setup polling interval (secs) for environment information (defaults to None and when unspecified collectl will use default to 300 seconds).

procfilt

Optional argument to procfilt. (defaults to None).

flush

Optional flush interval (defaults to None).

__init__(**kwargs)[source]
build_command_line()[source]
run(stdout=- 1, stderr=- 1)[source]

galaxy.job_metrics.collectl.processes module

Modules will run collectl in playback mode and collect various process statistics for a given pid’s process and process ancestors.

galaxy.job_metrics.collectl.processes.generate_process_statistics(collectl_playback_cli, pid, statistics=[('max', 'VmSize'), ('avg', 'VmSize'), ('max', 'VmRSS'), ('avg', 'VmRSS'), ('sum', 'SysT'), ('sum', 'UsrT'), ('max', 'PCT'), ('avg', 'PCT'), ('max', 'AccumT'), ('sum', 'RSYS'), ('sum', 'WSYS')])[source]

Playback collectl file and generate summary statistics.

galaxy.job_metrics.collectl.stats module

Primitive module for tracking running statistics without storing values in memory.

class galaxy.job_metrics.collectl.stats.StatisticsTracker[source]

Bases: object

__init__()[source]
track(value)[source]
property avg

galaxy.job_metrics.collectl.subsystems module

Abstractions describing collectl subsystems (specified with the collectl -s parameter).

Subsystems are essentially monitoring plugins available within collectl.

galaxy.job_metrics.collectl.subsystems.get_subsystem(name)[source]
>>> get_subsystem( "process" ).command_line_arg == "Z"
True