Warning

This document is for an in-development version of Galaxy. You can alternatively view this page in the latest release if it exists or view the top of the latest release's documentation.

galaxy.celery package

class galaxy.celery.GalaxyCelery(*args, **kwargs)[source]

Bases: Celery

fork_pool: ProcessPool
__init__(*args, **kwargs)[source]
gen_task_name(name, module)[source]
trim_module_name(module)[source]

Drop “celery.tasks” infix for less verbose task names: - galaxy.celery.tasks.do_foo >> galaxy.do_foo - galaxy.celery.tasks.subtasks.do_fuz >> galaxy.subtasks.do_fuz

class galaxy.celery.GalaxyTask[source]

Bases: Task

Custom celery task used to enforce per-user rate limits and concurrency limits on task executions.

before_start(task_id, args, kwargs)[source]

Set appropriate before start object from DI container.

after_return(status, retval, task_id, args, kwargs, einfo)[source]

Called after task returns (success, failure, revoked, or retry). Used to clean up concurrency tracking rows.

ignore_result = False

If enabled the worker won’t store task state and return values for this task. Defaults to the :setting:`task_ignore_result` setting.

priority = None

Default task priority. A number between 0 and 9, where the interpretation depends on the broker: with RabbitMQ, higher numbers denote higher priority; with Redis, priority 0 is the highest. See routing-options-rabbitmq-priorities and redis-message-priorities.

rate_limit = None

Rate limit for this task type. Examples: None (no rate limit), ‘100/s’ (hundred tasks a second), ‘100/m’ (hundred tasks a minute),`’100/h’` (hundred tasks an hour)

reject_on_worker_lost = None

Even if acks_late is enabled, the worker will acknowledge tasks when the worker process executing them abruptly exits or is signaled (e.g., :sig:`KILL`/:sig:`INT`, etc).

Setting this to true allows the message to be re-queued instead, so that the task will execute again by the same worker, or another worker.

Warning: Enabling this can cause message loops; make sure you know what you’re doing.

request_stack = <celery.utils.threads._LocalStack object>

Task request stack, the current request will be the topmost.

retry_on_worker_lost = False

When enabled, a task whose worker process exits abruptly is sent again the same way retry() would, so max_retries and default_retry_delay apply. Once the retries run out the task fails instead of being re-queued.

Takes precedence over reject_on_worker_lost, which is still used if sending the retry fails. The default is taken from the :setting:`task_retry_on_worker_lost` setting.

serializer = 'pydantic-aware-json'

The name of a serializer that are registered with kombu.serialization.registry. Default is ‘json’.

soft_time_limit = None

Soft time limit. Defaults to the :setting:`task_soft_time_limit` setting.

store_errors_even_if_ignored = False

When enabled errors will be stored even if the task is otherwise configured to ignore results.

time_limit = None

Hard time limit. Defaults to the :setting:`task_time_limit` setting.

track_started = False

If enabled the task will report its status as ‘started’ when the task is executed by a worker. Disabled by default as the normal behavior is to not report that level of granularity. Tasks are either pending, finished, or waiting to be retried.

Having a ‘started’ status can be useful for when there are long running tasks and there’s a need to report what task is currently running.

The application default can be overridden using the :setting:`task_track_started` setting.

typing = True

Enable argument checking. You can set this to false if you don’t want the signature to be checked when calling the task. Defaults to app.strict_typing.

galaxy.celery.set_thread_app(app)[source]
galaxy.celery.get_galaxy_app()[source]
galaxy.celery.build_app()[source]
galaxy.celery.get_app_properties()[source]
galaxy.celery.get_config()[source]
galaxy.celery.init_fork_pool()[source]
galaxy.celery.setup_worker_pool(sender=None, conf=None, instance=None, **kwargs)[source]
galaxy.celery.tear_down_pool(sig, how, exitcode, **kwargs)[source]
class galaxy.celery.GalaxyTaskFunction(*args, **kwargs)[source]

Bases: Protocol

A function decorated with galaxy_task.

Arguments are not checked: app.magic_partial injects the DI arguments when the task runs, so callers pass fewer arguments than the function takes.

name: str
run: Callable[[...], Any]
delay(*args: Any, **kwargs: Any) → Any[source]
apply_async(*args: Any, **kwargs: Any) → Any[source]
s(*args: Any, **kwargs: Any) → Any[source]
si(*args: Any, **kwargs: Any) → Any[source]
signature(*args: Any, **kwargs: Any) → Any[source]
__init__(*args, **kwargs)
galaxy.celery.galaxy_task(func: Callable[[...], Any], /) → GalaxyTaskFunction[source]
galaxy.celery.galaxy_task(*, action: str | None = None, **celery_task_kwd: Any) → Callable[[Callable[[...], Any]], GalaxyTaskFunction]
galaxy.celery.init_celery_app()[source]
galaxy.celery.config_celery_app(config, celery_app)[source]
galaxy.celery.setup_periodic_tasks(config, celery_app)[source]

Submodules

galaxy.celery.tasks module

galaxy.celery.tasks.cached_create_tool_from_representation(app: MinimalManagerApp, raw_tool_source: str, tool_source_class: Literal['XmlToolSource', 'YamlToolSource', 'CwlToolSource'], tool_dir: str | None = None, tool_id: str | None = None)[source]
galaxy.celery.tasks.recalculate_disk_usage_deduplicated(session: Session, user: User, object_store: BaseObjectStore) → bool[source]

Recalculate user’s disk usage unless an equivalent recalculation is already queued.

On PostgreSQL at most one recalculation per user runs and at most one waits for it. The waiting one starts computing after the changes that triggered this call were committed, so this call returns False without computing when another one is waiting. This relies on the READ COMMITTED isolation level, PostgreSQL’s default: each statement of the waiting recalculation sees the changes committed before it started. Under REPEATABLE READ or SERIALIZABLE the waiting recalculation would only see the snapshot taken by the transaction’s first query, before it waited.

lock_timeout bounds the wait for a running recalculation, Celery’s thread pool does not enforce task time limits. The recalculation itself runs under the connection’s own lock_timeout and statement_timeout.

All locks are taken on the session’s connection. The waiting lock is session-level, so it survives a rollback: if anything fails before it is released, the connection is invalidated, which releases it on the server instead of returning it to the pool.

galaxy.celery.tasks.is_aborted(session: galaxy_scoped_session, job_id: int)[source]
galaxy.celery.tasks.abort_when_job_stops(function: Callable, session: galaxy_scoped_session, job_id: int, **kwargs) → Any[source]