galaxy.files.sources package

class galaxy.files.sources.PluginKind(value)

Bases: str, Enum

Enum to distinguish between different kinds or categories of plugins.

rfs = 'rfs'

Remote File System

A remote file system is a file source that is backed by a remote file system that is accessible via a URI. Examples include FTP, SFTP, S3, etc.

drs = 'drs'

Data Repository Service

A data repository service is a file source that is backed by a remote data repository service implementing the (DRS) API which provides a generic interface to data repositories so data consumers, including workflow systems, can access data in a single, standard way regardless of where it’s stored and how it’s managed.

rdm = 'rdm'

Research Data Management

A research data management file source is a file source that is backed by a remote research data management system that can provide DOIs. Examples include InvenioRDM, Zenodo, etc.

stock = 'stock'

Stock Plugin

A stock plugin is a file source that is shipped with Galaxy and covers common use cases. Examples include the UserFTP, LibraryImport, UserLibraryImport, etc.

class galaxy.files.sources.FilesSourceProperties

Bases: TypedDict

Initial set of properties used to initialize a filesource.

Filesources can extend this typed dict to define any additional filesource specific properties.

file_sources_config: typing_extensions.NotRequired[ConfiguredFileSourcesConfig]

id: typing_extensions.NotRequired[str]

label: typing_extensions.NotRequired[str]

doc: typing_extensions.NotRequired[str | None]

scheme: typing_extensions.NotRequired[str]

writable: typing_extensions.NotRequired[bool]

requires_roles: typing_extensions.NotRequired[str | None]

requires_groups: typing_extensions.NotRequired[str | None]

uri_root: typing_extensions.NotRequired[str]

type: typing_extensions.NotRequired[str]

browsable: typing_extensions.NotRequired[bool]

class galaxy.files.sources.FilesSourceOptions(writeable: bool | None = False, extra_props: ~galaxy.files.sources.FilesSourceProperties | None = <factory>)

Bases: object

Options to control behavior of file source operations, such as realize_to, write_from and list.

writeable: bool | None = False

extra_props: FilesSourceProperties | None

__init__(writeable: bool | None = False, extra_props: ~galaxy.files.sources.FilesSourceProperties | None = <factory>) → None

class galaxy.files.sources.EntryData

Bases: TypedDict

Provides data to create a new entry in a file source.

name: str

class galaxy.files.sources.Entry

Bases: TypedDict

Represents the result of creating a new entry in a file source.

name: str

uri: str

external_link: typing_extensions.NotRequired[str]

class galaxy.files.sources.RemoteEntry

Bases: TypedDict

name: str

uri: str

path: str

class galaxy.files.sources.TDirectoryClass

Bases: TypedDict

class: typing_extensions.Literal[Directory]

class galaxy.files.sources.TFileClass

Bases: TypedDict

class: typing_extensions.Literal[File]

class galaxy.files.sources.RemoteDirectory

Bases: RemoteEntry, TDirectoryClass

name: str

uri: str

path: str

class: typing_extensions.Literal[Directory]

class galaxy.files.sources.RemoteFile

Bases: RemoteEntry, TFileClass

size: int

ctime: str

name: str

uri: str

path: str

class: typing_extensions.Literal[File]

class galaxy.files.sources.SingleFileSource

Bases: object

Represents a protocol handler for a single remote file that can be read by or written to by Galaxy. A remote file source can typically handle a url like https://galaxyproject.org/myfile.txt or drs://myserver/123456. The filesource abstraction allows programmatic control over the specific source to access, injection of credentials and access control. Filesources are typically listed and configured through file_sources_conf.yml or programmatically, as required.

Filesources can be contextualized with a user_context, which contains information related to the current user attempting to access that filesource such as the username, preferences, roles etc., which can then be used by the filesource to make authorization decisions or inject credentials.

Filesources are loaded through Galaxy’s plugin system in galaxy.util.plugin_config.

abstract get_writable() → bool

Return a boolean indicating whether this target is writable.

abstract user_has_access(user_context) → bool

Return a boolean indicating whether the user can access the FileSource.

abstract realize_to(source_path: str, native_path: str, user_context=None, opts: FilesSourceOptions | None = None)

Realize source path (relative to uri root) to local file system path.

Parameters:

• source_path (str) – url of the source file to copy from. e.g. https://galaxyproject.org/myfile.txt

• native_path (str) – local path to write to. e.g. /tmp/myfile.txt

• user_context (FileSourceDictifiable, optional) – A user context , defaults to None

• opts (Optional[FilesSourceOptions], optional) – A set of options to exercise additional control over the realize_to method. Filesource specific, defaults to None

abstract write_from(target_path: str, native_path: str, user_context=None, opts: FilesSourceOptions | None = None)

Write file at native path to target_path (relative to uri root).

Parameters:

• target_path (str) – url of the target file to write to within the filesource. e.g. gxfiles://myftp1/myfile.txt

• native_path (str) – The local file to read. e.g. /tmp/myfile.txt

• user_context (_type_, optional) – A user context , defaults to None

• opts (Optional[FilesSourceOptions], optional) – A set of options to exercise additional control over the write_from method. Filesource specific, defaults to None

abstract score_url_match(url: str) → int

Return how well a given url matches this filesource. A score greater than zero indicates that this filesource is capable of processing the given url.

Scoring is based on the following rules: a. The maximum score will be the length of the url. b. The minimum score will be the length of the scheme if the filesource can handle the file. c. The score will be zero if the filesource cannot handle the file.

For example, given the following file source config: - type: webdav

id: cloudstor url: “https://cloudstor.aarnet.edu.au” root: “/plus/remote.php/webdav”

• type: http id: generic_http

the generic http handler may return a score of 8 for the url https://cloudstor.aarnet.edu.au/plus/remote.php/webdav/myfolder/myfile.txt, as it can handle only the scheme part of the url. A webdav handler may return a score of 55 for the same url, as both the webdav url and root combined are a specific match.

Parameters:

url (str) – The url to score for a match against this filesource.

Returns:

A score based on the aforementioned rules.

Return type:

int

abstract to_relative_path(url: str) → str

Convert this url to a filesource relative path. For example, given the url gxfiles://mysource1/myfile.txt it will return /myfile.txt. Protocols directly understood by the handler need not be relativized. For example, the url s3://bucket/myfile.txt can be returned unchanged.

abstract to_dict(for_serialization=False, user_context=None) → FilesSourceProperties

Return a dictified representation of this FileSource instance.

If user_context is supplied, properties should be written so user context doesn’t need to be present after the plugin is re-hydrated.

class galaxy.files.sources.SupportsBrowsing

Bases: object

An interface indicating that this filesource is browsable.

Browsable filesources will typically have a root uri from which to start browsing. e.g. In an s3 bucket, the root uri may be gxfiles://bucket1/

They will also have a list method to list files in a specific path within the filesource.

abstract get_uri_root() → str

Return a prefix for the root (e.g. gxfiles://prefix/).

abstract list(path='/', recursive=False, user_context=None, opts: FilesSourceOptions | None = None) → dict

Return dictionary of ‘Directory’s and ‘File’s.

class galaxy.files.sources.FilesSource

Bases: SingleFileSource, SupportsBrowsing

Represents a combined interface for single or browsable filesources. The get_browsable method can be used to determine whether the filesource is browsable and implements the SupportsBrowsing interface.

abstract get_browsable() → bool

Return true if the filesource implements the SupportsBrowsing interface.

class galaxy.files.sources.BaseFilesSource

Bases: FilesSource

plugin_type: ClassVar[str]

plugin_kind: ClassVar[PluginKind] = 'rfs'

get_browsable() → bool

Return true if the filesource implements the SupportsBrowsing interface.

get_prefix() → str | None

get_scheme() → str

get_writable() → bool

Return a boolean indicating whether this target is writable.

user_has_access(user_context) → bool

Return a boolean indicating whether the user can access the FileSource.

property user_context_required: bool

get_uri_root() → str

Return a prefix for the root (e.g. gxfiles://prefix/).

to_relative_path(url: str) → str

Convert this url to a filesource relative path. For example, given the url gxfiles://mysource1/myfile.txt it will return /myfile.txt. Protocols directly understood by the handler need not be relativized. For example, the url s3://bucket/myfile.txt can be returned unchanged.

score_url_match(url: str) → int

Return how well a given url matches this filesource. A score greater than zero indicates that this filesource is capable of processing the given url.

Scoring is based on the following rules: a. The maximum score will be the length of the url. b. The minimum score will be the length of the scheme if the filesource can handle the file. c. The score will be zero if the filesource cannot handle the file.

For example, given the following file source config: - type: webdav

id: cloudstor url: “https://cloudstor.aarnet.edu.au” root: “/plus/remote.php/webdav”

• type: http id: generic_http

the generic http handler may return a score of 8 for the url https://cloudstor.aarnet.edu.au/plus/remote.php/webdav/myfolder/myfile.txt, as it can handle only the scheme part of the url. A webdav handler may return a score of 55 for the same url, as both the webdav url and root combined are a specific match.

Parameters:

url (str) – The url to score for a match against this filesource.

Returns:

A score based on the aforementioned rules.

Return type:

int

uri_from_path(path) → str

to_dict(for_serialization=False, user_context=None) → FilesSourceProperties

Return a dictified representation of this FileSource instance.

If user_context is supplied, properties should be written so user context doesn’t need to be present after the plugin is re-hydrated.

to_dict_time(ctime)

list(path='/', recursive=False, user_context=None, opts: FilesSourceOptions | None = None)

Return dictionary of ‘Directory’s and ‘File’s.

create_entry(entry_data: EntryData, user_context=None, opts: FilesSourceOptions | None = None) → Entry

write_from(target_path, native_path, user_context=None, opts: FilesSourceOptions | None = None)

Write file at native path to target_path (relative to uri root).

Parameters:

• target_path (str) – url of the target file to write to within the filesource. e.g. gxfiles://myftp1/myfile.txt

• native_path (str) – The local file to read. e.g. /tmp/myfile.txt

• user_context (_type_, optional) – A user context , defaults to None

• opts (Optional[FilesSourceOptions], optional) – A set of options to exercise additional control over the write_from method. Filesource specific, defaults to None

realize_to(source_path, native_path, user_context=None, opts: FilesSourceOptions | None = None)

Realize source path (relative to uri root) to local file system path.

Parameters:

• source_path (str) – url of the source file to copy from. e.g. https://galaxyproject.org/myfile.txt

• native_path (str) – local path to write to. e.g. /tmp/myfile.txt

• user_context (FileSourceDictifiable, optional) – A user context , defaults to None

• opts (Optional[FilesSourceOptions], optional) – A set of options to exercise additional control over the realize_to method. Filesource specific, defaults to None

galaxy.files.sources.uri_join(*args)

galaxy.files.sources.slash_join(*args)

Submodules

galaxy.files.sources.anvil module

class galaxy.files.sources.anvil.AnVILFilesSource(**kwd: ~typing_extensions.)

Bases: PyFilesystem2FilesSource

plugin_type: ClassVar[str] = 'anvil'

required_module: ClassVar[Type[FS] | None] = None

required_package: ClassVar[str] = 'fs.anvilfs'

galaxy.files.sources.basespace module

class galaxy.files.sources.basespace.BaseSpaceFilesSource(**kwd: ~typing_extensions.)

Bases: PyFilesystem2FilesSource

plugin_type: ClassVar[str] = 'basespace'

required_module: ClassVar[Type[FS] | None] = None

required_package: ClassVar[str] = 'fs-basespace'

galaxy.files.sources.dropbox module

class galaxy.files.sources.dropbox.DropboxFilesSource(**kwd: ~typing_extensions.)

Bases: PyFilesystem2FilesSource

plugin_type: ClassVar[str] = 'dropbox'

required_module: ClassVar[Type[FS] | None] = None

required_package: ClassVar[str] = 'fs.dropboxfs'

galaxy.files.sources.ftp module

class galaxy.files.sources.ftp.FtpFilesSource(**kwd: ~typing_extensions.)

Bases: PyFilesystem2FilesSource

plugin_type: ClassVar[str] = 'ftp'

required_module

alias of FTPFS

required_package: ClassVar[str] = 'fs.ftpfs'

score_url_match(url: str)

Return how well a given url matches this filesource. A score greater than zero indicates that this filesource is capable of processing the given url.

Scoring is based on the following rules: a. The maximum score will be the length of the url. b. The minimum score will be the length of the scheme if the filesource can handle the file. c. The score will be zero if the filesource cannot handle the file.

For example, given the following file source config: - type: webdav

id: cloudstor url: “https://cloudstor.aarnet.edu.au” root: “/plus/remote.php/webdav”

• type: http id: generic_http

the generic http handler may return a score of 8 for the url https://cloudstor.aarnet.edu.au/plus/remote.php/webdav/myfolder/myfile.txt, as it can handle only the scheme part of the url. A webdav handler may return a score of 55 for the same url, as both the webdav url and root combined are a specific match.

Parameters:

url (str) – The url to score for a match against this filesource.

Returns:

A score based on the aforementioned rules.

Return type:

int

galaxy.files.sources.galaxy module

Static Galaxy file sources - ftp and libraries.

class galaxy.files.sources.galaxy.UserFtpFilesSource(**kwd: ~typing_extensions.)

Bases: PosixFilesSource

plugin_type: ClassVar[str] = 'gxftp'

plugin_kind: ClassVar[PluginKind] = 'stock'

__init__(**kwd: ~typing_extensions.)

get_prefix() → str | None

get_scheme() → str

class galaxy.files.sources.galaxy.LibraryImportFilesSource(**kwd: ~typing_extensions.)

Bases: PosixFilesSource

plugin_type: ClassVar[str] = 'gximport'

plugin_kind: ClassVar[PluginKind] = 'stock'

__init__(**kwd: ~typing_extensions.)

get_prefix() → str | None

get_scheme() → str

class galaxy.files.sources.galaxy.UserLibraryImportFilesSource(**kwd: ~typing_extensions.)

Bases: PosixFilesSource

plugin_type: ClassVar[str] = 'gxuserimport'

plugin_kind: ClassVar[PluginKind] = 'stock'

__init__(**kwd: ~typing_extensions.)

get_prefix() → str | None

get_scheme() → str

galaxy.files.sources.googlecloudstorage module

class galaxy.files.sources.googlecloudstorage.GoogleCloudStorageFilesSource(**kwd: ~typing_extensions.)

Bases: PyFilesystem2FilesSource

plugin_type: ClassVar[str] = 'googlecloudstorage'

required_module: ClassVar[Type[FS] | None] = None

required_package: ClassVar[str] = 'fs-gcsfs'

galaxy.files.sources.googledrive module

class galaxy.files.sources.googledrive.GoogleDriveFilesSource(**kwd: ~typing_extensions.)

Bases: PyFilesystem2FilesSource

plugin_type: ClassVar[str] = 'googledrive'

required_module: ClassVar[Type[FS] | None] = None

required_package: ClassVar[str] = 'fs.googledrivefs'

galaxy.files.sources.onedata module

class galaxy.files.sources.onedata.OnedataFilesSource(**kwd: ~typing_extensions.)

Bases: PyFilesystem2FilesSource

plugin_type: ClassVar[str] = 'onedata'

required_module: ClassVar[Type[FS] | None] = None

required_package: ClassVar[str] = 'fs.onedatarestfs'

galaxy.files.sources.posix module

class galaxy.files.sources.posix.PosixFilesSource(**kwd: ~typing_extensions.)

Bases: BaseFilesSource

plugin_type: ClassVar[str] = 'posix'

__init__(**kwd: ~typing_extensions.)

score_url_match(url: str)

Return how well a given url matches this filesource. A score greater than zero indicates that this filesource is capable of processing the given url.

Scoring is based on the following rules: a. The maximum score will be the length of the url. b. The minimum score will be the length of the scheme if the filesource can handle the file. c. The score will be zero if the filesource cannot handle the file.

For example, given the following file source config: - type: webdav

id: cloudstor url: “https://cloudstor.aarnet.edu.au” root: “/plus/remote.php/webdav”

• type: http id: generic_http

the generic http handler may return a score of 8 for the url https://cloudstor.aarnet.edu.au/plus/remote.php/webdav/myfolder/myfile.txt, as it can handle only the scheme part of the url. A webdav handler may return a score of 55 for the same url, as both the webdav url and root combined are a specific match.

Parameters:

url (str) – The url to score for a match against this filesource.

Returns:

A score based on the aforementioned rules.

Return type:

int

to_relative_path(url: str) → str

Convert this url to a filesource relative path. For example, given the url gxfiles://mysource1/myfile.txt it will return /myfile.txt. Protocols directly understood by the handler need not be relativized. For example, the url s3://bucket/myfile.txt can be returned unchanged.

galaxy.files.sources.s3 module

galaxy.files.sources.s3fs module

class galaxy.files.sources.s3fs.S3FsFilesSource(**kwd: ~typing_extensions.)

Bases: BaseFilesSource

plugin_type: ClassVar[str] = 's3fs'

__init__(**kwd: ~typing_extensions.)

score_url_match(url: str)

Return how well a given url matches this filesource. A score greater than zero indicates that this filesource is capable of processing the given url.

Scoring is based on the following rules: a. The maximum score will be the length of the url. b. The minimum score will be the length of the scheme if the filesource can handle the file. c. The score will be zero if the filesource cannot handle the file.

For example, given the following file source config: - type: webdav

id: cloudstor url: “https://cloudstor.aarnet.edu.au” root: “/plus/remote.php/webdav”

• type: http id: generic_http

the generic http handler may return a score of 8 for the url https://cloudstor.aarnet.edu.au/plus/remote.php/webdav/myfolder/myfile.txt, as it can handle only the scheme part of the url. A webdav handler may return a score of 55 for the same url, as both the webdav url and root combined are a specific match.

Parameters:

url (str) – The url to score for a match against this filesource.

Returns:

A score based on the aforementioned rules.

Return type:

int

galaxy.files.sources.ssh module

class galaxy.files.sources.ssh.SshFilesSource(**kwd: ~typing_extensions.)

Bases: PyFilesystem2FilesSource

plugin_type: ClassVar[str] = 'ssh'

required_module: ClassVar[Type[FS] | None] = None

required_package: ClassVar[str] = 'fs.sshfs'

galaxy.files.sources.webdav module

class galaxy.files.sources.webdav.WebDavFilesSource(**kwd: ~typing_extensions.)

Bases: PyFilesystem2FilesSource

plugin_type: ClassVar[str] = 'webdav'

required_module: ClassVar[Type[FS] | None] = None

required_package: ClassVar[str] = 'fs.webdavfs'