January 2018 Galaxy Release (v 18.01)

Get Galaxy

Highlights

Performance and User Experience Improvements

We made Galaxy more lively and responsive. Homepage, published workflows, published/saved histories, and data libraries should all load much faster now. Importing data from FTP will also take less of your time. We achieved this by optimizing database queries, implementing cache, rethinking presentation, and adding progress bars and loading indicators where needed. See the list of related performance pull requests.

Web Server and Configuration

The default web server used by Galaxy has changed from Paste to uWSGI and the default configuration file for Galaxy is now config/galaxy.yml instead of config/galaxy.ini. To minimize the impact of this change on existing Galaxy instances, if a Galaxy has a galaxy.ini file configured, it will continue to use Paste by default unless additional steps are taken by the administrator. uWSGI is more production ready and allows Galaxy to scale better in its default configuration. Read more about the server, configuration, and documentation changes in the uWSGI details section of this document.

Dataset Collection Usability

This release has significantly improved the usability of Galaxy dataset collections. Dozens of improvements to collections have been made, some of the key highlights include:

Client Architecture

The architecture for the client code that powers the Galaxy user interface has been significantly overhauled. The code base has been converted to ES6, Yarn now powers the build and dependency management of the code, Prettier is now used to ensure consistent code formatting, and the VueJS framework has been integrated. Taken together, these changes are enabling Galaxy developers to write usable, responsive client code more quickly and concisely than previously possible. A big thanks goes out to community contributions from @anuprulez and @anatskiy that are already converting existing Galaxy components to reactive VueJS ones. See the list of related client pull requests.

New BAM datatypes

Previously Galaxy only supported coordinate sorted BAM files by default (the bam datatype). In addition, this release of Galaxy now supports three new types of BAM:

  • qname_sorted.bam, that ensures that the file is queryname sorted (e.g. SO:queryname);

  • qname_input_sorted.bam, that can be used to describe the output of aligners which generally keep mate pairs adjacent

  • unsorted.bam, that makes no assumptions about the sort order of the file.

A huge thanks goes out to @bgruening and @mvdbeek who implemented these datatypes.

Pull Request 5180, Pull Request 5589, Pull Request 5532, Pull Request 5644, Pull Request 5674

Experimental Job Caching

Galaxy can now be configured to allow users the option of skipping duplicated jobs if one with identical parameters has been previously executed and simply reuse the previously generated outputs. This contribution is thanks to @mvdbeek.

Pull Request 4690

Get Galaxy

The code lives at Github and you should have Git to obtain it.

To get a new Galaxy repository run:
$ git clone -b release_18.01 https://github.com/galaxyproject/galaxy.git
To update an existing Galaxy repository run:
$ git fetch origin && git checkout release_18.01 && git pull --ff-only origin release_18.01

See the community hub for additional details regarding the source code locations.

Security

Unsecure GenomeSpace token exposure

Tracked as GX-2018-0002.

We have found and fixed a medium-level security issue concering the GenomeSpace importer/exporter tools that were updated in the Galaxy release 17.09. These tools did not handle the GenomeSpace access token securely and stored it as a job parameter which made it accessible to anybody with access to the datasets created by these tools. This means that any user that used a GenomeSpace token to access these tools and subsequently shared the output dataset (or history that contains it) with another user shared their GenomeSpace token also.

These tools are both included in the tool_conf.xml.sample and are therefore enabled on every new Galaxy by default.

Administrators please see the GenomeSpace security sanitization section of this document for the details on how to sanitize the tokens stored in the Galaxy database created prior to this fix.

The vulnerability has been resolved by removing the token functionality until a proper implementation is in place. The GenomeSpace tools continue to work using the OpenID authentication as before.

The fix for this issue has been applied back to Galaxy release 17.09 and can be found in this pull request.

Breaking Changes

We have reworked the Galaxy Webhook interface so if you have custom webhooks at your instance you need to take the following steps in order to make them work:

  1. Rename the main config file: <name>.yml -> config.yml

  2. In the config file, rename the name attribute: name -> id

  3. Put all files into the plugin’s root folder, which should contain only four files: config.yml (mandatory), __init__.py (optional), script.js (optional), styles.css (optional)

Deprecation Notices

  • This is the last release we are shipping JavaScript source maps for Galaxy client.

Removal Notices

The following features that have been deprecated in the past releases are being removed in 18.01:

  • Sample tracking

  • Sample request and external services management

  • Legacy library interface for Administrators

In addition to that, the PlantTribes datatypes have been commented out in the source code. Uncomment them if you want to re-enable it.

uWSGI details

Galaxy can be forced to start under uWSGI even with an older configuration file by setting APP_WEBSERVER=uwsgi in the environment. As part of the transition to YAML-based configuration files, we have implemented a schema to validate Galaxy configuration files. Run make config-validate from Galaxy’s root directory to validate a schema and make config-lint to check for best practices. While there is no need to convert your configuration file (galaxy.ini hasn’t been deprecated), you can run make config-convert-dry-run and make config-convert to respectively test and perform the conversion of an ini configuration file to a YAML one.

In the future uWSGI will allow Galaxy to setup GIE proxies without additional configuration and use modern web technologies such as web sockets.

These are big changes that affect many parts of Galaxy’s administration documentation and makes this documentation very dependent on which Galaxy version they are targeting. To address this, we have moved a significant amount of administration documentation into Galaxy’s code base and made it available on a per-release basis. The latest administration documentation for the previous release of Galaxy (17.09) which reference galaxy.ini files and Paste servers can be found here, while documentation for this release can be found here.

Pull Request 4475, Pull Request 5135, Pull Request 5390, Pull Request 5373, Pull Request 5105, Pull Request 5441

Release Notes

Enhancements

Fixes

GenomeSpace security sanitization

Outputs of these tools may require sanitization: genomespace_importer and genomespace_exporter

The following SQL commands will help you identify the datasets in your Galaxy’s database.

Finding bad GenomeSpace importer params:

SELECT j.id,
       j.create_time,
       j.user_id,
       jp.value
FROM job_parameter jp
JOIN job j ON j.id = jp.job_id
WHERE jp.job_id IN
    (SELECT id
     FROM job
     WHERE tool_id = 'genomespace_importer')
  AND jp.name = 'URL'
  AND jp.value LIKE '%^%'
ORDER BY j.id DESC;

Finding bad GenomeSpace exporter params:

SELECT j.id,
       j.create_time,
       j.user_id,
       jp.value
FROM job_parameter jp
JOIN job j ON j.id = jp.job_id
WHERE jp.job_id IN
    (SELECT id
     FROM job
     WHERE tool_id = 'genomespace_exporter')
  AND jp.name = 'genomespace_browser'
  AND jp.value LIKE '%^%'
ORDER BY j.id DESC;

The following SQL commands will help you sanitize the datasets in your Galaxy’s database.

Sanitizing GenomeSpace importer params:

UPDATE job_parameter jp
SET value = split_part(jp.value, '^', 1) || '"'
FROM job j
WHERE jp.job_id = j.id
  AND j.tool_id = 'genomespace_importer'
  AND jp.name = 'URL'
  AND jp.value LIKE '%^%';

Sanitizing GenomeSpace exporter params:

UPDATE job_parameter jp
SET value = split_part(jp.value, '^', 1) || '"'
FROM job j
WHERE jp.job_id = j.id
  AND j.tool_id = 'genomespace_exporter'
  AND jp.name = 'genomespace_browser'
  AND jp.value LIKE '%^%';

To stay up to date with Galaxy’s progress, watch our screencasts; visit our community Hub; and follow us on Bluesky, Mastodon, and LinkedIn.

You can always chat with us on Matrix.

Thanks for using Galaxy!

The Galaxy Team