Warning
This document is for an in-development version of Galaxy. You can alternatively view this page in the latest release if it exists or view the top of the latest release's documentation.
May 2018 Galaxy Release (v 18.05)
Highlights
- New Interface Style Library
The design of many components of the Galaxy user interface has been subtly improved as the result of upgrading to the Bootstrap 4 style library. The upgrade process was a large community effort with commits from six contributors and testing and bug reports from many others. A huge thanks goes out to everyone who helped - it is a great step forward toward a more standardized Galaxy client infrastructure built on reactive VueJS components. Read more about whats new and changed in the underlying Bootstrap 4 JavaScript library here.
- Unlimited Browser Upload Size
Support for chunked and resumable file uploads out-of-the-box. This should effectively eliminate browser-based limitations on the size of files that can be uploaded to Galaxy.
- New Powerful Rule-Based Data Uploader
Galaxy now features a powerful new interface for manipulating lists of rules for processing metadata that can be used to describe the upload of arbitrarily complex collections as well as individual datasets. These rules can be used to build collections from lists of URLs, FTP directory contents, data library folders, and tabular sample sheets or exports from external data sources. The same technique can be applied to the metadata of an existing collection, enabling very flexible filtering, sorting, relabelling, grouping, flattening, and general re-organization of collections. This client widget is backed by a powerful new hierarchical upload API and a new collection operation tool.
Check out this tutorial describing new techniques for loading large numbers of datasets and complex collections into Galaxy.
Pull Request 5365, Pull Request 5220, Pull Request 5819, Pull Request 5823, Pull Request 5929, Pull Request 5940
- GDPR Beta Mode
We introduce beta GDPR privacy mode. Admins can enable it to help them comply with the European Union law known as General Data Protection Regulation, which goes into effect on May 25, 2018. Thanks to @hexylena.
- New HIV-TRACE Plugin
Integration of a new visualization plugin - HIV-TRACE. Thanks to @veg!
- Visualization Frameworks Unified
The charts and generic visualization frameworks have been merged together into a single coherent one. You can use a new menu option to list and search all available visualizations.
Get Galaxy
The code lives at Github and you should have Git to obtain it.
- To get a new Galaxy repository run:
$ git clone -b release_18.05 https://github.com/galaxyproject/galaxy.git
- To update an existing Galaxy repository run:
$ git fetch origin && git checkout release_18.05 && git pull --ff-only origin release_18.05
See the community hub for additional details regarding the source code locations.
Security
vcffilter by devteam allowed RCE
Tracked as GX-2018-0004
.
Older versions of the VCF filter tool in the vcffilter repository by devteam allowed remote code execution due to missing character escaping.
If you have this repository installed in your Galaxy, please update to the revision 4:6b935ab36d7b
or newer and disable/uninstall all older versions.
Breaking Changes
The
charts
repository that powers Galaxy built-in visualizations that need backend computation has been moved to the IUC account. In order for these visualization options to keep working, admins need to install the following repository: iuc/charts.Removed support for the Graphite statistics middleware. We continue to further support the
statsd
middleware.We reworked how compressed archives (zip, gz, bz etc.) are being defined and handled in Galaxy. As a result, if you have a custom
datatypes_conf.xml
configuration file, you should remove the compressed sniffers that used to be in the sample file but now are not. Otherwise you will encounter non-breaking errors in the logs.We have removed the deprecated symlink static/scripts/packed -> . If you used it please change your (proxy) configuration. An example can be seen here.
Release Notes
Enhancements
Allow user authentication using OpenID Connect protocol (thanks to @VJalili). Pull Request 4474, Pull Request 5908, Pull Request 5906
Add basic cgroup metric collection easing the collection of job memory data (thanks to @scholtalbers). Pull Request 5610
Enhance Galaxy’s API and test framework to allow running tool tests on running Galaxy instances. Check out the new Ephemeris functionality it is powering here. Pull Request 5628, Pull Request 5667
Global and profile-bound language configuration (thanks to @abretaud). Pull Request 5282
Tool that adds/sets tags for collection elements from a file (thanks to @mvdbeek). Pull Request 5462
Implement an extension point during workflow running to describe site specific job scheduling parameters (thanks to @JeffreyThiessen). Pull Request 4830
Update Dynamic Tool Destinations with configurable priorities and better validation (thanks to @tizzir). Pull Request 5645
More great Kubernetes job runner enhancements to improve robustness and configurability (thanks to @pcm32 and the PhenoMeNal project). Pull Request 5760, Pull Request 5761
Documentation version selector on docs.galaxyproject.org. Pull Request 5334
Update sort tool to allow ignoring headers (thanks to @lecorguille). Pull Request 4235
Add
GALAXY_MEMORY_MB_PER_SLOT
environment variable to job environment (thanks to @bernt-matthias). Pull Request 5625Allow sending slow request timing to Sentry. Pull Request 5479
Allow formatting StatsD data for InfluxDB (thanks to @hexylena). Pull Request 5742
Tool versions should follow PEP 440 (thanks to @nsoranzo). Pull Request 5746
Expand pipenv dependency management to include all Python dependencies (thanks to @nsoranzo). Pull Request 5786
Sniffing framework with constrained memory and I/O. Pull Request 5793
Infrastructure for dynamic compressed datatypes. Pull Request 5794
Capture more HDA state transitions in the history panel. Pull Request 5749
Add script for propagating hashtags in a Galaxy history (thanks to @anuprulez). Pull Request 5669
Add proteomics proBAM and proBED datatypes (thanks to @jj-umn). Pull Request 5360
Add fasta.gz datatype. Pull Request 5553
Add genbank.gz datatype (thanks to @Delphine-L). Pull Request 5772
Add RDS object and SingleCellExperiment datatypes (thanks to @mtekman). Pull Request 5716
Allow registration with empty email (thanks to @bernt-matthias). Pull Request 5320
Avoid silent tool upgrades during workflow imports. Pull Request 5821
Many awesome Python 3 related fixes (thanks to @nsoranzo and @mvdbeek). Pull Request 5359, Pull Request 5422, Pull Request 5453, Pull Request 5461, Pull Request 5492, Pull Request 5494, Pull Request 5458, Pull Request 5636,`Pull Request 5642`_, Pull Request 5427, Pull Request 5451
Install Node.js and Yarn on startup if running a non-release branch. Pull Request 5375
Add
--skip_flakey_fails
flag to CI script for API tests (thanks to @nsoranzo). Pull Request 5388Remove unused mapping code. Pull Request 5450
Show parameter name in
DataToolParameter
exception (thanks to @peterjc). Pull Request 5465Unify config-rebuild
Makefile
targets (thanks to @nsoranzo). Pull Request 5488Rev SQLAlchemy from 1.0.15 to 1.2.2. Pull Request 5538
Add middleware and instrumentation to allow per-request SQL debugging. Pull Request 5539
Overhaul UCSC tools with more specific, working Conda compatible requirements (with help from @nsoranzo). Pull Request 5544, Pull Request 5973
More and more robust Selenium tests. Pull Request 5559, Pull Request 5882, Pull Request 5886, Pull Request 5876, Pull Request 5836, Pull Request 5701, Pull Request 5771, Pull Request 5754, Pull Request 5714, Pull Request 5705, Pull Request 5708, Pull Request 5710, Pull Request 5706, Pull Request 5707
Implement basic cache on toolbox
to_dict
objects. Pull Request 5570Log errors from external chown script. (thanks to @Petraea). Pull Request 5581
Updates against galaxy-lib changes. Pull Request 5602, Pull Request 5621, Pull Request 5936, Pull Request 5653
Performance optimization to avoid producing deep-copies of tool input dictionaries. Pull Request 5646
Disable tool input datatype validation for dropped history items. Pull Request 5657
Add a script for dumping library file paths for backup. Pull Request 5436
Rebuild client only if there are changes (thanks to @nsoranzo). Pull Request 5679
Populate column selector for datasets which are not ready yet. Pull Request 5680
Fix role handling library manager. Pull Request 5681
GFF updates to handle
##FASTA
directive. Pull Request 5686Memory and performance fixes for upload tool. Pull Request 5689
Fixes for testing
Dockerfile
recipe. Pull Request 5693Various improvements for API tests. Pull Request 5697, Pull Request 5699
Small log level improvement (thanks to @hexylena). Pull Request 5715
Swap password checking for the constant time comparison algorithm (thanks to @hexylena). Pull Request 5719
Upgrade clone failure logging to error (thanks to @pvanheus). Pull Request 5724
Improve galaxy update commands, update some bootstrap checkpoints. Pull Request 5731
Remove
extract_genomic_dna
tool from tool sample. Pull Request 5741Revise uWSGI defaults and display. Pull Request 5743
Documentation typo fixes and small refactors (thanks to @nsoranzo). Pull Request 5747
More backend speed ups for user’s workflow index query. Pull Request 5755
Fix all bundled scripts for the switch to YAML configuration. Pull Request 5769
Rev and re-lock Galaxy dependencies (thanks to @nsoranzo). Pull Request 5776
Lock pysam to 0.14.1 in Pipfile. Pull Request 5798
Client related readme and
Makefile
clarifications. Pull Request 5800Use VueJS and API to handle workflow imports. Pull Request 5804
Add link that explains quota management to quota indicator. Pull Request 5811
Improve startup interoperability with Conda. Pull Request 5854
Implement the API version of the Docker containers interface. Pull Request 5861
Use
extras
to specify additional social_auth_core dependencies (thanks to @nsoranzo). Pull Request 5877Update cluster docs for a Virtualenv issue in DRMAA real user jobs (thanks to @moskalenko). Pull Request 5881
Add Mocha/Chai client test framework. Pull Request 5888
Small fix to run more integration tests (thanks to @nsoranzo). Pull Request 5892
Use random password generation function of user class (thanks to @VJalili). Pull Request 5904
Remove unused client route. Pull Request 5907
Fix to provide
ToolDataTableManager
class for backwards compatibility (thanks to @mvdbeek). Pull Request 5924Document Github bug reporter (thanks to @hexylena). Pull Request 5928
Rewrite history structure display in VueJS (thanks to @anuprulez). Pull Request 5317
Remove
static/scripts/*
from the dev branch, add to .gitignore. Pull Request 5434History contents client code refactoring. Pull Request 5448
Drop short-lived
make client-production
warning from build scripts. Pull Request 5508Use
with
statement inlib/galaxy/datatypes/
(thanks to @nsoranzo). Pull Request 5623Update tool conf samples for recent changes. Pull Request 5637
Extend grid framework with formal support for delayed columns. Pull Request 5789
Fixes for plugin directory staging. Pull Request 5810
Remove defunct CasperJS tests. Pull Request 5567
Eliminate unused pre-created datasets concept in upload API. Pull Request 5604
Fixes
Fix workflow step cloning. (thanks to @mvdbeek). Pull Request 6085
Fix tool state performance for large collections. Pull Request 5997
Fix overly strict sanitizing in the filter tool (thanks to @abretaud). Pull Request 5872
Allow whitelisted symlinks to be imported (thanks to @bwlang). Pull Request 5596
Fix no dataset found issue (thanks to @osallou). Pull Request 5829
Fix active tab highlighting (thanks to @qiagu). Pull Request 5675
Fix initial binding for slow query logging. Pull Request 5401
Default slow request tracking to disabled. Pull Request 5513
Fixed stale link (thanks to @bernt-matthias). Pull Request 5548
Fix active tab highlighting issue #4846 (thanks to @qiagu). Pull Request 5593
Small misc fixes and cleanups (thanks to @nsoranzo). Pull Request 5638
Fix for running Yarn in Docker for tests that do not need it. Pull Request 5690
Various test fixes (thanks to @nsoranzo). Pull Request 5691
Hack to fix bug in uploader where it sometimes posts with empty “file_type”. Pull Request 5692
Bump psycopg2 to 2.7.4. Pull Request 5698, Pull Request 5825
Do not show composite datatypes for regular dataset upload (thanks to @nsoranzo). Pull Request 5700
Relax validation condition for dynamic parameters. Pull Request 5702
Fix issue where some elements of collections wouldn’t render in the UI. Pull Request 5703
Fix default role type on auto created roles (thanks to @scholtalbers). Pull Request 5709
Fix to properly order
@flakey
and@selenium_test
Selenium test decorators (thanks to @nsoranzo). Pull Request 5711Clarify that nginx-extras needs to be install for the upload_store documentation (thanks to @mvdbeek). Pull Request 5717
Remove
subclass="true"
from Phylip datatype definition (thanks to @nsoranzo). Pull Request 5718Sentry bug reports working for anonymous users (thanks to @hexylena). Pull Request 5721
Add Galaxy lib directory to Python path for 2 scripts (thanks to @nsoranzo). Pull Request 5725
Fix LDAP when
role_search_attribute
is not defined (thanks to @nsoranzo). Pull Request 5732Fix mulled Docker caching. Pull Request 5735
Drop old charts building from makefile/package scripts Pull Request 5737
Configuration fixes (thanks to @nsoranzo). Pull Request 5745
Add additional validation to numeric input element. Pull Request 5750
Various scatterplot fixes. Pull Request 5752
Fix linking option for library dataset uploads (thanks to @nsoranzo). Pull Request 5758
Fix copying tags for collections when copying histories. Pull Request 5766
Fix
/api/whoami
API endpoint for master key. Pull Request 5773Log the pid of the first uWSGI process (thanks to @nsoranzo). Pull Request 5774
Fix SQL query for “workflows shared with me” as part of workflow index. Pull Request 5782
Fix model operation tools that produce standalone datasets. Pull Request 5784
Fix missing “from Import Directory” button in data libraries (thanks to @nsoranzo). Pull Request 5797
Fix metadata setting for CRAM files (thanks to @mvdbeek). Pull Request 5805
Use
app.datatypes_registry
inValidationContext.from_app()
(thanks to @nsoranzo). Pull Request 5806Fix
upload_common.py
ensure hda is added to a session. Pull Request 5816Exclude the JS GIE proxy’s Node.js modules from linting. Pull Request 5826
Do not append ‘/’ to ‘js’ in GIE static path, this breaks uWSGI static routing. Pull Request 5859
Fix conditional parameter handling. Pull Request 5864
Increase Yarn’s network timeout. Pull Request 5870
Move the memory statement below _galaxy_setup_environment (thanks to @moskalenko). Pull Request 5880
Remove deprecated controller endpoint and link to preferred implementation. Pull Request 5887
Fix update_repository when hg repo has local modifications (thanks to @nsoranzo). Pull Request 5890
Fix admin data library dataset permissions. Pull Request 5899
Fix display of the tour index names. Pull Request 5900
Fix Trackster links. Pull Request 5916
Fix uploads of link-only datasets to data library (thanks to @mvdbeek). Pull Request 5917
Correct signal number in admin documentation (thanks to @hexylena). Pull Request 5921
Fix error with data managers when trying to move_merge nonexistent extra_files dir (thanks to @abretaud). Pull Request 5922
Fix bug with removing workflow outputs from a node. Pull Request 5935
Fix for running in containers when
job_tmp
is set injob_conf.xml
. Pull Request 5937Fix for running jobs in containers. Pull Request 5938
Fix dropdown click and highlighting behavior. Pull Request 5945
Resort to
os.path.exists()
also for paths not ending in.loc
(thanks to @nsoranzo). Pull Request 5947Remove ini config parameter from cleanup scripts (thanks to @bernt-matthias). Pull Request 5956
Fix shared history list bug. Pull Request 5962
Pin the version of Node.js used in CI. Pull Request 5972
Fix 2 typos in the docs (thanks to @nsoranzo). Pull Request 5986
To stay up to date with Galaxy’s progress, watch our screencasts; visit our community Hub; and follow us on Bluesky, Mastodon, and LinkedIn.
You can always chat with us on Matrix.
Thanks for using Galaxy!