Warning
This document is for an in-development version of Galaxy. You can alternatively view this page in the latest release if it exists or view the top of the latest release's documentation.
May 2017 Galaxy Release (v 17.05)¶
Highlights¶
- Tag your data with propagating hashtags
Large Galaxy histories used to be messy. Hashtags make it easy to track dataset (and collection) relationships. These two movies (both under a minute) explain how to use this with datasets and with collections.
Learn more about Galaxy histories in our updated tutorial
- Drag & Drop datasets into tool inputs
- Interface now allows dragging datasets from history panel into the content selectors of the tool form. Implemented in Pull Request 3871.
- Upload directly to a collection
You can now bypass the history manipulation and upload your data straight into a collection for convenience.
Learn more about collections and how to use them in a new tutorial.
We extend special thanks to the 64 New Contributors to Galaxy in the past year.
Get Galaxy¶
The code lives at Github and you should have Git to obtain it.
- To get a new Galaxy repository run:
$ git clone -b release_17.05 https://github.com/galaxyproject/galaxy.git
- To update an existing Galaxy repository run:
$ git checkout release_17.05 && git pull --ff-only origin release_17.05
See the community hub for additional details regarding the source code locations.
Additionaly there are New Configuration Options and New Datatypes sections.
Release Notes¶
Enhancements¶
- Recalculate user disk usage on logout. Pull Request 3600
- Overlay search for tools, workflows, libraries and histories (thanks to @anuprulez and @bgruening). Pull Request 3460
- Webhook plugin: tool_list (thanks to @blankclemens). Pull Request 3753
- Add Cytoscape as the Galaxy charts plugin (thanks to @anuprulez and @bgruening). Pull Request 3817
- Ethercalc Interactive Environment (thanks to @shiltemann). Pull Request 3699
- Add a simple UI to manage tool dependencies (thanks to @mvdbeek). Pull Request 3479
- Add nrrd image datatype (thanks to @ThomasWollmann). Pull Request 3385
- NGL Viewer for molecule representation (thanks to @anuprulez and @bgruening). Pull Request 3601
- Add script to encode or decode the IDs that Galaxy exposes to users and admins (thanks to @erasche). Pull Request 3622
- Support Docker Engine swarm mode for Interactive Environments. Pull Request 3666
- Interactive Environment container waiting and swarm management. Pull Request 3740
- Phylocanvas as a BioJS component for the Galaxy charts (thanks to @anuprulez @bgruening). Pull Request 3850
- Container interface abstraction. Pull Request 3863
- Graphite support (thanks to @erasche). Pull Request 3964
- Normalize the handling of steps and missing tools for the workflow editor. Pull Request 3181
- Add possibility to load collection into data library (thanks to @mvdbeek). Pull Request 3559
- Prefer versioned Conda dependencies to unversioned Galaxy ones. Pull Request 3708
- Speedup startup and toolbox operations (thanks to @mvdbeek). Pull Request 3909
- Add missing imports for documentation builds (thanks to @athos-ribeiro). Pull Request 3104
- Workflow editor UI touch up. Pull Request 3407
- Add an example group filter for displaying tool panel sections (thanks to @gregvonkuster). Pull Request 3484
- Python3: use @six.add_metaclass() decorator (thanks to @nsoranzo). Pull Request 3491
- Improvements to build sites configuration. Pull Request 3503
- Add peek and preview for compressed FASTQ datatype (thanks to @mvdbeek). Pull Request 3512
- Add
refresh_on_change
to tool XSD. Pull Request 3515 - Store module step states as nested dictionaries, fix export for backward compatibility. Pull Request 3522
- Add Jenkins documentation tasks to the release issue. Pull Request 3523
- Allow decompressing tool test outputs. Pull Request 3550
- Add a task to protect the release branch in GitHub. Pull Request 3562
- Remove duplicated/unused display_migration_details() functions (thanks to @nsoranzo). Pull Request 3564
- Display the genbank datatype in upload. Pull Request 3569
- Database migrations: improvements and fixes (thanks to @nsoranzo). Pull Request 3570
- Add d3_hierarchy subclass (thanks to @jj-umn). Pull Request 3580
- Clean up docker-compose resources better for Selenium tests. Pull Request 3583
- Add option to send API key in header (thanks to @jvolkening). Pull Request 3584
- Increase patience of Galaxy tests throughout. Pull Request 3589
- Introduce label and annotation attributes to subworkflows, fix annotation display, unify module form update handler. Pull Request 3593
- Align architecture slides with training-material remarkjs template. Pull Request 3594
- Revision of tool XML reloading feature. Pull Request 3596
- Implement collection operation to relabel list identifiers from contents of a file. Pull Request 3603
- Slow query logging. Pull Request 3616
- Remove un-needed check in histories manager code. Pull Request 3618
- Improve interaction between job state and purged datasets. Pull Request 3619
- Usable slow query logging. Pull Request 3625
- Allow composite datatypes of type auto_primary_file to automatically generate dataset. Pull Request 3643
- Introduce labels for all workflow modules, remove name handling field for data inputs. Pull Request 3644
- When using the new slow query logger, include parameters. Pull Request 3649
- Allow nested parameters in the dynamic_tool rule -> argument (thanks to @lecorguille). Pull Request 3657
- By default, do not allow workflow invocations to schedule indefinitely. Pull Request 3659
- Add the mzData datatype (thanks to @lecorguille). Pull Request 3661
- Display an error message when GIE loading raises an exception. Pull Request 3662
- Set LDAP options only once (thanks to @nsoranzo). Pull Request 3663
- Bump Jupyter container version to 16.01.1 . Pull Request 3665
- Improve wording of getgalaxy in bootstrapping script for release notes. Pull Request 3671
- Filter hda tags for tool parameters. Pull Request 3688
- Add datatypes for soon coming PlantTribes Galaxy tools (thanks to @gregvonkuster). Pull Request 3698
- Touch up Galaxy testing documentation. Pull Request 3715
- Add API and UI to remove requirements installed by Conda (thanks to @mvdbeek). Pull Request 3725
- Quota recalculation logic consolidation. Pull Request 3734
- Percolator out datatypes (thanks to @glormph). Pull Request 3737
- Modify our docs in preparation for restructuring. Pull Request 3738
- Update codefile documentation in tool XSD with info from old Wiki. Pull Request 3739
- Mirror latest galaxy-lib changes for Conda and CWL improvements. Pull Request 3742
- Add the possibility to uninstall Conda environments that are not in use by Galaxy (thanks to @mvdbeek). Pull Request 3750
- New Authentication section of the docs. Pull Request 3751
- Bump routes requirement to 2.4.1 . Pull Request 3752
- Abstract uWSGI handling into new galaxy.web.stack package. Pull Request 3759
- i18n improvements and French localization. Pull Request 3762
- Should exit immediately to avoid index error (thanks to @erasche). Pull Request 3770
- Made collection operations help sections slightly more readable. Pull Request 3784
- Avoid potential inconsistencies between differently stored labels. Pull Request 3793
- Remove custom builds mako. Pull Request 3794
- Small enhancements for collection builders. Pull Request 3806
- Small improvements for driving Qunit tests. Pull Request 3810
- Implement more upload testing in Selenium. Pull Request 3811
- API & reports documentation improvements. Pull Request 3812
- Make ToolConfWatcher watch ToolCache (thanks to @mvdbeek). Pull Request 3814
- Disable tool shed test function that fails frequently on Jenkins. Pull Request 3819
- Restrict workflow scheduling within a history to a fixed, random handler. Pull Request 3820
- Order processing evaluation of workflow invocations when scheduling. Pull Request 3830
- Stricter checks for GFF/GFF3/GTF datatypes sniffing (thanks to @nsoranzo). Pull Request 3832
- Add root redirect for backbone router. Pull Request 3837
- Allow explicit configuration of handlers for workflow scheduling. Pull Request 3844
- Log reason when workflow scheduling of a step is delayed. Pull Request 3849
- Update datatypes_conf.xml.sample (thanks to @lecorguille). Pull Request 3852
- Clean up old TODO into history tests. Pull Request 3855
- Small changes to links and add Gitter (thanks to @bgruening). Pull Request 3861
- Use detect_errors=”exit_code” in Trim tool (thanks to @nsoranzo). Pull Request 3869
- Synchronise against latest UCSC versions (thanks to @bgruening). Pull Request 3874
- Hardklor and kronik datatypes for proteomics (thanks to @glormph). Pull Request 3884
- History drag-and-drop feature with iframe support for run workflow form. Pull Request 3886
- Update to a new Docker test image to fix dependency bugs. Pull Request 3890
- Python3: Use shlex_quote() from six.moves instead of pipes.quote() (thanks to @nsoranzo). Pull Request 3891
- Add a validator that ensures a dataset’s extra_files_path (thanks to @gregvonkuster). Pull Request 3918
- Update text.py Add interaction annotations to SnpEffDb metadata (thanks to @jj-umn). Pull Request 3938
- Collection operation: Filtering from a file. Pull Request 3940
- Remove Google from OpenID providers in the sample. Pull Request 3953
- Change ‘User’ menu to ‘Login or Register’ for anonymous users. Pull Request 3955
- Stats docs (thanks to @erasche). Pull Request 3968
- Enhance filenames with collection context when downloading datasets from GUI. Pull Request 3985
- Displaying a message instead of just redirecting. Pull Request 4000
- Implicitly set use_volumes = False if using Docker swarm. Pull Request 4006
- Update Jupyter image to 17.05 . Pull Request 4007
- Inline CSS in error report email (thanks to @erasche). Pull Request 3445
- Improve routing for user preference views. Pull Request 3446
- Tool documentation and testing improvements. Pull Request 3457
- Update readme badges. Pull Request 3465
- Lint some tools (thanks to @nsoranzo). Pull Request 3468
- Improve release issue docs. Pull Request 3496
- Watch tool data table
*.loc
files, and reload if modified (thanks to @mvdbeek). Pull Request 3533
- Attempt work around for race-y sort of condition in Selenium sizzle checkers. Pull Request 3451
- Webhooks: add tests (thanks to @anatskiy). Pull Request 3469
- Disable transiently failing tour test. Pull Request 3599
- Selenium testing improvements. Pull Request 3716
- Merge ConfiguresGalaxyMixin.reload_toolbox() in _configure_toolbox() (thanks to @nsoranzo). Pull Request 3732
- Abstraction for re-use between collection creators. Pull Request 3781
- Merge ToolConfWatchers and separate from Toolbox (thanks to @mvdbeek). Pull Request 3821
- Attempt to improve the robustness and verboseness of the test_sharing_valid Selenium test. Pull Request 3841
- Routes cleanup. Pull Request 3895
- Standardize usage of log.exception . Pull Request 3933
- Followup to exception refactoring. Pull Request 3959
- Use container-based infrastructure (sudo: false) in TravisCI (thanks to @nsoranzo). Pull Request 3977
- Improve implicit workflow connection test case. Pull Request 3991
Fixes¶
- Fix tag escaping. Pull Request 3720
- Fix kombu documentation link in galaxy.ini.sample . Pull Request 3629
- Fix for EtherCalc IE image specification. Pull Request 3825
- Fix margin style values for color picker element. Pull Request 3965
- 17.05 rstudio fixes / one small GIE fix (thanks to @erasche). Pull Request 3975
- Fix transiently failing Selenium upload test. Pull Request 3587
- Fix webhook search overlay interfering with masthead load. Pull Request 3731
- Fix collection operations in workflows. Pull Request 3848
- Remove old, unused search controller. Pull Request 3894
- Deduplicate obsolete API readme. Pull Request 3777
- Assorted fixes for dockerized test setup. Pull Request 3755
- Another fix for Selenium testing on Jenkins. Pull Request 3422
- Fix most broken Selenium tests. Pull Request 3440
- Fix for transiently failing workflow test. Pull Request 3668
- Fix selenium workflow tests broken with #3181. Pull Request 3504
- Fix integration test for default Conda mapping. Pull Request 3508
- Various fixes for test image Docker user switching. Pull Request 3443
- Fix Travix builds which use flake8-docstrings (thanks to @nsoranzo). Pull Request 3981
- Azure - Leasing container sometimes breaks things in Docker Galaxy (thanks to @zfrenchee). Pull Request 3263
- run_tests.sh: a small typo (thanks to @anatskiy). Pull Request 3322
- Fix cycle detection for single node workflows. Pull Request 3374
- Small selenium hack to fix occasionally failing collection upload tests. Pull Request 3967
- Bug fix in mulled.py (thanks to @abdulrahmanazab). Pull Request 3434
- Do not select first value as default if select value is optional. Pull Request 3472
- jsonify initial value when returned through to_dict caller. Pull Request 3513
- Fix <is_valid_xml> output test assert check (thanks to @jvolkening). Pull Request 3530
- Fix output tests for checksum example tools. Pull Request 3545
- Update Nagios check. Pull Request 3568
- Set ‘deleted’ to True when purging a history dataset association (thanks to @nsoranzo). Pull Request 3572
- Update tool XSD for (hopefully deprecated) change_format syntax. Pull Request 3579
- Remove broken tour step. Pull Request 3582
- Avoid step duplication when ordering steps on the fly. Pull Request 3590
- Minor fix of core.galaxy-ui tour. Pull Request 3608
- Fix RELEASE_ISSUE_TEMPLATE in bootstrap_history.py (thanks to @nsoranzo). Pull Request 3653
- Update dynamic_tool_destination.py (thanks to @lecorguille). Pull Request 3655
- Ignore Node modules installed for the GIE proxy when linting. Pull Request 3664
- Logout recalc improvements. Pull Request 3669
- Strip broken autocomplete feature, webbrowsers console error is an artifact. Pull Request 3676
- Generate primary composite file after extra files collection. Pull Request 3678
- Fix tool search index building for tools with slash in id and not from ToolShed. Pull Request 3680
- Set $HOME to a temp directory rather than conda_prefix because conda_prefix may not be writable either. Pull Request 3701
- Properly handle boolean values when parsed to a string-based/default tool parameter. Pull Request 3702
- Mothur datatypes: fix metadata failure for large count tables (thanks to @shiltemann). Pull Request 3706
- Filename handling for FTP upload. Pull Request 3722
- Do not require stored workflow id for subworkflows. Pull Request 3723
- Remove outdated items from tool cache when reloading the toolbox (thanks to @mvdbeek). Pull Request 3729
- Change API path in docstrings to correct /api/dependency_resolvers/* (thanks to @mvdbeek). Pull Request 3749
- Wiki to hub link update. Pull Request 3754
- Fix KeyError when getting request and response content-type headers (thanks to @nsoranzo). Pull Request 3757
- Fix for uploading rdata datasets when explicitly declaring datatype. Pull Request 3766
- Fixes for users API. Pull Request 3771
- Fix XSS vulnerability in grids. Pull Request 3776
- Layered routing fix for when controller route fails and client route does not exist. Pull Request 3783
- Fix collection operations not having a history. Pull Request 3786
- Fix problem with cleaning up jobs in local runner. Pull Request 3803
- Have tool-search listen to change as well as keyup. Pull Request 3808
- Fix BED dataset display in Trackster (thanks to @hrhotz). Pull Request 3822
- More CONVERTER fixes for restricting tools’ Python environment. Pull Request 3824
- Have IGV bigWig display application make use of build_sites/ site_type config instead of direct filename. Pull Request 3828
- Properly handle multiple hidden datasets when populating data select options. Pull Request 3842
- Fix for direct message routing using a ‘real’ exchange like RabbitMQ. Pull Request 3846
- Fix setting UUID on steps when copying steps from another workflow. Pull Request 3856
- Fix certain aspects of collection reductions in conditionals/repeats. Pull Request 3864
- Fix multiple collection reductions on multiple data parameters. Pull Request 3875
- Fix inability for viz to raise loading with non-ascii messages (thanks to @erasche). Pull Request 3881
- Fix new history endpoint providing custom builds metadata. Pull Request 3887
- Remove unused model import from users API. Pull Request 3888
- Cytoscape bugfix for SIF files (thanks to @bgruening). Pull Request 3896
- Add missing trigger event to drop handler in content selector. Pull Request 3897
- Fix Instance is not bound to a Session (thanks to @mvdbeek). Pull Request 3904
- Fix library parameter style width to avoid second line. Pull Request 3926
- More careful inspection of fixes for MAF tools requiring Galaxy. Pull Request 3935
- Containers interface and GIE bugfixes. Pull Request 3941
- Show default value for runtime values. Pull Request 3943
- Fix JJ’s column_join tool depending on Galaxy at runtime. Pull Request 3961
- Fix edge case for workflow mapping handling when tools not available. Pull Request 3962
- Tour fixes (thanks to @nsoranzo). Pull Request 3970
- Fix some typos in config/galaxy.ini.sample (thanks to @nsoranzo). Pull Request 3971
- Add two tools from str_fm repository to galaxy module whitelist. Pull Request 3973
- Remote user logout fixes, rework header handling a bit. Pull Request 3976
- Fixes to get GIEs working on Mac OS X. Pull Request 3978
- Rework Jupyter IE for additional_ids handling. Pull Request 3980
- Update Babel to a version compat. with sphinx. Pull Request 3986
- Use the configured tool_path when loading data managers. Pull Request 3989
- Typo in remote user / user preferences exception (thanks to @erasche). Pull Request 4015
- Fix setting up explicit container resolution from a container resolution config file. Pull Request 4020
- Fix spelling error in error message (thanks to @pvanheus). Pull Request 4026
- Fixes for containerized job executions. Pull Request 4033
- Try activating Conda environment again if previous activation failed (thanks to @mvdbeek). Pull Request 4035
- HTCondor fix for running containers (thanks to @bgruening). Pull Request 4038
- Fix destructive tag rerender to be a simpler, more precise html replacement. Pull Request 4052
- Fix for S3ObjectStore _clean_cache method. Pull Request 4055
- Correct webhooks to use Galaxy.root and be compatible with proxy_prefix (thanks to @erasche). Pull Request 4065
- Fix global grid action event handler. Pull Request 4068
- Fix various transiently failing Selenium tests. Pull Request 3450
- Do not modify self.app.config.tool_data_path and self.app.config.tool_data_table_config_path during metadata setting (thanks to @mvdbeek). Pull Request 4089
- Open Babel update for datatype converters (thanks to @bgruening). Pull Request 4094
- Change to only inheriting ‘name’ tags. Pull Request 4104
- Fixes repeat handling in workflow editor, store values as passed from UI. Pull Request 4107
New Contributors¶
We thank these new contributors that helped with the Galaxy codebase since 16.04:
- Evgeny Anatskiy
- Abdulrahman Azab
- balto
- Bérénice Batut
- Marius van den Beek
- Maria Bernard
- Jean-Frédéric Berthelot
- Léo Biscassi
- Clemens Blank
- Jorrit Boekel
- Christian Y. Brenninkmeijer
- Freek de Bruijn
- Matt Chambers
- Peter Cock
- Matthias Desmet
- Shane Dowling
- Ignacio Eguinoa
- Mark Einon
- fescudie
- Anne Fouilloux
- Jean-Frédéric
- Maximilian Friedersdorff
- Ben Fulton
- Carrie Ganote
- Ryan Golhar
- Peter van Heusden
- Saskia Hiltemann
- Hans-Rudolf Hotz
- Manabu Ishii
- Xiaoqian Jiang
- Iyad Kandalaft
- Edward Kirton
- Jasper Koehorst
- Anup Kumar
- Delphine Lariviere
- Yvan Le Bras
- Gildas Le Corguillé
- Alexander Lenail
- Pierre Lindenbaum
- Mikael Loaec
- Thoba Lose
- Zipho Mashologu
- Hervé Ménager
- Pablo Moreno
- Takao Nakaguchi
- Balthazar Pavot
- Matt Ralston
- ramezrawas
- Athos Ribeiro
- Devon Ryan
- Michael Sauria
- Varun Shankar
- Timur Shtatland
- Sourav Singh
- Eteri Sokhoyan
- Nick Stoler
- Scott Szakonyi
- Jesse c j van Dam
- Ashok Varadharajan
- Jeremy Volkening
- Pavan Videm
- Junzhou Wang
- Joachim Wolff
- Thomas Wollmann
- Jay Young
New Configuration Options¶
Below are the newly added options you can configure in galaxy.ini
.
To see the whole sample file visit our GitHub repository.
# Set to True to enable monitoring of the tool_data and shed_tool_data_path
# directories. If changes in tool data table files are found, the tool data
# tables for that data manager are automatically reloaded.
# Watchdog ( https://pypi.python.org/pypi/watchdog ) must be installed and
# available to Galaxy to use this option. Other options include 'auto'
# which will attempt to use the watchdog library if it is available but won't
# fail to load Galaxy if it is not and 'polling' which will use a less
# efficient monitoring scheme that may work in wider range of scenarios
# than the watchdog default.
#watch_tool_data_dir = False
# To run interactive environment containers in Docker Swarm mode (on an
# existing swarm), set this option to True and set `docker_connect_port` in the
# IE plugin config (ini) file(s) of any IE plugins you have enabled and ensure
# that you are not using any `docker run`-specific options in your plugins'
# `command_inject` options (swarm mode services run using `docker service
# create`, which has a different and more limited set of options). This option
# can be overridden on a per-plugin basis by using the `swarm_mode` option in
# the plugin's ini config file.
#interactive_environment_swarm_mode = False
# Galaxy can run a "swarm manager" service that will monitor utilization of the
# swarm and provision/deprovision worker nodes as necessary. The service has
# its own configuration file.
#swarm_manager_config_file = config/swarm_manager_conf.yml
# Log to graphite
# Graphite is an external statistics aggregator (https://github.com/graphite-project/carbon)
# Enabling the following options will cause galaxy to log request timing and
# other statistics to the configured graphite instance. The graphite_prefix is
# useful if you are running multiple Galaxy instances and want to segment
# statistics between them within the same aggregator.
#graphite_host=
#graphite_port=2003
#graphite_prefix=galaxy
# Enable the new container interface for Interactive Environments
#enable_beta_containers_interface = False
New Datatypes¶
Below are the newly added datatypes and sniffers you can configure in datatypes_conf.xml
.
To see the whole sample file visit our GitHub repository.
<datatype extension="d3_hierarchy" type="galaxy.datatypes.text:Json" mimetype="application/json" subclass="true" display_in_upload="false"/>
<datatype extension="mzdata" type="galaxy.datatypes.proteomics:MzData" mimetype="application/xml" display_in_upload="true" />
<datatype extension="percin" type="galaxy.datatypes.tabular:Tabular" subclass="true" />
<datatype extension="percout" type="galaxy.datatypes.xml:GenericXml" subclass="true" />
<datatype extension="hardklor" type="galaxy.datatypes.tabular:Tabular" subclass="true" />
<datatype extension="kronik" type="galaxy.datatypes.tabular:Tabular" subclass="true" />
<datatype extension="ptalign" type="galaxy.datatypes.text:PlantTribesMultipleSequenceAlignment" />
<datatype extension="ptortho" type="galaxy.datatypes.text:PlantTribesOrtho" />
<datatype extension="ptorthocs" type="galaxy.datatypes.text:PlantTribesOrthoCodingSequence" />
<datatype extension="pttree" type="galaxy.datatypes.text:PlantTribesPhylogeneticTree" />
<sniffer type="galaxy.datatypes.proteomics:MzData"/>
To stay up to date with Galaxy’s progress watch our screencasts, visit our community hub, and follow @galaxyproject on Twitter.
You can always reach us on Gitter or IRC.
Thanks for using Galaxy!