January 2016 Galaxy Release (v 16.01)¶
- Interactive Tours
The interactive tours framework allows developers and deployers to build interactive tutorials for users superimposed on the actual Galaxy web front end. Unlike video tutorials, these will not become stale and are truly interactive (allowing users to actually navigate and interact with Galaxy). Galaxy 16.01 ships with two example tours and new ones can easily be added by creating a small YAML file describing the tour. Try the Galaxy UI tour on Main.
Galaxy’s Python dependencies have traditionally been distributed as eggs using custom dependency management code to enable Galaxy to distribute binary dependencies (enabling quick downloads and minimal system requirements). With this release all of that infrastructure has been replaced with a modern Python infrastructure based on pip and wheels. Work done as part of this to enable binary dependencies on Linux has been included with the recently released pip 8.
Detailed documentation on these changes and their impact under a variety of Galaxy deployment scenarios can be found in the Framework Dependencies section of the Admin documentation.
- Nested Workflows
Workflows may now run other workflows as a single abstract step in the parent workflow. This allows for reusing or subworkflows in your analyses.
% git clone -b master https://github.com/galaxyproject/galaxy.git
- Update to latest stable release
% git checkout master && pull --ff-only origin master
- Update to exact version
% git checkout v16.01
% hg pull % hg update latest_16.01
See our wiki for additional details regarding the source code locations.
Barring a strong outcry from deployers, 16.01 will be the last release of Galaxy to support Python 2.6. For more information, see Galaxy Github Issue #1596.
Multiple security vulnerabilities were identified during this release cycle and fixed concurrently with the release. In addition, the fixes have been backported to older releases.
The Galaxy Committers would like to thank Youri Hoogstrate at the Erasmus MC, Rotterdam, who initially
hg push vulnerability. Through additional auditing based on
this attack vector, we discovered the other vulnerabilities.
Multiple security vulnerabilities were discovered in Galaxy that allow malicious actors to read and write files on the Galaxy server. Additionally, Galaxy servers on which a rarely used feature has been enabled are vulnerable to an arbitrary code execution exploit.
A write vulnerability exists in the history import mechanism. It is possible to create a history tar archive that contains files with parent directory components in the file path (e.g.
foo/../../barwould extract to
../bar), and these archive members would be written if the user running the Galaxy server had write permission to the given path.
A read vulnerability exists in the object store path composition code. Galaxy allows clients to add elements to the end of a path to “extra” files associated with a dataset (as is the case with composite datatypes). These elements were not being checked to ensure they did not contain relative parent references (
..) or did not start with an absolute path character (
/). Because of this, the dataset display methods could be manipulated to return the contents of any files for which the Galaxy server user had read permission.
An arbitrary code execution vulnerability exists in the Galaxy sample tracking system. The sample tracking system included a feature which allowed administrators to browse remote “external services” (such as sequencers) to choose files to transfer to the Galaxy server. This browsing code used a shell invocation which did not sanitize user input. However, this code is only reachable if at least one external service has ever been defined.
Fixes for all three issues have been applied to Galaxy releases back to v14.10.
- Tool Shed
Multiple security vulnerabilities were discovered in the Tool Shed that allow malicious actors to read and write files on the Tool Shed server outside of normal Tool Shed repository directories.
A write vulnerability exists in the Tool Shed tarball and capsule upload functionality. It is possible to create a tar archive that contains files with parent directory components in the file path (e.g.
foo/../../barwould extract to
../bar), and these archive members would be written if the user running the Tool Shed had write permission to the given path. The Tool Shed tarball handling code checked for invalid characters (
..) at the beginning of the path but not for
..in the middle of a path.
A read vulnerability exists in multiple places. The first is in the (now deprecated)
hg pushfunctionality for updating Tool Shed repositories. This method allows malicious actors to push symlinks whose targets are outside the repository (either via an absolute or relative path). The contents of the targets would then be visible in the Tool Shed repository contents viewer, if the Tool Shed user has read permission on the target.
A second read vulnerability exists in the Tool Shed repository contents viewer. The viewer would allow a malicious actor to specify a path outside the repository, and if the Tool Shed system user had read permissions on that path, it would be displayed. The viewer also did not check to ensure that the targets of symlinks in a repository did not point outside the repository.
The repository contents viewer read vulnerability also exists in Galaxy, but is only reachable/exploitable by admin users. Fixes for vulnerability #3 have been applied to Galaxy/Tool Shed releases back to v14.10, and issues #1 and #2 have been applied to releases back to v15.01.
Replace Galaxy eggs dependency management with wheels. Pull Request 428, Pull Request 989, Pull Request 988, Pull Request 1389, Pull Request 1485, Pull Request 995, Pull Request 996, Pull Request 1006, Pull Request 1017, Pull Request 1037, Pull Request 1495
Implement nested workflows. Pull Request 1306
Many ToolBox improvements including allowing specifying labels on individual tools, monitoring and automatic reloading of
tool_conf.xmlfiles, and allowing specification of such files in JSON/YAML. Pull Request 1012, Pull Request 1398
Configurable client side logging. Pull Request 1011
Allow input collections to specify multiple
collection_types. Pull Request 1308
Allow multiple collections to be supplied to a multiple data parameter. Pull Request 805
type_sourceon output collections. Pull Request 1153
Allow tools to request a special configuration file containing tool parameters dumped to json. Pull Request 1405
Avoid confusion by adding the PDF datatype to the list of uploadable file types. Pull Request 901
Add Google analytics tracking to new data libraries. Pull Request 959
Finish swapping unencoded ids with order_index in workflows API. Pull Request 1137
Eliminate LWR from Galaxy (you will need to upgrade to Pulsar). Pull Request 857
Allow override of job shell (for conda resolver). Pull Request 1473
Slight tweak to sidebar section naming from ‘security’ to ‘user management’. Pull Request 877
Enable dockerized toolshed tests. Pull Request 942
Improvements and fixes for abstract tool interface. Pull Request 955
Updates to default welcome page. Pull Request 1013
Revise deferred queue, use jQuery promises. Pull Request 1018
Keep select field open in tool form multi select fields to ease selecting many options quickly. Pull Request 1019
Allow resizing of regular multi-select boxes. Pull Request 1025
Add biocrusoe to contributors Pull Request 1067
Outline “Format 2” workflow definitions. Pull Request 1096
Improve the API attributes display on parameters page. Pull Request 1098
Show welcome page with required login. Pull Request 1105
Add nginx config for GIE proxy to documentation. Pull Request 1123
Enforce client build deps are up-to-date. Pull Request 1130
Menu onclick addition. Pull Request 1142
Ensure confirmation when leaving GIE windows. Pull Request 1157
Add local grunt-cli dependency for qunit tests. Pull Request 1159
Tighten up GG sniffing Pull Request 864
Use common exceptions in tools. Pull Request 874
Do not check for tool migrations when running tests. Pull Request 1176
Add debug statement in output checker for why job is failing. Pull Request 1213
Small enhancements in workflow inputs and outputs. Pull Request 1214
More small workflow tweaks. Pull Request 1216
Add Workflow editor UI for step labels. Pull Request 1251
Add beta run workflow form based on the newer tool form code. Pull Request 1249
In the users API, properly return a boolean from has_requests Pull Request 1262
Remove history_options and options.mako (history options as a page). Pull Request 1271
Minor mail config cleanup Pull Request 1299
Replace uses of history.imp and history.copy web methods for API create Pull Request 1303
More tightening up of parameter validation during workflow stuff. Pull Request 1319
Upgrade Paste to 2.0.2. Pull Request 1344
A series of small refactoring enabling use of certain Galaxy modules a stand-alone Python library with minimal dependencies and Python 3 compatibility. Pull Request 1350, Pull Request 1351, Pull Request 1352, Pull Request 1359, Pull Request 1362, Pull Request 1376, Pull Request 1413, Pull Request 1427, Pull Request 1363, Pull Request 1367, Pull Request 1377, Pull Request 1388, Pull Request 1448
Unify and abstract code for checking if file looks like a tool definition. Pull Request 1368
Change scratchbook close icon. Pull Request 1425
Fix data library test case. Pull Request 898
test_map_over_two_collections_legacytest case as it is obsolete. Pull Request 924
Version the testing-base docker image. Pull Request 938
Update casperjs functional tests. Pull Request 944
Fix the docker db client_encoding to not be ascii (default). Pull Request 952
Use the database temp directory to store the Mako template cache when running framework tests. Pull Request 956
lib/galaxy/main.pyto scripts/galaxy-main Pull Request 994
Fix deferred dom removal. Pull Request 997
Options to more easily test esoteric tooling options. Pull Request 1014
Fix log statements and link to logger. Pull Request 1040
Add more data libraries API tests. Pull Request 1074
Update testing docker image. Pull Request 1083
Small tool and workflow refactorings. Pull Request 1097
Revise and fix waiting for tool tests. Pull Request 1119
Small Tool and Workflow Refactoring and Fixes Pull Request 1202
Wait on jobs and history in certain API test cases. Pull Request 1226
Fix qunit shim to match base_panels.mako shim. Pull Request 1233
Improved logging related tool test timeouts. Pull Request 1243
Refactor generic side workflow editor panel toward backbone. Pull Request 1247
Attempt to fix transiently failing tool test on Jenkins. Pull Request 1248
Set client_encoding for TS dockerized test db Pull Request 1276
Check for sessionStorage using a more cross-browser way. Pull Request 1279
Small API test improvements. Pull Request 1285
Enforce metrics related to moving toward modern client infrastructure. Pull Request 1292
Refactor tool stuff for generic model actions. Pull Request 1307
Remove workflow casperjs API test duplicating test coverage of API tests. Pull Request 1316
Lint with Python 3 several modules. Pull Request 1354
Fix and potential fix for transiently failing tests. Pull Request 1401
Add a safe_relpath util function for ensuring a path does not reference an absolute or parent directory. Commit f540a16
Security fixes for history imports. Commit bf1c77d
Security fixes for object store paths. Commit 5da91bd
Remove sample tracking manual external service transfer due to security concerns. Commit cd8b965
Security fixes for tool shed repository browsing. Commit e4a1d57
Security fixes for tool shed hg push and capsule/tarball uploads. Commit e845d64
Various Quota bug fixes. Pull Request 907
Always set ‘tests’ for a visualization plugin to avoid an attribute error. Pull Request 908
Change user preference datatype to text. Pull Request 916
Revise tool URL building. Pull Request 947
Fix base panels to include scripts as the last element of the body. Pull Request 969
Grid batch operation fixes. Pull Request 971
Fix for reloading tools that have non-standard tool_ids/versions. Pull Request 1050
Improved encoding handling for Jobs. Pull Request 1052
Fix a bug in IEs when proxying the proxy. Pull Request 1076
Fix 500 error when attempting to update installed repository. Pull Request 1082
Resolve conflicting label CSS class for trackster. Pull Request 1086
Fix bug with referrer attribute type change in WebOb. Pull Request 1091
$inputhang cheetah evaluation. Pull Request 1117
Fix interface and usage of
WorkflowModule.get_runtime_inputs. Pull Request 1174
Fix for updating tool parameter dicts when a new parameter has been added to a section. Pull Request 1215
Replace the defunct readthedocs badge. Pull Request 1229
Fix passing of nginx_upload_path and ftp_upload_site. Pull Request 1250
Open select2 drop down on caret click. Pull Request 1298
Improved validation of tools during workflow execution. Pull Request 1302
Properly remove datasets from the filtered lists when pairing datasets for the paired dataset list creator. Pull Request 1310
Update Kombu and AMQP wheels to fix problems with El Capitan’s System Integrity Protection. Pull Request 1327
Fix for creating workflow outputs on initial workflow upload. Pull Request 1330
GALAXY_SLOTSis defined in the environment, use it for the local runner. Pull Request 1346
Fix for loading workflows that have tool version / step upgrade messages. Pull Request 1348
Copy workflow objects when importing them. Pull Request 1474
Undo user icon in masthead. Pull Request 1493
Fix mime type when previewing certain tabular data. Pull Request 1498
Fix disabled CSS. Pull Request 1501
catch Exception and properly log errors Pull Request 1511
Keep track of hidden datasets. Pull Request 1551
--skip-venvif we can detect that Python is Conda Python. Pull Request 1554
Thanks for using Galaxy!