July 2015 Galaxy Release (v 15.07)

Highlights

Interactive Environments

The interactive environments (IE) framework features several updates this release and Galaxy is now distributed with a new IE for RStudio implemented by Helena Rasche.

Workflow Editor Enhancements

The workflow editor has been significantly revamped - it now uses newer tool form components and allow construction of workflows with tools explicitly producing output collections.

Policies for Committers and Pull Requests

The process for adding committers to the project and fielding pull requests has been formalized and documented in the source code. Three new committers have been added to the project - Björn Grüning, Nicola Soranzo, and Helena Rasche.

Github

New

% git clone -b master https://github.com/galaxyproject/galaxy.git

Update to latest stable release

% git checkout master && pull --ff-only origin master

Update to exact version

% git checkout v15.07

BitBucket

Upgrade

% hg pull
% hg update latest_15.07

See our wiki for additional details regarding the source code locations.

Deprecation Notice

Galaxy’s built-in cloudlaunch functionality is being replaced by CloudLaunch and will be removed from the core codebase in the next release.

Release Notes

• Interactive environment (IE) framework updates including a new IE for RStudio. (Thanks to a team effort spearheaded by @hexylena.) Pull Request 229, Pull Request 230

• Rework workflow editor to use newer tool form components and allow construction of workflows with tools explicitly producing output collections. Pull Request 286, Pull Request 328, Pull Request 311, Pull Request 397, Pull Request 233, Pull Request 625, Pull Request 602

• Implement policies for committers and pull requests. Pull Request 295, Pull Request 314

• Allow exporting a library folder to a history. (Thanks to @ericenns.) Pull Request 426

• Fix and update documentation available on the ReadTheDocs with particular focus on the API documentation. (Thanks to @nsoranzo.) Pull Request 399, Pull Request 405

• Add EDAM IDs to Galaxy datatypes. (Thanks to a team effort initiated by @hexylena.) Pull Request 80, Pull Request 342

• Improved reports app sorting and bug fixes thanks to Daniel Bouchard (@Airistotal). Even more reports enhancements will be included in the next release of Galaxy including new landing page, pagination, and sparkline graphs. Pull Request 303, Pull Request 330, Pull Request 338

• Significant speedup of tool panel/homepage loading. (Thanks to @abretaud.) Pull Request 159

• Allow parameterization of tool XML macros. Pull Request 362

• Allow use of tool-style XML macros in job_conf.xml. Pull Request 362

• Update and clarify admin menu. Pull Request 302

• Add usage information to python cleanup scripts. (Thanks to @dpryan79.) Pull Request 274

• Added default indexing for bcf files. Pull Request 163

• Implement merging of BAM files for the parallelism framework. (Thanks to @roalva1.) Pull Request 149

• Send signal to reload data tables when adding an entry (Thanks to @abretaud.) Pull Request 250

• Add display for Interval files at IGV via bedstrict. Pull Request 259

• Include more information in default welcome.html file. Pull Request 190

• Implement a command line script for collecting stats on job runtimes. Pull Request 151

• Implement an authentication provider for PAM (Pluggable Authentication Modules) (Thanks to @pvanheus.) Pull Request 458

• Re-implement library data and FTP file parameters for new tool form. Pull Request 132, Pull Request 546

• Adding tool error tracking. (Thanks to @kellrott.) Pull Request 322

• Implement <filter> tag for output collections. Pull Request 455

• Allow environment variable definitions in tool XML. Pull Request 395

• Add new exclude_min and exclude_max attributes to InRangeValidator for numeric tool parameters. (Thanks to @gregvonkuster.) Pull Request 339

• Added unlinked Tool Shed dependency resolver. (Thanks to @Christian-B.) Pull Request 441

• Add emboss datatypes (previously migrated to Tool Shed). (Thanks to @bgruening.) Pull Request 148

• Add new Hmmer & Stockholm datatypes. (Thanks to @hexylena.) Pull Request 218

• Add an RData datatype. (Thanks to @hexylena.) Pull Request 239

• Add mauve XMFA datatype (Thanks to @hexylena.) Pull Request 270

• Add ConnectivityTable (*.ct) (RNA 2D structure format) datatype. (Thanks to @yhoogstrate.) Pull Request 439

• Add Illumina methylation datatype (IDAT). Pull Request 335

• Add Dot-Bracket notation (dbn) datatype. (Thanks to @yhoogstrate.) Pull Request 366

• Add proteomics datatypes. (Thanks to @iracooke.) Pull Request 389

• Add new MzSQlite datatype for aggregating MS-based protein identification information. (Thanks to @jj-umn.) Pull Request 427

• Allow display_in_upload by default for h5 files. Pull Request 470

• Accept history id in tool state construction API. Pull Request 268

• Adding dataset uuid information to job and workflow invocation calls. (Thanks to @kellrott.) Pull Request 181

• Allow workflows executed via the API to optionally run with tool upgrades. Pull Request 402

• Various PEP-8 fixes, more to come next release. (Thanks to efforts lead by @nsoranzo, @remimarenco, and @bgruening.) Pull Request 421, Pull Request 422, Pull Request 424, Pull Request 433, Pull Request 449, Pull Request 350, Pull Request 453, Pull Request 394, Pull Request 204

• Nose update with --debug option added to run_tests.sh. Pull Request 172

• Mention -installed option in run_tests.sh help. Pull Request 323

• Allow config of shed tool configuration file used with run_tests.sh -installed. Pull Request 267

• Allow running tests from outer directories (Thanks to @yhoogstrate.) Pull Request 137

• Enhancement allowing use of DRMAA when Galaxy instance isn’t on a shared volume with compute nodes. (Thanks to @vavrusa.) Pull Request 160

• Enhancements for Condor. (Thanks to @dpryan79.) Pull Request 289

• Update gsummary dependencies. (Thanks to @bgruening.) Pull Request 182

• Refactor visualization plugins to promote implementation to a class. Pull Request 186

• Switch visualizations registry to entry_point config definition for built-in visualizations. Pull Request 243

• Set default value for IE sig proxy (Thanks to @hexylena.) Pull Request 197

• Other smaller interactive environment updates. (Thanks to @hexylena.) Pull Request 412

• Refuse to load interactive debugging middleware under uwsgi. Pull Request 200

• Move stylesheet source files from static/style/src to client/galaxy/style. Pull Request 208

• Fix sprite paths by using hardcoded path in task. Pull Request 279

• Improve various focus handling behavior in the UI. Pull Request 280

• Fixes and updates to various JavaScript-based unit tests. Pull Request 219

• Update various client libs (e.g. jQuery, Raven, d3, requirejs). Pull Request 221, Pull Request 241

• Ignore sample files from toolshed installed tools. (Thanks to @Unode.) Pull Request 248

• Add test option for disabling truncation of test diffs. Pull Request 249

• Various documentation and wording improvements for various files. (Thanks to @nsoranzo.) Pull Request 277

• Allow external display applications optional access to files within a dataset’s extra_files_path. Pull Request 284

• Improve the data manager JSON datatype’s class to be more specific. Pull Request 285

• Allow setting connection options for LDAP authentication. (Thanks to @Unode.) Pull Request 287

• Fix for gzip being imported twice. (Thanks to @bgruening.) Pull Request 296

• Export environment variables from all dependent repositories. (Thanks to @bgruening.) Pull Request 297

• Improve auth_conf.xml.sample documentation. (Thanks to @nsoranzo.) Pull Request 299

• Preserve unix permissions, if any, when extracting .zip files during tool shed installations. Pull Request 308

• Update mailing lists to use lists.galaxyproject.org addresses. Pull Request 316

• Add new argument to inject volumes in docker_cmd() and launch(). (Thanks to @bgruening.) Pull Request 318

• Create a separate application entry point to be used by uWSGI without --ini- paste and PasteDeploy. Pull Request 325

• Implement groups in Tool Shed models and UI. Pull Request 326, Pull Request 374

• Improved datatype registry debugging. Pull Request 329

• Add app option to client process to allow building Tool Shed scripts. Pull Request 332

• Add grunt command grunt decompress to copy decompressed artifacts. Pull Request 173

• Improvements to Tool Shed search API. Pull Request 334, Pull Request 376, Pull Request 420, Pull Request 563

• Restore link to mercurial browsing of repositories. Pull Request 340

• Enhance tool testing to only cleanup successful jobs. Pull Request 341

• Implement metadata size limiting. Pull Request 345, Pull Request 416

• Add tool parameter argument to parameter help. Pull Request 346, Pull Request 351

• Refactor Tool Shed JavaScript files to be separated from Galaxy Java Script Pull Request 347

• Fixed object store path - job_work. (Thanks to @charz.) Pull Request 354

• Make debugging output nicer for IE’s (Thanks to @bgruening.) Pull Request 355

• Add configurable Docker hostname (Thanks to @bgruening.) Pull Request 356

• Improve XML styling of display applications. (Thanks to @hexylena.) Pull Request 364

• README improvements including - adding shields for Biostar, IRC, and Travis. Pull Request 368, Pull Request 369

• Use passed in URL for getbuilds method of parse_builds.py (Thanks to @dctrud.) Pull Request 384

• Add error message for bad dependency declaration (Thanks to @kaktus42.) Pull Request 419

• Refactor JavaScript set up code for tagging toward reuse. Pull Request 429

• Improved md5sum/sha256 in Tool Shed actions. (Thanks to @yhoogstrate.) Pull Request 444

• Include owner annotations of HDAs in published history display. Pull Request 490

• Added log error for config file not found. (Thanks to @Christian-B.) Pull Request 443

• Fix for installing dependencies from source when download of precompiled binaries has failed. Pull Request 531

• Use a password field for deprecated cloudlaunch’s secret key field. Pull Request 551

• Normalize datatypes_by_ext upon entry. Pull Request 567

• Stop creating unneeded database/message.mako.py. Pull Request 575

Fixes

• During the 15.07 development cycle dozens of fixes were pushed to the release_15.05 branch of Galaxy. These are all included in 15.07 and summarized here (with special thanks to @bgruening, @mvdbeek, @nsoranzo, and @abretaud).

• Stop Trackster location wrapping in Chrome. (Thanks to @dctrud.) Pull Request 440

• Fix return integer values when ‘round to nearest integer’ is selected in Group tool. Pull Request 452

• Fix external set_meta under sqlalchemy 1.0.0. Pull Request 150

• Fix base.util to get the changeset from git. Pull Request 161

• Remove biomart_test tool, http://test.biomart.org/ is down. (Thanks to @nsoranzo.) Pull Request 183

• Fix broken browser tests. Pull Request 202

• Fix assertions in test_VisualizationsRegistry.py. Pull Request 223

• Fix run_tests.sh exit code. Pull Request 225

• Use unittest2 to allow Python 2.7 assertions under Python 2.6. Pull Request 246

• Have trans.user_ftp_dir return None instead of error if upload dir unset. Pull Request 257

• Allow IGV external displays to use a name with a comma in it. Pull Request 261

• Fix bug in history sharing. (Thanks to @bgruening.) Pull Request 263

• Convert tools directory to only use unix newline characters. (Thanks to @nsoranzo.) Pull Request 264

• Remove spurious numpy requirement on maf_stats.xml tool. (Thanks to @nsoranzo.) Pull Request 264

• Use more actionable package requirement type on some tools. (Thanks to @nsoranzo.) Pull Request 264

• Small doc fix for config/plugins/visualizations/README.txt. (Thanks to @nsoranzo.) Pull Request 264

• Fix function names for dynamic tool test functions. Pull Request 266

• Fixes to dataset chooser. Pull Request 272

• Fix Trackster load order of require and d3. Pull Request 291

• Remove bad HTML tags from login template. (Thanks to @Unode.) Pull Request 294

• Exclude curly braces from API-key to fix toolshed bootstrapping. (Thanks to @mvdbeek.) Pull Request 301

• Fix repeated error messages with identical text. Pull Request 306

• Fix shared mako templates that would not work properly with the Tool Shed. Pull Request 307

• Fix json sniffer for large files. Pull Request 309

• Fix a typo error in visualization/plugins/resource_parser.py. (Thanks to @remimarenco.) Pull Request 312

• Fix for generating metadata on toolshed tools tests, under certain conditions. Pull Request 319

• Fix Tool Shed metadata generation bug. (Thanks to @bgruening.) Pull Request 333

• Fixing string appending error in library dictification. (Thanks to @kellrott.) Pull Request 336

• Hide constantly broken Tool Shed test results. Pull Request 337

• Fix for fix for handling of unix permissions in .zip files. Pull Request 344

• Fix common API scripts so they don’t fail when response has no URL param. Pull Request 352

• Fix for dynamic test data fetching. Pull Request 371

• Fix bug in Text.split() method. (Thanks to @nsoranzo.) Pull Request 373

• Fix and update serializer call to non-trans form for histories. Pull Request 390

• Fix getting admin key when bootstrapping toolshed. (Thanks to @galaxyproject.) Pull Request 392

• Fix error when displaying job params for tools containing a section. (Thanks to @abretaud.) Pull Request 398

• Fix for setting environment variable paths for installed tools. (Thanks to @gregvonkuster.) Pull Request 403

• Quota calculation fixes for managers. Pull Request 406

• Fix paging issue with whoosh due to missing str to int conversion. Pull Request 407

• Use web.url_for( '/', qualified=True ) instead of '/' to generate URLs (for correctness). (Thanks to @mvdbeek.) Pull Request 414

• Fix for async datasources. Pull Request 423

• Fix UI bug related to ID collisions for display of certain collections. Pull Request 431

• Fix run_tests.sh example API test command. Pull Request 432

• Fixed typo in Tool Shed docstring. (Thanks to @peterjc.) Pull Request 438

• Various Trackster fixes. Pull Request 446

• Fix job query to allow admins to view other histories. Pull Request 465

• Fix dbkey filtering of multiple input targets. Pull Request 534

• Fixes for DeleteIntermediatesAction. Pull Request 547

• Fix overriding the name variable in Tool Shed index code. Pull Request 566

• Fix batch mode input in new tool form UI. (Thanks to @ericenns.) Pull Request 568

• CSV/SAM sniff order fix. Pull Request 571

• Show field for entering remote/home url when managing a Tool Shed repository. Pull Request 580

• Fix regenerating repository metadata when coming from batch interface. Pull Request 600

To stay up to date with Galaxy’s progress, watch our screencasts; visit our community Hub; and follow us on Bluesky, Mastodon, and LinkedIn.

You can always chat with us on Matrix.

Thanks for using Galaxy!

The Galaxy Team