May 2020 Galaxy Release (v 20.05)

Highlights

Many new Interactive Tools Galaxy interactive tools allow a greater depth of analysis via access to GUI-based tools inside an instance. 10 new interactive tools (some previously available only on the UseGalaxy.eu server) join the standard base toolset. Improvements to the Higlass interactive tool now also allow for a more in-depth visualization of multiple datasets in various formats simultaneously (Pull Request 9559).

Data Tables can now be backed by refgenie Refgenie manages storage, access, and transfer of reference genome resources. Galaxy can now fill data tables that were received from a refgenie installation (Pull Request 9600).

Tool Shed is now Python3 ready As the last component in the Galaxy Codebase the Tool Shed has been ported to Python 3. This concludes a 4 year effort to port the Galaxy codebase to Python 3 (Pull Request 9528).

Workflow Editor and Workflow Run Form in Vue.js The Workflow Editor and Workflow Run Form have been re-written in Vue.js. While the functionality has been preserved this lays the groundwork for creating beautiful and reactive custom variants of these components in the future (Pull Request 9346, Pull Request 9332, Pull Request 9324, Pull Request 9249, Pull Request 9201, Pull Request 9147, Pull Request 9138, Pull Request 9471).

Accelerated Galaxy Startup Galaxy now caches expanded tool documents, delays creating the tool search index until after startup and creates search indexes incrementally (Pull Request 9610).

Also check out the 20.05 user release notes

Get Galaxy

The code lives at GitHub and you should have Git to obtain it.

To get a new Galaxy repository run:

$ git clone -b release_20.05 https://github.com/galaxyproject/galaxy.git

To update an existing Galaxy repository run:

$ git fetch origin && git checkout release_20.05 && git pull --ff-only origin release_20.05

See the community hub for additional details regarding the source code locations.

Upgrade Warning for Admins

We have been simplifying the organization of our static and client content. As a result, the /static directory now only requires a single rule in your proxy (nginx, Apache, etc.) configuration. Previously, additional rules for /static/scripts and /static/style were required. Both of these additional rules should be removed from your proxy server configuration when upgrading your Galaxy server to release 20.05.

More information about proxy server configuration for Galaxy can be found in Proxying Galaxy with NGINX and Proxying Galaxy with Apache.

Release Notes

Enhancements

• New data types for OpenMS (thanks to @bernt-matthias). Pull Request 8109

• Refactor Workflow Run Form Toward Vue (part 1) Pull Request 9138

• Refactor Workflow Run Form Toward Vue (part 2) Pull Request 9147

• Replace Workflow Editor Tool Panel with Vue component Pull Request 9201

• Replace right panel in Workflow Editor with Vue component Pull Request 9249

• Replacing Workflow Editor Node with Vue-component Pull Request 9332

• Render workflow node connections using d3 Pull Request 9471

• Make Toolshed Python 3 compatible & tests Pull Request 9528

• Migrate interactive tools from EU server (thanks to @astrovsky01). Pull Request 9559

• Allow Tool Data Tables to be backed by refgenie Pull Request 9600

• Startup speedups (XML doc cache, delay non-essential tool param parsing & indexing) Pull Request 9610

• Enable admin to resend activation email Pull Request 9154

• Create Python3 virtualenv by default (thanks to @nsoranzo). Pull Request 9344

• iRODS object store rewrite (thanks to @kxk302). Pull Request 9438

• Tool recommendation (thanks to @anuprulez). Pull Request 9440

• Tool recommendations in workflow editor (thanks to @anuprulez). Pull Request 9569

• Added whitelisting option for user registration (thanks to @MiguelJulia). Pull Request 9501

• Editor as visualization (thanks to @OlegZharkov). Pull Request 9531

• Chira visualization for Chimeras (thanks to @OlegZharkov). Pull Request 9562

• Datasource tool for SRA manifest Pull Request 9613, Pull Request 9892

• Skip visualization build, if no changes were registered (thanks to @OlegZharkov). Pull Request 9634

• Add tpr datatype (thanks to @simonbray). Pull Request 9244

• Add datatypes for xcms and camera (thanks to @computational-metabolomics). Pull Request 9470

• index with -c flag for large bam (one region larger than 2**29 - 1) (thanks to @FredericBGA). Pull Request 9570

• Sniffer for multi-member zip archives (thanks to @lecorguille). Pull Request 9595

• Followup to shed tool static fixes Pull Request 9450

• Remove deprecated interpreter attribute from tools <command> (thanks to @nsoranzo). Pull Request 9490

• Resize D3 tree and make it responsive (thanks to @anuprulez). Pull Request 9547

• Fix ITs to work in remote Kubernetes Pull Request 9614

• Fix galaxy_tool_util for composite file input testing in workflows. Pull Request 9312

• Refactor toward testing workflow collection outputs. Pull Request 9340

• Remove Backbone and Globals dependency from Workflow client code Pull Request 9346

• Allow anonymous downloading of public workflows Pull Request 9736

• MS Edge Fixes Pull Request 9289

• Refactor more of jobs admin component toward API. Pull Request 9093

• make more tool parsing errors contain id (thanks to @bernt-matthias). Pull Request 9139

• “Normalize” more custom functions in admin jobs index. Pull Request 9157

• Job Lock VueJS component. Pull Request 9181

• add a new function to get the local ip address (thanks to @gmauro). Pull Request 9353

• Integration test for GxITs against remote proxies. Pull Request 9504

• paramiko: load system host keys (thanks to @eancelet). Pull Request 9553

• Optimize setting external ID for jobs. Pull Request 9691

• Enable manual user activation Pull Request 9143

• Skip migration if uninitialized database exists Pull Request 9491

• Add Okta as OIDC authentication option (thanks to @selten). Pull Request 7497

• Added trigger to dataset table to update history.update_time Pull Request 8187

• Notifications on job completion (thanks to @astrovsky01). Pull Request 8427

• Issue #8445: Clicking on user/email in User menu triggers a page refresh - fix (thanks to @janani-prasad). Pull Request 8644

• Third Party Identity Management (thanks to @juleengraham). Pull Request 9060

• Disable the admin_toolshed controller (except for tests). Pull Request 9072

• Upgrade to Fontawesome 5 Pull Request 9101

• Add a test for a tool with from_working_dir, outputs_to_working_directory and pulsar (thanks to @bgruening). Pull Request 9108

• Client build work (HMR, build upgrades, refactoring asset placement) Pull Request 9123

• Correct typos, update location of api tests Pull Request 9158

• Add more repository installation test cases Pull Request 9160

• Add support for logging out of OIDC Identity Provider (thanks to @machristie). Pull Request 9162

• Vue-ify Upload Dialog w/small fixes Pull Request 9167

• Update galaxy.tool_util.cwl for latest changes in CWL branch. Pull Request 9175

• More progress on Tool shed cleanup Pull Request 9186

• Full Python dependencies update (thanks to @nsoranzo). Pull Request 9188, Pull Request 9402, Pull Request 9529, Pull Request 9697

• Allow mixed store_by objectstore configurations. Pull Request 9197

• implement workflow bookmark selenium test (thanks to @OlegZharkov). Pull Request 9208

• Use a Galaxy root path with a space for integration tests (thanks to @nsoranzo). Pull Request 9223

• implementing a global tools-view component + schema.org support (thanks to @OlegZharkov). Pull Request 9261

• Add icons to dropdowns in dataset and workflow list Pull Request 9273

• extend config file inputs tests (thanks to @bernt-matthias). Pull Request 9279

• Do not strip tab characters from data_column reference and clarify documentation for use_header_names (thanks to @bernt-matthias). Pull Request 9291

• Update CONTRIBUTING.md (thanks to @nsoranzo). Pull Request 9299

• Add more info to debug logging for loading datatypes. Pull Request 9301

• Add do_not_cache decorator, apply to codebase. Pull Request 9311

• Use Bootstrap components to build the Workflow Editor Toolbar Pull Request 9324

• implement selenium test User->Preferences->Manage information (thanks to @OlegZharkov). Pull Request 9355

• Fix success message in user preferences, add help text to toolbox filter page Pull Request 9368

• CSS changes for a modern look and feel (thanks to @bedroesb). Pull Request 9386

• Added conda python update (thanks to @astrovsky01). Pull Request 9390

• Replace editor grid image with css, add editor client unit tests Pull Request 9392

• Vuefy interactive tools (thanks to @OlegZharkov). Pull Request 9409

• Add delta_frac for sim_size to specify relative file size differences (thanks to @bernt-matthias). Pull Request 9425

• Make mutable_config_dir configurable + rename Pull Request 9444

• Remove virtualenv<20 requirement for mulled tests (thanks to @nsoranzo). Pull Request 9447

• Remove mapping related abstraction classes from workflow client, increase node header contrast Pull Request 9474

• Add eslint rule to check for invalid slot specifications Pull Request 9481

• Another undocumented option: job metrics path (thanks to @hexylena). Pull Request 9482

• Document maxpctfull objecto store option (thanks to @hexylena). Pull Request 9498

• Selenium Multi-History View (thanks to @OlegZharkov). Pull Request 9505

• Adjust dataset error reporting page, remove email option, remove long info text Pull Request 9508

• Drop remaining Python 2.7 CircleCI tests (thanks to @nsoranzo). Pull Request 9518

• Implementation selenium test for user change password (thanks to @Realjuan). Pull Request 9521

• Rename and modify current_history_contents method Pull Request 9526

• Improvement to the selenium test to change password from user preferences (thanks to @Realjuan). Pull Request 9545

• Implementation of Selenium test for signout from user preferences (thanks to @Realjuan). Pull Request 9546

• Move OSX startup tests to github CI, drop travis tests Pull Request 9563

• Check if path is set before joining Pull Request 9568

• Sort Tool Search Result by ranking (thanks to @juleengraham). Pull Request 9579

• TS performance fixes Pull Request 9580

• Vuefy Job Destination Parameters (thanks to @OlegZharkov). Pull Request 9583

• Fix kubernetes test failures again Pull Request 9585

• Enabled various object store types to store data objects by uuid. (thanks to @kxk302). Pull Request 9609

• Add IDs to admin links and selenium navigation. Pull Request 9616

• Feature/refactor rule builder (thanks to @assuntad23). Pull Request 9619

• Add selenium test for admin toolshed operations. Pull Request 9628

• Add unit tests for all schema-defined config properties Pull Request 9631

• Officially allow adding new datatypes to the latest tagged release (thanks to @nsoranzo). Pull Request 9632

• Update the main readme with new Python version Pull Request 9635

• Minor IT doc clarification Pull Request 9668

• Enhance toolshed selenium test. Pull Request 9678

• Optimize history_content union and filtering query Pull Request 9680

• Add CSS class to messages in error report pages to make errors more visible (thanks to @hexylena). Pull Request 9690

• Feature/3937 advanced search tooltip (thanks to @assuntad23). Pull Request 9696

• Rename function, params in ConfigSerializer Pull Request 9247

• Make str.join()s more efficient (thanks to @nsoranzo). Pull Request 9258

• Faster and cleaner set creation (thanks to @nsoranzo). Pull Request 9300

• Remove broken script (thanks to @nsoranzo). Pull Request 9302

• Clean up old ‘blue’ style path, update documentation and samples Pull Request 9306

• Consistent “import” of maketrans (thanks to @nsoranzo). Pull Request 9351

• Move schema loading/validating code out of config Pull Request 9371

• Remove more duplication from TS, galaxy config Pull Request 9379

• Unify default values used for delta (sim_size) (thanks to @bernt-matthias). Pull Request 9439

• Vue slot refactoring Pull Request 9443

• Move sanitize_whitelist into mutable_config_dir Pull Request 9455

• Use auto_create functionality of get_private_user_role() (thanks to @nsoranzo). Pull Request 9509

• Combine selenium tests for creating and switching histories Pull Request 9527

• Eslint one-var spec / airbnbstyle Pull Request 9532

• unify usage of Popen (thanks to @bernt-matthias). Pull Request 9544

• Prettier 2.0 Pull Request 9552

• Abstract ObjectStore interface implementation Pull Request 9554

• Misc. refactoring of galaxy+TS config Pull Request 9582

• Use PyGithub for issue creation Pull Request 9637

• clean up code to use helper function instead (thanks to @lawrence14701). Pull Request 9643

• Respect retry-after header, raise on unexpected status codes Pull Request 9659

• restore tool_util.deps.commands (thanks to @bernt-matthias). Pull Request 9707

Fixes

• Update charts entrypoint inclusion of dist artifacts Pull Request 9511

• Python3 compatible SnpEff and SnpSift metadata detection (thanks to @wm75). Pull Request 9282

• Add PAF, GFA1 and odgi datatypes (thanks to @nsoranzo). Pull Request 9592

• Add xg, vg and protobuf datatypes (thanks to @bgruening). Pull Request 9593

• Fix gff3.gz and gff3.bz2 datatypes Pull Request 9731

• Use right extension in bz2_to_uncompressed.xml Pull Request 9734

• Fix disappearing datatypes after restart Pull Request 9835

• fix error in bundled tool: shrimp_color_wrapper (thanks to @bernt-matthias). Pull Request 9665

• extract_genomic_dna.py: import Comment, Header from bx.tabular.io instead of bx.intervals.io (thanks to @cat-bro). Pull Request 9748

• Move some tools that no longer need galaxy on the PATH (thanks to @bernt-matthias). Pull Request 9850

• Fix workflow list filtering dropdown Pull Request 9468

• Fix workfow parameter connections and default values Pull Request 9648

• Fix ChangeOutputDatatype being dropped on node update Pull Request 9676

• Trigger onchange event, fixes tool version changes being lost Pull Request 9695

• Input dataset module format filtering fix; Pull Request 9884

• fix for python3 and GEDA, was getting “Connection closed at byte 0” Pull Request 9507

• Checkbox bugfix Pull Request 9667

• Fix none type exception in provided metadata handling. Pull Request 9173

• Handles floats correctly in ByteSize (thanks to @pcm32). Pull Request 9418

• unicodify message (thanks to @ieguinoa). Pull Request 9494

• Fixes for embedded Pulsar with MQ (with tests) Pull Request 9519

• Fix outputs written to storage despite outputs_to_working_directory Pull Request 9683

• Fix collection mapping in extended metadata mode. Pull Request 9709

• Force container image names to lower case to satisfy Docker (thanks to @nuwang). Pull Request 9733

• Make watcher more resilient to missing files (thanks to @nuwang). Pull Request 9738

• Improved exception handling for kubernetes runner (thanks to @nuwang). Pull Request 9751

• Check dataset is accessible and use payload in job error API (thanks to @nsoranzo). Pull Request 9753

• Fix the cleanup condition when job state is OK and cleanup_job is set as “onsuccess’ (thanks to @gmauro). Pull Request 9761

• Migrate condor runner to jwd for job-files (thanks to @bgruening). Pull Request 9833

• Fix incorrect variable assignment Pull Request 9907

• Fix vue helper in admin router, fixes toolshed installation, avoids console errors Pull Request 9220

• Fix default quota creation message. Pull Request 9479

• Fix data manager table headers Pull Request 9480

• Save integrated tool panel once on toolbox reload Pull Request 9487

• Cannot find migrate directory for tools database (thanks to @phac-nml). Pull Request 9699

• Fix S3 issue discovered during admin training (thanks to @hexylena). Pull Request 7294

• Make failing to load tools from a tool_dir non-fatal. Pull Request 9106

• Fix incorrect reference to app (thanks to @nuwang). Pull Request 9159

• Missing env in AskOmics IT (thanks to @xgaia). Pull Request 9177

• Check return value of quick_load Pull Request 9185

• Fix sending email error on python-3.7 (thanks to @shenweiyan). Pull Request 9204

• Properly quote paths in upload tool (thanks to @bernt-matthias). Pull Request 9222

• Fix reference in workflows API docs (thanks to @nsoranzo). Pull Request 9259

• Fix target_str_to_targets Pull Request 9268

• Do not test config_dir default value Pull Request 9288

• Remove a dead symlink (thanks to @abretaud). Pull Request 9292

• Fixes reports app under py3 Pull Request 9305

• Allow setting container path for docker_volumes (thanks to @AndreasSko). Pull Request 9310

• Skip test_config_option[build_sites_config_file] (thanks to @nsoranzo). Pull Request 9357

• fix special characters in citation field (thanks to @OlegZharkov). Pull Request 9373

• Use ES6 function declaration to handle error response in workflow form Pull Request 9412

• Fix first_startup Travis tests (thanks to @nsoranzo). Pull Request 9423

• Use a Galaxy root path with a space for converters and mulled tests (thanks to @nsoranzo). Pull Request 9429

• Fix test_run_simple() test broken on Python 2 by virtualenv 20.0.5 (thanks to @nsoranzo). Pull Request 9458

• Use browser locale over global default Pull Request 9460

• Fix login handling w/ redirects and user creation disabled. Pull Request 9462

• Add missing pygithub dev-requirement (thanks to @nsoranzo). Pull Request 9463

• Drop manage_dependency_relationships option Pull Request 9465

• fix options_filter_attribute: needs to be str (thanks to @galaxyproject). Pull Request 9466

• History Dropdown Fix Pull Request 9469

• Print error message if no action is selected in pgcleanup script (thanks to @nsoranzo). Pull Request 9489

• Drop custom json encoding for tool test API endpoint (thanks to @nvk747). Pull Request 9496

• Keep status code on message exception and history import/export fixes Pull Request 9500

• Fixed tool locations (thanks to @astrovsky01). Pull Request 9512

• Update the current_history method in navigates_galaxy.py Pull Request 9514

• Free space before running tests Pull Request 9533

• History import/export fixes Pull Request 9535

• Retry reading tool conf on IOError Pull Request 9537

• Fix citation ids and vue unit test in ToolsView (thanks to @OlegZharkov). Pull Request 9538

• Bump tensorflow from 1.12.2 to 1.15.2 in /lib/galaxy/dependencies (thanks to @galaxyproject). Pull Request 9540

• Incremental indexing for tool shed whoosh indexes Pull Request 9541

• fix documentation for data checksum attribute (thanks to @bernt-matthias). Pull Request 9543

• Prevent exception in toolshed if userid is ‘current’ Pull Request 9548

• Fix setting annotation that contains only integers Pull Request 9549

• Fix docs to update conda from conda-forge (thanks to @nsoranzo). Pull Request 9550

• Disable workflow parameter switch for data inputs, augment form handling Pull Request 9551

• Hide LDDA when creating collection in library interface Pull Request 9561

• Fix indexing script Pull Request 9565

• Assign random password in user manager if none specified (thanks to @nuwang, @nsoranzo). Pull Request 9587, Pull Request 9851

• Fix removal of workflow outputs from workflow nodes in editor Pull Request 9606

• Fix ELIXIR AAI (thanks to @lecorguille). Pull Request 9620

• Fix tensorflow bug (thanks to @anuprulez). Pull Request 9622

• Change session timeout redirect URL to login entrypoint. Pull Request 9626

• Changed CSS overflow property from scroll to hidden to make table scrollable (thanks to @assuntad23). Pull Request 9627

• fix links in Galaxy xsd docs (thanks to @bernt-matthias). Pull Request 9633

• Fix TypeError: OSError object is not subscriptable under Python 3 (thanks to @nsoranzo). Pull Request 9640

• Fix vcf_bgzip mimetype, allow cors, tbi from metadata element Pull Request 9653

• Hide “python: not found” error when Python 2 is not installed (thanks to @nsoranzo). Pull Request 9657

• Fix for time.clock() removal in Python 3.8 (thanks to @nsoranzo). Pull Request 9666

• Fixed an issue in object store integration tests (thanks to @kxk302). Pull Request 9677

• Plugin build hash bugfix Pull Request 9679

• Backport auth process lock fix Pull Request 9681

• Update documentation (thanks to @nsoranzo). Pull Request 9693

• Rename galaxy.util.logging to galaxy.util.custom_logging Pull Request 9705

• Make collection import from library import into correct history Pull Request 9715

• Fix workflow steps import feature in workflow editor Pull Request 9724

• Fix custos credential fetching in python3 Pull Request 9726

• oidc-based authnz provider overhaul; breaking changes for keycloak Pull Request 9730

• Sort Workflow list by last updated time instead of created time Pull Request 9740

• Remove dummy tool sequence (thanks to @anuprulez). Pull Request 9752

• Switch to lxml for xml parsing Pull Request 9759

• Set UTF-8 for parsing tool xml (thanks to @AndreasSko). Pull Request 9760

• inputs config file generation depend on profile (thanks to @bernt-matthias). Pull Request 9776

• Pass through IOError/OSError in tool_shed parse_xml Pull Request 9781

• Maintain whitespace when parsing token from xml Pull Request 9815

• Log exception if reading tool file failed unexpectedly Pull Request 9816

• Drop custom in-process lock class Pull Request 9817

• Update NoseHTML to 0.4.6 for Python 3.8 compatibility (thanks to @nsoranzo). Pull Request 9821

• Tolerate exception in container resolution Pull Request 9824

• Invalidate tool cache when tool modified outside of Galaxy Pull Request 9866

• Custos spec fixes Pull Request 9889

• Fixes for flake8 3.8 and bugbear (thanks to @nsoranzo). Pull Request 9896

• Fix webpack import proxy prefix Pull Request 9898

• Fix extra preference values leaking between requests (thanks to @nuwang). Pull Request 9913

• Rewrite check_html() and Text.count_data_lines() to read a chunk of each line (thanks to @nsoranzo). Pull Request 9920

• Make cookie handling more robust Pull Request 9921

• Enable regex validator to validate lists of values. Pull Request 9925

• oidc: fix dict_keys does not support indexing (thanks to @almahmoud). Pull Request 9931

• Move the --debug flag after the conda subcommand (thanks to @nsoranzo). Pull Request 9932

To stay up to date with Galaxy’s progress, watch our screencasts; visit our community Hub; and follow us on Bluesky, Mastodon, and LinkedIn.

You can always chat with us on Matrix.

Thanks for using Galaxy!

The Galaxy Team