19.09

Enhancements

• Add InteractiveTools. Pull Request 7494, Pull Request 8506, Pull Request 8574, Pull Request 8589

• Implement workflow invocation reports. Pull Request 8543, Pull Request 8664, Pull Request 8683

• VueJS rewrite of Toolshed install interface - with substantial API extensions to enable this. Pull Request 7402, Pull Request 8090, Pull Request 8621, Pull Request 8635, Pull Request 8681, Pull Request 7825 Pull Request 8638, Pull Request 8035, Pull Request 8076, Pull Request 8585, Pull Request 8622

• VueJS rewrite of admin jobs page (thanks to @scrathat). Pull Request 7790

• VueJS rewrite of tools panel (resolves #8262) (thanks to @inkuzmin). Pull Request 8368

• VueJS rewrite of dataset job properties tables - with API extensions to enable this. Pull Request 8022, Pull Request 8044

• VueJS rewrite of Page management components - with API extensions to enable this. Pull Request 8047

• New visualization - free style image annotation (thanks to @anuprulez). Pull Request 6891

• New visualization - OpenLayers maps. (thanks to @anuprulez). Pull Request 7888

• New visualization - hyphy-vision (thanks to @veg). Pull Request 8227

• Implement frameworks for datatype validation and tool-based QC. Pull Request 7913

• New Pulsar runner for Kubernetes allowing file staging with Kubernetes. Pull Request 7835, Pull Request 8195

• Implement option to delete or preserve Kubernetes pods after completion (thanks to @nuwang). Pull Request 8091

• Implement walltime_limit option for the Kuberentes jobs (thanks to @telukir). Pull Request 8311

• Implement UI for Authnz API. Pull Request 7665

• Extend dataset search and add search API endpoint (thanks to @mvdbeek). Pull Request 7745

• Add Matthias Bernt to the committers group. Pull Request 8446

• Add filtering and sorting for folders in data libraries (with help from @bwlang). Pull Request 8281, Pull Request 8287, Pull Request 8277

• Implement xref tag on the tool structure (with visualization) (thanks to @ValentinMarcon). Pull Request 7905

• Substantial progress toward refactoring and decomposing Galaxy into properly installable packages. (with help from @dglaetzer, @nsoranzo and @gmauro). Pull Request 921, Pull Request 7992, Pull Request 8114, Pull Request 8234, Pull Request 8241, Pull Request 8266, Pull Request 8276, Pull Request 8284, Pull Request 8293, Pull Request 8306, Pull Request 8305, Pull Request 8307, Pull Request 8308, Pull Request 7971, Pull Request 8051, Pull Request 8062, Pull Request 8280, Pull Request 8310, Pull Request 8302, Pull Request 8298, Pull Request 8098, Pull Request 8208, Pull Request 8235, Pull Request 8365, Pull Request 8357, Pull Request 8342, Pull Request 8345, Pull Request 8431, Pull Request 8269, Pull Request 8314, Pull Request 8687, Pull Request 8702, Pull Request 8610, Pull Request 8315

• Enable dynamic reloading of select config options. Pull Request 8475

• Improved architecture for Galaxy’s configuration object Pull Request 8502, Pull Request 8526, Pull Request 8462, Pull Request 8433, Pull Request 8545

• Allow YAML/JSON job configuration - including embedding right in Galaxy config, validation, better defaults, and better section naming. Pull Request 7854, Pull Request 8616, Pull Request 7882 Pull Request 8609

• Allow declaring dependency resolvers right in Galaxy config, validation, and serialization. Pull Request 8118, Pull Request 8152

• Add external display application plugin for iCn3D. Pull Request 8347

• Enable external display application plugin parameters to declare allow_cors="true". Pull Request 8539

• Add XML Schema Definition (XSD) for Galaxy External Dispaly Applications (GEDAs). Pull Request 8461

• Adding Shapefile composite datatype (thanks to @Alanamosse). Pull Request 7615

• Add mcool datatype and extend h5 tool output testing (thanks to @msauria). Pull Request 7993

• Add hyphy_results.json datatype (thanks to @veg). Pull Request 8186

• Add pgp and osw datatypes as used in OpenSWATH experiments (thanks to @bgruening). Pull Request 8458

• Add ckpt datatype (thanks to @astrovsky01). Pull Request 8086

• Add dada2 datatypes (thanks to @bernt-matthias). Pull Request 7895

• Add rDock active site datatype (thanks to @bgruening). Pull Request 8727

• Add Kallistobus datatypes (thanks to @astrovsky01). Pull Request 8229

• Sniff fastqsanger more aggressively (thanks to @wm75). Pull Request 8322

• Various collection operation tool improvements (thanks to @mvdbeek). Pull Request 8038

• Enhance unzip/zip collection collection operation to annotate output format properly (thanks to @bernt-matthias). Pull Request 8239

• Implement option to strip whitespace from tool environment_variables. Pull Request 8296

• Add tool output test assertion for n_lines (thanks to @bgruening). Pull Request 8440

• Add zoom reset feature for workflows (thanks to @juleengraham). Pull Request 8301

• Fix tool run output to display more data. Pull Request 8018

• Track basename at the time of creation for datasets. Pull Request 7670

• Remove GFF headers during conversion (backported to 19.05). Pull Request 7808

• Enable mulled containers by default for 19.09. Pull Request 8569

• Enhanced GitLab integration (thanks to @selten). Pull Request 8455, Issue #8183

• Make admin users grid filter on purged Pull Request 8303

• Add links to admin entry page. Pull Request 8351

• Add ELIXIR OIDC backend, upgrade social-auth-core requirement version (thanks to @kidanem). Pull Request 7852

• Run futurize for python2 cheetah templates, fix scope issue for Python 3 comprehensions (thanks to @mvdbeek). Pull Request 7867

• Improve look and feel for data libraries (thanks to @inkuzmin). Pull Request 7878

• Raise detailed exception for broken citation (thanks to @nsoranzo). Pull Request 7891

• Remove file attribute of bibtex citation (thanks to @nsoranzo). Pull Request 7911

• Use transparent background in default welcome page. Pull Request 7929

• Cellxgene GIE - use API to get datasets. Pull Request 7945

• Change “username” to “public name” throughout the user interface. Pull Request 8009

• History menu (cog) UI updates. Pull Request 8024

• Delegate config file watching to single handler (thanks to @mvdbeek). Pull Request 8078

• Allow admins to disable the visualizations tab in the masthead. Pull Request 8450

• Update all Python dependencies (thanks to @nsoranzo). Pull Request 8120, Pull Request 8126

• Enable authorization to Google Cloud Platform Pull Request 8123

• Update requirements and use pyuwsgi (thanks to @mvdbeek). Pull Request 8489

• Update cloudauthz dependency to v0.4.0. Pull Request 8155

• Improve accessibility of certain UI elements. Pull Request 8133

• Round-off the time taken for CPU execution (thanks to @janani-prasad). Pull Request 8144

• Add API to tag library datasets (thanks to @abretaud). Pull Request 8530

• Improve API docs. Pull Request 8541

• Update repository issue handling text in docs. Pull Request 8542

• Improved tool parameter error reporting (thanks to @bernt-matthias). Pull Request 8163

• Configuration API cleanup. Pull Request 8206

• New regular expression filter, and efficient sort filter (thanks to @bernt-matthias). Pull Request 8211

• Data managers - allow removal of entries (thanks to @selten). Pull Request 8250, Pull Request 8503

• Adjust minimum tag length. Pull Request 8256

• Some improvements to tool menu styles. Pull Request 8285

• Make styleguide work for VueJS components. Pull Request 8291

• Library date propagate (thanks to @bwlang). Pull Request 8294

• Skip database migrations on entirely clean new databases (thanks to @mvdbeek). Pull Request 8300

• Implement undelete and purge API for users (thanks to @cganote). Pull Request 8309

• Use lowercase true/false in config_schema.yml (thanks to @nsoranzo). Pull Request 8319

• Check correct permission for allowing a user to modify lib folder (thanks to @scholtalbers). Pull Request 8326

• Issue`#8105 <https://github.com/galaxyproject/galaxy/issues/8105>`__: Make tool favourite query a toggle (thanks to @janani-prasad). Pull Request 8331

• Update anchor tags and enhance navigation cues for screen readers Pull Request 8334

• Better config isolation for integration tests. Pull Request 8355

• catch exceptions occuring in expression validators (thanks to @bernt-matthias). Pull Request 8359

• Update RST files to reflect code structure changes. Pull Request 8363

• make cleanup script use logging instead of print (thanks to @bernt-matthias). Pull Request 8394

• Mention IGB in the docs Pull Request 8404

• Automatically replace dashes by underscores for argument attribute on tool parameters. (thanks to @bernt-matthias). Pull Request 8409

• Add basic unit test for lint_inputs (thanks to @mvdbeek). Pull Request 8410

• Various improvements designed to optimize ToolBox loading time. Pull Request 8421, Pull Request 8422, Pull Request 8426

• Fix authentication provider exception not logged (thanks to @BioinformaticsPlatformWIV-ISP). Pull Request 8424

• Optimize unrestricted library dataset permission checks. Pull Request 8430

• Change ‘a when tag has not been defined for’ level to debug (thanks to @mvdbeek). Pull Request 8439

• Fix database migrations for MySQL but mention it is unsupported in the docs (thanks to @nsoranzo). Pull Request 8407, Pull Request 8640, Pull Request 8473

• Auto convert python 2 code files using 2to3 (thanks to @mvdbeek). Pull Request 8499

• Update sass version to support node 12. Pull Request 8507

• Integration test for outputs_to_working_directory with Pulsar. Pull Request 8335

• Fix ‘postgres’ dialect name has been renamed to ‘postgresql’ deprecat… (thanks to @mvdbeek). Pull Request 7804

• Use markupsafe.escape instead of cgi.escape (thanks to @mvdbeek). Pull Request 7837

• Use getfullargspec on Python 3 (thanks to @mvdbeek). Pull Request 7838

• Align popup/dropdown menu templating. Pull Request 7871

• Change keyboard input for floats and switch to ES6 features Pull Request 7927

• Tool output_checker - fix superficial condition (thanks to @bernt-matthias). Pull Request 7980

• Enhance galaxy.js and entry folder for ES6. Pull Request 8025, Pull Request 8034

• Switch var to let and const for VueJS components Pull Request 8028, Pull Request 8036

• es6 upgrades in entry folder

• Remove unresolved function causing console error. Pull Request 8101

• Swap out galaxy.util.odict for core python collections.OrderedDict objects. Pull Request 8104

• Enhance make config-rebuild to update config/docs. Pull Request 8220

• Refactor setup_user_get_key method to return the created test user and its API key. Pull Request 8233

• Fix various small UI issues. Pull Request 8275

• Replace in-python quota calc with a SQL query (thanks to @erasche). Pull Request 8452

• Remove deprecated scripts (thanks to @erasche). Pull Request 8465

• Fix markup in ToolEntryPoints, format client Pull Request 8549

• Allow specifying maxseconds in verify_tool (thanks to @mvdbeek). Pull Request 7828

• Call tool_script.sh with shell (thanks to @mvdbeek). Pull Request 8077

• Drop “Manage” from admin workflow Invocation link. Pull Request 8693

• Record Galaxy version a job is created with. Pull Request 8414

• Git ignore files in database/shed_tools directory (thanks to @ashvark). Pull Request 8268

• Added unit testing section to the client README Pull Request 7890

• Update Sphinx RST files for lib/. Pull Request 8270

• Allow re-populating objectstore from to_dict form. Pull Request 8213

• Testing framework fixes (thanks to @nsoranzo). Pull Request 8147

• Extend ObjectStore integration tests. Pull Request 8042

• Fix link formats for mulled_containers.rst, minor rewording. Pull Request 8075

• Add missing files from galaxy-lib (thanks to @nsoranzo). Pull Request 8113

Fixes

• Add file extensions when downloading from data libraries (thanks to @selten). Pull Request 8456

• Fix tool assert_command tag test not executed (thanks to @bernt-matthias). Pull Request 7730

• Fix inconsistent defaults for cleanup scripts - fixing #975 (thanks to @erasche). Pull Request 8470

• Allow missing H5 groups in Loom files (thanks to @nsoranzo). Pull Request 8560

• Document undocumented workflow_schedulers_config_file (thanks to @erasche). Pull Request 8437

• Use bootstrap classes to fix form css issue. Pull Request 8371

• Fix Pulsar job files API to allow transferring directory content. Pull Request 8124

• Fix gops_intersect_1 importing galaxy (thanks to @mvdbeek). Pull Request 7805

• Quit gsummary script with non-zero exit code upon error (thanks to @mvdbeek). Pull Request 8214

• Sort Collection - add missing when tags (thanks to @bernt-matthias). Pull Request 8366

• Remove gops from venv list (thanks to @bgruening). Pull Request 8492

• Don’t let parameter_input steps be workflow outputs (thanks to @mvdbeek). Pull Request 7799

• Fix column assign bug (thanks to @mvdbeek). Pull Request 8023

• Quick hack to allow delayed invocations followed by step that consumes an input parameter (thanks to @mvdbeek). Pull Request 8249

• Expression tool fix - include workdir path when reading env var from file (thanks to @mvdbeek). Pull Request 8253

• Fix workflow tags lost on save (thanks to @mvdbeek). Pull Request 8398

• Fix failure in resubmitting job more than once (thanks to @pcm32). Pull Request 7832

• Add file missing from #7694 Pull Request 7873

• Strip unicode control characters from TEXT before entering the database (thanks to @mvdbeek). Pull Request 7889

• Fix Kubernetes test for job I/O separation. Pull Request 7892

• In output_checker, fixed wrong comparison. (thanks to @bernt-matthias). Pull Request 7966

• Fix typo grap -> grab. Pull Request 8061

• Fix wording for pod retries while maintaining backward compatibility (thanks to @nuwang). Pull Request 8082

• Kubernetes / CLI runner integration test fixes (thanks to @mvdbeek). Pull Request 8154

• Fail job if queueing job fails (thanks to @mvdbeek). Pull Request 8161

• Fix job wrapper fail for queued jobs (thanks to @mvdbeek). Pull Request 8179

• Fix failure to fail job when tool is unavailable (thanks to @mvdbeek). Pull Request 8181

• Fix external_metadata_set_successfully when metadata file doesn’t exist (thanks to @mvdbeek). Pull Request 8182

• Fix portable metadata datatypes config. Pull Request 8207

• Fix cancelling jobs with a message from the admin panel (thanks to @mvdbeek). Pull Request 8231

• Fix Kubernetes runner on Python 3 (thanks to @mvdbeek). Pull Request 8321

• Fix univa runner check for drmaa python module (thanks to @bernt-matthias). Pull Request 8385

• Fix memory statement for drmaa (SGE/UNIVA) (thanks to @bernt-matthias). Pull Request 8451

• Fix setting metadata externally (thanks to @mvdbeek). Pull Request 8497

• Fix Pulsar version checking logic. Pull Request 8524

• Hack to fix Pulsar for containerized jobs. Pull Request 8533

• Improve and refactor process group handling in the local job runners (thanks to @nsoranzo). Pull Request 8538

• Fix container property parsing for Singularity. Pull Request 8614

• Data Manager interface - fix to use standard card header styles without extra markup. Pull Request 7811

• Allow looking up tool section using label (thanks to @mvdbeek). Pull Request 8225

• Backport tool shed and data manager fixes (thanks to @mvdbeek).

• Create repository clone path if it doesn’t exist. Pull Request 7770

• Remove unused requirejs imports from templates. Pull Request 7810

• Small bugfix to GRT for ancient data, deprecated params table (thanks to @erasche). Pull Request 7830

• Fix help text in ftp-uploader dialog. Pull Request 7908

• Update instructions on conda upgrade (thanks to @nsoranzo). Pull Request 7918

• Remove view button, fix library flag in data dialog. Pull Request 7935

• Fix non-eager object store with nested object stores (thanks to @mvdbeek). Pull Request 7939

• Fix extra newlines in peek code. Pull Request 7952

• Fix populators.py workflow testing for non-text inputs. Pull Request 7953

• Only show colon when a port exists in URL. Pull Request 7985

• Remove href from a tag to prevent backbone page reload and change sometimes. Pull Request 7991

• Various datatype I/O fixes (thanks to @mvdbeek). Pull Request 7995

• Don’t close popup menu if header item is clicked. Pull Request 8001

• Refactor comments in sample object stores to remove nested comments. (thanks to @simpsonw). Pull Request 8016

• Fix Selenium DatasetPopulator for more interesting uploads. Pull Request 8046

• Fix stream_to_open_named_file for non-utf8 text (backported to 19.05) (thanks to @mvdbeek). Pull Request 8050

• Small testing Dockerfile fixes (thanks to @nsoranzo). Pull Request 8094

• Report a lint error if a citation is empty (thanks to @nsoranzo). Pull Request 8112

• Ensure DEFAULT_TOOL_TEST_WAIT is an integer (thanks to @mvdbeek). Pull Request 8136

• Fix for tool test data in sub directories (thanks to @bernt-matthias). Pull Request 8162

• Increase default font size to 0.85rem. Unify default welcome page fonts. Pull Request 8168

• Use IdEncodingHelper to encode/decode user ids (thanks to @nsoranzo). Pull Request 8201

• Fix prefix usage in history copy dialog. Pull Request 8205

• Fix UnicodeEncodeError when verifying test results (thanks to @mvdbeek). Pull Request 8221

• Fix migration if WorkflowStepConnection index not present (thanks to @mvdbeek). Pull Request 8247

• Fix regex rule count (thanks to @mvdbeek). Pull Request 8254

• Fix and unify search bar handling. Pull Request 8257

• Typo fix comunity -> community (thanks to @yochannah). Pull Request 8267

• Fix typo in run_tests.sh help text. Pull Request 8289

• Don’t show the pip progress bar when installing under CI Pull Request 8290

• Use temporary file for sniff tests (thanks to @nsoranzo). Pull Request 8304

• Adjustments for panel spacing changes Pull Request 8316

• Restore docs building (thanks to @nsoranzo). Pull Request 8323

• Updates interactor to not ignore numbers in user email address Pull Request 8324

• Test removing uwsgi weird python package from circleci Pull Request 8337

• Fix use of PIP_PROGRESS_BAR env variable (thanks to @nsoranzo). Pull Request 8343

• Startup fixes (thanks to @nsoranzo). Pull Request 8346

• Fix default involucro path (thanks to @mvdbeek). Pull Request 8360

• Fix autodetection of zip file format (including xlsx) (thanks to @wm75). Pull Request 8362

• Hide references card if array is empty. Pull Request 8373

• Fix exception thrown with invalid session key (thanks to @mvdbeek). Pull Request 8377

• Better isolation of embedded Pulsar tests. Pull Request 8386

• Fix absolutely positioned elements and children from displaying off-screen. Pull Request 8466

• Don’t cleanup integration tests twice. Pull Request 8468

• Try separate amqp connection from database for integration testing. Pull Request 8469

• Fix for NodeJS executable being called nodejs in Conda. (thanks to @bgruening). Pull Request 8474

• Fix startup checks when using a Conda env (thanks to @nsoranzo). Pull Request 8477

• Use default backdrop color for modals. Pull Request 8479

• Fix shed_tool_conf isolated tool (not in section) removal Pull Request 8485

• Do not warn for not configured user_preferences_extra_conf_path (thanks to @nsoranzo). Pull Request 8494

• Use default background for scratchbook. Pull Request 8527

• Explicitly declare expression as sql.text. Pull Request 8544

• Update calculate_or_set_disk_usage to support sqlite (thanks to @qiagu). Pull Request 8552

• Fix menu option classes (and thus event bindings) for histories in th… Pull Request 8575

• Modify tool panel section text color, size and margin to improve readability. Pull Request 8579

• Replaces tabs with spaces in apache.md (thanks to @nsoranzo). Pull Request 8590

• Capture missing log messages Pull Request 8605

• Regenerate ts search indexes in tmp directory. Pull Request 8611

• Only schedule file watching if an observer exists (thanks to @mvdbeek). Pull Request 8629

• Fix issue with xml comment handling on python 3 (thanks to @mvdbeek). Pull Request 8643

• Fix tool JSON wrapper fail to wrap paths (thanks to @qiagu). Pull Request 8647

• Bugfix - kill testing container when test script is killed. Pull Request 8679

• Build dependency manager only for galaxy app (thanks to @nsoranzo). Pull Request 8696