19.05

Enhancements

• Implement the ability to favorite tools. Pull Request 7209

• Add data dialog option to tool form data selector. Pull Request 7553, Pull Request 7460

• Many workflow editor connection fixes and enhancements making them more correct, accessible, and transparent (with huge thanks to @mvdbeek). Pull Request 7466, Pull Request 7435, Pull Request 7972, Pull Request 7979, Pull Request 7642, Pull Request 7989, Pull Request 7254

• Add a Galaxy IE for cellxgene. Pull Request 7268

• Implement expression tools and non-data tool outputs (with thanks to @mvdbeek). Pull Request 7556, Pull Request 7796, Pull Request 7797, Pull Request 7944

• Add Galaxy cloud runner support (thanks to @nuwang). Pull Request 7226

• More robust task messaging (thanks to @mvdbeek). Pull Request 7548

• Allow implicit conversion for input collection parameters (thanks to @mvdbeek). Pull Request 7683

• Allow import of “Format 2” workflows by default. Pull Request 7659

• Decompose Galaxy into Python libraries in-tree. Pull Request 7524

• Rewrite and generalize History import and export - dozens of fixes, support for collections, Python library support, etc.. Pull Request 7367, Pull Request 7214, Pull Request 7366, Pull Request 7369, Pull Request 7370, Pull Request 7358, Pull Request 7505, Pull Request 7507, Pull Request 7363, Pull Request 7193, Pull Request 7192, Pull Request 7537, Pull Request 7540, Pull Request 7684, Pull Request 7704

• User facing release notes (thanks to @hexylena). Pull Request 7527

• Implement dataset source and hash tracking. Pull Request 7487, Pull Request 4659, Pull Request 7549

• Custos integration for AuthNZ (thanks to @machristie). Pull Request 7195

• Flush out API support for AuthNZ and CloudAuthz Pull Request 7598, Pull Request 7609, Pull Request 7592, Pull Request 7597, Pull Request 7639

• Update to PSA 3.1 and support login with Globus identity. Pull Request 7463

• Swap from ‘slug.js’ to ‘slugify.js’ - saving 10 MB in client bundles. Pull Request 7538

• Update all client dependencies - including to Backbone 1.4, Bootstrap 4.2, Vue 2.6. Pull Request 7544, Pull Request 7637

• Allow admins to import dynamic JSON-based tools. Pull Request 7545

• Implement environment modules mapping files (thanks to @FredericBGA). Pull Request 7398

• Allow tool testing of multiple files within a zip output (thanks to @thermokarst). Pull Request 7400

• Add datatype converters between tabular and CSV. Pull Request 7246

• Add Azure and GCP support for ObjectStore Cloud backend. Pull Request 7272

• Add a validator that ensure metadata value is on a range. (thanks to @fmareuil). Pull Request 7288

• Fix post job actions for “Database Operation” tools, implement change_datatype for collection output (thanks to @mvdbeek). Pull Request 7480

• Implement and enforce ESLint linting of the entire client code base. Pull Request 7202, Pull Request 7860

• Multiple enhancements to the LSF cli-plugin job runner - support out of memory handling, project specification, and Spectrum LSF. (thanks to @pcm32 and @selten). Pull Request 6866, Pull Request 7581, Pull Request 7486

• Drop support for Python 3.4 and officially support Python 3.5 or greater. (thanks to @nsoranzo). Pull Request 7291

• Batch bulk hide and delete operations when multiple selected. Pull Request 5495

• Implement client initialization pipeline. Pull Request 7117

• Allow storing objects in the object store by UUID. Pull Request 7154, Pull Request 7650

• Replace login and registration mako templates with VueJS components. Pull Request 6621, Pull Request 7047, Pull Request 7329, Pull Request 7442, Pull Request 7756, Pull Request 7347

• Track messages generated tool stdio tags in the database instead of just appending them to tool standard error. Structured UI in reporting also. (with help from @bernt-matthias) Pull Request 7095

• Track job script stdout and stderr separately from the tool’s stdout and stderr in the database. Pull Request 7095

• Support loading/using tools that are not in the install database (with help from @mvdbeek). Pull Request 7316, Pull Request 8021

• Git providers error reporting plugins (thanks to @selten). Pull Request 7485

• Rename _future_expose_api to expose_api and expose_api to legacy_expose_api (thanks to @mvdbeek). Pull Request 7743

• Add mtx datatype (thanks to @bebatut and @nsoranzo). Pull Request 7569, Pull Request 7702, Pull Request 7717

• Add geojson datatype (thanks to @bgruening). Pull Request 7773

• Add shapefile (shp) datatype (thanks to @bgruening). Pull Request 7819

• Add imgt.json datatype for IMGT immune system libraries (thanks to @jj-umn). Pull Request 7587

• Add microarrays data types gpr and gal (thanks to @bensellak). Pull Request 7457

• Add spaln database type (thanks to @pvanheus). Pull Request 7718

• Redefine cel datatype (thanks to @bensellak). Pull Request 7514

• Run most tests formerly ran on Travis on CircleCI (with help from @nsoranzo). Pull Request 7424, Pull Request 7573, Pull Request 7451, Pull Request 7207, Pull Request 7539, Pull Request 7574

• Toolshed API enhancements. Pull Request 6652

• Kubernetes job runner integration test and enhancements (thanks to @mvdbeek). Pull Request 6958

• Implement sniffing of datatypes for library datasets. Pull Request 7379

• Rewrite tool output metadata collection - now portable and more correct, tested, and documented. Pull Request 7470, Pull Request 7459, Pull Request 7156 Pull Request 7483, Pull Request 7158, Pull Request 7213, Pull Request 7694, Pull Request 7596, Pull Request 7186, Pull Request 7471

• Add python to Count1 tool requirements (thanks to @mvdbeek). Pull Request 7185

• Fixes and enhancements for the grouping1 tool - including not removing empty cells at the begin/start of lines (thanks to @bernt-matthias and @mvdbeek). Pull Request 7205, Pull Request 7844

• Workflow editor accessibility fixes. Pull Request 7750

• Python dependency updates and documentation fixes (thanks to @nsoranzo). Pull Request 7215, Pull Request 7247, Pull Request 7285, Pull Request 7740

• Remove requirement pin on specific Sphinx version (thanks to @nsoranzo). Pull Request 7365

• Drop conditional pygmentz requirement (autoinstalled as necessary), minor formatting in conditional-reqs Pull Request 7219

• Fix spacing between checkbox and buttons in history sharing view. Pull Request 7222

• Use regular font-size and styling in uploader. Pull Request 7232

• Remove underline from history datasets. Pull Request 7235

• Add headless testing option for Selenium tests. Pull Request 7287

• Build tool XML schema docs using sphinx_markdown_tables (thanks to @nsoranzo). Pull Request 7313

• Refactor dataset controller to reuse input validation checks between get/set. Pull Request 7322

• Add web server documentation notes about Range support (thanks to @pvanheus and @nsoranzo). Pull Request 7323, Pull Request 7334

• Fall back to binary comparison if BAM conversion fails during tool tests (thanks to @bernt-matthias). Pull Request 7342

• Update area issue/PR labels (thanks to @nsoranzo). Pull Request 7344

• Remove deprecated collect_outputs_from configuration option. Pull Request 7144

• Remove ToolsController._rerun_tool() API method (thanks to @nsoranzo). Pull Request 7345

• Refactor item_attrs.py for reuse without mixin inheritance. Pull Request 7346

• Add --force_path_paste argument to tool test verify script. Pull Request 7356

• Separate client node version requirement into individual dotfile for reuse. Pull Request 7357

• Improve tool testing for counting datasets in paired collections (thanks to @bernt-matthias). Pull Request 7359

• Do not default to DEV_WHEELS=1 if no git is found (thanks to @jdavcs). Pull Request 7371

• Restore gray scale difference between panel vs portlet background color for contrast. Pull Request 7387

• Specify exceptions more precisely in user API controller. Pull Request 7397

• Automatically set cookie_path using url_for(). Pull Request 7404

• Allow configuration location of user preferences extra configuration (thanks to @hexylena). Pull Request 7428

• Add hover styling to target tool in tool panel (thanks to @hexylena). Pull Request 7437

• Update Docker testing image (thanks to @mvdbeek). Pull Request 7439, Pull Request 7187

• Remove unused code (thanks to @nsoranzo). Pull Request 7445

• Note cannot import name _remove_dead_weakref error and solution (thanks to @pvanheus). Pull Request 7461

• Add option to pass in individual globs to client unit test watcher Pull Request 7477

• Update of tours overview according to #5860 (thanks to @selten). Pull Request 7478

• Add new metadata and data table tool validators (thanks to @bernt-matthias). Pull Request 7500

• Updated documentation in admin/scaling.md for Systemd (thanks to @ooobik). Pull Request 7515

• Replace backbone tagging views with VueJs components. Pull Request 7516

• Eliminate import dependency of datatypes on galaxy.web (url_for). Pull Request 7521

• Static plugin staging as a part of client build instead of startup. Pull Request 7532

• Implement dockerized variant of update.sh for project Python dependencies. Pull Request 7546

• Refactor XSD output elements to share common output attributes. Pull Request 7555

• Improve linking between job conf handler assignment and documentation (thanks to @hexylena). Pull Request 7558

• Refactor galaxy.web.security into galaxy.security.idencoding. Pull Request 7560

• Move the ‘create new history’ out of the history context menu Pull Request 7565, Pull Request 7606

• Improve workflow cog menu user interface. Pull Request 7594, Pull Request 7858

• Make galaxy.model.dataset_collections more usable outside manager context. Pull Request 7595

• Webpack bundle overhaul and initial dynamic loading. Pull Request 7605

• Don’t un-hide mapped-over outputs when job fails (thanks to @mvdbeek). Pull Request 7630

• Continue gracefully in set_meta if unable to count number of dataset lines. Pull Request 7641

• Implement functional dict-ifiable tool outputs and error detection. Pull Request 7651

• Refactor IntegrationTestCase to allow integration test functions. Pull Request 7657

• Drop .pyc files before attempting to migrate (thanks to @mvdbeek). Pull Request 7677

• Add docs to OIDC backends configuration file. Pull Request 7691

• Update CITATION to latest Galaxy update paper (thanks to @nsoranzo). Pull Request 7698

• Update pull request procedures to exempt notes, changelogs, and packages from mandatory review. Pull Request 7706

• Update procedures to relax the vote requirement for fixing bugs in releases Pull Request 7707

• Improve error message if GFF file is missing attribute col (thanks to @nsoranzo). Pull Request 7708

• Add hostname to task message queues (thanks to @mvdbeek). Pull Request 7712

• OIDC login buttons based on configuration (thanks to @machristie). Pull Request 7720

• Enable the selection of library datasets in the tool form Pull Request 7746

• Replace enable_beta_export_format2_default with default_workflow_export_format (thanks to @nsoranzo). Pull Request 7748

• Report a couple common errors in dataset/job report. Pull Request 7755

• Add singularity to valid types, allow customization of test history name (thanks to @mvdbeek). Pull Request 7760

• Update parameter_input type when changing parameter type (thanks to @mvdbeek). Pull Request 7780

• Don’t clean up jobs when running tests if $GALAXY_TEST_NO_CLEANUP is set Pull Request 7798

• Add option to wait for the database to become available (thanks to @jdavcs). Pull Request 7827

• Remove recursion from expand_nested_tokens() (thanks to @nsoranzo). Pull Request 7834

• Allow composite upload if all non-optional files are selected (thanks to @mvdbeek). Pull Request 7872

• Cleanup integration test configuration code. Pull Request 7874

• Add “Reset” button to composite upload interface (thanks to @mvdbeek). Pull Request 7875

• Add local namespaced resolver to default container resolvers (thanks to @mvdbeek). Pull Request 7896

• Use node-watch instead of fs.watch to watch for db changes (thanks to @mvdbeek). Pull Request 7967

• Remove reset handler from ruler input element. Pull Request 6321

• Declutter the history ‘cog’ menu. Pull Request 6437

• Remove deprecated OpenID features. Pull Request 7028, Pull Request 7395

• Restores OpenID post authentication protocols. Pull Request 7676

• Replace custom UI colors with equivalent theme colors, consolidate color handling. Pull Request 7203

• Modernize library toolbar client code. Pull Request 7234

• Large refactoring of model migrations (thanks to @nsoranzo). Pull Request 7621, Pull Request 7248, Pull Request 7737, Pull Request 7695

• Implement a general purpose tagging component in the Galaxy client code. Pull Request 7270

• Define doctype method on XMLParser target (thanks to @mvdbeek). Pull Request 7303

• Refactor galaxy.util.handlers -> galaxy.web.stack.handlers. Pull Request 7339

• Move create_history_template out of galaxy.util. Pull Request 7522

• Refactor galaxy.web.form_builder into galaxy.util.form_builder. Pull Request 7529

• Drop unused install keyword and load TS workflows with utf-8 encoding (thanks to @mvdbeek). Pull Request 7423

• Remove biostar integrations. (with help from @nsoranzo). Pull Request 7443, Pull Request 7510

• Refactor TagManager (back) to TagHandler. Pull Request 7530

• Refactor output_collect.py toward a database session-less modality. Pull Request 7541

• Drop D3 from publicly provided libs Pull Request 7542

• Fix dependency cycle between galaxy.security and galaxy.model. Pull Request 7554

• Refactor galaxy.dataset_collections into galaxy.model.dataset_collections. Pull Request 7588

• Toastr debowerization and refactoring. Pull Request 7640

• Tweak popup-menu.js error handling. Pull Request 7643

• Update Conda version that new Galaxies will install Pull Request 8048

Fixes

• Explicitly specify system user under which Kubernetes jobs should run. Pull Request 8103

• Ensure Kubernetes job querries filter the same namespace as defined in the job destination. Pull Request 8092

• Fix client styleguide asset handling. Pull Request 7664

• Handle subworkflows in view workflow mako (thanks to @mvdbeek). Pull Request 7276

• Fix removing tags from a dataset previously only the first tag could be removed (thanks to @gtrack). Pull Request 7674, Pull Request 7680

• Fix displaying and editing SelectTagParameters in workflow editor (thanks to @mvdbeek). Pull Request 7534

• Fix workflow output rectification. Pull Request 7101

• Interactive Environment Fixes Pull Request 7917

• Make job script actually return tool exit code (thanks to @bernt-matthias). Pull Request 7147

• Fix client galaxy object instance access in visualizations. Pull Request 7198

• Assorted component bugfixes. Pull Request 7200

• Update IE proxy dependencies. Pull Request 7208

• Cleanup code for univa job runner (thanks to @bernt-matthias). Pull Request 7210

• Verify and update .venv’s node version. Pull Request 7220

• Update use_interactive default in docs (thanks to @nsoranzo). Pull Request 7221

• Fix hotdata method access. Pull Request 7256

• Fix saving imported subworkflows (thanks to @mvdbeek). Pull Request 7262

• Fix “occured” typo everywhere (thanks to @nsoranzo). Pull Request 7266

• Fix flakey history import metadata test (thanks to @mvdbeek). Pull Request 7279

• Do not skip the client build in dockerized selenium tests (thanks to @nsoranzo). Pull Request 7280

• Fixes for the cluster documentation (thanks to @nsoranzo). Pull Request 7281

• Fix ObjectNotFound exception when exporting an history (thanks to @abretaud). Pull Request 7286

• Do not remove tools from other installed revisions when uninstalling TS repo (thanks to @nsoranzo). Pull Request 7300

• Remove unused keyword in resend email helper. Pull Request 7315

• Fix popupmenu creation. Pull Request 7318

• Fix trackster toolbox filter for logged-in users (thanks to @nsoranzo). Pull Request 7319

• Remove overlapping tooltip from dropdown button in library toolbar. Pull Request 7321

• UI fixes for libraries. Pull Request 7337

• Fix test_import_metadata_regeneration API test (thanks to @nsoranzo). Pull Request 7340

• Possible fix for fasta file never getting filesize (thanks to @Slugger70). Pull Request 7374

• Fix upload button style. Pull Request 7381

• Fix link doc syntax (thanks to @galaxyproject). Pull Request 7388

• Update application documentation (thanks to @nsoranzo). Pull Request 7391

• Fix pie chart and portlet height in flexboxes. Pull Request 7396

• Fix LocalShellRunner (thanks to @mvdbeek). Pull Request 7438

• Use a lock on cleanup, expire_tool and cache_tool (thanks to @mvdbeek). Pull Request 7446

• Fix loading workflows with steps without default label (thanks to @mvdbeek). Pull Request 7448

• Remove broken configuration option tool_submission_burst (thanks to @mvdbeek). Pull Request 7449

• Restore get_filename to galaxy.tools.verify:verify() Pull Request 7452

• Fix documentation for statsd and uwsgi (thanks to @abretaud). Pull Request 7453

• Fix hidden parameter use in workflows (thanks to @mvdbeek). Pull Request 7458

• Fix syntax of XSD tool schema. Pull Request 7462

• Fix workflow building typo (thanks to @mvdbeek). Pull Request 7472

• Correct virtualenv download with no wget/curl (thanks to @hmenager). Pull Request 7476

• Use a relative output.publicPath to avoid 404 errors when serving with a url prefix (thanks to @abretaud). Pull Request 7481

• Restrict loading of JSON files on the server via the workflow API to Galaxy admins (thanks to @nsoranzo). Pull Request 7499

• Fix warning message formatting in sharing controller (thanks to @mvdbeek). Pull Request 7504

• Make sure temp_output_dir path is absolute (thanks to @nsoranzo). Pull Request 7517

• Create directory before copying file there (thanks to @pvanheus). Pull Request 7519

• Fix routing when serving Galaxy at a prefix. Pull Request 7523

• Small object store fixes. Pull Request 7536

• Fix entering workflow parameter in workflow run form (thanks to @mvdbeek). Pull Request 7543

• Extend reload test case logging (thanks to @mvdbeek). Pull Request 7547

• Correct db-self handler assignment method (thanks to @hexylena). Pull Request 7561

• Fix database config doctests when env vars are in use for dburi Pull Request 7562

• Fix help warnings style. Pull Request 7566

• Fix to run bootstrap_history.py inside Galaxy’s virtualenv (thanks to @nsoranzo). Pull Request 7584

• Update cloud imports to adhere with cloudbridge interface changes. Pull Request 7586

• Fix RStudio GIE (thanks to @hexylena). Pull Request 7590, Pull Request 7925

• Fix styleguide. Pull Request 7602

• Fix history options menu href navigation Pull Request 7612

• Minor cli and metadata fixes (thanks to @mvdbeek). Pull Request 7614

• Improve string escaping and changeset validation on toolshed. Pull Request 7616

• Delay workflow step execution for discovered & mapped-over input (thanks to @mvdbeek). Pull Request 7633

• Fix float to int casting (thanks to @mvdbeek). Pull Request 7634

• Adjust test_run_with_numeric_input_connection to extra line (thanks to @mvdbeek). Pull Request 7645

• Fix incorrect LDDAs being used as collection members. Pull Request 7646

• Fix GenomeSpace tools. Pull Request 7647, Pull Request 7697

• Bugfix for input selection with tag representation in ui-select-default. Pull Request 7658

• Drop now unused import endpoint from workflow controller. Pull Request 7660

• Fix for trans.redirect not setting cookies/headers. Pull Request 7663

• Fix shutdown for Pulsar MQ Runner. Pull Request 7667

• Fix “Run workflow” jumps when clicking (thanks to @gtrack). Pull Request 7675

• Detect errors using exit code for sort tool (thanks to @mvdbeek). Pull Request 7679

• Fix format_source for implicit conversion inputs and fasta.gz upload (thanks to @mvdbeek). Pull Request 7682

• Plugin static staging fix. Pull Request 7689

• Fix side panel styling, adjust appearance in reports. Pull Request 7690

• Patch for metadata setting of minimal BIOM1 files (thanks to @bebatut). Pull Request 7696

• Only check for exit code on actual sort command (thanks to @mvdbeek). Pull Request 7699

• Update pytest-posgresql to 1.4.0, which removes a hard dependency on psycopg2 (thanks to @mvdbeek). Pull Request 7703

• Fix runner param validation in the DRMAA runner. Pull Request 7709

• Restrict workflow invocation index to current user (thanks to @mvdbeek). Pull Request 7711

• Copy tools_by_id before collecting dependency status (thanks to @mvdbeek). Pull Request 7713

• Fix uWSGI startup with separate ini file under Python 3 (thanks to @nsoranzo). Pull Request 7716

• Fix (re)starting Galaxy using paste, GALAXY_RUN_ALL=1 and --wait under macOS (thanks to @nsoranzo). Pull Request 7722

• Fix check on links in CompressedFile.safemembers() (thanks to @nsoranzo). Pull Request 7723

• Unpause dependent jobs when resuming jobs (thanks to @mvdbeek). Pull Request 7724

• Commented-out cloudbridge requirement for the cloud/send tool. Pull Request 7727

• Fix comments (thanks to @nsoranzo). Pull Request 7738

• Upgrade to latest Pulsar release for various remote job running fixes. Pull Request 7744, Pull Request 7865

• Fix automatic user registration. Pull Request 7749

• Sanitize HTML in messages using galaxy.util.sanitize_html. Pull Request 7751

• Fix Kubernetes integration tests for Docker on Mac. Pull Request 7754

• Fix BootstrapVue component specification in Citations. Pull Request 7778

• Symlink two more images that are accessed via non-webpack’d applications. Pull Request 7779

• Make url_get return unicode (thanks to @mvdbeek). Pull Request 7782

• Disable flexible spreading of radio buttons by default. Pull Request 7788

• Fix default value handling of color parameter. Pull Request 7475

• Fix various Python deprecations (thanks to @nsoranzo). Pull Request 7800

• Return HTTP 400 for GET /api/datasets/nonexistent_id (thanks to @nsoranzo). Pull Request 7806

• Encode email messages using UTF-8 (thanks to @nsoranzo). Pull Request 7812

• Fix nested macro/token expansion on Python 3 (thanks to @mvdbeek). Pull Request 7823

• Possible fix for WSGI process issue #7758 (thanks to @tmcgowan). Pull Request 7824

• Fix InputValueWrapper gt/ge/lt/le comparisons (thanks to @mvdbeek). Pull Request 7826

• Fix trackster issue with constraining (overflow:non-visible) containers. Pull Request 7845

• Improve upload accessibility. Pull Request 7848

• Adjust appearance of user creation form in admin panel. Pull Request 7851

• Use just the env name when checking CONDA_DEFAULT_ENV (thanks to @mvdbeek). Pull Request 7853

• Fix workflow extraction for jobs whose JobToDatasetOutputAssociation references a discovered dataset (thanks to @mvdbeek). Pull Request 7855

• Fix flag for library dataset option in data selector. Pull Request 7859

• Fix bug preventing handler runner plugin handling. Pull Request 7870

• Fix click targeting for quota usage details Pull Request 7877

• Allow typing e in float param field. Pull Request 7880

• Update Cheetah dependency, fixes AssertionError (thanks to @mvdbeek). Pull Request 7887

• Fix application of history default permissions to anonymous histories carried over upon login. Pull Request 7904

• Fix multi-history copying of collections. Pull Request 7906

• Remove GFF headers during conversion and test fix (thanks to @mvdbeek). Pull Request 7916

• Fix community tags showing JSON file. Pull Request 7923

• Fix upload of gzipped VCF files (thanks to @nsoranzo). Pull Request 7933

• Fix documentation building (thanks to @nsoranzo). Pull Request 7934

• Listen to history changes in tool form until job has been submitted. Pull Request 7943

• Fixed wrong example for amqp_ack_republish_time in job_conf.xml.sample_advanced (thanks to @AndreasSko). Pull Request 7983

• Backport various upload encoding fixes from #7995 (thanks to @mvdbeek). Pull Request 8000, Pull Request 8004

• Disable create user button while request is being processed Pull Request 8029

• Fix multipanel history search Pull Request 8085