February 2022 Release Notes


Verification for Alluxio Copy Job Size

Alluxio copy jobs will now fail if the output file size isn’t the same before and after the job. Previously, there was no validation check, which could result in loss of data without the user knowing. Now, the system will throw the error DispatcherDataIntegrityError if the sizes of the pre- and post-copy dataset don’t match.

Sorting Consistency

List pages for Models, Datamarts, Processes, Queries and Secrets have been updated to have sorting by name and updated_at. The default sort is by updated_at.

Environment Variables added to Data Lab

Added AU_ORGANIZATION and AU_PROJECT as environment variables to Data Lab.

Dataflow Schema Panel Resizing

While users did like the addition of the schema panel to the Dataflow builder, they found that it could take up too much space. Users will now be able to resize the schema panel by clicking and dragging from the lower left corner.

Job Duration Tracking

The main details page for Workflow, Dataflow, Pipeline, Process, and Query jobs will now show how long the job has been running in the Duration space. Jobs must be in running state and the duration will update automatically every second. No manual refresh is needed. Duration information will persist on the details page for these jobs.

Small Feature Enhancements

  • Workflow builder enhancements
  • Override port validation with Shift and Control keys
  • Load Datamart component giving selector list for datamart port and form modal for tables port
  • Pre-fill operation panel with user selected relation

Aunsight Golden Record

Small Feature Enhancements

  • Show/Hide All Properties for Golden Records > Browse Data screen
  • Go-Between updates for Job Trigger Resources
  • View background processing progress for Golden Records and Transactional Workflows to show when data is in flight, in-process, or static.

Bug Fixes

Description Client Request
Update to Workflow Job History filters
Resolve hanging Alluxio uploads
Failure to open parameters port in Evaluate Script component of Workflows
Exasol connection may generate mutual-locking states for schemas during progress checks
Webapp storybook unable to open
Updated shape and job level user interface purges to execute as quickly as shape and job level API purges.
Update workflow port validation logic T20220110.0388
Dataflow change causes parameters set in the dataflow logic to change default values T20220204.0066