Release Overview

Aunalytics is excited to announce the February 2021 Daybreak release to our clients. This release will provide clients with model and site enhancement information along with any fixes to existing functionality we have included.


Query Wizard: New “In the Range of” Operator

The Query Wizard in the Daybreak Data Builder now supports a new operator for dates: “In the range of.” This operator serves as a placeholder for relative dates calculated from the present date when the query is executed, allowing query creators to define a range of relative dates. For example, a query could search for accounts created in the week immediately preceding when the query is run:

first in the range of screenshot

The Query Wizard interface provides a number of predefined date ranges:

  • Past ranges:
    • Yesterday
    • Last week/month/quarter/year
    • The last seven/thirty days
  • Present ranges:
    • Today
    • This week/month/quarter/year
  • Future ranges:
    • Tomorrow
    • Next week/the next two weeks/next month/next quarter

In addition to the predefined ranges, users can select “special” from the drop-down menu and define their own ranges using three criteria. Possible special ranges include examples such as the first quarter of last year or the week five weeks into the future.

second in the range of screenshot

SQL Builder: Autocomplete functionality

The SQL Builder mode of Data Builder now has an autocomplete feature alongside syntax highlighting for assistance in writing SQL query code. The autocomplete feature presents a popup menu with suggestions matching the characters the user has typed. Suggestions are drawn from SQL reserved words (e.g. “SELECT”, “AVG”, or “JOIN”) as well the field names for the Daybreak datamart being queried. So for example, typing “Date” will bring up a list of suggestions beginning with the “DATE” keyword in SQL as well as the datamart field names “DateJoined” and “DateofBirth” from the Daybreak Customer Table. This feature makes writing SQL queries against the Daybreak datamarts much easier since users not need to rely as much on the datamart field names browser in the SQL editor’s schema panel.

sql editor syntax highlight example

Data Builder: Streamlined Interface Buttons

The Daybreak Data Builder app is undergoing minor interface changes.

The “New Query” button previously displayed a dropdown menu from which the user could choose the desired query type: Query Wizard or SQL Editor. This drop-down has been replaced by a set of icons for Query Wizard, and SQL editor. Clicking each icon will create a new query in that mode.

streamlined interface buttons

Similarly, the action button to close the current query has been renamed from “Cancel” to “Back.” Users will still be prompted to save any modifications before the current query is closed.

databuilder action buttons

Aunsight™ Data Platform

Parquet Support

Aunsight Atlas records now support a new dataset format: Parquet. Parquet is a columnar dataset format developed by the Apache Software Foundation for Hadoop-based big-data platforms ( Unlike CSV files commonly used by Aunsight’s Hadoop storage clusters to date, Parquet structures data internally as columns rather than rows. Accessing and storing data in this way enhances the ability to store and operate on subsets of columns in a table. More importantly, it also allows datasets to be copied directly between the Alluxio object acceleration engine used to store and process data in pipelines and Exasol, the relational database used to process queries from the Daybreak webapp.

In addition to improving the efficiency of Aunsight data pipelines, Parquet will be an important component of a new Aunsight concept: datamarts. Parquet datasets will be used as the basis of datamarts, Aunsight objects which define and manage the structure of data as it moves between Aunsight’s big data storage infrastructure and the relational database structures which Daybreak imposes on data in order to render its structure more familiar to data consumers.

Aunsight™ Golden Record

Transactional Workflows

Aunsight Golden Record now allows users to configure a Transactional Workflow that can move larger datasets from a source database to the datastore of their choice while bypassing the Mapping, Matching, and Merging configuration steps. This new type of workflow supports bulk data transfers to provide fast, cloud-native data transfer from client data sources into both Amazon S3 and the Aunsight platform.

Minor Improvements: Daybreak

Ticket Number Ticket Name Business Value
AUN-14206 Daybreak: Create tooltip for gear icon on query wizard The gear icon in the query wizard now shows “Select columns to show in results” when users hover the cursor over it.
AUN-14379 Daybreak: Add one-to-many to query wizard Users can now join from the account or transaction table back to the customer table.

Minor Improvements: Aunsight

Ticket Number Ticket Name Business Value
AUN-11623 Expression Builder refers to "Bag" but proper nomenclature is "Collection" Twelve expressions were updated to refer to “Collections” rather than the deprecated term “bags.”
AUN-13251 Dataflow: STRSPLIT Expression improvement The third argument can now be a field or a static integer.
AUN-13770 Sample rows operation should allow 100% (and at 100%, it should pass through all rows, not sample them) The operation "Sample Rows" will now work at 100%, allowing dataflow engineers to use “Sample Rows” as a flow control gate.
AUN-13840 Consistent labels across webapp for 'Updated' fields Various field titles were updated to consistently use the term “updated at”.
AUN-14025 Add undo/redo actions in the expression builder User will be able to use undo and redo within the expression builder.
AUN-14033 Cached Mode for CopyMulti in the workflow components Workflow CopyMulti components now have an attribute field to enable cached mode as has already been the case for the Copy component.
AUN-14121 Make runtime images easy to use and maintain in the registry Packaged process runtimes are now pushed to the Aunsight Harbor registry and made available as base runtimes for other processes.
AUN-14354 Secret Service: Convert password input into multiline text input field The Secret Service Web UI now uses a multi-line input field allowing multi-line text input for secret content (e.g. for storing SSH keys).
AUN-14378 Processes: Sync file selector and form validator with eachother Users can now upload files with any contents when the selected runtime does not specify a main file format.

Minor Improvements and Bug Fixes: Aunsight Golden Record

Change Type Business Value
Enhancement Scheduled input jobs will now detect records that have been deleted at the source, and propagate those deletions to the appropriate Golden Records.
Enhancement File Reader Plugin - Additional modes 'FTP' and 'SFTP' are now supported which allows users to ingest files directly from an FTP/SFTP server.
Enhancement Oracle Plugin - Users can now bypass discovery to improve performance when connecting to extremely large databases while retaining the ability to create a custom schema input from the selected database.
Bug Fix Improved error message display in the UI for large error messages including the ability to scroll which allows users to view the entire error message directly in the UI.
Bug Fix Updated the Version Browser display to keep the Golden Record in the first position of displayed records when sorting (applies to both horizontal and vertical browsing modes.)
Bug Fix Improved performance of deleted Version and Golden Records reflected in the UI.
Bug Fix Input/Output Graphs on the Monitor page will now default to display the last 5 minutes of data to improve UI performance.
Bug Fix Changing the display name of a Matching step will no longer require a full purge and reload in order to Preview/Publish.
Bug Fix Improved recoverability for scheduled jobs that when they become stuck in a pending state.
Bug Fix Improved reliability of download option for the Rejected Records report.

Bug Fixes: Aunsight

Ticket Number Ticket Name
AUN-14523 Activity subscriptions not working
AUN-14445 Workflow name disappears when user in view/modify mode in the project context
AUN-14295 Workflow Builder- multiselecting things twice is possible, causing multiple issues
AUN-14283 Clean up GA event tracking
AUN-14248 Webapp CopyButton - add `type="button"` to copy button
AUN-14220 Jupyter cannot upload or save large files
AUN-14205 Address CloneDeep of Attributes in Data Lab
AUN-14204 Address CloneDeep of Attributes in Data Pipeline
AUN-14203 Address CloneDeep of Attributes in Core Services
AUN-14065 ReadProcess and ListMembers tests are taking unnecessarily long
AUN-13268 Process does not update user task jobs with env vars set by k8s adapter
AUN-12834 Select and Rename Operation not working as expected