Release Overview: April 2021

Aunalytics® is excited to announce the April 2021 Daybreak™ release to our clients. This release will provide clients with model and site enhancement information along with any fixes to existing functionality we have included.

Daybreak

Daybreak™ for Financial Services 3.0

Daybreak for Financial Services 3.0 (DFS 3.0) represents a new generation of Daybreak data model for clients in the financial services industry. Based on insights gleaned from the experiences of numerous client banks and credit unions, Aunalytics has identified numerous improvements that will now be delivered to future and existing clients as Daybreak for Financial Services 3.0.

At its core, DFS 3.0 starts with a next-generation datamart core to support a more streamlined datamart structure and delivery pipeline. Whereas previous versions of Daybreak for Financial Services relied on a monolithically structured datamart, 3.0 has streamlined the logic used to build derived fields into a tiered structure so that datamarts can be implemented faster and customizations handled more dynamically for clients.

In addition to the data model, DFS 3.0 takes advantage of next-generation developments in the Aunsight™ Data Platform to deliver faster and more resilient datamarts. Improvements such as the microprocessing of client data and delivery via storage infrastructure built for ultra-fast query response times adds efficiency and performance to our next-generation data model.

Finally, Aunalytics operations and product documentation teams are working behind the scenes to provide documented standard processes for client implementation that will help educate existing clients on the steps needed to migrate to this version and enable our teams to deliver these upgrades faster and more reliably.

Datamart Restructuring

An improved structure for datamarts enables us to deliver the core analytical database tables to new clients faster while enabling clients to build out additional custom fields as desired.

Tier Levels

One of the major changes to Daybreak for Financial Services 3.0 is the introduction of a tiered system for organizing fields and tables in the datamart:

Tier	Purpose	Requirement	Example fields
Tier 0	Required System fields	Required	CustomerID
Tier 1	Foundational fields	Required	DateofBirth, TaxpayerID, AddressKey (foreign key)
Tier 2	Enhancing Fields	Recommended	Gender, Email, CurrentCreditScore
Tier 3	Customer selected fields	Elective	HasBillPay, HasEStatements, CheckingAccountBalance

The purpose of this tiering system is to decrease the time to implementation of a standard set of tables common across all clients. Once the initial two tiers are complete, customers can immediately access a basic datamart featuring additional data from tier 2 as that becomes available. Tier 3 enables clients to request custom fields for ad-hoc or client-specific use cases or to add Smart Features on a subscription basis.

By decoupling the layers of the data model, Daybreak for Financial services offers faster time delivery to clients and enhanced ability to provide custom services without complicating the implementation of the foundational datamart structure.

New Tables

As part of the restructuring of datamarts into a tiered system, Daybreak for Financial Services will be adding new tables that enhance the relational structure of datamarts:

• Products: Information on the products offered by the client
• Services: Information about the different services a customer has subscribed to

• Milestones: Information about the stages of the loan approval process

• Addresses: Customer address and geolocation data

Many of these new tables serve to normalize relationships that had not always been clearly reflected in the 2.0 datamart versions. For example, customers in the previous model were limited to one address, which created various problems when the same customer had accounts listed under different addresses. The new structure makes it easier to accomodate customers who may have a business account listed under one address, and personal accounts under a home address.

Datamart Microprocessing

Daybreak for Financial Services 3.0 enables an evolutionary leap in the processing of data for Daybreak: datamart micro-processing. In past versions of the Daybreak for Financial Services model, data is refreshed nightly by processing complete datasets, including fields whose values change infrequently. Datamart microprocessing enables much faster and more frequent datamart refreshes by only building where source data has changed.

Datamart microprocessing leverages transactional workflows in Aunsight™ Golden Record to transfer new or updated data from a source in the form of a dataset log. These small chunks can then be processed on their own and the results integrated into the datamart, greatly reducing unnecessary resource load since only new data is processed in every batch.

In addition to greatly reduced resource footprint for cost and energy savings, microprocessing opens up the possibility of refreshing Daybreak datamarts multiple times in a day. If implemented for clients, this approach would offer a more dynamic experience as datamarts would be accurate to within hours of when their source data has changed.

Point-in-Time Snapshots

Daybreak for Financial Services 3.0 also includes datamart snapshots, a new database dimension providing point-in-time information about datamart fields. Snapshots provide an extra dimension to the Daybreak for Financial Services experience, allowing users to retrieve trend data about a field, such as credit score or account balances. In addition to exposing historical data to users in search of trends, the Innovation Lab data science team will have access to historical data for exploration and use in developing new SmartFeatures built on data trends over time.

Dynamic Subscription to Smart Features™

Clients on the 3.0 data model will now be able to dynamically subscribe to machine learning generated Smart Features as a part of their tiered datamart. Previously, delivery of new Smart Features required an extensive overhaul of the data pipeline used to generate that datamart. Thanks to its tiered data model, Daybreak for Financial Services 3.0 enables the activation of Smart Features by simply changing the configuration objects for a datamart pipeline.

Dynamic subscription enables new business models for delivering industry intelligent data points as a service. Clients gain the ability to personalize their Daybreak experience by subscribing to the individual machine learning scores and insights they want in their datamart.

Daybreak for Financial Services 2.2 Patch

Daybreak for Financial Services version 2 clients have a new update to their data model available to correct a change in the way the transaction summary table is built. Based on changes to Accounts table, the DFS 2.2 patch will change the foreign key for the Transactions summary to point to a uniquely identifying field, AccountPrimaryKey, rather than the previous AccountID which was a non-unique identifier in many client data sources. This change will resolve certain issues for some banking clients whose core systems did not provide unique account IDs (account numbers) for all sub-accounts (e.g. checking versus savings).

Download Insights

Last month Aunalytics released Insights, graphical representations of query results in the Daybreak Data Builder. At that time, users could create Insights in one of four types, save them to their query's Insights dashboard and share the query and its Insights with other Daybreak users in the app. This month, users can download the visualizations as PNG or JPG files for inclusion in other applications, or data (CSV) files to create additional visualizations manually. This allows users to import Daybreak Insights into documents and presentations using other tools directly from the app.

Read more about how to download Insights here.

is one of/is not one of Operator for Structured Queries

Many Daybreak datamart fields hold enumerated values; in other words, a field can only have one of a limited set of possible values such as ProductType being one of "Mortgage", "Checking," "Savings," etc. To support new query capabilities on such fields, Daybreak now offers is one of and is not one of operators which allow a user to select a subset of values that they would like to find.

For example, if a user wants to select mortgage or HELOC accounts, they can now use the is one of operator on the ProductType field and check the "Mortgage" and "HELOC" boxes from the operands field.

is not one of works similar to is one of but in reverse: it will exclude fields that are checked and return all other results.

Direct Notifications

The April release features in-app notifications as a new Aunsight platform component. Daybreak users will experience this as an in-app notification area that provides updates about changes to queries that are shared with them. By default, users will be automatically subscribed to notifications for all queries shared with them. Notifications will enable users to be updated when these queries are changed by their creators.

Notifications appear as an extension of the webapp in the lower left corner of the screen above the account/log out information. When a user visits this part of the app, they will see the last thirty days of activity in a sorted stream of notifications about changes to queries shared with them. Clicking on a message will mark it as read, or users can mark all as read.

Insights for Natural Language Answers

Last month Aunalytics released Insights, graphical representations of query results in the Daybreak Data Builder. At that time, users could create Insights in one of four types, save them to their query's Insights dashboard and share the query and its Insights with other Daybreak users in the app. This month, Daybreak Natural Language Answers will feature automatic Insights, graphical representations of the results generated by a natural language question. At present, Daybreak Natural Language Answers will show a summary Insight displaying the number of records in the result set returned by the query. In the future, Insights for Natural Language Answers could be trained to display different visualizations, such as a breakdown of customers by generation (“Show me a list of customers with checking accounts by generation”) or zip code (“Show me mortgage accounts by ZIP code”). Insights for Natural Language Answers lays the groundwork for applications of machine learning to provide answers to clients using a simple, natural language interface.

Aunsight

Direct Notifications

As with Daybreak, the Aunsight web interface now features notifications as a platform concept. As in Daybreak, certain platform events can be configured to generate notification objects that can be viewed using the new web UI notifications interface.

Aunsight notifications appear in two places. The first is the notifications tool, which is accessed from the User Dashboard (the landing page when a user first logs in). The second location is a notifications quicklink in the upper right corner of the screen near the documentation and service desk links. This quicklink is visible from whatever context a user is working in, and so provides instant visual alerting when new notifications are received.

New Connectors: SmartyStreet and Character Cleaner

This month Aunsight premieres two new connectors: SmartyStreet and Character Cleaner.

The SmartStreet geolocation database is used in many Daybreak datamarts to retrieve geolocation information (GPS coordinates and distance calculations) from street addresses using the SmartyStreet API. To further streamline the retrieval of this data for client solutions, Aunsight now has a SmartStreet connector. Engineers can run this connector with a dataset of street addresses in a standard format and recieve a result dataset with geolocation data for those addresses. By using a connector, engineers can now get geolocation data without having to understand or interact with the SmartyStreet API or write custom processes to obtain their data for Daybreak datamarts.

The Character Cleaner connector is a utility process to aid solution engineers with a common use case: finding and replacing certain characters using regular expressions. Previously, data engineers would write custom processes to use the Unix sed utility to perform these transformations. Now, engineers can feed a raw dataset into the Character Cleaner connector with configuration options to find and remove or replace undesired characters (e.g. removing quotation marks like '' or '"') and write the output to a cleaned dataset record. This will provide a valuable tool to increase solution developer productivity.

Job Tracing V1 Framework

This release previews the first version of a job tracing framework for Aunsight workloads. The job tracing API provides enhanced capabilities for tracing related Aunsight jobs within a workload. In addition to the platform API changes, this initial release of job tracing will add a trace panel to the jobs page in the Aunsight web interface enabling users to see links to the job showing the root job (usually a workflow), the immediate parent job (another workflow or other component) and any child jobs (jobs kicked off by the present job) as well as the job type and state. This feature provides an easier way for operations to investigate pipeline failures and isolate the root cause of failing components quickly and easily, enabling fast responses to failures in production solutions.

Dataflow Operator Improvements

This month a number of new dataflow operators and expressions were created and certain changes were made to existing operators to make them easier to use.

Issue	Type	Expression/Operator	Purpose
AUN-9784	New Operator	NonEqui.Join	This operator allows user to perform a non-equal join operation (a "Theta join") using comparison rather than equality as the condition.
AUN-9758	New Operator	Median	This operator computes a statistical median from a collection.
AUN-12188	New Expression	CastToBigdecimal	Casting a double type to chararray transformed the number to scientific notation, which prevented the field from being used as an operand for arithmetical operations. CastToBigdecimal casts numbers to a string (chararray) type, but preserves the decimal notation needed to perform arithmetic.
AUN-12617	New Operator	join.InnerCompound	Users can now perform joins multiple fields without creating a compound key field
AUN-13969	Updated Operator	A confusing parameter name ('alias') has been changed to 'prefix.'
AUN-14525	New Expression	ToDateFormatTimezoneCorrected	A new operator to converts datetime strings to objects with correctly adjusted DST

Aunsight Golden Record

Snowflake Plugin for Transactional Workflows

Last month, Aunsight Golden Record released Transactional Workflows, a new capability that can move large datasets from a source database to the datastore of their choice while bypassing the Mapping, Matching, and Merging configuration steps. Initially, this feature came with plugins for performing bulk transfers of data with Amazon S3 and the Aunsight platform. This month, Aunsight Golden record has released another plugin for the Snowflake cloud analytics platform. This feature adds additional capabilities for transactional workfldaows that will enable Aunsight Golden Record to provide enhanced support for customers using that platform.

Release Contents

Issue ID	Description
ILT-155	Data Download by Query
DATAINT-522	Update TX shapes to not use MDM provider in UI
DATAINT-515	Refactor Job Scheduling Logic
DATAINT-509	TXWF - Destination timing options
DATAINT-507	TXWF Statistic Improvements
DATAINT-487	Update UI to validate schemas in draft if connection settings is changed
DATAINT-480	AU TXWF Date format standardization (8601)
DATAINT-475	Improved Password Reset UX
DATAINT-447	Text Area version storage
DATAINT-180	AWS Redshift Transactional Job
DATAINT-38	Aon Mongo Accessibility
DATAINT-17	I/O/R Error Message UX
AUN-14840	Exasol Connector Update: Make DROP transaction safe
AUN-14774	Datamarts: Show names for created by and updated by
AUN-14731	Implement design system font in remaining UIs
AUN-14661	Model service: Make "Publish" button reactive
AUN-14657	Add backend support for tag validation throughout Aunsight
AUN-14646	Add option fields to copy job (update_schema, etc)
AUN-14645	"Move" mode for dispatcher copy
AUN-14643	Daybreak NLP: Display warning when value is empty in enum field
AUN-14542	Design System: Update font throught Aunsight to match the one in Daybreak
AUN-14493	Using entityMart to get links in wf builder, fixing view mode and resource link bugs
AUN-14489	Data Lab: Create more timeframes for resource usage
AUN-14488	Data Lab: Ability to create time frame for resource usage
AUN-14486	Data Lab: Expose more data points in activity history
AUN-14461	Execute functionality for AuQL scripts using lib-aunsight-py
AUN-14415	Create "load datamart" workflow component
AUN-14047	Add restrictions on custom id field for characters and length
AUN-13779	Encompass Refactor
AUN-13463	Processes: Update runtimes automatically when toolbelt or lib-aunsight is updated
AUN-13417	Add additional fields to pull from Encompass
AUN-11964	Data Lab Notebook: Add code snippet title to dialog
AUN-11730	Toolbelt should not execute when invalid options are provided
AUN-6178	Toolbelt create/update secret add field options

Bug Fixes

Issue ID	Description
DATAINT-526	UI Session Expiration not being enforced
DATAINT-516	[BUG] UI Not properly filtering out connections from Replication and Outputs
DATAINT-490	Advnced tab disappears when editing property
DATAINT-485	Profiling Suggestions Missing
DATAINT-482	UI calling plugin twice for the same request
DATAINT-477	Profiling UI Hotfix
DATAINT-463	[BUG] - Merge Rule Dropdown weird behavior when > 1 rule exists
DATAINT-461	Publish modal shows different TXWF source names on editing table name
DATAINT-460	Ensure resources are cleaned up when draft is deleted
DATAINT-456	AuGR Agent leaving rogue plugin processes running
DATAINT-152	Scheduled Read Jobs getting stuck / not running correctly
DATAINT-20	[PROD-ERROR] MDM Activities getting cancelled and throwing exception across tenants
AUN-14864	Changing the tables port type component for load datamart component
AUN-14812	DF Parquet loader ignores load schema in favor of record's internal schema
AUN-14770	NLP Feedback breaks if NLP model URL does not end with /
AUN-14736	Sonarbug fix for Expression builder
AUN-14719	Query tool queries fail if they end with comment
AUN-14628	Daybreak-Place Holder content for "in the range of, special" needs different formatting
AUN-14589	Workflows - Notification type for roles/members not working in Run WF from modify view
AUN-14532	fixing missing column cause out of bound error
AUN-14426	GraphQL WF References should point to specific job types not TrackerJob
AUN-14384	Losing information while saving WF builder for the exact same state
AUN-13849	Dataflow: Undefined parameter causes launch to hang at "Preparing..."
AUN-13824	Aunsight de-duplication not working
AUN-13723	Dataflow Failure - ERROR 1025 Invalid field projection. Projected field does not exist in schema:
AUN-13066	Dataflow: Improve error message when a record is being used multiple times (for store).
AUN-8771	Lib-Py: dataflow upload loses parameters