May Release Notes¶
Aunalytics is excited to announce the May 2021 Daybreak release to our clients. This release will provide clients with model and site enhancement information along with any fixes to existing functionality we have included.
Daybreak¶
Context Selector¶
Users of Daybreak have always been able to access more than one Daybreak datamart through the use of separate solution URLs communicated to clients. This month, a change to the login screen user experience now allow all users to access Daybreak through a single URL and select the Daybreak datamarts they would like to access from a context selector screen displayed after successfully authenticating with the platform.
Context selection enhances the Daybreak login experience and simplifies the way clients access Daybreak from a single, memorable URL. Moreover, context selection enables users who subscribe to more than one Daybreak instance to easily switch between instances without navigating to a separate site and authenticating again.
Natural Language Answers Model Updates¶
In April, Aunalytics released Daybreak™ Natural Language Answers, a natural language processing model that generates structured queries from questions in natural language. This model is developed by Aunalytics’ Innovation Lab and is updated to support changes to the underlying data model used to power our Daybreak datamarts. This month, Innovation Lab will release a new model version trained to better recognize answers that involve external relationship fields—Smart Features™ generated by reading through a customer’s transaction data to detect financial products they may have with external entities (other financial institutions, auto dealerships, etc.) After release, Natural Language Answers will be able to understand questions about customers with external relationships by answering questions like:
- “Give me a list of customers who have an external mortgage and don’t have a HELOC.”
- “Show customers who have external brokerage accounts”
- “Customers born between 1950 – 1955"
- “Customers who are millennials with active checking accounts”
- “Number of accounts created in the last year”
These natural language phrases are supported by retraining the model with data to support the following fields:
Customer_HasExternalCarLeaseCustomer_HasExternalCarPaymentCustomer_HasExternalCreditCardCustomer_HasExternalDealerPaymentCustomer_HasExternalInvestmentCustomer_HasExternalMerchantProcessingCustomer_HasExternalMortgageCustomer_DateofBirthCustomer_Generation
Additionally, the model now uses a dictionary of synonyms and acronyms as a training source to better understand technical terms and natural language expressions used in questions. For example, the model now understands “create” to be synonymous with “opened” in the context of financial institution accounts. Searches for "women with mortgages" now understands that "women" refers to a subset of customers (Gender == "Female"). Finally, a technical acronyms have been added so that these terms can be understood by the model. For example, "DDA" ("demand account") is understood to be a subset of accounts of a certain type (ProductCategory == "Demand"). This new technique will allow the model to easily be trained to accommodate more natural forms of expression and technical terms that could not easily be understood by the current machine learning techniques used to generate the model.
Daybreak™ for Financial Services Version 2.2 Patch¶
This month Daybreak for Financial Services users on version 2 of the data model will receive a patch update with several minor changes. Clients should be aware, however, that some user impact should be expected, particularly for Tableau users.
One change with no negative impact are that certain fields now support negative integer values:
Lend_AmountPastDueLend_BookBalanceLend_FASBCostLend_NetDeferredFees
This change should not negatively impact users, but will enhance the analytical value of these fields.
One change with possible impact to users is that the data type of Lend_ParticipationStatus has been changed from integer to string.
Tableau users using this field in dashboards have likely used casting operations to display this field's values correctly. However, now that the underlying data type has changed, those casting operations will cause an error in the dashboard that can easily be resolved by simply removing the casting operation and treating the field as a string.
Finally, two spelling errors in the names of fields have been corrected:
Lend_CurrentAmoritizationPeriodcorrected toLend_CurrentAmortizationPeriodLend_LeinPositioncorrected toLend_LienPosition
While this correction should not confuse users, clients should be made aware that may impact field references in Tableau dashboards. Any dashboards referencing these fields will need to be updated to use the corrected field names.
Aunsight¶
Aunalytics is excited to announce also the May 2021 Aunsight platform release to our internal users. This release will provide users with enhancements, new features, as well as any fixes to existing functionality we have included.
Machine Learning Tool Improvements¶
This month's release includes some feature enhancements to Aunsight's machine learning tools, Data Lab and Model service.
Data Lab Git Integration¶
Data scientists use Data Lab to create machine learning model code within containerized Jupyter notebooks. In most cases, machine learning code is stored in repositories such as Bitbucket, Github, or Gitlab using the git version control system.
Git integration enables Data Lab users to automate the management and versioning of machine learning code by handling the authentication and transfer of code into and out of Data Lab containers and their persistent storage mounts. This month's new feature rolls out git credential validation to ensure that authentication tokens are valid for remote repository used to version container code so that users do not need to recreate a Data Lab if their credentials are incorrect.
Model Service Job Tracing¶
Machine learning model deployment is managed using the Model service, an object store for serialized machine learning models. When users retrieve a model from the service (for use in a pipeline to score datasets, for example), a platform copy job is performed in the Aunsight system services. This month, these copy jobs will now be traceable so that failures in model retrieval can be discovered and remedied by operations staff. This feature builds off of recent work to better manage jobs in the Aunsight platform in order to offer clients more resilient data pipelines.
NL2SQL as a Service¶
The Aunsight NL2SQL service developed last year was initially designed to provide a stateful API for specific language models developed for a particular client. As such, the NL2SQL service was deployed for clients using certain hard-coded contexts for the use of that service. As our client base for NL2SQL has grown, it has become evident that deploying machine learning language models in this manner presented a number of difficulties.
This month, the NL2SQL backend has been reconfigured to be deployed in a less stateful manner, meaning instances of the NL2SQL service will be initialized independent of any particular Aunsight organization/project context. State-based data such as the Aunsight context are now supplied with the request, meaning NL2SQL now runs as a platform service rather than a component within particular Aunsight projects. These changes make it easier for us to deploy, manage, and test NL2SQL models for client work.
Aunsight™ Golden Record¶
Aunalytics is excited to announce also the May 2021 Aunsight Golden Record data integration platform release to our clients. This release will provide users with enhancements, new features, as well as any fixes to existing functionality we have included.
Agent Notifications¶
Last month AuGR introduced a new notifications framework to enable better communication with system operators. This month, AuGR agent notifications have been released, enabling system operators to receive email notifications when there are failures with an AuGR agent deployment.
AuGR agent notifications will enable those who support AuGR agents to monitor the health of these deployments and take action should there be a failure.
Release Contents¶
| Issue ID | Description |
|---|---|
| ILT-136 | Replace error message for erroneous input |
| DATAINT-547 | Disable operator from setting feature flags. |
| DATAINT-546 | API Request - Add option to trigger input job reads |
| DATAINT-535 | Add paging component to Pending/Failed WB queues |
| DATAINT-523 | Support CIFI 3.0 Change log Format |
| DATAINT-513 | AuGR Microservice Consolidation: Merge Dataflow into Nucleus |
| DATAINT-498 | Short-circuit read if bad record ratio is too high |
| DATAINT-487 | Update UI to validate schemas in draft if connection settings is changed |
| DATAINT-447 | Text Area version storage |
| AUN-14980 | Martin's UI changes |
| AUN-14928 | Getting rid of 'is not a recognized counter' logs in dataflows |
| AUN-14914 | Datamarts: Load from DSV |
| AUN-14892 | Add pagination for Exasol in Daybreak v2 |
| AUN-14779 | Add a system info page to webapps that show library versions |
| AUN-14734 | Show Org/Workflow Names in Workflow Notification Emails |
| AUN-14733 | Workflow: "validate_schema" option in copy |
| AUN-14715 | add EAI_AGAIN to the list of connection retries |
| AUN-14649 | Add whitelist of cases where workflow retries immediately fail |
| AUN-14647 | "validate_schema" option for Copy |
| AUN-14435 | Remove the gl2_message_id key from logs |
| AUN-14415 | Create "load datamart" workflow component |
| AUN-14406 | Update UI to accept batched queries |
| AUN-14402 | Update query service to use lib-query v2 |
| AUN-14307 | Metric service failure handling |
Bug Fixes¶
| Issue ID | Description |
|---|---|
| DATAINT-554 | mdm notifications not working for tep |
| DATAINT-550 | Dataflow Statistics API is returning 0 for the current month when it should have data. |
| DATAINT-548 | Hide lookup shapes in Monitor UI |
| DATAINT-541 | Test Exact Match on Hashed Property |
| DATAINT-538 | Matching breaks on data type 'Decimal' |
| DATAINT-526 | UI Session Expiration not being enforced |
| DATAINT-505 | Matching failure not apparent in UI |
| DATAINT-503 | Do not auto-discover on Input edit |
| DATAINT-495 | Auto-provisioning should set the UI host suffix specific to the cluster. |
| DATAINT-494 | UI logging doesn't seem to be logging errors to Graylog |
| DATAINT-491 | Update charts with correct logging variables |
| DATAINT-152 | Scheduled Read Jobs getting stuck / not running correctly |
| AUN-14953 | Workflow Trace Info - Child job links in WF jobs can be overwhelming |
| AUN-14931 | Notifications count fetch even when user is logged out |
| AUN-14918 | Difference between row count shown in the data results page and an Insights summary card for number of records |
| AUN-14802 | While editing details of a fixture, hitting enter refreshes the page and loses changes |
| AUN-14756 | Unable to run stored queries through lib_aunsight_py |
| AUN-14714 | Workflow Builder disabled components should disable required fields |
| AUN-14426 | GraphQL WF References should point to specific job types not TrackerJob |
| AUN-14276 | Formations Updating Roles is not working as expected. |