November/December Release Notes

Aunalytics is excited to announce the last release of 2021 to our clients. This release will provide clients with model and site enhancement information along with any fixes to existing functionality we have included.


Natural Language Answers Autocomplete

This month we are releasing an exciting new feature to Daybreak users: autocomplete for Natural Language Answers™. Similar to the autocomplete behavior of search engines like Google, users will now see a list of their ten most recent searches when they place the cursor in the natural language search box, and when they begin typing, Daybreak will make suggestions based on what the user enters.

NLA autocomplete example

This feature will make it easier for users to ask questions both because it will suggest commonly searched after questions and offer autocompletion for field names that a user might not remember (for example, "age of account" versus account age").

Enhanced Information for Field Definitions

Daybreak users working with the structured Query Wizard and SQL editor have access to a field definitions tab (formerly called "Data dictionary") that lists the names of fields available in the datamart. This feature has helped users correctly identify the names of fields, but will now also show the data type (integer, floating point, string, etc.), and a brief description of the data point. These field descriptions or definitions help explain what a data point is and in some cases, how it is determined. For example, "CustomerAge" is "Integer representing the customer's age in years, dynamically generated each day from the customer's date of birth." This additional information will help Daybreak users understand their data marts so that they can create richer and more useful queries from them.

Workspace Descriptions

As we prepare to support increased numbers of workspaces for Daybreak in 2022, we've made a minor enhancement to the Daybreak app to help guide users to the workspaces they want to use. Now, when a user hovers the mouse cursor over the icon for a workspace, there will be a pop-up showing that workspaces description. While at present most Daybreak users only access one or two workspaces, our forthcoming release of scoped datamart views will increase the number of workspaces users can access and these descriptions will help them to better understand what view of the datamart each workspace offers.

Natural Language Answers Model Retrain

This month we have updated the Natural Language Answers model to support four new tables: Collateral, Branch, Lending, and Cards. While at present, not all Daybreak subscribers have chosen to implement all the tables in the data model, those who have subscribed to these four tables or a subset of them will have NLA support for all their fields, while subscribers without them will still be able to use NLA for the subset of fields that they have implemented.

We're excited about the large number of fields supported in this update and the new kinds of questions our users will now be able to ask!

Table Field Example Question
Collateral AccountValue Customers with 0 collateral account value
Collateral AccountValue Customers with collateral account value 0
Collateral ApplicationCode Collateral with an application code of 70
Collateral AppraisalDate collateral appraisal date last month
Collateral AppraisalValue collateral appraisal value above 5000
Collateral IsPledged Customers with pledged collateral
Collateral IsPrimary Customers with no primary collateral
Collateral MSACode Collateral with MSA Code 0
Collateral Make Collateral make is CHEVROLET
Collateral Model Collateral Model is FUSION
Collateral OriginalAppraisalDate Collateral with Original Appraisal Date this year
Collateral OriginalAppraisalValue Customers with Original Appraisal Value above 10000
Collateral RecordingDate Collateral with Recording Date last week
Collateral RelatedAccountType Collateral with Related Account Type 125
Collateral TotalValue Collateral with Total Value below 20000
Collateral TypeCode collateral type code is A1
Collateral VIN Collateral with VIN
Collateral Year Collateral with Year 2016
Branch N/A I want a list of branches
Branch Branch_CloseDate branches closed date is last week
Branch Branch_OpenDate branch opened date is last month
Branch Branch_Name show me Topeka branch
Branch WasAcquired branches are acquired
Branch WasAcquired acquired branches
Lending BoardingDate Loans with boarding date last month
Lending BookBalance Loans with Book Balance over 1000
Lending ApplicationCode Loans with Federal Call Code 300
Lending NonAccrualDate Loans with Non Accrual Date last week
Account OfficerID Loans with officer id 697
Lending OriginalPrincipalAmount Loans with Original Principal Amount above 500
Lending OriginalTerm Loans with Original Term below 120
Lending PaymentFrequencyCode Loans with Payment Frequency Code 4
Cards ATMDailyWithdrawalLimit Customer with ATM daily withdrawal limit 5000
Cards ATMTransactionWithdrawalLimit Customer with ATM Transaction withdrawal limit above 5000.00
Cards BIN Customer with bin number 551336
Cards StatusChangeDate Customers changed cards status after 2019
Cards Cards_Status cards status is active
Cards CloseReason Cards with close reason 0
Cards LastTransactionDate cards expired last week
Cards IsCorporateCard Customers with Corporate Cards
Cards IsPrimaryAccount show me primary cards
Cards IsSavingsCard Customers with savings cards
Cards LastTransactionDate Customers with cards last transaction this week
Cards LastUsageAmount Cards with last usage amount below 100
Cards OriginalFirstUsageDate Customers with original first usage after 2000
Cards OriginalCardIssueDate Customers with Original Card Issue Date this week
Lending IsInterestRateVariable How many loans with variable interest rate
Lending OriginationDate loans with an origination date between April 1st, 2020 and March 31st, 2021
Lending LoanType Loans with loan types 135
Lending CurrentLoanStatus delinquent loans
Lending CurrentLoanStatus chargeoff loans
Lending ProductCode number of credit card loans
Lending ProductCode list of agricultural loans 
Cards IsDebitCard How many members have debit cards 
Cards IsDebitCard How many debit cards do we have
Cards IsCreditCard Number of credit cards
Cards ProductType Cards are ATM type
Cards ProductType I want a list of cards with product type ATM card


Workflow Improvements

Time Between Retries for Workflow Components

Workflow users have long made use of the number of retries parameter to add resilience to their data pipelines. Because many job failures are linked to transient issues such as waiting for a dataset lock to be released, Aunsight determines a time to wait that increases with each subsequent retry. Previously, Aunsight determined this interval automatically, but because the appropriate time to wait depends on the type of failure anticipated, we now allow users to specify a time to wait between retries for a component.

Time to wait between retries is now configured via the UI based on four user supplied parameters:

  • Minimum delay: The minimum time to wait, in milliseconds.
  • Maximum delay: The maximum time to wait, in milliseconds.
  • Backoff factor: An exponential factor to increase subsequent retry waiting periods. Expressed as a floating point with a maximum of two decimal places (e.g. 2.0).
  • Randomize delay: Set to true or false to allow a randomize delay between the minumum and maximum to be chosen.

Workflow Component Input Dependencies

The Workflow Builder allows users to configure automated workloads by connecting the inputs and outputs of tasks (sometimes called "ports" generically) and configuring settings based on those inputs. Many input ports depend logically on others. For example, selecting a process version to run depends on the ID of the selected process. For this reason, this month our UI now shows workflow component input dependencies in the component sidebar. These inputs will now indicate the name of their dependency so that users can quickly identify which inputs need to be connected or configured first before connecting or configuring dependent ports.

Workflow Input Validation

In addition to workflow component dependencies, Workflow Builder now includes component input validation. Previously, users could connect incompatible connections between components--for example, connecting a JSON object to an input that requires a simple string input. As this leads to workflow failures, the Workflow Builder now has awareness of data types required by component inputs, and will not allow a user to connect incompatible inputs. Within the UI, this is indicated by the cursor changing shape when a connection is about to be made:


Datamart Engine Optimizations

This month we are releasing the first part of several months worth of back-end improvements to our Datamart engine. This month, we have refactored the microservice that executes Daybreak user queries into a stateful service. This first step will yield immediate performance increases, as well as allowing for subsequent improvements such as connection pooling that will further enhance query efficiency and execution times.

Aunsight™ Golden Record

Log4j Vulnerability Patch

This month, we have reviewed our platform infrastructure to address an evolving security threat in one of our software dependencies: Apache Log4j, a Java logging utility. A recently discovered security vulnerability with this library can allow a bad-actor to execute arbitrary code injected into log messages generated by an attacker-controlled LDAP server. After a thorough review of our infrastructure, it was determined that the Log4j vulnerability only affected one of our underlying systems, and a patch has been applied to secure our platform from this ongoing security threat. As reports of successful attacks continue to appear in the news, users can rest assured that AuGR has been hardened to prevent an attack against the platform.

Release Contents

Issue ID Description
AUN-11620 In Parameters box, "Text" should be changed to chararray
AUN-14862 Move workflow mail service from Google to Microsoft
AUN-14993 Add NLP metadata sections to Datamart tables and Datamart columns
AUN-15021 Move Walleye Tasks to K8s & Alluxio Internal Storage
AUN-15183 Okta: Implement Aunsight-lib-login in Sightglass Mobile
AUN-15258 Okta: Clean up legacy frontend authentication functionality
AUN-15316 Gateway Augment Communication Timeout - Indicate Rendezvous Worker Chosen
AUN-15374 Require description: (Front end)
AUN-15375 Require description (backend)
AUN-15376 Add default messaging on existing/active workspaces
AUN-15386 Add req_id/trace/span/client to activity events
AUN-15388 Clear Insights when changes are made to the core table
AUN-15395 Lib-ui Error component, pass error messages through DOMPurify, with tests
AUN-15428 Remove inactive users shown in sharing window in the UI
AUN-15442 Daybreak: Enforce limit of 10,000 enums v2
AUN-15444 Improve lib-au-js error reporting
AUN-15453 Update Webpack to v5 in Aunsight Webapp
AUN-15457 Update supported browser/requirements
AUN-15476 Resource UI warn on pending changes or deletion
AUN-15497 Datamart Migration and Load events don't include datamart id
AUN-15501 Move query list to right side of page & add Data Dictionary Tab
AUN-15503 Add Data Dictionary Events for UserPilot
AUN-15535 Define CSP in webapp
DATAINT-617 Go-Between updates for Job Trigger Resources
DATAINT-637 Agent Notification Subscription - UX tweak
DATAINT-640 Fully Deprecate Old Matching
DATAINT-681 Aunalytics Idea Portal - AuGR - Prevent AuGR from logging out while a test query is running
DATAINT-787 Ability to Hotfix Specific Agents
DATAINT-789 Add API to request Agent Log Upload - Go-Between
DATAINT-790 Provide upload endpoint for Agent logs
DATAINT-791 Agent - Download logs from UI option
DATAINT-794 Session Expiration UX improvement
DATAINT-842 Password Requirements Listed on Change Password Screen
PZ-1266 CIFI 3.0 - Deploy Service Assembly Line
PZ-1268 CIFI 2.2 - Deploy Exception table addition to LastSync table

Bug Fixes

Issue ID Client Request Description
AUN-13182 Fix Workflow Builder read only view in Safari
AUN-13705 removed modal-backdrop and modal-open classes when modal is destroyed
AUN-15210 Aunsight WF and DF job detail pages no longer shows update timestamp
AUN-15228 Fixed bug around validation message not showing for invalid output.
AUN-15275 Fix bug when there are no recent searches
AUN-15318 Added a check for an empty string before fetch, show 'recent searches' otherwise.
AUN-15337 service_locks and global_locks are set to null instead of removed
AUN-15365 Added a form wrapper with a keyup.enter to the NLPWidget
AUN-15387 fixing the landing page redirection when a user only has one workspace.
AUN-15410 Fixes a race condition in start up which could cause the context menu to not load correctly
AUN-15433 added a .fail() to fetch
AUN-15437 T20211013.0542 Date Cast Creates Timezone Difference and DaysBetween is off by 1 day.
AUN-15441 T20211014.0201 Sightglass Web browser - Breadcrumb behavior repeats "root" with change in Search criteria
AUN-15445 Create user from System Dashboard>User throws handled error (type mismatch)
AUN-15465 Making sure errors are displayed on insights when queries are batched
AUN-15472 Print test results in Daybreak Jenkins builds
AUN-15478 Fixing bug where tool would go blank if a query was opened that referenced a deleted table
AUN-15480 Assuring labels for markdown fields are aligned correctly.
AUN-15486 Allow webapp unit tests to run consistently regardless of timezone of host machine.
AUN-15536 Fixing a bug where if you change contexts before config finishes loading, it might override the current context with previous one
AUN-15539 T20211117.0280 Copy jobs deleting records in ONB
DATAINT-282 Trim Whitespace on Deletes
DATAINT-537 Make append-only resilient against job stopping
DATAINT-705 Agent Display Name vs. Agent Name being Displayed
DATAINT-757 T20210901.0625 Ascent - Opening a Well Resorts Version Browser Results Erratically
DATAINT-786 Resuming Writebacks displays big red error
DATAINT-796 SSO User Lockout expiration context missing
DATAINT-81 [BUG UAT]Inconsistencies count doesn't get updated
DATAINT-813 Batch History page missing virtual scroll or paging option
DATAINT-848 T20211104.1294 TEP Prod - Output Mapping Requiring Date Format for String Object
DATAINT-857 Nucleus shape level purge triggers multiple source level purge
DATAINT-714 T20210811.0332 Default attribute ordering/sort
DATAINT-776 T20210830.0480 Include Pilot Logs in Auto Clean-up
PRODSUP-987 T20211213.0647 Log4j Vulnerability Fix