November/December Release Notes¶
Aunalytics is excited to announce the last release of 2021 to our clients. This release will provide clients with model and site enhancement information along with any fixes to existing functionality we have included.
Daybreak¶
Natural Language Answers Autocomplete¶
This month we are releasing an exciting new feature to Daybreak users: autocomplete for Natural Language Answers™. Similar to the autocomplete behavior of search engines like Google, users will now see a list of their ten most recent searches when they place the cursor in the natural language search box, and when they begin typing, Daybreak will make suggestions based on what the user enters.
This feature will make it easier for users to ask questions both because it will suggest commonly searched after questions and offer autocompletion for field names that a user might not remember (for example, "age of account" versus account age").
Enhanced Information for Field Definitions¶
Daybreak users working with the structured Query Wizard and SQL editor have access to a field definitions tab (formerly called "Data dictionary") that lists the names of fields available in the datamart. This feature has helped users correctly identify the names of fields, but will now also show the data type (integer, floating point, string, etc.), and a brief description of the data point. These field descriptions or definitions help explain what a data point is and in some cases, how it is determined. For example, "CustomerAge" is "Integer representing the customer's age in years, dynamically generated each day from the customer's date of birth." This additional information will help Daybreak users understand their data marts so that they can create richer and more useful queries from them.
Workspace Descriptions¶
As we prepare to support increased numbers of workspaces for Daybreak in 2022, we've made a minor enhancement to the Daybreak app to help guide users to the workspaces they want to use. Now, when a user hovers the mouse cursor over the icon for a workspace, there will be a pop-up showing that workspaces description. While at present most Daybreak users only access one or two workspaces, our forthcoming release of scoped datamart views will increase the number of workspaces users can access and these descriptions will help them to better understand what view of the datamart each workspace offers.
Natural Language Answers Model Retrain¶
This month we have updated the Natural Language Answers model to support four new tables: Collateral, Branch, Lending, and Cards. While at present, not all Daybreak subscribers have chosen to implement all the tables in the data model, those who have subscribed to these four tables or a subset of them will have NLA support for all their fields, while subscribers without them will still be able to use NLA for the subset of fields that they have implemented.
We're excited about the large number of fields supported in this update and the new kinds of questions our users will now be able to ask!
| Table | Field | Example Question |
|---|---|---|
Collateral |
AccountValue |
Customers with 0 collateral account value |
Collateral |
AccountValue |
Customers with collateral account value 0 |
Collateral |
ApplicationCode |
Collateral with an application code of 70 |
Collateral |
AppraisalDate |
collateral appraisal date last month |
Collateral |
AppraisalValue |
collateral appraisal value above 5000 |
Collateral |
IsPledged |
Customers with pledged collateral |
Collateral |
IsPrimary |
Customers with no primary collateral |
Collateral |
MSACode |
Collateral with MSA Code 0 |
Collateral |
Make |
Collateral make is CHEVROLET |
Collateral |
Model |
Collateral Model is FUSION |
Collateral |
OriginalAppraisalDate |
Collateral with Original Appraisal Date this year |
Collateral |
OriginalAppraisalValue |
Customers with Original Appraisal Value above 10000 |
Collateral |
RecordingDate |
Collateral with Recording Date last week |
Collateral |
RelatedAccountType |
Collateral with Related Account Type 125 |
Collateral |
TotalValue |
Collateral with Total Value below 20000 |
Collateral |
TypeCode |
collateral type code is A1 |
Collateral |
VIN |
Collateral with VIN |
Collateral |
Year |
Collateral with Year 2016 |
Branch |
N/A | I want a list of branches |
Branch |
Branch_CloseDate |
branches closed date is last week |
Branch |
Branch_OpenDate |
branch opened date is last month |
Branch |
Branch_Name |
show me Topeka branch |
Branch |
WasAcquired |
branches are acquired |
Branch |
WasAcquired |
acquired branches |
Lending |
BoardingDate |
Loans with boarding date last month |
Lending |
BookBalance |
Loans with Book Balance over 1000 |
Lending |
ApplicationCode |
Loans with Federal Call Code 300 |
Lending |
NonAccrualDate |
Loans with Non Accrual Date last week |
Account |
OfficerID |
Loans with officer id 697 |
Lending |
OriginalPrincipalAmount |
Loans with Original Principal Amount above 500 |
Lending |
OriginalTerm |
Loans with Original Term below 120 |
Lending |
PaymentFrequencyCode |
Loans with Payment Frequency Code 4 |
Cards |
ATMDailyWithdrawalLimit |
Customer with ATM daily withdrawal limit 5000 |
Cards |
ATMTransactionWithdrawalLimit |
Customer with ATM Transaction withdrawal limit above 5000.00 |
Cards |
BIN |
Customer with bin number 551336 |
Cards |
StatusChangeDate |
Customers changed cards status after 2019 |
Cards |
Cards_Status |
cards status is active |
Cards |
CloseReason |
Cards with close reason 0 |
Cards |
LastTransactionDate |
cards expired last week |
Cards |
IsCorporateCard |
Customers with Corporate Cards |
Cards |
IsPrimaryAccount |
show me primary cards |
Cards |
IsSavingsCard |
Customers with savings cards |
Cards |
LastTransactionDate |
Customers with cards last transaction this week |
Cards |
LastUsageAmount |
Cards with last usage amount below 100 |
Cards |
OriginalFirstUsageDate |
Customers with original first usage after 2000 |
Cards |
OriginalCardIssueDate |
Customers with Original Card Issue Date this week |
Lending |
IsInterestRateVariable |
How many loans with variable interest rate |
Lending |
OriginationDate |
loans with an origination date between April 1st, 2020 and March 31st, 2021 |
Lending |
LoanType |
Loans with loan types 135 |
Lending |
CurrentLoanStatus |
delinquent loans |
Lending |
CurrentLoanStatus |
chargeoff loans |
Lending |
ProductCode |
number of credit card loans |
Lending |
ProductCode |
list of agricultural loans |
Cards |
IsDebitCard |
How many members have debit cards |
Cards |
IsDebitCard |
How many debit cards do we have |
Cards |
IsCreditCard |
Number of credit cards |
Cards |
ProductType |
Cards are ATM type |
Cards |
ProductType |
I want a list of cards with product type ATM card |
Aunsight¶
Workflow Improvements¶
Time Between Retries for Workflow Components¶
Workflow users have long made use of the number of retries parameter to add resilience to their data pipelines. Because many job failures are linked to transient issues such as waiting for a dataset lock to be released, Aunsight determines a time to wait that increases with each subsequent retry. Previously, Aunsight determined this interval automatically, but because the appropriate time to wait depends on the type of failure anticipated, we now allow users to specify a time to wait between retries for a component.
Time to wait between retries is now configured via the UI based on four user supplied parameters:
- Minimum delay: The minimum time to wait, in milliseconds.
- Maximum delay: The maximum time to wait, in milliseconds.
- Backoff factor: An exponential factor to increase subsequent retry waiting periods. Expressed as a floating point with a maximum of two decimal places (e.g.
2.0). - Randomize delay: Set to
trueorfalseto allow a randomize delay between the minumum and maximum to be chosen.
Workflow Component Input Dependencies¶
The Workflow Builder allows users to configure automated workloads by connecting the inputs and outputs of tasks (sometimes called "ports" generically) and configuring settings based on those inputs. Many input ports depend logically on others. For example, selecting a process version to run depends on the ID of the selected process. For this reason, this month our UI now shows workflow component input dependencies in the component sidebar. These inputs will now indicate the name of their dependency so that users can quickly identify which inputs need to be connected or configured first before connecting or configuring dependent ports.
Workflow Input Validation¶
In addition to workflow component dependencies, Workflow Builder now includes component input validation. Previously, users could connect incompatible connections between components--for example, connecting a JSON object to an input that requires a simple string input. As this leads to workflow failures, the Workflow Builder now has awareness of data types required by component inputs, and will not allow a user to connect incompatible inputs. Within the UI, this is indicated by the cursor changing shape when a connection is about to be made:
Datamart Engine Optimizations¶
This month we are releasing the first part of several months worth of back-end improvements to our Datamart engine. This month, we have refactored the microservice that executes Daybreak user queries into a stateful service. This first step will yield immediate performance increases, as well as allowing for subsequent improvements such as connection pooling that will further enhance query efficiency and execution times.
Aunsight™ Golden Record¶
Log4j Vulnerability Patch¶
This month, we have reviewed our platform infrastructure to address an evolving security threat in one of our software dependencies: Apache Log4j, a Java logging utility. A recently discovered security vulnerability with this library can allow a bad-actor to execute arbitrary code injected into log messages generated by an attacker-controlled LDAP server. After a thorough review of our infrastructure, it was determined that the Log4j vulnerability only affected one of our underlying systems, and a patch has been applied to secure our platform from this ongoing security threat. As reports of successful attacks continue to appear in the news, users can rest assured that AuGR has been hardened to prevent an attack against the platform.
Release Contents¶
| Issue ID | Description |
|---|---|
| AUN-11620 | In Parameters box, "Text" should be changed to chararray |
| AUN-14862 | Move workflow mail service from Google to Microsoft |
| AUN-14993 | Add NLP metadata sections to Datamart tables and Datamart columns |
| AUN-15021 | Move Walleye Tasks to K8s & Alluxio Internal Storage |
| AUN-15183 | Okta: Implement Aunsight-lib-login in Sightglass Mobile |
| AUN-15258 | Okta: Clean up legacy frontend authentication functionality |
| AUN-15316 | Gateway Augment Communication Timeout - Indicate Rendezvous Worker Chosen |
| AUN-15374 | Require description: (Front end) |
| AUN-15375 | Require description (backend) |
| AUN-15376 | Add default messaging on existing/active workspaces |
| AUN-15386 | Add req_id/trace/span/client to activity events |
| AUN-15388 | Clear Insights when changes are made to the core table |
| AUN-15395 | Lib-ui Error component, pass error messages through DOMPurify, with tests |
| AUN-15428 | Remove inactive users shown in sharing window in the UI |
| AUN-15442 | Daybreak: Enforce limit of 10,000 enums v2 |
| AUN-15444 | Improve lib-au-js error reporting |
| AUN-15453 | Update Webpack to v5 in Aunsight Webapp |
| AUN-15457 | Update supported browser/requirements |
| AUN-15476 | Resource UI warn on pending changes or deletion |
| AUN-15497 | Datamart Migration and Load events don't include datamart id |
| AUN-15501 | Move query list to right side of page & add Data Dictionary Tab |
| AUN-15503 | Add Data Dictionary Events for UserPilot |
| AUN-15535 | Define CSP in webapp |
| DATAINT-617 | Go-Between updates for Job Trigger Resources |
| DATAINT-637 | Agent Notification Subscription - UX tweak |
| DATAINT-640 | Fully Deprecate Old Matching |
| DATAINT-681 | Aunalytics Idea Portal - AuGR - Prevent AuGR from logging out while a test query is running |
| DATAINT-787 | Ability to Hotfix Specific Agents |
| DATAINT-789 | Add API to request Agent Log Upload - Go-Between |
| DATAINT-790 | Provide upload endpoint for Agent logs |
| DATAINT-791 | Agent - Download logs from UI option |
| DATAINT-794 | Session Expiration UX improvement |
| DATAINT-842 | Password Requirements Listed on Change Password Screen |
| PZ-1266 | CIFI 3.0 - Deploy Service Assembly Line |
| PZ-1268 | CIFI 2.2 - Deploy Exception table addition to LastSync table |
Bug Fixes¶
| Issue ID | Client Request | Description |
|---|---|---|
| AUN-13182 | Fix Workflow Builder read only view in Safari | |
| AUN-13705 | removed modal-backdrop and modal-open classes when modal is destroyed |
|
| AUN-15210 | Aunsight WF and DF job detail pages no longer shows update timestamp | |
| AUN-15228 | Fixed bug around validation message not showing for invalid output. | |
| AUN-15275 | Fix bug when there are no recent searches | |
| AUN-15318 | Added a check for an empty string before fetch, show 'recent searches' otherwise. | |
| AUN-15337 | service_locks and global_locks are set to null instead of removed |
|
| AUN-15365 | Added a form wrapper with a keyup.enter to the NLPWidget | |
| AUN-15387 | fixing the landing page redirection when a user only has one workspace. | |
| AUN-15410 | Fixes a race condition in start up which could cause the context menu to not load correctly | |
| AUN-15433 | added a .fail() to fetch | |
| AUN-15437 | T20211013.0542 | Date Cast Creates Timezone Difference and DaysBetween is off by 1 day. |
| AUN-15441 | T20211014.0201 | Sightglass Web browser - Breadcrumb behavior repeats "root" with change in Search criteria |
| AUN-15445 | Create user from System Dashboard>User throws handled error (type mismatch) | |
| AUN-15465 | Making sure errors are displayed on insights when queries are batched | |
| AUN-15472 | Print test results in Daybreak Jenkins builds | |
| AUN-15478 | Fixing bug where tool would go blank if a query was opened that referenced a deleted table | |
| AUN-15480 | Assuring labels for markdown fields are aligned correctly. | |
| AUN-15486 | Allow webapp unit tests to run consistently regardless of timezone of host machine. | |
| AUN-15536 | Fixing a bug where if you change contexts before config finishes loading, it might override the current context with previous one | |
| AUN-15539 | T20211117.0280 | Copy jobs deleting records in ONB |
| DATAINT-282 | Trim Whitespace on Deletes | |
| DATAINT-537 | Make append-only resilient against job stopping | |
| DATAINT-705 | Agent Display Name vs. Agent Name being Displayed | |
| DATAINT-757 | T20210901.0625 | Ascent - Opening a Well Resorts Version Browser Results Erratically |
| DATAINT-786 | Resuming Writebacks displays big red error | |
| DATAINT-796 | SSO User Lockout expiration context missing | |
| DATAINT-81 | [BUG UAT]Inconsistencies count doesn't get updated | |
| DATAINT-813 | Batch History page missing virtual scroll or paging option | |
| DATAINT-848 | T20211104.1294 | TEP Prod - Output Mapping Requiring Date Format for String Object |
| DATAINT-857 | Nucleus shape level purge triggers multiple source level purge | |
| DATAINT-714 | T20210811.0332 | Default attribute ordering/sort |
| DATAINT-776 | T20210830.0480 | Include Pilot Logs in Auto Clean-up |
| PRODSUP-987 | T20211213.0647 | Log4j Vulnerability Fix |