What are Processes

Processes represent one of the main ways that users can extend the Aunsight platform capabilities with custom code that interacts with the platform through its Javascript or Python SDKs. Processes are docker containers that run code within the platform cluster but can also access the Global internet. As such, processes can serve as a point of contact between Aunsight and other IT and data analytics tools.

In Aunsight, processes are managed as records that contain metadata about each process and its versions. Process versions represent a unique Docker image and can have different parameters and metadata. Each Aunsight process can have multiple process versions and . Because other Aunsight objects like workflows and AuQL scripts can interact with processes without having to manage the details of Docker image versions, processes provide an abstracted way to extend a data analytics process.

This article discusses the process workspace, a means of managing process records and the versions each process contains.

The Processes Workspace

The Aunsight web interface provides a process workspace to users who have the AU-PROCESS:view-process permission available to them through a role or group they have in a given context.

To access this workspace, log in to the web interface and select the context you wish to work in through the context selector. From that context's dashboard, click the "Process" icon () in the palette on the right.

The process workspace is a standard list-based view of processes available in the present context. You can search () the list to find a process by clicking the appropriate icon at the top of the list. To create a new process, click the "plus" icon (). If a process is not appearing in the list, or its details seems out of date, click the refresh () icon to update the data in the list.

Creating a Process

Clicking the "plus" icon () from the process list will bring up a process creation dialog.

To create a new process, it is necessary to specify a name for the process and click the "create" button ().

In addition to the process name, there are additional settings that you may wish to specify:

• Description The description field is a text field that will be used to generate the process's description text. Though the field accepts only plain text, it will render markdown formatted content as rich text. The description field can be edited at any time after the process is created.

• Solutions Solutions are custom tags (32 characters) denoting that a process relates to a particular data solution. This helps to group a process together with the other Aunsight objects that relate to it.

• Dev Mode Dev mode tags are used by solutions architects to describe specific stages for a process and other Aunsight objects. For example, a DEV tag usually means the creator of this process intends it as a part of a solution development process, whereas the PROD tag signifies a production process delivered to data consumers. Use of these tags for specific purposes is arbitrary, but teams should be consistent in their use of Dev Mode tag nomenclature.

Viewing and Editing a Process

Once a process is created or a user has selected one from the process list, the Web interface will display the process record in the main view of the interface. The web application interface displays information and tools in several tabs to group similar information and functions logically. Additionally, the process record has a group of action buttons that perform tasks relating to the process record as a whole.

Details Tab

By default, the web interface displays the details tab. Details are displayed in sections.

The Process info section displays basic metadata for the workflow, such as ID and description of the workflow and any tags it may have (Dev mode and solution). To edit these these descriptions, click the edit icon () in the upper right corner of this section to change the name, description, and tagging information.

The Context section displays context ownership details, as well as providing a tool to share the process with a new context. Clicking "Share with new context" will bring up the context sharing dialog.

Using this dialog, users can add a new share by specifying a context and a set of policies and permissions to grant to that context.

Note

Granting permissions to a context only grants the possibility of access to members of the specified context; members still need to have those permissions within that context.

The Version Info section displays information about the currently specified version.

The Process Activity History displays a record of activity regarding the process record. For information about process job runs, see the jobs tab.

Run Tab

The run tab allows users to run the current process version on demand and specify parameters for that job. Users can provide an alternate name for the job (by default, Aunsight uses the name of the process as the job name) as well as specifying a job token. In addition to these settings, users can specify other parameters in the various sections:

The Resource section allows uses to specify which compute resource they wish to run the process against. Resources also allow the user to configure settings, such as the amount of RAM and CPU cores allocated to the job.

The Max Duration section specifies a timeout limit for the process in seconds. This can be useful if there is the possibility that the job may fail ungracefully and require automatic process termination after a set amount of time.

The Capabilities section configures what platform capabilities will be made available to the process container; for example, a scratch space volume for non-persistent storage, or a loggerstream to allow logging of essential information.

The Environment Variables section specifies environmental variables passed on to the job as key/value pairs. If defaults are specified in the process version parameters, these can be temporarily overridden by changing the key/value pairs in this section.

New Version Tab

Each version of an Aunsight process record permits users to upload a new Docker image which will be associated with that process.

Each new process version can be created by specifying a description (process names are assigned automatically by sequence number) and default environmental variables that will be sent to the process. When done specifying options, click "Create" to create a new process version and set it as the current process version.

Note

Users can set the process record to any version by using the versions action button.

Jobs Tab

The jobs tab provides a convenient way to monitor the state of jobs. The jobs tab provides a filtered view of data tracked in the jobs workspace so that users can easily monitor when and how a process was run, and what the results were. The jobs list shows the job ID, the name of the process, activity dates, state ("In Progress", "Succeeded", or "Failed"), and the name of the user who ran the process job.

Clicking the arrow icon () on the job will leave the process workspace for the jobs workspace where you can review more details about that job's status.

Process Action Buttons

In addition to the features contained on the tabs of the process workspace, an action button group provides tools that affect the entire process record.

Versions List

The versions action button allows users to select a different version of the process to edit in the record display from a dropdown list of processes. Since each version of a process contains different metadata, this button will change the data displayed on the tabs of the record view to reflect the specified process version.

Delete

The delete action button allows users to delete a process version or the entire process record itself along with its associated Docker images. If just one version of a process exists, clicking delete will bring up a message asking the user to confirm if they wish to delete the entire process record.

Warning

Process deletion is irreversible!

If more than one version of the process exists, however, clicking the delete action button will allow the user to delete either the record as a whole or just the currently selected process version.

Selecting "Delete version" will only delete the current process version and its associated Docker image.

Warning

Process version deletion is irreversible!

Note

When deleting versions, the version will be lost forever, but the version numbers of all subsequent versions will not be updated. This means users can still tell there was an intermediate version, even though it is no longer recoverable.