Skip to main content
Version: 10.x (Current)

Real-Time Updater Low Code configuration

Here, low-code specific configuration will be described. All the documentation regarding generic real time updater features in the manual configuration are still valid and applicable.

Low Code Real Time Updater is available since version 4.2.0

Environment variables

If the System of Records has been created using the automatic configuration, the Real-Time updater has all the environments variables already prepared.

info

You can quickly convert a System of Records from Manual to Low code by changing the USE_AUTOMATIC_STRATEGIES to true. Then, you should follow the next steps to set up you Fast Data Low Code project properly.

danger

When you create a new configmap, remember to use the same Mount Path of your environment variables STRATEGIES_FOLDER, ER_SCHEMA_FOLDER, PROJECTION_CHANGES_FOLDER

For the Automatic Real-Time Updater the kafka_adapter, map_table and cast_function variables are configured by the Mia-Platform console. However, it is fundamental to define the erSchema.json to describe the strategies' path.

ER schema configuration

caution

When a new Real-Time Updater is generated, a base erSchema.json file is generated with the following content:

{ "version": "1.0.0", "config": { } }

This is an empty configuration: the Real-Time Updater Microservice could be deployed without pod restart, but this file must be modified according to the projections associated to this microservice to work properly.

When creating a low code system, its service will have a link to the er-schema configmap. If other microservices already had this configmap they will share it with the new real time updater. If you do not make changes to the default configmaps of low code real time updaters you will have all of them sharing the same Er schema. But if you need a different er-schema (e.g. you have created a new real time updater configured to a different system of records), then you have to unlink the er-schema folder and create a new configmap with its own unique identifier and create a new erSchema.json file in it.

The erSchema.json configmap defines the relationship between tables and projections.

Here you can find a deep explanation of how ER Schema configuration works.

Projection Changes Schema

Differently from the Manual Configuration, the projection changes configurations are described with a json file aimed to reduce the developing effort.

caution

When a new Real-Time Updater is generated, a base projectionChangesSchema.json file is generated with the same content of the erSchema.json file. Despite this configuration will not throw any error during the deploy, the file must be customized according to the related projections.

A single view example:

{
"idCustomer": "ebc12dc8-939b-447e-88ef-6ef0b802a487",
"taxCode": "tax_code",
"name": "MARIO",
"surname": "ROSSI",
"email": "email_mario",
"address": "address_1",
"telephone": "phone_number_1000",
"allergens": [{
"id": "eggs",
"comments": "this is another comment change",
"description": "this is the description"
}]
"__STATE__": "PUBLIC"
}

The data of this single view comes from 3 projections:

  • pr_registry: which contains users personal data
  • pr_allergens: which contains allergens details
  • pr_allergens_registry: that is the table which describes the relationship between users and allergens

Given that idCustomer and ID_USER are the same, the single view is focused on the user. For this reason if an allergen is updated, it will be necessary to identify all the users affected by this change in order to revise the single view.

Therefore, we know that if pr_allergens changes, the path to update the single view is: pr_allergenspr_allergens_registrypr_registry.

This configuration is described with the projectionChangesSchema.json:

projectionChangesSchema.json
{
"version": "1.0.0",
"config": {
"sv_customers": {
"paths": [
{
"path": [ "pr_allergens", "pr_allergens_registry", "pr_registry"],
"identifier": {
"ID_USER": "ID_USER"
}
}
]
}
}
}

The version field holds the current configuration version, which determines the syntax and semantics of the rest of the configuration. For version 1.0.0 these are the fields and their meaning, all inside the config field:

  • sv_customers: it is the single view name
  • paths: it is the path list that can update the single view
  • path: it is the single path
  • identifier: it defines the projection changes identifier, where ID_USER is the name of the field which contains the primary key in the collection pr_registry.
info

Given the pr_allergenspr_allergens_registrypr_registry path, both paths pr_allergen_registrypr_registry and pr_registry are sub-paths of the first one. You can avoid the definition of the sub-paths because they will be automatically recognized.

danger

If automatic aggregation is used, the reference object key (in this case ID_USER) is required to have the same name as the projection changes identifier, which is the starting point of the aggregation. Otherwise, if the automatic aggregation is not used, it is possible naming the key as preferred coherently with the singleViewKey.js and mapper.js file in the Single View Creator. It is possible to write the projectionChangesSchema.json in ConfigMaps & Secret area of the Automatic Single view Plugin.

At the end of the configuration be sure to register the projection in the Strategies' page of your single view:

  • Go to single view page
  • Click on Strategies
  • Add the projection with the newly defined strategies if they do not already exist
  • Click on the checkbox for the automatic strategies management

Custom path of strategies

If you need to manually handle specific strategies you have two choices:

  • You can write your own strategy function. In this case you have to write the whole strategy on your own
  • You can let the Low Code handle the initial path of the strategy, and then make it execute your own custom function to handle it from there

To do that you have to specify in the projectionChangesSchema.json that the identifier will be handled "from file", which is your custom file that exports your custom function. The syntax is __fromFile__[myCustomFunction].

Let's see it in the configuration file below:

projectionChangesSchema.json
{
"version": "1.0.0",
"config": {
"sv_users": {
"paths": [
{
"path": [ "pr_products", "pr_companies", "pr_registry"],
"identifier": {
"ID_USER": "ID_USER"
}
},
{
"path": [ "pr_selling", "pr_clients"],
"identifier": "__fromFile__[myCustomFunction]"
}
]
}
}
}

When the second path is crossed, the path pr_selling → pr_clients is passed through automatically. Once the Real Time Updater reaches the projection pr_clients, it invokes myCustomFunction, where you can write your own custom logic. The custom function has to be placed in a javascript file inside the same configmap as the projectionChangesSchema.json file, which is the one whose mount path is the same as the value of the environment variable called PROJECTION_CHANGES_SCHEMA_FOLDER. The file must export a function matching the following signature:

async function myCustomFunction (logger, mongodbInstance, document)

The value returned by the function has to be an array of objects representing the identifiers of the strategy.

Let's see an example of a custom function:


async function someCustomLogin (value) {
// Custom logic here
}

module.exports = async function myCustomFunction (logger, mongodbInstance, document) {
const query = { ID_USER: document.ID_USER }
const documentFound = await mongodbInstance.collection('pr_allergens').findOne(query)
const response = await someCustomLogin(documentFound)
return [{
name: documentFound.NAME,
idUser: documentFound.ID_USER
}]
}