Fabric BFF
Fabric BFF, as suggested by the name, works as Backend For Frontend for Control Plane UI and it mediates requests between the frontend and Control Plane service. Besides, it exposes the Websocket interface that enables live updates on Control Plane UI and the JSON RPC APIs for controlling Fast Data state.
Configuration
Configuration of Fabric BFF is a straightforward process that involves setting up a ConfigMap and specifying essential environment variables.
Environment Variables
Fabric BFF can be customized using the following environment variables:
Name | Required | Description | Default Value |
---|---|---|---|
HTTP_PORT | - | This variable determines the TCP port where the HTTP controller binds its listener | 3000 |
LOG_LEVEL | - | Specify the centralized application log level, choosing from options such as debug , error , info , trace or warn | info |
BFF_CONFIGURATION_FILEPATH | - | Set the location of the configuration file | ~/.fd/data-fabric-bff/config.json |
OTEL_EXPORTER_OTLP_ENDPOINT | - | The URL to a GRPC endpoint of an OpenTelemetry Collector. Specifying this value enables the support for OpenTelemetry tracing |
Config Map
The configuration of the service is handled by a JSON file whose location is defined by the BFF_CONFIGURATION_FILEPATH
. When instantiating
Control Plane application from Marketplace, Fabric BFF service configuration is generated with
a dedicated Config Map, named fabric-bff-configuration
. This file contains a template configuration that should help in configuring the service.
- Schema Viewer
- Raw JSON Schema
- Example
{
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "Configuration",
"description": "BFF Service configuration",
"type": "object",
"required": [
"persistence"
],
"examples": [
{
"console": {
"rest": {
"target": {
"type": "env",
"key": "CONSOLE_BASE_URL"
},
"auth": {
"type": "oauth2",
"flow": "client_credentials",
"tokenEndpoint": "/api/m2m/oauth/token",
"credentials": {
"clientId": {
"type": "file",
"path": "/run/secrets/data-fabric/fabric-bff.ini",
"key": "FABRIC_CLIENT_ID"
},
"clientSecret": {
"type": "file",
"path": "/run/secrets/data-fabric/fabric-bff.ini",
"key": "FABRIC_M2M_CLIENT_SECRET"
}
}
}
}
},
"controlPlane": {
"rest": {
"target": "control-plane"
},
"grpc": {
"target": "http://control-plane:50051"
}
},
"persistence": {
"type": "mongodb",
"configuration": {
"url": {
"type": "file",
"path": "/run/secrets/data-fabric/fabric-bff.ini",
"key": "MONGODB_URL"
},
"database": "data-fabric-db"
}
}
}
],
"properties": {
"console": {
"anyOf": [
{
"$ref": "#/definitions/ConsoleSettings"
},
{
"type": "null"
}
]
},
"controlPlane": {
"anyOf": [
{
"$ref": "#/definitions/ControlPlaneSettings"
},
{
"type": "null"
}
]
},
"openLineage": {
"anyOf": [
{
"$ref": "#/definitions/OpenLineageSettings"
},
{
"type": "null"
}
]
},
"persistence": {
"$ref": "#/definitions/Persistence"
},
"settings": {
"$ref": "#/definitions/Settings"
}
},
"definitions": {
"AuthContext": {
"oneOf": [
{
"type": "object",
"oneOf": [
{
"type": "object",
"required": [
"credentials",
"flow",
"tokenEndpoint"
],
"properties": {
"credentials": {
"$ref": "#/definitions/ClientCredentials"
},
"flow": {
"type": "string",
"enum": [
"client_credentials"
]
},
"tokenEndpoint": {
"type": "string"
}
}
}
],
"required": [
"type"
],
"properties": {
"type": {
"type": "string",
"enum": [
"oauth2"
]
}
}
}
]
},
"BasicCredentials": {
"type": "object",
"required": [
"clientId",
"clientSecret"
],
"properties": {
"clientId": {
"$ref": "#/definitions/secret"
},
"clientSecret": {
"$ref": "#/definitions/secret"
}
}
},
"ClientCredentials": {
"anyOf": [
{
"$ref": "#/definitions/JWTBearerCredentials"
},
{
"$ref": "#/definitions/BasicCredentials"
}
]
},
"ConsoleSettings": {
"description": "Settings describing the connections with Control Plane service",
"type": "object",
"required": [
"rest"
],
"properties": {
"rest": {
"description": "Configuration related to the exposed REST APIs that are proxied",
"allOf": [
{
"$ref": "#/definitions/RestProxy"
}
]
}
}
},
"ControlPlaneSettings": {
"description": "Settings describing the connections with Control Plane service",
"type": "object",
"required": [
"grpc",
"rest"
],
"properties": {
"grpc": {
"description": "Configuration related to the inter-service communication with Control Plane service",
"allOf": [
{
"$ref": "#/definitions/GrpcProxy"
}
]
},
"rest": {
"description": "Configuration related to the exposed REST APIs that are proxied",
"allOf": [
{
"$ref": "#/definitions/RestProxy"
}
]
}
}
},
"GrpcProxy": {
"description": "Configuration regarding the GRPC services called by the BFF",
"type": "object",
"required": [
"target"
],
"properties": {
"target": {
"description": "An structured representation of a URI employed internally to contact target services",
"type": "string"
}
}
},
"JWTBearerCredentials": {
"type": "object",
"required": [
"clientId",
"clientKeyId",
"privateKey"
],
"properties": {
"clientId": {
"$ref": "#/definitions/secret"
},
"clientKeyId": {
"$ref": "#/definitions/secret"
},
"privateKey": {
"$ref": "#/definitions/secret"
}
}
},
"MongodbPersistence": {
"type": "object",
"required": [
"url"
],
"properties": {
"database": {
"description": "Optional database name. It selects which database to be employed for storing data (it overrides the one provided in the connection string when it is set)",
"default": null,
"type": [
"string",
"null"
]
},
"url": {
"description": "MongoDB connection string",
"allOf": [
{
"$ref": "#/definitions/secret"
}
]
}
}
},
"OpenLineageSettings": {
"description": "Settings describing the connections with OpenLineage service",
"type": "object",
"required": [
"rest"
],
"properties": {
"rest": {
"description": "Configuration related to the exposed REST APIs that are proxied",
"allOf": [
{
"$ref": "#/definitions/RestProxy"
}
]
}
}
},
"Persistence": {
"oneOf": [
{
"type": "object",
"required": [
"configuration",
"type"
],
"properties": {
"configuration": {
"$ref": "#/definitions/MongodbPersistence"
},
"type": {
"type": "string",
"enum": [
"mongodb"
]
}
}
}
]
},
"RestProxy": {
"description": "Configuration regarding the Control Plane APIs that are proxied by the BFF",
"type": "object",
"required": [
"target"
],
"properties": {
"auth": {
"anyOf": [
{
"$ref": "#/definitions/AuthContext"
},
{
"type": "null"
}
]
},
"target": {
"description": "An structured representation of a URI employed internally to contact target services",
"type": "string"
}
}
},
"Settings": {
"description": "Service specific configurations",
"type": "object",
"properties": {
"apiPrefix": {
"description": "Prefix path that it is applied to all the exposed APIs, except from the status ones",
"default": "",
"type": "string"
},
"auditUserHeader": {
"description": "Header containing the user unique identifier",
"type": [
"string",
"null"
]
}
}
},
"secret": {
"oneOf": [
{
"type": "string"
},
{
"type": "object",
"required": [
"key",
"type"
],
"properties": {
"encoding": {
"type": "string",
"enum": [
"base64"
]
},
"key": {
"type": "string"
},
"type": {
"const": "env"
}
}
},
{
"type": "object",
"required": [
"path",
"type"
],
"properties": {
"encoding": {
"type": "string",
"enum": [
"base64"
]
},
"key": {
"type": "string"
},
"path": {
"type": "string"
},
"type": {
"const": "file"
}
}
}
]
}
}
}
{
"console": {
"rest": {
"target": {
"type": "env",
"key": "CONSOLE_BASE_URL"
},
"auth": {
"type": "oauth2",
"flow": "client_credentials",
"tokenEndpoint": "/api/m2m/oauth/token",
"credentials": {
"clientId": {
"type": "file",
"path": "/run/secrets/data-fabric/fabric-bff.ini",
"key": "FABRIC_CLIENT_ID"
},
"clientSecret": {
"type": "file",
"path": "/run/secrets/data-fabric/fabric-bff.ini",
"key": "FABRIC_M2M_CLIENT_SECRET"
}
}
}
}
},
"controlPlane": {
"rest": {
"target": "control-plane"
},
"grpc": {
"target": "http://control-plane:50051"
}
},
"persistence": {
"type": "mongodb",
"configuration": {
"url": {
"type": "file",
"path": "/run/secrets/data-fabric/fabric-bff.ini",
"key": "MONGODB_URL"
},
"database": "data-fabric-db"
}
}
}
In the paragraphs below are explained the main properties of the Fabric BFF configuration file.
Console Communication
In order for Control Plane application to know which Runtimes are available for monitoring, and therefore which ones can be included in Runtime Views, the service needs to contact Mia-Platform Console and retrieve the list of Projects that should be accessible from this Control Plane instance.
To achieve so, it is first necessary that your Company Owner creates a dedicated Service Account on your Mia-Platform Console instance and assign to it the proper permissions for listing the Console projects of interest.
Pay attention to the level of access to the resources that you assign to the Service Account.
For Control Plane use case, a good practice may be to assign the role of guest
at Company level while granting
the reporter
role to all the projects that should be visible by Control Plane.
About permissions assignment, it is possible to go even more granular in case you want to allow visibility only to a subset of runtime environments of a specific project.
In fact, to do that, you may opt to assign the role of guest
even at Project level while granting
the reporter
role solely to those runtime environments that should be visible by Control Plane.
Once the service account has been registered, your Company Owner needs to hand over to you its credentials, which are:
client-id
→ unique identifier generated for this the service account- [Client Secret Basic]
client-secret
→ a string representing the secret needed by the service account to authenticate - [Private Key JWT]
client-key-id
→ unique identifier of the key employed for generating for this the service account - [Private Key JWT]
private-key
→ private key in PEM format
These details then should be inserted in your Fabric BFF service configuration under the console
property. In particular, the fields to be set are:
target
→ specifies the base URL of your Mia-Platform Console instanceauth
→ defines how the Fabric BFF service should authenticate on Mia-Platform Console APIs, that is the service account credentials
It is responsibility of your Company Owner to ensure that service account credentials are properly processed according to your company security policies.
Furthermore, it is of extreme importance understanding that any Control Plane user will be able to list the project name and available environments of all the projects that can be accessed by the service account configured on Fabric BFF.
This is and example of console
property configuration:
- M2M Client Secret Basic
- M2M Private Key
{
"console": {
"target": "https://<your-mia-platform-console-url>",
"auth": {
"type": "oauth2",
"flow": "client_credentials",
"tokenEndpoint": "/api/m2m/oauth/token",
"credentials": {
"clientId": "<service-account-client-id>",
"clientSecret": "<service-account-client-secret>"
}
}
}
// ...other fabric bff configurations
}
{
"console": {
"target": "https://<your-mia-platform-console-url>",
"auth": {
"type": "oauth2",
"flow": "client_credentials",
"tokenEndpoint": "/api/m2m/oauth/token",
"credentials": {
"clientId": "<service-account-client-id>",
"clientKeyId": "<service-account-client-key-id>",
"privateKey": "<service-account-private-key>"
}
}
}
// ...other fabric bff configurations
}
The following properties support secret resolution:
console.target
console.auth.credentials.clientId
console.auth.credentials.clientKeyId
console.auth.credentials.privateKey
Control Plane Communication
Communication between Fabric BFF and Control Plane services occur both via gRPC and HTTP REST requests. For this reason its necessary to
configure on the Fabric BFF the addresses where to reach Control Plane service. This can done by setting the properties rest
and grpc
of controlPlane
field in the Fabric BFF configuration.
In both properties, the target
field should be set to the address where Control Plane service exposes the corresponding one.
Here can be found an example of configuration that assumes Fabric BFF and Control Plane services are deployed within the same K8s namespace:
{
// ...other fabric bff configurations
"controlPlane": {
"rest": {
"target": "<control-plane-service-name>" // when protocol is http, it is not necessary specifying it. When port is not specified, it is assumed the 80 is employed
},
"grpc": {
"target": "http://<control-plane-service-name>:50051" // it is important to notice that GRPC connection uses a different port from the REST target
}
},
// ...other fabric bff configurations
}
Persistence Layer
Currently only MongoDB is supported as persistence layer for storing relevant data, such as the one related to operations' auditing.
The MongoDB database selected for storing Control Plane data must be configured to have replicaSet
enabled, since
Fabric BFF exploits features that can be used only when a replicaSet
is available.
In order to carry out all its operations, Fabric BFF requires a persistence layer where relevant information, such as auditing details, are stored. This configuration can be set under
the persistence.configuration
key of the configuration file. The main properties are:
url
→ the connection string to your MongoDB instance;database
→ the database name where to search for the collections relevant to Fabric BFF service. Please notice that setting this property will override the database name potentially set in the connection string;
An example of persistence configuration can be seen below:
{
// ...other fabric bff configurations
"persistence": {
"type": "mongodb",
"configuration": {
"url": "mongodb://<server>:27017/<default-database>?replicaSet=local",
"database": "<data-fabric-database-name>"
}
},
// ...other fabric bff configurations
}
The following properties support secret resolution:
persistence.configuration.url
persistence.configuration.database
Service Settings
Additionally, the Fabric BFF service itself has a set of properties for changing its behavior. Here are listed the available ones within settings
properties:
apiPrefix
→ the base path applied to all the exposed routes. It defaults to/
;auditUserHeader
→ specifies in which HTTP header can be found the user identifier set by the authentication system. The value of this header will be employed to correlate requests stored by the auditing system with the user that performed them. When using Mia-Platform Authentication and Authorization services this property can be set tomiauserid
.
In case it is not set the auditing system does not correlate users with requests;
Here can be found a configuration example:
{
// ...other fabric bff configurations
"settings": {
"apiPrefix": "/",
"auditUserHeader": "miauserid"
}
}
Endpoints
In the table below is provided the list of endpoints that should be defined in Console and assigned to Fabric BFF service.
Endpoint | Rewrite Base Path | Microservice | Description |
---|---|---|---|
/fast-data | /fast-data | fabric-bff | Groups all the requests related to Control Plane operations |
Routes
Under the endpoint specified above, the following routes are served by Fabric BFF
Route | Type | Method | Description |
---|---|---|---|
/fast-data/feedback | Websocket | GET | Opens a websocket connection with the client to receive updates of runtimes and pipelines |
/fast-data/control | REST | HEAD | Verifies whether the pipeline change state action can be carried out by current user |
/fast-data/control | REST | POST | Receives JSON-RPC from the frontend to change pipelines state |
/fast-data/* | REST | * | Other routes are forwarded towards Control Plane service. For more details please read corresponding documentation |