Skip to main content
Version: 13.x (Current)

Control Plane Workloads Configuration

This section helps you configure new or existing Fast Data services so that they can communicate with the Control Plane.

Compatibility Matrix

To see if your microservices have support for your Control Plane version, give a look at the compatibility matrix section.

As mentioned in the Overview section, a service configured to interact with the Control Plane will have all its artifacts (topics, projections or single views) in a paused state, meaning that no data will be processed until a resume interaction will be requested by the user.

danger

If you are configuring Control Plane configurations for existing Fast Data workloads, they will be released in a paused state after deployment and they will not process data.

This may cause some delay in your architecture.

To reduce as much possible this delay, you can relay on Bulk actions to quickly resume entire pipeline stages.

Configuration​

To enable a Fast Data service to communicate with a Control Plane instance, you have to define a JSON configuration that should match the one given to the Control Plane.

The JSON object has the same properties for all Fast Data microservices and must be compliant to the following JSON Schema.

Loading ....

Choose below the service you want to configure with Control Plane to see where to place this configuration.

Projection Storer​

To configure the Control Plane connection for the Projection Storer, first you need to attach the microservice to a System of Record.

Then, you can scroll over the Control Plane section and write your JSON configuration. More details on the Control Plane configuration for Projection Storer service can be found in the service configuration page.

PS Configuration

gRPC Host Resolution

To enable gRPC communication between the Control Plane and the Fast Data runtime, services within the Fast Data runtime should reference the gRPC server contained in the Control Plane using its corresponding Kubernetes hostname.

For example, consider a use case where we have a Control Plane microservice named fd-control-plane-grpc. Then, workloads should have the settings properties configured as follows:

{
  // ...
  "settings": {
    "grpc": {
      "host": "fd-control-plane-grpc" 
      // or fd-control-plane-grpc.svc.cluster.local, etc...
    }
  }
  // ...
}

Secrets Resolution​

The following fields contain secrets that can be managed using the Fast Data Secrets Resolution mechanism:

  • runtime.feedback.configuration.['bootstrap.servers']
  • runtime.feedback.configuration.['sasl.username']
  • runtime.feedback.configuration.['sasl.password']
  • runtime.feedback.configuration.['ssl.truststore.certificates']
  • runtime.feedback.configuration.['ssl.keystore.key']
  • runtime.feedback.configuration.['ssl.key.password']
  • runtime.feedback.configuration.['ssl.keystore.certificate.chain']
  • runtime.state.configuration.['bootstrap.servers']
  • runtime.state.configuration.['sasl.username']
  • runtime.state.configuration.['sasl.password']
  • runtime.state.configuration.['ssl.truststore.certificates']
  • runtime.state.configuration.['ssl.keystore.key']
  • runtime.state.configuration.['ssl.key.password']
  • runtime.state.configuration.['ssl.keystore.certificate.chain']

Examples​

Here are listed some configuration examples that you can use as a starting point for your workloads.

Workload Configuration with Kafka Channels

{
    "state": {
      "type": "kafka",
      "channel": "actionTopic",
      "configuration": {
        "bootstrap.servers": "localhost:9092",
        "client.id": "test-factory",
        "sasl.mechanism": "plain",
        "group.id": "groupId",
        "ssl.enabled": false
      }
    },
    "feedback": {
      "type": "kafka",
      "configuration": {
        "bootstrap.servers": "localhost:9092",
        "client.id": "test-factory",
        "sasl.mechanism": "plain",
        "heartbeat.ms": 2500,
        "ssl.enabled": false
      },
      "channel": "heartbeatTopic"
    }
}

Workload Configuration with gRPC Channels

{
  "state": {
    "type": "grpc"
  },
  "feedback": {
    "type": "grpc"
  },
  "settings": {
    "grpc": {
      "host": "fd-control-plane"
    }
  }
}

Bind Microservices to Runtime​

Once a Control Plane configuration has been defined for one of the workloads above mentioned, upon saving, the Console will create an additional configuration, that we refer to as bindings, that will contain the mapping between the Projections or the Single Views with a specific pipeline and its own artifacts.

This configuration will be read-only and:

  • for the Projection Storer, the configuration will be managed internally by the microservice;
  • for other microservices, an additional environment variable CONTROL_PLANE_BINDINGS_PATH managed by Mia-Platform Console will be set-up with the full path of the new configuration.