Skip to main content
Version: 11.7.x

Unmapped Fields

This dashboard will help you to monitor unmapped fields. With unmapped field we mean a field which is in a Kafka message, but it is not included in the projection definition or it can not be casted to the desired field type.

In order to use this dashboard you'll need to set the environment variable COLLECTION_NAME_UNMAPPED_FIELDS_STORAGE in any Real Time Updater service you want to keep track of the unmapped fields.

The dashboard relies on the MongoDB Query Exporter service and as such you should check out the MongoDB Query Exporter Configuration section. Once you have the service set up you need to add the two metrics displayed in the following example to the main configuration file.

...
metrics:
- name: unknown_fields
type: counter #Can also be empty, the default is gauge
servers: [main] #Can also be empty, if empty the metric will be used for every server defined
help: 'Count the number of Unknown fields'
value: total
labels: []
mode: pull
cache: 0
constLabels:
collection: unmapped-fields
database: {{FAST_DATA_PROJECTIONS_DATABASE_NAME}}
collection: unmapped-fields
pipeline: |
[
{
"$match": {
"type": "UNKNOWN_FIELD",
"__STATE__": "PUBLIC"
}
}, {
"$count": "total"
}
]
- name: uncastable_fields
type: counter #Can also be empty, the default is gauge
servers: [main] #Can also be empty, if empty the metric will be used for every server defined
help: 'Count the number of Uncastable fields'
value: total
labels: []
mode: pull
cache: 0
constLabels:
collection: unmapped-fields
database: {{FAST_DATA_PROJECTIONS_DATABASE_NAME}}
collection: unmapped-fields
pipeline: |
[
{
"$match": {
"type": "UNCASTABLE_FIELD",
"__STATE__": "PUBLIC"
}
}, {
"$count": "total"
}
]

After having configured everything you can download the dashboard's json here and import it on your Grafana portal.