The objective of these guidelines is to guarantee uniformity in the logs generated by the platform and, thus, to ensure an easy usage.
Logs are generated according to Elastic Common Schema specifications.
The following are guidelines, drawn from guidelines and best practices suggested by Elastic, which have to be followed when application logs are generated:
Field names have to be in
Field names does not have to contain special characters;
Field names have to properly be used in the singular or plural form according to the content of the field itself;
For custom fields use, where it makes sense, nesting in objects;
Avoid using abbreviations in field names as much as possible.
The text message passed to the log is, by default, inserted in the key
message, if only one object is passed to the logger. Make sure that the key used for the message is the same.
When an error log is generated, the passed object must necessarily have the key
error, whose value has to be an object of the form specified by ECS for the Error Fields, which are shown in the table below:
|Error code describing the error. type: ||core|
|Unique identifier for the error. type: ||core|
|Error message. type: ||core|
|The stack trace of this error in plain text. type: ||extended|
|The type of the error, for example the class name of the exception. type: ||extended|
NB: none of the fields is mandatory, however it is important not to use custom keys.
There are 6 debug levels:
It is important to always use the correct level; to choose which is the appropriate level, one should rely on the following criteria:
fatal: in cases of unexpected and not recoverable error, and as a result of which the service must stop its execution;
error: in cases of not recoverable error and error for which the processing of the request (but not the service) must be interrupted;
warning: in cases of recoverable error, the service can continue to process the request;
info: in cases where the service has to give information about the branch of code in execution, in general this type of log should be strictly necessary;
debug: to report information that may be useful in troubleshooting (do not use in production);
trace: to trace the operations flow of the application (do not use in production).
It is important to enter in the logs the right information that is used to track operations, highlight problems and allow troubleshooting. It is equally important to pay close attention to what is entered in the logs, avoiding, in any case, to insert in the messages or in the custom properties of the log, any value that can lead to privacy data.
Each core service must necessarily generate the logs of the table below.
Generally, the management of these logs is entrusted to internal libraries:
If the service is not REST but takes its inputs from another source, the same criterion is applied; use a log
trace when the processing event starts and a log
info when the event ends; the final log has to contain as much useful information as possible.
In each log, the following fields have to be always present:
request_id: taken from the platform headers and necessary to trace the flow of each request;
time: that signals the moment when it was generated in ISO 8601 format.
user_agent have to maintain the format specified in ECS; the available parameters are reported here.
None of the fields below is mandatory.
|Hostname of the host. It normally contains what the ||core|
|Name of the host. It can contain what ||core|
|Seconds the host has been up. type: ||extended|
|Size in bytes of the request body. type: ||extended|
|Total size in bytes of the request (body and headers). type: ||extended|
|HTTP request method. The field value must be normalized to lowercase for querying. See the documentation section "Implementing ECS". type: ||extended|
|Referrer for this HTTP request. type: ||extended|
|Size in bytes of the response body. type: ||extended|
|Total size in bytes of the response (body and headers). type: ||extended|
|HTTP response status code. type: ||extended|
|HTTP version. type: ||extended|
|Domain of the url, such as www.elastic.com. In some cases a URL may refer to an IP and/or port directly, without a domain name. In this case, the IP address would go to the ||extended|
|Portion of the url after the ||extended|
|If full URLs are important to your use case, they should be stored in ||extended|
|Path of the request, such as "/search". type: ||extended|
|Port of the request, such as 443. type: ||extended|
|The query field describes the query string of the request, such as "q=elasticsearch". The ||extended|
|Scheme of the request, such as "https". Note: The ||extended|
|Name of the device. type: ||extended|
|Name of the user agent. type: ||extended|
|Unparsed user_agent string. type: ||extended|
|Version of the user agent. type: ||extended|
For backwards compatibility issues, FluentBit will be configured to remap certain keys:
@timestampwill be taken from the field
errwill be remapped with the
erris a string);
reqId(and possible variants, such as
request_id) will be remapped on
ECS expects fields related to information on the container from which the logs were generated; we expect to be able to extract this information during the log recovery phase, in particular