Version: 8.x (Current)

Microservice Gateway performance

PRE and POST hooks#

It is not convenient to use them to define and model business flows and to manage logics with high performances.

caution

If you are having performance problems with PRE and POST Hooks, it can be due to:

  • slowness of cache management;
  • debugging difficulties.

Requests and limits#

Here are listed different requests and limits configurations that should be applied to the CPU and memory of Microservice Gateway in order to reach optimal performances on different scenarios.

note

Following these guidelines allows you to bring Microservice Gateway to an optimal state where:

  • performances are not restricted by K8s CPU throttling policies
  • the minimum amount of resources are used on the cluster.

Results have been obtained through specific tests implementing a series of requests to the Microservice Gateway.

Specifically, four endpoints have been identified for the tests:

  • an endpoint with a pre decorator
  • an endpoint with 5 pre decorators
  • an endpoint forced through the Microservice Gateway by an apposite flag reachable from the design section
  • an endpoint with a post decorator

These endpoints have been used to stress the Microservice Gateway with requests and monitor its performances.

Factors such as the amount of data involved or the way used to comunicate with the Microservice Gateway may have a strong impact on performances, creating the need of finding specific requests and limits levels for different situations. For these reasons, We reported results coming from different scenarios, and we recommend applying the configuration that is more appropriate to your case.

Each scenario includes multiple categories of users and the number of requests per second the Microservice Gateway can serve for that userbase.

note

In order to scale to even higher number of users, we suggest increasing the number of replicas and treat each replica as one of the identified user categories.


Passing by API Gateway#

When accessing the Microservice Gateway using the Console, requests are normally redirected through API Gateway. This service helps correctly rerouting requests and keeping only necessary connection, optimizing the usage of cluster resources.

Here we list requests and limits levels that should be applied for a standard use of Microservice Gateway:

UsersRequests Per SecondCPU RequestsCPU LimitsMemory RequestsMemory Limits
50507025050300
10010012040050300
25025023080050300
500~500380120050300

Only Microservice Gateway#

If you need to directly access to the Microservice Gateway, you should consider a small loss of optimization that was provided by API Gateway.

The following are the limits and requests that should be applied to Microservice Gateway in this scenario:

UsersRequests Per SecondCPU RequestsCPU LimitsMemory RequestsMemory Limits
505015070050300
100100260100050300
250250560200050300
500~5001000350050300

For more than 250 users request levels start to reach the core, so we advise scaling horizontally, increasing replicas to lower the load on a single instance of the service.


Using Large Configmaps#

When complex configurations including elevated numbers of endpoints and routes are required, the Microservice Gateway will mount a very large configmap. This factor can have some influence on the required setup for the service, hence we need to consider higher levels of requests and limits for the Microservice Gateway.

note

In our tests, we considered the hypothesis of having a configmap of 500 kilobytes (that basically corresponds to having more than a thousand routes set for your service).

In this scenario, CPU throttling may arise much earlier than normal. We recommend keeping high levels of limits and verify the presence of throttling when applying these values:

UsersRequests Per SecondCPU RequestsCPU LimitsMemory RequestsMemory Limits
5050150100060300
100100300200080400
250~250560300080400
500~5001000400090400

Even in this case, we recommend to opt for an horizontal scaling when considering more than 250 users, since the request levels start to be close to a single core.


Using Large Payloads#

When we start to increase the size of the payload sent with requests, things start to complicate and we should be aware of the limitations characterizing the Microservice Gateway. In those circumstances, CPU usage drastically increase and CPU throttling may arise much earlier than a standard configuration.

note

In our tests, we considered having a payload of 1 Megabyte for each request.

In this scenario, CPU requests are over the single core even for 50 users. When considering more users, we recommend switching to multiple replicas and lowering the number of users for each instance of the service.

Only Microservice Gateway#
UsersRequests Per SecondCPU RequestsCPU LimitsMemory RequestsMemory Limits
504012004000160500
Passing through API Gateway#

In this scenarios, passing through the API Gateway may help in lowering memory requests and levels:

UsersRequests Per SecondCPU RequestsCPU LimitsMemory RequestsMemory Limits
50401300400050300