Skip to main content
Version: 13.x (Current)

User Manager performance

This document contains information about performance tests of the User Manager Service.

Job scheduler concurrency tests

A couple of concurrency tests are available for the job scheduler that handles the import and activation in Auth0 of multiple users. The tests goal is to verify that with many replicas of the microservice running there are no concurrency issues with the multiple job schedulers accessing the same job resources.

note

The bulk user import and activation, and so the job scheduler, are available since version 1.5.0 of the User Manager Service.

The tests consist in sending many requests in parallel to activate users or fetch jobs details to multiple User Manager replicas.

Run the tests

Set the number of replicas of the User Manager service to at least four units. Run the scripts saved in the microservice repository in the /scripts/tests/concurrency folder.

Tests output

Everything works smoothly, and no concurrency issues are found.

In particular the job scheduler is able to access and update the jobs correctly. When a job handler fails, the job details persisted in MongoDB are not updated, the job is still pending and it is processed again in the next scheduled execution.

CSV Bulk import load/stress tests

Description

The tested endpoint is the PATCH /users/import. It allows to import users from a CSV file, without activating them through an authentication service.

The results are collected from multiple tests sending a series of requests to the endpoint, and varying some conditions:

  • number of imported users
  • the USER_IMPORT_BATCH_SIZE environment variable
  • the CSV data, which may contain or not arrays
  • the CPU resources
  • the RAM resources

Results

UsersFile sizeResponse TimeBatch sizeCSV arrayCPURAMOutcome: Success/Total
101.5k3.88s250yesR=0m, L=50mR=50Mi, L=70Mi10/10
507.4K4.73s250yesR=0m, L=50mR=50Mi, L=70Mi50/50
10015k13.43s250yesR=0m, L=50mR=50Mi, L=70Mi99/100
20030k24.27s250yesR=0m, L=50mR=50Mi, L=70Mi199/100
30045K25.88s250yesR=0m, L=50mR=50Mi, L=70Mi299/300
50075K6.83s250yesR=0m, L=50mR=50Mi, L=70Mi0/500
1000150k10.37250yesR=0m, L=50mR=50Mi, L=70Mi0/1000
10015k15.80s50yesR=0m, L=50mR=50Mi, L=70Mi99/100
30045K28.73s50yesR=0m, L=50mR=50Mi, L=70Mi299/300
50075K7.73s50yesR=0m, L=50mR=50Mi, L=70Mi0/500
1000150K1000yesR=0m, L=50mR=50Mi, L=70Mi0/1000
50075K1000yesR=0m, L=50mR=50Mi, L=70Mi0/500
30045K1000yesR=0m, L=50mR=50Mi, L=70Mi0/300
30045K1000yesR=0m, L=50mR=50Mi, L=70Mi299/100
30045K1000noR=0m, L=50mR=50Mi, L=70Mi299/100
50075K44s1000noR=0m, L=50mR=50Mi, L=70Mi130/500
50075K57s1000noR=0m, L=50mR=50Mi, L=70Mi499/500
10000787k22s1000noR=0m, L=50mR=50Mi, L=70Mi0/10000
2000154K22s1000noR=0m, L=50mR=50Mi, L=70Mi0/2000
2000154K250noR=50m, L=100mR=100Mi, L=100Mi250/500
2000154K250noR=300m, L=300mR=100Mi, L=100Mi0/2000
info

The result outcome indicates the ratio between successfully imported users and the total amount of users to be imported.

Notes

The endpoint shows various problems when importing more than 50 users at a time due to performance issues.

Auth0 bulk activation load/stress tests

Description

The tested endpoint is the POST /jobs/bulk-activation. It allows to activate multiple users in Auth0 with an async operation.

Results

UsersResponse TimeCPURAMOutcome: Success/Total
200asyncR=0m, L=50mR=50Mi, L=70Mi200/200
300asyncR=0m, L=50mR=50Mi, L=70Mi200/300
1000asyncR=0m, L=50mR=50Mi, L=70Mi0/1000
info

The results outcome indicates the ratio between successfully activated users and the total amount of users to be activated.

Notes

The endpoint can handle comfortably up to 200 users at a time, but it will fail to activate more than 200 users due to performance issues.

Auth0 bulk change password load/stress tests

Description

The tested endpoint is the POST /jobs/change-password/bulk. It allows to trigger multiple change-password events in Auth0 with an async operation.

Results

UsersTime to completionRate Limit per secondCPURAMOutcome: Success/Total
200227s1R=0m, L=50mR=50Mi, L=70Mi200/200
500502s1R=0m, L=50mR=50Mi, L=70Mi500/500
10001602s1R=0m, L=50mR=50Mi, L=70Mi1000/1000
200102s2R=0m, L=50mR=50Mi, L=70Mi200/200
1000503s2R=0m, L=50mR=50Mi, L=70Mi1000/1000
20068s3R=0m, L=50mR=50Mi, L=70Mi143/200
20043s5R=0m, L=50mR=50Mi, L=70Mi106/200
20022s10R=0m, L=50mR=50Mi, L=70Mi146/200
1000111s10R=0m, L=50mR=50Mi, L=70Mi223/1000

Notes

When the rate limit per second is too high some change-password events fail. This happens since the global rate limit of Auth0 Authentication API is exceeded. These tests uses the free tier of Auth0, which allows a maximum of 300 requests per minute.

The rate limit value can be configured with the environment variable AUTH0_BULK_CHANGE_PASSWORD_RATE_LIMIT_PER_SECOND. The default value is equal to 2 requests per second in order to avoid failures due to the Auth0 free tier rate limit. Higher values up to 10 can be set for this variable, but are only safe for bulk operations with few users.