Adapt pid files handling for API auth process pool #28757

jnasselle · 2025-03-21T22:32:23Z

Related issue
Closes #28754

Description

This issue aims to create pid files (/var/ossec/var/run/wazuh-apid*) as processes being created by the API. Change motivated by #28653 improvements that could spawn one to 50 processes to handle auth requests.

Testing

https://jenkins-staging.qa.wazuh.info/job/Test_integration_endpoints/22/

root@wazuh-dev:~/repos/wazuh# grep authentication_pool_size /var/ossec/api/configuration/api.yaml
authentication_pool_size: 50
root@wazuh-dev:~/repos/wazuh# ps -aux | grep wazuh_apid.py | grep -v grep
wazuh     143023 18.9  1.0 848928 107308 ?       Sl   10:43   0:09 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143024  0.0  0.5 139668 60944 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143025  0.0  0.5 139668 60164 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143026  0.0  0.5 139668 60164 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143027  0.0  0.5 139668 60164 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143028  0.0  0.5 139668 60168 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143029  0.0  0.5 139668 60168 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143030  0.0  0.5 139668 59864 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143031  0.0  0.5 139668 59864 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143032  0.0  0.5 139668 59864 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143033  0.0  0.5 139668 59868 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143034  0.0  0.5 139668 60180 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143035  0.0  0.5 139668 60176 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143036  0.0  0.5 139668 60176 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143037  0.0  0.5 139668 60176 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143038  0.0  0.5 139668 60180 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143039  0.0  0.5 139668 60180 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143040  0.0  0.5 139668 60180 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143041  0.0  0.5 139668 60180 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143042  0.0  0.5 139668 60188 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143043  0.0  0.5 139668 60188 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143044  0.0  0.5 139668 60188 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143045  0.0  0.5 139668 60188 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143046  0.0  0.5 139668 60192 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143047  0.0  0.5 139668 60196 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143048  0.0  0.5 139668 60196 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143049  0.0  0.5 139668 60196 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143050  0.0  0.5 139668 60196 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143051  0.0  0.5 139668 60200 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143052  0.0  0.5 139668 60204 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143053  0.0  0.5 139668 60204 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143054  0.0  0.5 139668 60204 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143055  0.0  0.5 139668 60204 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143056  0.0  0.5 139668 60204 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143057  0.0  0.5 139668 60204 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143058  0.0  0.5 139668 60204 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143059  0.0  0.5 139668 60212 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143060  0.0  0.5 139668 60212 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143061  0.0  0.5 139668 60212 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143062  0.0  0.5 139668 60212 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143063  0.0  0.5 139668 60212 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143064  0.0  0.5 139668 60152 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143065  0.0  0.5 139668 60156 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143066  0.0  0.5 139668 60156 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143067  0.0  0.5 139668 60224 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143068  0.0  0.5 139668 60228 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143069  0.0  0.5 139668 60228 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143070  0.0  0.5 139668 60228 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143071  0.0  0.5 139668 60100 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143072  0.0  0.5 139668 60164 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143073  0.0  0.5 139668 60040 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143076  0.0  0.6 287132 61248 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
wazuh     143079  0.0  0.6 369060 61284 ?        S    10:43   0:00 /var/ossec/framework/python/bin/python3 /var/ossec/api/scripts/wazuh_apid.py
root@wazuh-dev:~/repos/wazuh# ps -aux | grep wazuh_apid.py | grep -v grep | wc -l
53
root@wazuh-dev:~/repos/wazuh# ls /var/ossec/var/run/wazuh-apid*.pid
/var/ossec/var/run/wazuh-apid-143023.pid       /var/ossec/var/run/wazuh-apid_auth-143041.pid  /var/ossec/var/run/wazuh-apid_auth-143059.pid
/var/ossec/var/run/wazuh-apid_auth-143024.pid  /var/ossec/var/run/wazuh-apid_auth-143042.pid  /var/ossec/var/run/wazuh-apid_auth-143060.pid
/var/ossec/var/run/wazuh-apid_auth-143025.pid  /var/ossec/var/run/wazuh-apid_auth-143043.pid  /var/ossec/var/run/wazuh-apid_auth-143061.pid
/var/ossec/var/run/wazuh-apid_auth-143026.pid  /var/ossec/var/run/wazuh-apid_auth-143044.pid  /var/ossec/var/run/wazuh-apid_auth-143062.pid
/var/ossec/var/run/wazuh-apid_auth-143027.pid  /var/ossec/var/run/wazuh-apid_auth-143045.pid  /var/ossec/var/run/wazuh-apid_auth-143063.pid
/var/ossec/var/run/wazuh-apid_auth-143028.pid  /var/ossec/var/run/wazuh-apid_auth-143046.pid  /var/ossec/var/run/wazuh-apid_auth-143064.pid
/var/ossec/var/run/wazuh-apid_auth-143029.pid  /var/ossec/var/run/wazuh-apid_auth-143047.pid  /var/ossec/var/run/wazuh-apid_auth-143065.pid
/var/ossec/var/run/wazuh-apid_auth-143030.pid  /var/ossec/var/run/wazuh-apid_auth-143048.pid  /var/ossec/var/run/wazuh-apid_auth-143066.pid
/var/ossec/var/run/wazuh-apid_auth-143031.pid  /var/ossec/var/run/wazuh-apid_auth-143049.pid  /var/ossec/var/run/wazuh-apid_auth-143067.pid
/var/ossec/var/run/wazuh-apid_auth-143032.pid  /var/ossec/var/run/wazuh-apid_auth-143050.pid  /var/ossec/var/run/wazuh-apid_auth-143068.pid
/var/ossec/var/run/wazuh-apid_auth-143033.pid  /var/ossec/var/run/wazuh-apid_auth-143051.pid  /var/ossec/var/run/wazuh-apid_auth-143069.pid
/var/ossec/var/run/wazuh-apid_auth-143034.pid  /var/ossec/var/run/wazuh-apid_auth-143052.pid  /var/ossec/var/run/wazuh-apid_auth-143070.pid
/var/ossec/var/run/wazuh-apid_auth-143035.pid  /var/ossec/var/run/wazuh-apid_auth-143053.pid  /var/ossec/var/run/wazuh-apid_auth-143071.pid
/var/ossec/var/run/wazuh-apid_auth-143036.pid  /var/ossec/var/run/wazuh-apid_auth-143054.pid  /var/ossec/var/run/wazuh-apid_auth-143072.pid
/var/ossec/var/run/wazuh-apid_auth-143037.pid  /var/ossec/var/run/wazuh-apid_auth-143055.pid  /var/ossec/var/run/wazuh-apid_auth-143073.pid
/var/ossec/var/run/wazuh-apid_auth-143038.pid  /var/ossec/var/run/wazuh-apid_auth-143056.pid  /var/ossec/var/run/wazuh-apid_events-143079.pid
/var/ossec/var/run/wazuh-apid_auth-143039.pid  /var/ossec/var/run/wazuh-apid_auth-143057.pid  /var/ossec/var/run/wazuh-apid_exec-143076.pid
/var/ossec/var/run/wazuh-apid_auth-143040.pid  /var/ossec/var/run/wazuh-apid_auth-143058.pid
root@wazuh-dev:~/repos/wazuh# ls /var/ossec/var/run/wazuh-apid*.pid | wc -l
53
root@wazuh-dev:~/repos/wazuh# /var/ossec/bin/wazuh-control stop
wazuh-clusterd not running...
Killing wazuh-modulesd...
Killing wazuh-monitord...
Killing wazuh-logcollector...
Killing wazuh-remoted...
Killing wazuh-syscheckd...
Killing wazuh-analysisd...
wazuh-maild not running...
Killing wazuh-execd...
Killing wazuh-db...
Killing wazuh-authd...
wazuh-agentlessd not running...
wazuh-integratord not running...
wazuh-dbd not running...
wazuh-csyslogd not running...
Killing wazuh-apid...
Wazuh v4.11.2 Stopped
root@wazuh-dev:~/repos/wazuh# ^C
root@wazuh-dev:~/repos/wazuh# ls -la /var/ossec/var/run/
total 8
drwxrwx--- 2 root wazuh 4096 Mar 26 10:49 .
drwxr-x--- 9 root wazuh 4096 Mar 26 10:49 ..
root@wazuh-dev:~/repos/wazuh# ps -aux | grep wazuh_apid.py | grep -v grep | wc -l
0

api.log

2025/03/26 10:43:23 INFO: Checking RBAC database integrity...
2025/03/26 10:43:23 INFO: /var/ossec/api/configuration/security/rbac.db file was detected
2025/03/26 10:43:23 INFO: RBAC database integrity check finished successfully
2025/03/26 10:43:23 DEBUG2: Creating 'authentication_pool' process pool
2025/03/26 10:43:23 DEBUG2: Creating 'process_pool' process pool
2025/03/26 10:43:23 DEBUG2: Creating 'events_pool' process pool
2025/03/26 10:43:31 DEBUG: Loaded API configuration: {'host': ['0.0.0.0', '::'], 'port': 55000, 'drop_privileges': True, 'experimental_features': False, 'max_upload_size': 10485760, 'authentication_pool_size': 50, 'intervals': {'request_timeout': 10}, 'https': {'enabled': True, 'key': '/var/ossec/api/configuration/ssl/server.key', 'cert': '/var/ossec/api/configuration/ssl/server.crt', 'use_ca': False, 'ca': '/var/ossec/api/configuration/ssl/ca.crt', 'ssl_protocol': 'auto', 'ssl_ciphers': ''}, 'logs': {'level': 'debug2', 'format': 'plain', 'max_size': {'enabled': False, 'size': '1m'}}, 'cors': {'enabled': False, 'source_route': '*', 'expose_headers': '*', 'allow_headers': '*', 'allow_credentials': False}, 'access': {'max_login_attempts': 50, 'block_time': 300, 'max_request_per_minute': 300}, 'upload_configuration': {'remote_commands': {'localfile': {'allow': True, 'exceptions': []}, 'wodle_command': {'allow': True, 'exceptions': []}}, 'limits': {'eps': {'allow': True}}, 'agents': {'allow_higher_versions': {'allow': True}}, 'indexer': {'allow': True}, 'integrations': {'virustotal': {'public_key': {'allow': True, 'minimum_quota': 240}}}}}
2025/03/26 10:43:31 DEBUG: Loaded security API configuration: {'auth_token_exp_timeout': 900, 'rbac_mode': 'white'}
2025/03/26 10:43:32 INFO: Listening on ['0.0.0.0', '::']:55000.
2025/03/26 10:43:32 INFO: Getting installation UID...
2025/03/26 10:43:32 INFO: Getting updates information...
2025/03/26 10:49:53 INFO: Shutdown wazuh-apid server.

javiersanchz

Review

LGTM !

framework/wazuh/core/common.py

framework/wazuh/core/cluster/utils.py

GGP1

LGTM

nico-stefani

I'm requesting some minor fixes

api/scripts/wazuh_apid.py

framework/wazuh/core/pyDaemonModule.py

nico-stefani

LGTM

fdalmaup

LGTM

jnasselle self-assigned this Mar 21, 2025

jnasselle linked an issue Mar 21, 2025 that may be closed by this pull request

API integration tests fail as disabling privilege dropping does not impact #28754

Closed

3 tasks

javiersanchz self-requested a review March 24, 2025 10:47

javiersanchz previously approved these changes Mar 24, 2025

View reviewed changes

fdalmaup requested changes Mar 25, 2025

View reviewed changes

framework/wazuh/core/common.py Outdated Show resolved Hide resolved

fdalmaup requested changes Mar 25, 2025

View reviewed changes

framework/wazuh/core/common.py Outdated Show resolved Hide resolved

jnasselle dismissed javiersanchz’s stale review via 7328b0d March 25, 2025 12:06

jnasselle requested review from fdalmaup, javiersanchz and a team and removed request for javiersanchz March 25, 2025 16:26

This comment was marked as outdated.

Sign in to view

Force the creation of pid files

a6cfff0

jnasselle force-pushed the bug/28754-api-auth-multiple-processes branch from 7328b0d to a6cfff0 Compare March 25, 2025 20:18

jnasselle added 2 commits March 26, 2025 09:05

Force processes creation from execution pools

a200e5f

Fix naming and remove unused imports

84b56a2

jnasselle requested a review from nico-stefani March 26, 2025 12:21

GGP1 reviewed Mar 26, 2025

View reviewed changes

framework/wazuh/core/cluster/utils.py Outdated Show resolved Hide resolved

Create generic spawn function and move it

c49f5ac

jnasselle force-pushed the bug/28754-api-auth-multiple-processes branch from a3c2028 to c49f5ac Compare March 26, 2025 13:23

jnasselle requested review from javiersanchz and GGP1 March 26, 2025 13:50

GGP1 previously approved these changes Mar 26, 2025

View reviewed changes

nico-stefani requested changes Mar 26, 2025

View reviewed changes

api/scripts/wazuh_apid.py Outdated Show resolved Hide resolved

framework/wazuh/core/pyDaemonModule.py Outdated Show resolved Hide resolved

Rearrange imports to be PEP8 compilant

e9efdda

jnasselle dismissed GGP1’s stale review via e9efdda March 26, 2025 14:21

nico-stefani approved these changes Mar 26, 2025

View reviewed changes

fdalmaup approved these changes Mar 26, 2025

View reviewed changes

fdalmaup merged commit 641f3c2 into 4.11.2 Mar 26, 2025
3 checks passed

fdalmaup deleted the bug/28754-api-auth-multiple-processes branch March 26, 2025 15:05

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adapt pid files handling for API auth process pool #28757

Adapt pid files handling for API auth process pool #28757

jnasselle commented Mar 21, 2025 •

edited

Loading

javiersanchz left a comment

This comment was marked as outdated.

GGP1 left a comment

nico-stefani left a comment

nico-stefani left a comment

fdalmaup left a comment

Adapt pid files handling for API auth process pool #28757

Adapt pid files handling for API auth process pool #28757

Conversation

jnasselle commented Mar 21, 2025 • edited Loading

Description

Testing

javiersanchz left a comment

Choose a reason for hiding this comment

Review

This comment was marked as outdated.

GGP1 left a comment

Choose a reason for hiding this comment

nico-stefani left a comment

Choose a reason for hiding this comment

nico-stefani left a comment

Choose a reason for hiding this comment

fdalmaup left a comment

Choose a reason for hiding this comment

jnasselle commented Mar 21, 2025 •

edited

Loading