Memory leak on nginx -s reload, even after mem leak fixes of v3/master

[baudneo]

Describe the bug

I am either misunderstanding how to configure modsec or there is still a memory leak.

I have created an Nginx-Proxy-Manager (openresty) image that includes modsecurity (built from latest 'v3/master') which includes a pull request that was ment to fix some memory leaks. OWASP-CRS is the default ruleset.

I was compiling modsec with lmdb support which caused HUGE memory leaks (200+MB on every nginx -s reload). I have since changed compiling modsec to use default configuration and the huge memory leak seems to have reduced to only increasing 10-12MB per reload. Sometimes some memory seems to get freed, perplexing!

Before, the modsec directives for enabling and the rule file were in each server {} block and on top of that were also included in some location {} blocks. I read that, that might be causing the memory leak so I changed the directives to be placed in the root http {} block with users having to disable modsec specifically. With modsec directives in http {} block and lmdb support disabled the memory only increases about 12MB every reload.

I have a user that has their own custom ruleset and when they load their custom rules into one of their server {} or location {} blocks they get a huge memory leak of 300+MB every reload.

I have had my server up for roughly 16 hours and my memory usage is at 400MB where before it would be 3+ GB after 12 hours. On a fresh restart memory is around 110MB and starts growing on every nginx -s reload. If I do not enable modsec and do not have any rules_file directives, no memory leak is present.

So my 2 questions are:

1. what is the officially recommended way to enable modsec in nginx and how are we supposed to enable different rulesets per host or location?
2. How can I debug or help diagnose the memory leak?

Logs and dumps

Output of:

1. DebugLogs (level 9)
2. AuditLogs
3. Error logs
4. If there is a crash, the core dump file.

Notice: Be carefully to not leak any confidential information.

To Reproduce

Steps to reproduce the behavior:

Enable modsecurity in root http {} block of nginx.conf

http {

Other stuff;

modsecurity on;
#OWASP-CRS
modsecurity_rules_file /etc/nginx/modsec/main.conf;

server {}
# etc
}

Then nginx -s reload several times over a time period and watch the memory increase.

Expected behavior

Reloading nginx does not cause memory to increase.

Server (please complete the following information):

• ModSecurity version (and connector): Modsec 3.0.8 [82f75dc] and nginx connector 1.0.3
• WebServer: Openresty 1.21.4.1
• OS (and distro): debian buster (docker slim image)

Rule Set (please complete the following information):

• OWASP-CRS 3.3.4

**Additional context **
It's perplexing that some of the memory seems to be freed (i.e. it gets to 1GB and then suddenly drops to 400MB) at random intervals.

The Nginx-Proxy-Manager image runs nginx -s reload quite a bit, it has a timer to check SSL certs and renew them which causes a reload. When a new host is added or modified via the webapp a reload is also done.

[martinhsv]

@baudneo ,

This is a known issue.

The recent pull request to which refer made no pretense of resolving all reload-without-restart memory issues.

[Clemv95]

So is there a way to bypass this ? Since there is no official fix even with this PR

[baudneo]

From the sounds of the reply from the maintainer I'm going to assume we are to be restarting nginx instead of reloding it for the time being. I can add a restart timer to npm to free up memory.

[martinhsv]

What I expect most users choose is to do a restart, rather than reload-without-restart.

That said, some installations might choose to prefer some reloads-without-restart as long as the memory leakage is manageable and then to periodically prefer restart.

[vncloudsco]

Dear: @martinhsv

Restart nginx will sometimes cause problems for users. Users may lose connectivity on systems where multiple domains are running. Restart is really a bad idea for running systems

[martinhsv]

@vncloudsco ,

I was asked what people do instead. I did not claim that restarting nginx has no disadvantages -- of course it does.

[baudneo]

Nginx is giving up on modsec in favor of Coraza

https://www.nginx.com/blog/f5-nginx-modsecurity-waf-transitioning-to-eol/

[martinhsv]

Closing as duplicate

[baudneo]

Lol

[martinhsv]

Huh?

[alexanderadam]

Closing as duplicate

Can you please refer to the duplicated issue? 🤔

[martinhsv]

There were many open issues fundamentally about the same thing. The oldest (until recently) open one was #2502 .

I have closed all 6 older issues and replaced them with a new item ( #2848 )