Bug 218 - Eliminate buffer memory waste for UDP netio handlers
Eliminate buffer memory waste for UDP netio handlers
Product: NSD
Classification: Unclassified
Component: NSD Code
All Linux
: P2 normal
Assigned To: NSD team
Depends on:
  Show dependency treegraph
Reported: 2008-11-14 01:42 CET by John Lightsey
Modified: 2009-01-05 13:29 CET (History)
1 user (show)

See Also:

Share single query between all UDP handlers (1.03 KB, patch)
2008-11-14 01:48 CET, John Lightsey
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description John Lightsey 2008-11-14 01:42:25 CET
Hi there,

In comparing the memory efficiency of NSD and BIND, NSD does much better when there are very few interfaces configured and BIND starts to do better with 30 or more interfaces.  I believe the problem is the way the UDP netio handlers are constructed.

Each NSD server process creates a complete set of TCP and UDP netio handlers for each interface that has been configured.  For TCP handlers, the memory to hold the request isn't allocated until the connection has been accepted and it should be cleared when the connection closes.  For UDP handlers, the memory to hold the request is allocated when the server_child process starts up and isn't released until the server_child process exits.

Since the buffer is 128K, this gets expensive very rapidly.  With a single interface configured, the server_child process on my system has a RSS of 484K adn a VSZ of 6652 (mostly in copy on write memory.)  When I increase the number of interfaces to 20 I end up with a RSS of 2188K and a VSZ of 11116K (much larger than the parent, so it's definitely not copy on write.)  This all happens before any connections come in to raise the numbers further.  If I keep going up on the number of interfaces, memory will keep increasing in a linear fashion.

From what I can see the problem is that query_create() gets called for each UDP netio handler.  It doesn't look like this is necessary though.  handle_udp() processes a UDP packet in one pass, so a single query buffer could be used for all of the UDP netio handlers.  On my system, switching to a single UDP query buffer makes the memory usage with 20 interfaces nearly identical to the memory usage with a single interface, 540K RSS 6656 VSZ.

I'd be very interested in knowing if there is any reason the query buffer can't be shared among all of the UDP netio handlers in this fashion to eliminate the memory waste on machines with many IP addresses.

Comment 1 John Lightsey 2008-11-14 01:48:06 CET
Created attachment 91 [details]
Share single query between all UDP handlers
Comment 2 Matthijs Mekking 2009-01-05 13:29:45 CET
Fixed in trunk, r2833. Thanks for your patch!