Bug 216 - zone transfer memory leak
zone transfer memory leak
Status: RESOLVED FIXED
Product: NSD
Classification: Unclassified
Component: NSD Code
other
Other All
: P3 normal
Assigned To: NSD team
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2008-10-15 20:54 CEST by Srinivasa
Modified: 2011-03-02 16:30 CET (History)
4 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Srinivasa 2008-10-15 20:54:14 CEST
The nsd version: 3.1.1

It was found when NSD gets xfers from bind 9.3. Did testing on ‘linux Red Hat 3.4 & 3.2’ the memory leak is reproducible every time when zone updates are pulled from master. Once OS cannot allocate memory NSD simply forwards updates to xfer db file. Only way NSD can serve new data is by restarting.

The log messages shows:
[1222475586] nsd[13390]: info: Handle incoming notify for zone test.com
[1222475586] nsd[13390]: info: Handle incoming notify for zone biz.com
[1222475586] nsd[13390]: info: Handle incoming notify for zone biz.com
[1222475593] nsd[15201]: warning: signal received, reloading...
[1222475593] nsd[15211]: info: memory recyclebin holds 573484 bytes
[1222475594] nsd[15211]: error: fork failed: Cannot allocate memory
[1222475594] nsd[15201]: warning: Reload process 15211 failed with status 256, continuing with old database
[1222475594] nsd[13390]: error: xfrd: zone biz.com: soa serial 5930174 update failed restarting transfer (notified zone)
[1222475594] nsd[13390]: info: Handle incoming notify for zone biz.com
[1222475643] nsd[15202]: info: Notify received and accepted, forward to xfrd
[1222475643] nsd[13390]: info: Handle incoming notify for zone test.com
[1222475643] nsd[13390]: info: Handle incoming notify for zone test.com
[1222475643] nsd[15201]: warning: signal received, reloading...
[1222475644] nsd[15214]: info: memory recyclebin holds 573484 bytes
[1222475644] nsd[15214]: error: fork failed: Cannot allocate memory

I am not running nsd-patch so frequently though. Not sure it has any impact.
Comment 1 Matthijs Mekking 2008-10-21 09:25:13 CEST
Could you try to put nsdc patch in a cron job, so that it runs regularly and see if the problem still occurs?

Regards, 

Matthijs
Comment 2 Srinivasa 2008-11-04 17:22:05 CET
Yes, I have setup cronjob to run every hour with out any luck. (Not sure how nsdc process can help).
Comment 3 Vicky Shrestha 2008-11-24 21:31:31 CET
I am testing both nsd 3.1.1 and 3.2.0 on x86_64(CentOS 5.2) and seeing the memory leak as well. With each zone notify and update, there is an increase of around 30M(in my case).

I am running nsdc patch regulary (tried with 1 hour and one day).
I tried disabling nsec3 and tsig, the but problem persists.


First startup
# ps ax -o rss,size,command |grep nsd; free -m
  536   264 syslogd -m 0 -a /var/nsd/dev/log
3234068 3240528 /usr/sbin/nsd -c /var/nsd/nsd.conf
3396196 3400328 /usr/sbin/nsd -c /var/nsd/nsd.conf
3396236 3400868 /usr/sbin/nsd -c /var/nsd/nsd.conf
             total       used       free     shared    buffers     cached
Mem:         11769       8889       2880          0        182       2064
-/+ buffers/cache:       6642       5127
Swap:         1023          6       1017

Notify and reload
# ps ax -o rss,size,command |grep nsd
  536   264 syslogd -m 0 -a /var/nsd/dev/log
3234068 3240528 /usr/sbin/nsd -c /var/nsd/nsd.conf
3426188 3430320 /usr/sbin/nsd -c /var/nsd/nsd.conf
3426124 3430736 /usr/sbin/nsd -c /var/nsd/nsd.conf 

# ps ax -o rss,size,command |grep nsd
  536   264 syslogd -m 0 -a /var/nsd/dev/log
3234068 3240528 /usr/sbin/nsd -c /var/nsd/nsd.conf
3456180 3460312 /usr/sbin/nsd -c /var/nsd/nsd.conf
3456116 3460728 /usr/sbin/nsd -c /var/nsd/nsd.conf

# ps ax -o rss,size,command |grep nsd
  536   264 syslogd -m 0 -a /var/nsd/dev/log
3234068 3240528 /usr/sbin/nsd -c /var/nsd/nsd.conf
3486172 3490304 /usr/sbin/nsd -c /var/nsd/nsd.conf
3486212 3490844 /usr/sbin/nsd -c /var/nsd/nsd.conf

After running for an hour or so
# ps ax -o rss,size,command |grep nsd; free -m
  536   264 syslogd -m 0 -a /var/nsd/dev/log
3234096 3240528 /usr/sbin/nsd -c /var/nsd/nsd.conf
6365412 6369536 /usr/sbin/nsd -c /var/nsd/nsd.conf
6365448 6370076 /usr/sbin/nsd -c /var/nsd/nsd.conf
  712   252 grep nsd
             total       used       free     shared    buffers     cached
Mem:         11769      11666        103          0         98       2015
-/+ buffers/cache:       9552       2217
Swap:         1023          6       1017

Comment 4 Matthijs Mekking 2009-09-03 14:58:57 CEST
Unfortunately, we were not able to reproduce this bug until so far.
Comment 5 Wouter Wijngaards 2011-03-02 16:30:40 CET
Hi Srinivasa,

We have since reproduced a memory leak, and it is fixed in svn for the next release.  See the upcoming 3.2.8RC1 if you wish to test, and let us know if the issue persists.

Best regards, Wouter