Bug 743 - DNS Spoofing - Max number of local-zone: / local-data: entries allowed?
DNS Spoofing - Max number of local-zone: / local-data: entries allowed?
Status: ASSIGNED
Product: unbound
Classification: Unclassified
Component: server
1.5.4
x86_64 FreeBSD
: P5 enhancement
Assigned To: unbound team
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2016-02-13 06:50 CET by bbcan177
Modified: 2016-02-17 00:35 CET (History)
2 users (show)

See Also:


Attachments
Unbound-checkconf - partial truss output (15.25 KB, text/plain)
2016-02-13 06:50 CET, bbcan177
Details

Note You need to log in before you can comment on or make changes to this bug.
Description bbcan177 2016-02-13 06:50:35 CET
Created attachment 323 [details]
Unbound-checkconf - partial truss output

Is there a limit to the number of entries allowed in Unbound for the following DNS Spoofing lines?

   local-data: "ads.xyz.com 60 IN A 127.0.0.1"
   local-zone: "zzz.com" redirect local-data: "zzz.com 60 IN A 127.0.0.1"

In my testing with "Unbound-checkconf, it seems to hit a limit at around 1.02 million entries at which time it faults to a "Signal 9" error and crashes Unbound, requiring a reboot.  Testing on FreeBSD 10.1-RELEASE-p24 w/4GB memory. I assume that its crashing due to memory exhaustion, but would like to see if there is any chance to optimize/improve the code for this scenario? See attachment for a partial truss output.

When "local-zone" entries are not used, there doesn't seem to be any issue with entries over 1.3 million. Haven't tested beyond that.

(Include file can be provided if required)
Comment 1 bbcan177 2016-02-13 18:03:28 CET
Update: unbound-checkconf completes with "no errors" when testing in a box w/8GB ram. It seems to hit just over 4GB of ram usage at its peak. Execution time of approx 25 seconds.

When testing w/4GB ram, it crashes unbound and a reboot is necessary to restore unbounds functionality.

Are there any optimizations that can be made to either FreeBSD or Unbound to improve this?
Comment 2 Wouter Wijngaards 2016-02-15 09:08:55 CET
Hi bbcan177,

This is a memory limit on your system.  Unbound does not have a set limit in the code, but it uses memory for the configuration.

If you compile in 32-bit mode (for 4Gb machine), then pointers are half the size and maybe you can fit more entries in that 4Gb machine?

Otherwise, I can only advise you to install more memory if you want to configure more entries.

Best regards, Wouter
Comment 3 bbcan177 2016-02-15 17:57:44 CET
Hi Wouter,

I have tested unbound-checkconf in a FreeBSD 10.1-RELEASE-p25 (32 Bit) machine; however it fails on both 4GB and 8Gb machines with: 

   " ERR#12 'Cannot allocate memory' "


truss unbound-checkconf unb.conf

w/4Gb

mmap(0x0,4194304,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON,-1,0x0) ERR#12 'Cannot allocate memory'
break(0x28000000)                                = 0 (0x0)
mmap(0x0,4194304,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON,-1,0x0) ERR#12 'Cannot allocate memory' break(0x28400000)                                ERR#12 'Cannot allocate memory'
getpid()                                         = 66822 (0x10506)
[1455553829] unbound-checkconf[66822:0] error: out of memory
write(2,"[1455553829] unbound-checkconf[6"...,61) = 61 (0x3d)
getpid()                                         = 66822 (0x10506)
[1455553829] unbound-checkconf[66822:0] error: could not enter zone xgmajdckxxgqxdqxe.cXoXm redirect
write(2,"[1455553829] unbound-checkconf[6"...,99) = 99 (0x63)
getpid()                                         = 66822 (0x10506)
[1455553829] unbound-checkconf[66822:0] fatal error: failed local-zone, local-data configuration
write(2,"[1455553829] unbound-checkconf[6"...,97) = 97 (0x61)
process exit, rval = 1


w/8Gb

mmap(0x0,4194304,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON,-1,0x0) ERR#12 'Cannot allocate memory'
break(0x28000000)                                = 0 (0x0)
mmap(0x0,4194304,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANON,-1,0x0) ERR#12 'Cannot allocate memory'
break(0x28400000)                                ERR#12 'Cannot allocate memory'
getpid()                                         = 96371 (0x17873)
[1455554021] unbound-checkconf[96371:0] error: out of memory
write(2,"[1455554021] unbound-checkconf[9"...,61) = 61 (0x3d)
getpid()                                         = 96371 (0x17873)
[1455554021] unbound-checkconf[96371:0] error: could not enter zone xgmajdckxxgqxdqxe.cXoXm redirect
write(2,"[1455554021] unbound-checkconf[9"...,99) = 99 (0x63)
getpid()                                         = 96371 (0x17873)
[1455554021] unbound-checkconf[96371:0] fatal error: failed local-zone, local-data configuration
write(2,"[1455554021] unbound-checkconf[9"...,97) = 97 (0x61)
process exit, rval = 1


I checked the include file, and there is only one line for that particular domain. It failed at line 681,715.

grep -n "xgmajdckxxgqxdqxe.cXoXm" file

681715:local-zone: "xgmajdckxxgqxdqxe.cXoXm" redirect local-data: "xgmajdckxxgqxdqxe.cXoXm 60 IN A 127.0.0.1"


The good news was atleast the 32bit version didn't crash Unbound completely, whereas, the 64Bit version needs a reboot to allow unbound to restart correctly. Is it possible to fail more gracefully?

I am the developer for a package that downloads various Domain based feeds and processes those for an Unbound DNS sinkhole application. With the 25 Domain based lists available, it amounts to just over 1 million malicious/AD server domains... It would be great to find a workaround for this issue.

Currently, its working fine but since adding "local-zone" entries, it requires a more substantial amount of memory to process.

Would you have any other recommendations?
Comment 4 Wouter Wijngaards 2016-02-16 09:25:13 CET
Hi bbcan177,

So, the 32bit version is not big enough; you'll have to use the 64bit version and get a machine with more memory. (16Gb, 32Gb ... more).

local-zone entries are hierarchical, and if you want to cover all possible space underneath a name you can use a single local-zone entry to do so.  That might reduce the number of local-zone entries that you need, and save memory for you.

I cannot make your system deal with a single large out of memory process more gracefully, it is not an unbound thing.  (perhaps search for linux OOM killer settings, it has kernel options, but I do not think they are really going to help).

Another option is to create a string of resolvers, with part of the dataset, that forward to another, with the one end a full recursor and the other end a caching machine ...

Best regards, Wouter
Comment 5 Wouter Wijngaards 2016-02-16 09:32:58 CET
Hi bbcan177,

Another option, you want to redirect to 127.0.0.1 now, but you could also make these names NXDOMAIN perhaps?  You can do this with local-zone: zzz.com static and no local-data entry.  Without the local-data entry, that might save you some space.

Best regards, Wouter
Comment 6 Wouter Wijngaards 2016-02-16 10:43:53 CET
Hi bbcan177,

Another option.  This one only blocks the names as exactly written in the config.  So the subdomains are not blocked unless specifically mentioned.  That is different from the redirect and static options.  You create only one local-zone, higher up, like local-zone: "com" "transparent".  And then add all the local-data statements that you have (www.zzz.com A 127.0.0.1).  This then creates one local-zone structure in memory and all the local-data structures are packed into memory fairly tightly (in a memory region).  This construction may therefore save a lot of memory; but I am not sure if it will block the right contents.

Best regards, Wouter
Comment 7 bbcan177 2016-02-17 00:35:41 CET
Thanks Wouter,

I am away for a week, but will try these options and see what I can come up with. Ideally, adding local-zone: "com" "transparent" would be ok as long as it blocks all of the sub-domains. It will take some effort to automate the formatting of 1 million domains to fit this scheme :)  Will update you later on my finding. If any other solutions come up, I'd be happy to hear them...

Thanks!