[ldns-users] pyldns - memory leaks and double freeing

Wed Apr 13 14:54:46 UTC 2011

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Bedrich,

I've applied your patches in trunk.
The tests run all fine with me.
Thanks for fixing this.

Best regards, Willem

On 04/13/11 08:45, Bedrich Kosata wrote:
> Hi again,
> 
> as nobody seemed to have anything against my proposal, I prepared the
> patches that should fix the problems I outlined before :)
> 
> When the patches are applied, the python bindings should always have
> rr_list and rr structs separated in memory. This is accomplished by
> always cloning rr data when either adding them to an rr_list or
> retrieving them from the rr_list.
> Most of the changes should be transparent and should not require
> modifications to existing code. The only outside change I had to make
> was in the "ldns_verify_rrsig_keylist_notime" and
> "ldns_verify_rrsig_keylist" functions, because it copies pointers to the
> correct keys into a second rr_list. I solved this by replacing each of
> these functions by two new ones (thus creating four functions). All
> versions take only 3 arguments and differ in the way they treat the
> good_keys value. One versions name ends with "_status_only" (e.g.
> ldns_verify_rrsig_keylist_notime_status_only) and this does not report
> the good_keys at all, just the result of verification. The other ends
> with "_" (e.g. ldns_verify_rrsig_keylist_notime_) and returns a tuple
> (status, good_keys) where good_keys are indexes of the keys in the key
> list supplied to the function. Thus the whole business of two rr_lists
> sharing the same data is sidestepped.
> 
> The patches are made for each file separately because the fixes are
> mostly independent. I also included several test scripts which should
> show how the situation was fixed. They either show a situation where the
> old version would crash with a segmentation fault or display the amount
> of memory the app has used thus pointing to a memory leak. When used
> with patched sources, none of the scripts should crash and none should
> report more that ~30MB of memory used.
> 
> Best regards
> 
> Beda
> 
> p.s.- the patch for ldns_rdf.i is in fact not related to the rest and
> fixes and unrelated memory leak in ldns_rdf.new_frm_str
> 
> p.p.s.- I did not touch a related issue of rr and rdf relationship. In
> current version, the rdf data does not live beyond the lifetime of the
> corresponding rr. This is demonstrated in the attached
> test-rrlist-get-rdf.py. The problem could again be solved by cloning all
> rdfs on retrieval and addition from/into an rr.
> However, I will wait how the current patch is received before I start
> thinking about fixing it.
> 
> 
> On 03/25/2011 09:48 AM, Bedrich Kosata wrote:
>> Hi everybody,
>>
>> while trying to find a cause of a memory leak in a simple script, I
>> found a nest of memory related issues in the python bindings.
>> The problems are all related to one common problem - who takes care of
>> memory of composite objects, such as ldns_rr_lists or ldns_pkt.
>> For example, in the current version, ldns_pkt bindings use ldns_pkt_free
>> to free a packet structure, however, when a rr_list is taken from the
>> packet and returned from a function, the packet gets out of scope, is
>> freed and the rr_list refers to already freed memory.
>> On the other hand, a rr_list only frees its own memory, not memory of
>> the stored rrs. This can lead to memory leaks.
>> I am attaching two scripts that demonstrate these problems (it might be
>> necessary to have the sources patched with the "freeing None" patch I
>> sent last week).
>> I would be willing to have a stab at the problems (provided I get
>> clearance from my boss :)), but the only solution I think would be clean
>> enough, is to clone the necessary bits where needed. This might lead to
>> some inefficiency and slowdowns (probably not big), so I would like to
>> ask if this is ok.
>> If there is anyone else willing to fix this, I would be happy to act as
>> a tester.
>>
>> Cheers
>>
>> Beda
>>
>> p.s.- test-pkt-free.py crashes with a segmentation fault,
>> test-rr-list.py ends up eating about 130 MB of memory and more than
>> 400000 python objects which python cannot free.
>>
>>
>>
>> _______________________________________________
>> ldns-users mailing list
>> ldns-users at open.nlnetlabs.nl
>> http://open.nlnetlabs.nl/mailman/listinfo/ldns-users
> 
> 
> 
> 
> _______________________________________________
> ldns-users mailing list
> ldns-users at open.nlnetlabs.nl
> http://open.nlnetlabs.nl/mailman/listinfo/ldns-users

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (FreeBSD)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQIcBAEBAgAGBQJNpbk2AAoJEOX4+CEvd6SYMx0P/i4X059sn9fXRk7WzEdOdsz4
AtDbi1B/rf2D26WN2rjrQwJRhmaC6s20YKeo9h4cFBqcZrROaOWseeMqN24iOYsI
Rt1ltWPbe+2th+G1o9r6F7keZU3i1PUxPGEgzpTe8o8xicq5tP9pJlpI/43ISj2T
KHKLi1Ps6B/TPOM519q50FCapTFU/IlaybONGJ12Ts3QhtwUB/Q0uUL5DmJWM8B6
iz2Jqy5/ABJYmyiVDup2rtVW/nZI0t032PCP8qHBEVlfZvRyX7Cm0TMLpXLH80cH
w6AXkdj+FUOD7uDi8cxGxMeQnYg/OtgjerTaVqc0+1v3XgwxflReIxpZ1rudUz81
YGf04XFuzk8+2/X3wukfXLoNAMQCSaqsQLtR5N1u/JMDt5KlBRcpzEPQJjLbxTBc
ABsCuCuhrVgVclMTkLXZ/aIVAcx1Rjwjgppxr3Rc0Ixp+gPEWZb8SZc5ZAvucAI1
lVHVi7gohiFdQ9tILByxkFVvbzTYvbkojUJ2zgobJuGdfJ/KnIQ/mt9lZ3PF3IEu
jkPpNRKGqY9diTuMQt9hQvKbXQF0KedP1kjUYFDhHql2li0xpdVzvHq+Xpgc/jGj
owj4faYKHo9QSxbMhshuypV3+QF3Ck9l2IwGCTWw2qM9ffXKH+eB8HA5aXsuz70p
dSfoo4W83DQRqjdS3Vk/
=LBN9
-----END PGP SIGNATURE-----