Results 1 to 8 of 8

Thread: Looking for a freelance ClanLib developer, experienced in ClanLib & memory leaks...

  1. #1

    Exclamation Looking for a freelance ClanLib developer, experienced in ClanLib & memory leaks...

    Hey,

    I work at a games company, located in Caracas, Venezuela. At the moment we are developing a time-management game using ClanLib 0.8. The game is almost finished, but we are having some serious issues with some memory leaks in the application (we are thinking it has to do with ClanLib leaking memory ). And we are interested in hiring someone as freelancer for some hours that can help us solve this problem. So we need someone with A LOT of experience developing applications in ClanLib, and also experienced in finding and fixing memory leaks. Please let me know if you are interested so we can formally contact you in order to reach an agreement. Thanks in advance.

    Cheers,

    eduardo...

  2. #2
    ClanLib Developer
    Join Date
    Sep 2006
    Location
    Denmark
    Posts
    554

    Default

    Which platform are you running the software on?

    If the platform is Windows, the easiest way to find your memory leaks is to use the UMDH (User mode debug heap) tool from Microsoft, it is part of the Debugging Tools for Windows package, which you can download from:



    The tool is simple to use. The steps are:

    • Run gflags.exe /i myapp.exe +ust
    • set _NT_SYMBOL_PATH=srv*C:\SymbolsDB*http://msdl.microsoft.com/download/symbols
    • Start myapp.exe
    • Run umdh.exe -p:pid_of_myapp -d -f:allocations1.txt
    • Wait for the process to leak some memory
    • Run umdh.exe -p:pid_of_myapp -d -f:allocations2.txt
    • Finally make UMDH generate a report comparing the memory allocations: Run umdh.exe -d -l allocations1.txt allocations2.txt > report.txt


    The file report.txt now includes a list of all memory allocations that are new between the time you generated allocations1.txt and allocations2.txt, including a full stack backtrace showing what allocated that data.

    The only catch about using UMDH is that the tool must be able to locate all the program database (PDB) files used by the application. If the application was compiled on the same machine as you run UMDH, this usually works without further changes - otherwise you must ensure that _NT_SYMBOL_PATH includes the path to all your exepdb, exe and dll files used by the software.

    If you are unsure on how to parse the report.txt file, you could post it here on the forum and I'll have a look at it.

    If you are still stuck after this, I can help you have a look at it. You can contact me by sending me a personal message on the forum - I'll give you my email address from there.
    Last edited by Judas; 08-21-2008 at 12:51 AM. Reason: disabled smilies in text

  3. #3

    Default

    Hey Magnus thanx for the quick response.. IŽll try the UMDH tool to see if it helps.. IŽll let you know the result of this later on.. But now I just noticed something pretty weird in our game.. If you load the first level the memory usage in the Windows Task Manager will reach 108.800 Kbs.. but now if I minimize the gameŽs window, a couple of times, the mem usage will drop to 17.000 Kbs.. So now my question is are there any events/methods called in ClanLib when you minimize a window?? if so do you think it would be wise to call these events/methods every once in a while in order to reduce memory usage.. thanks in advance for your help..

    Cheers,

    eduardo...

  4. #4

    Default Finally got UMDH working..

    Hey Magnus, I finally got UMDH to work and got a proper report out of it. Now IŽm having a bit of a hard time understanding what it says. IŽll attach the report just in case you want to take a look at it.. many many thanks for your help mate.. I greatly appreciated..

    Cheers,
    eduardo
    Attached Files Attached Files

  5. #5
    ClanLib Developer
    Join Date
    Sep 2006
    Location
    Denmark
    Posts
    554

    Default

    The task manager is generally a very poor tool for determining how much memory your application uses.

    The reason for this is that the VM Size it displays is not the amount of memory the application has allocated, but the amount of memory currently paged in. When you minimize the game, Windows uses some algorithm to page out a lot of the memory, so that is why the memory usage suddenly seem to drop in the process.

    What you want to look for is the Private Bytes used by a process. I'm not entirely sure if you can see that in the normal Task Manager, but either way you probably will prefer to download the Process Explorer application from Microsoft: http://technet.microsoft.com/en-us/s.../bb896653.aspx. This program is basically how the task manager in Windows really should have looked like.

    I've looked at your report1.txt, and if we look at the first memory difference it displays it looks like this:

    Code:
    + 48103600 ( 48103600 -      0)     16 allocs	BackTrace350B
    +      16 (     16 -      0)	BackTrace350B	allocations
    
    	ntdll!RtlDebugAllocateHeap+000000E1
    	ntdll!RtlAllocateHeapSlowly+00000044
    	ntdll!RtlAllocateHeap+00000E64
    	kernel32!GlobalAlloc+00000066
    	igldev32!devProcessAttach+000046C9
    This needs to read as follows: The function devProcessAttach in igldev32.dll allocated 48 megabytes totally. The allocation happened 16 times, so thats 3 megabytes per allocation.

    igldev32.dll is the OpenGL driver for the Intel graphics driver. Either the driver is leaking, which is less likely so we instead assume the application did something that made OpenGL allocate 48 mb of data. My guess would be some textures being loaded.

    Ok, the next leak presented by report1.txt:

    Code:
    +  653208 ( 653208 -      0)   4803 allocs	BackTrace2584
    +    4803 (   4803 -      0)	BackTrace2584	allocations
    
    	ntdll!RtlDebugAllocateHeap+000000E1
    	ntdll!RtlAllocateHeapSlowly+00000044
    	ntdll!RtlAllocateHeap+00000E64
    	NurseTycoon!_heap_alloc_base+0000005E (f:\dd\vctools\crt_bld\self_x86\crt\src\malloc.c, 105)
    	NurseTycoon!_heap_alloc_dbg_impl+000001F6 (f:\dd\vctools\crt_bld\self_x86\crt\src\dbgheap.c, 427)
    	NurseTycoon!_nh_malloc_dbg_impl+0000001F (f:\dd\vctools\crt_bld\self_x86\crt\src\dbgheap.c, 239)
    	NurseTycoon!_nh_malloc_dbg+0000002C (f:\dd\vctools\crt_bld\self_x86\crt\src\dbgheap.c, 296)
    	NurseTycoon!malloc+0000001B (f:\dd\vctools\crt_bld\self_x86\crt\src\dbgmalloc.c, 56)
    	NurseTycoon!operator new+00000011 (f:\dd\vctools\crt_bld\self_x86\crt\src\new.cpp, 59)
    	NurseTycoon!CL_DomNode::CL_DomNode+00000043 (c:\clanlib\clanlib-0.8.1\sources\core\xml\dom_node.cpp, 66)
    	NurseTycoon!CL_DomAttr::CL_DomAttr+00000042 (c:\clanlib\clanlib-0.8.1\sources\core\xml\dom_attr.cpp, 44)
    	NurseTycoon!CL_DomElement::set_attribute+000001B3 (c:\clanlib\clanlib-0.8.1\sources\core\xml\dom_element.cpp, 114)
    	NurseTycoon!CL_DomDocument::load+000005FE (c:\clanlib\clanlib-0.8.1\sources\core\xml\dom_document.cpp, 198)
    	NurseTycoon!CL_ResourceManager_Generic::load+000002E4 (c:\clanlib\clanlib-0.8.1\sources\core\resources\resource_manager_generic.cpp, 285)
    	NurseTycoon!CL_ResourceManager::CL_ResourceManager+000000BF (c:\clanlib\clanlib-0.8.1\sources\core\resources\resource_manager.cpp, 45)
    	NurseTycoon!Level::Level+000001CC (c:\ermania\nursetycoonlatest\nursetycoon\source\level.cpp, 17)
    	NurseTycoon!StoryModeModule::StoryModeModule+00000124 (c:\ermania\nursetycoonlatest\nursetycoon\source\storymodemodule.cpp, 22)
    	NurseTycoon!ModuleManager::update+000001D1 (c:\ermania\nursetycoonlatest\nursetycoon\source\modulemanager.cpp, 39)
    	NurseTycoon!ERMania::main+00000212 (c:\ermania\nursetycoonlatest\nursetycoon\source\ermania.cpp, 73)
    653 KB of memory leaked in 4803 allocations.

    This leak is obviously CL_DomNode that allocated some memory that was never released. Now the question is whether CL_DomNode leaks, or if something further up the system leaks. Try to verify in the code whether Level destroys the CL_DomDocument it loads at line 17 in nursetycoon\source\level.cpp. If it does not, then the leak is in Level, otherwise the leak can be caused by Level not being properly destroyed (unless it only kept that CL_DomDocument in a local function scope).

    If nothing else can explain why this memory wasn't released, then the leak must be in CL_DomNode / CL_DomDocument.

    Next follows a lot of stack backtraces that all seem to be the same source: the DOM classes -> Level -> StoryModeModule -> ModuleManager::update -> ERMania.

    Totally all these leaks seem to have leaked 3.2 MB of memory.

    After these ClanLib DOM leaks, report1.txt mentions things like:

    Code:
    +   49896 (  49896 -      0)     66 allocs	BackTrace3A83
    +      66 (     66 -      0)	BackTrace3A83	allocations
    
    	ntdll!RtlDebugAllocateHeap+000000E1
    	ntdll!RtlAllocateHeapSlowly+00000044
    	ntdll!RtlAllocateHeap+00000E64
    	NurseTycoon!_heap_alloc_base+0000005E (f:\dd\vctools\crt_bld\self_x86\crt\src\malloc.c, 105)
    	NurseTycoon!_heap_alloc_dbg_impl+000001F6 (f:\dd\vctools\crt_bld\self_x86\crt\src\dbgheap.c, 427)
    	NurseTycoon!_nh_malloc_dbg_impl+0000001F (f:\dd\vctools\crt_bld\self_x86\crt\src\dbgheap.c, 239)
    	NurseTycoon!_nh_malloc_dbg+0000002C (f:\dd\vctools\crt_bld\self_x86\crt\src\dbgheap.c, 296)
    	NurseTycoon!malloc+0000001B (f:\dd\vctools\crt_bld\self_x86\crt\src\dbgmalloc.c, 56)
    	NurseTycoon!operator new+00000011 (f:\dd\vctools\crt_bld\self_x86\crt\src\new.cpp, 59)
    	NurseTycoon!std::_Allocate<AStarAlgorithm::AStarCell>+00000064 (c:\program files\microsoft visual studio 9.0\vc\include\xmemory, 43)
    	NurseTycoon!std::allocator<AStarAlgorithm::AStarCell>::allocate+0000002E (c:\program files\microsoft visual studio 9.0\vc\include\xmemory, 145)
    	NurseTycoon!std::vector<AStarAlgorithm::AStarCell,std::allocator<AStarAlgorithm::AStarCell> >::_Insert_n+00000133 (c:\program files\microsoft visual studio 9.0\vc\include\vector, 1178)
    	NurseTycoon!std::vector<AStarAlgorithm::AStarCell,std::allocator<AStarAlgorithm::AStarCell> >::resize+000000B2 (c:\program files\microsoft visual studio 9.0\vc\include\vector, 719)
    	NurseTycoon!std::vector<AStarAlgorithm::AStarCell,std::allocator<AStarAlgorithm::AStarCell> >::resize+00000045 (c:\program files\microsoft visual studio 9.0\vc\include\vector, 714)
    	NurseTycoon!AStarAlgorithm::initialize+000000B0 (c:\ermania\nursetycoonlatest\nursetycoon\source\astaralgorithm.cpp, 208)
    	NurseTycoon!Level::Level+00003587 (c:\ermania\nursetycoonlatest\nursetycoon\source\level.cpp, 404)
    	NurseTycoon!StoryModeModule::StoryModeModule+00000124 (c:\ermania\nursetycoonlatest\nursetycoon\source\storymodemodule.cpp, 22)
    	NurseTycoon!ModuleManager::update+000001D1 (c:\ermania\nursetycoonlatest\nursetycoon\source\modulemanager.cpp, 39)
    	NurseTycoon!ERMania::main+00000212 (c:\ermania\nursetycoonlatest\nursetycoon\source\ermania.cpp, 73)
    In this case its 49 KB of memory seemingly leaked from AStarAlgorithm::initialize when it puts something into a vector. Since it is unlikely that we have found a memory leak in the std::vector code, the more likely explanation is that the vector is never destroyed afterward. Once again this points at the possibility that Level is perhaps never destroyed?

    After these local leaks, we have some CL_SoundBuffer objects owned by Nurse that seemed to have leaked, but once again those are owned by classes that are called by Level.

    My guess would be that Level is never destroyed and that's what may be causing the memory leaks in the application.

    The UMDH tool works by comparing memory allocations between the two points you ran it at, so there is a certain chance of it showing 'false positives', depending on when you run it. For instance, if you ran it the first time before it had loaded any levels at all, and run it second time before the level is destroyed, then it will show all the allocations done by the level. Since I do not know at what time in the app you ran the heap allocation snapshots, I cannot tell whether all these allocations are definitive memory leaks, but hopefully this walk through gave you a good idea on how to read and interpret the output from UMDH.

    Oh, one last thing. I'm not sure if you ran the program in debug build or release build but the results may be better for a release version of the app. This is because the debug C/C++ runtime library from Microsoft doesn't always free the memory immediately (it does this to try detect dangling pointers in your application). I am not sure how much this affects the output from UMDH, but if you want to play it safe, run it with a release build.
    Last edited by Judas; 08-22-2008 at 12:28 PM. Reason: As usual I should check for typos before clicking submit :)

  6. #6

    Default

    Mate thanks for the GREAT info.. IŽll try all of it and IŽll let you know how I did.. I really appreciate your help, I was really stuck before I got all your advice.. THANKS!!!!

  7. #7
    ClanLib Developer
    Join Date
    Sep 2006
    Location
    Denmark
    Posts
    554

    Default

    You're welcome. I hope it helps you track down those memory leaks.

  8. #8

    Default Some more help....

    Hey Magnus,
    Mate IŽve been able to improve the memory use in my game a lot.. but IŽm still having some problems fixing a leak related to SoundBuffers.. I donŽt have a clue why this leaks are happening since IŽm not declaring my soundBuffers variables as pointers.. I just use the method prepare().. IŽm attaching my latest report to see if you can please take a look at it a lend me a hand with if.. thanx for your help..

    Cheers,
    eduardo...
    Attached Files Attached Files

Similar Threads

  1. Replies: 3
    Last Post: 06-29-2011, 01:50 AM
  2. Main Developer ?
    By zyklo in forum Official ClanLib SDK Forums
    Replies: 2
    Last Post: 08-21-2008, 03:00 AM
  3. Installing ClanLib 0.8 and ClanLib 0.9 on linux
    By rombust in forum Official ClanLib SDK Forums
    Replies: 4
    Last Post: 07-15-2008, 09:51 AM
  4. clanlib.m4 too old
    By Qwerto in forum Official ClanLib SDK Forums
    Replies: 0
    Last Post: 03-30-2008, 11:35 AM
  5. Memory Leaks
    By in forum Funeral Quest
    Replies: 9
    Last Post: 03-06-2007, 11:00 AM

Bookmarks

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •