Results 1 to 11 of 11

Thread: Unloading and Re-loading CL_ResourceManager

  1. #1
    Lesser Wizard
    Join Date
    Jun 2007
    Posts
    109

    Default Unloading and Re-loading CL_ResourceManager

    Hi. I'm needing the functionality to basically unload all of a resource manager's known resources, then re-load the XML file it reads from.

    It seems to removed the entries when I do the following:

    Code:
    // Free all resources
    		std::vector<CL_String> res = game_resources.get_resource_names();
    		for(std::vector<CL_String>::iterator it = res.begin(); it != res.end(); it++)
    		{
    			game_resources.destroy_resource((*it));
    		}
    		// Regnerate XML data -- this reconstructs and saves a new XML file
    		regenerate_xml();
    
    		// Recreate resources
    		game_resources = CL_ResourceManager("Resources/data.xml");
    However, if I use a third party editor to change one of my image files while the application is executing, then "reload" them, the old image persists. It's as if it doesn't actually purge the pixel data from the resource manager?

    Considering I reconstruct the manager (and I've tried making it a pointer and deleted and re-allocated it, which does the same thing), it's either a bug in my code or a bug in ClanLib?

    I don't see how though, delete and new should totally free the memory, but alas, the image remains until I relaunch the application. Maybe something with windows and the file handle itself?

    Any idears?

  2. #2
    ClanLib Developer
    Join Date
    May 2007
    Posts
    1,824

    Default

    If the file "graphics/mypicture.png" is used multiple times, the texture is still only loaded once, sharing the same memory and cached.

    This code is not clever enough to either:
    1) detect if the file has changed
    2) remove the image from the cache if it is no longer being used

    I believe the cache is held in CL_SharedGCData

    I'm not sure of the best way to fix this

  3. #3
    Lesser Wizard
    Join Date
    Jun 2007
    Posts
    109

    Default

    Maybe I misunderstand the way the Resource Manager works, I would think destroying the resource and having it reload the XML file it would try to re-allocate the image.

    If I iterate through the list of resource names and then afterward immediately save (using the save method of ResourceManager), the file is blank. So it is indeed destroying the resources on some level. The destroy resource function does erase from the container, which should call the destructor.

    But somehow it "remembers" the old image....

    I'm not sure either. You're talking about the bowels of ClanLib, something I am not fortunate enough to be familiar with

  4. #4
    ClanLib Developer
    Join Date
    May 2007
    Posts
    1,824

    Default

    Yes, it is in the bowels of ClanLib - At the point where CL_Sprites and CL_Image's are loaded internally.

    However, it can be fixed .

    At the moment, (internally) the picture is loaded via CL_VirtualDirectory and a filename.

    iirc, we create a unique identifier for a filename using CL_VirtualFileSystem::get_name() and CL_VirtualDirectory::get_name()

    (Note the documentation for these functions say: "The exact format of this name depends on the implementation of the underlying filesystem and mounts")

    CL_VirtualFileSystem::get_name() could be adjusted to append the file date stamp

    Thus solving the problem

  5. #5
    ClanLib Developer
    Join Date
    Sep 2006
    Location
    Denmark
    Posts
    554

    Default

    Why is it that when we make a hack that someone always find it?

    I cannot remember anymore why exactly we made the hack the way we did it, but here's what is going on:

    To avoid excessive memory consumption (and to optimize the batching of images), we cache CL_Sprite and CL_Image images in a class called CL_SharedGCData.

    My guess is that we do this mainly for the GUI, since we here have a situation where a lot of CL_DisplayWindow's can be created, lots of CL_GUIComponent derivates that do not communicate well together about resource sharing. No matter, the end result currently is that once you load an image, it stays loaded until all CL_GraphicContext objects for the loading thread is destroyed.

    We currently have no API function for clearing this cache, so to solve your problem we have to make a change to ClanLib. The simplest and fastest solution would be to add a function allowing you to clear the cache. But I am unsure if this is the best approach since app developers then need to know about the cache, and if they clear the cache it might clear other cached objects that they do not want cleared (i.e. if you only wanted to reload certain resources).

    The original design for loaded resource management was that everyone would use CL_ResourceManager and then objects that wanted to share resources would use CL_ResourceDataSession. That system turned out to be too naive, too annoying to use and generally just totally missing its target. This left us with the situation we have now where the GUI absolutely needed resource management, but our current solution sucked for the GUI and this hack was tempting.

    Too bad someone had to find out about it

  6. #6
    Lesser Wizard
    Join Date
    Jun 2007
    Posts
    109

    Default

    Well, the program I'm currently working on is for the eyes of developers only, and their ability reload the art they change isn't drastically important, I can just tell them to relaunch the application.

    It was my attempt at convenience, that's all.

    I do know if I load things outside of a resource manager this isn't really a problem, at least it doesn't seem to be because I have a small preview function that does that, and the image does indeed "refresh." Not a pratical solution for the amount of art I will be dealing with, though.

    So I guess what I'm saying is, this isn't something totally unlivable, however, if anyone wanted to do elaborate caching to save memory during their game execution, it might be a problem?

    Considering I'm on track to be doing that (to a mild degree, not very extensive), I could wonder, but I'm not terribly concerned with it. Clearing the cache would probably be alright, if it was somehow linked to just ResourceManagers (I keep my GUI and game data separate, anyway). From the sound of it, it's GC based, and that is a little bit of a problem.

    Maybe it's too simple or obvious to point out, but couldn't you only clear items in the cache with a reference to the ResourceManager that loaded it? Apologies if that's a terribly silly thing to request, as I say, I don't know much about the bowels of ClanLib =)

    So, anyway, for the time being I will just tell my artist to quit complaining and re-open the app

  7. #7
    ClanLib Developer
    Join Date
    May 2007
    Posts
    1,824

    Default

    This problem is now fixed in SVN (For ClanLib 2.0.4 release)

    I have used a different approach:

    Previously, the shared texture cache (In CL_SharedGCData) indirectly stored the CL_Texture's using the "CL_SharedPtr<CL_Texture_Impl>" handle

    I have fixed this by modifying the shared texture cache to use "CL_WeakPtr<CL_Texture_Impl>" instead.

    So, when the application (or clanlib library) has finished using the CL_Texture, the CL_Texture is totally deallocated. The next call to load a texture, will load the file as expected.

  8. #8
    Lesser Wizard
    Join Date
    Jun 2007
    Posts
    109

    Default

    Nice, thanks for the update!

    However, I updated to the SVN and re-built my libraries, although the issue still remains...

  9. #9
    ClanLib Developer
    Join Date
    May 2007
    Posts
    1,824

    Default

    Can you post a full example so I can test it?

  10. #10
    Lesser Wizard
    Join Date
    Jun 2007
    Posts
    109

    Default

    Okay I feel silly now

    It does indeed work, I just had a memory leak, which once repaired, fixed the issue.

    Ironic I actually stumbled upon this at all, since it was kind of my fault since the beginning

    No matter, since Rombust asked for an example, I made a small demo application.

    I know you all just added a recent SQL example, but I had the code handy (and surprise to me, it was fairly modular!) so I made a little "How to load resources from SQL" which includes writing an XML file and reading from SQLite. I was prompted to do things this way because the resource system is pretty strongly based off XML, so it was easier for me to adapt to it by generating XML entries for my SQL tables rather than writing my own custom resource system.

    (EDIT: And a quick hat tip to whomever did the SQL and XML modules; they are quite savvy!)

    It's a quick hack; less than an hour! Maybe useful though.

    Thanks again all!
    Attached Files Attached Files
    Last edited by catch22; 07-22-2009 at 11:41 PM.

  11. #11
    ClanLib Developer
    Join Date
    May 2007
    Posts
    1,824

    Default

    Many thanks, i'll have a look later

    That web site (in the READ_ME.txt file) for free graphics is very good resource, and it also has a friendly license - http://reinerstileset.4players.de/readmeE.html#b
    Last edited by rombust; 07-23-2009 at 07:09 AM. Reason: I cannot spell

Similar Threads

  1. Tileset loading...
    By Sirius2D in forum Novashell Game Creation System
    Replies: 3
    Last Post: 10-10-2008, 03:34 PM
  2. Script loading sequence
    By bullno1 in forum Novashell Game Creation System
    Replies: 2
    Last Post: 09-03-2008, 12:23 PM
  3. Loading
    By Cobra in forum Novashell Game Creation System
    Replies: 2
    Last Post: 06-26-2008, 09:43 AM
  4. CL_ResourceManager init problem. (OSX)
    By Otto (Strange) Halmén in forum Official ClanLib SDK Forums
    Replies: 12
    Last Post: 11-02-2007, 11:15 AM
  5. Font loading and the «» characters
    By Anvilfolk in forum Official ClanLib SDK Forums
    Replies: 0
    Last Post: 09-14-2007, 05:08 PM

Bookmarks

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •