partial serialize/deserialize a structure
Hello everyone,
I have a large number of strcture instances, the I can not have all of them in memory. So I have the idea of serialize/deserialize partial of them to/from disk. The operation I need on these structure instances is to enumerate elements one by one, and I want to provide applications which use this interface to enumerate elements be transparent to how and which parts of structure instances is written/read to/from disk -- and the application only will feel read from memory.
I am wondering whether there are any samples/tutorials to make a reference?
thanks in advance,
George
[603 byte] By [
George2] at [2007-12-24]
Out of memory caching can be a complex matter, and not necessarily covered by all solutions which provide database persistent storage. The easiest (less expensive in terms of $'s) route to take, is to use a combination of variable getters and setters, boost serialize or a similar (free) library which provides effective storage of a number of types.
The key to a well working system isn't really the storage mechanism, though, but rather the mechanism which controls the lifetime of the objects. The obvious model: If an object is left dorment for a period of time, it should be serialized, and whenever it's accessed it should be deserialized. To take it one step further, you could try to anticipate the use of objects, and deserialize them prior to access -- eliminating latency when a getter or setter is called. Optimistic object responsibility chaining can help solve this -- indicating to the cache mechanism that class B is likely to be accessed upon a call to class A.
Techniques such as this is often used in game programming, so literature on that field is likely to yield some clues.
einaros wrote: