A VRFileMap is backed by a disk file. The file is split into records as records are requested by the caller. Record size is defined to be a multiple of a block size to increase the likelyhood of record-reuse. Free'd records are tracked for re-use. Overtime it is possible for holes to occur in the file of record sizes that are never re-used. A compact() method is provided to compact() the backing file -- this could be an expensive operation depending on the amount of data in the file.
The initial design will support no record splitting or coelesceing (except via compact).
Nio is used for I/O to the file. In particular MappedByteBuffers are used to reduce memory usage and data copies.
How is it implemented?
JDK 1.4 java.nio MappedByteBuffer classe is used to map the backing file into memory and ByteBuffer.slice() is used to create slices that are returned by allocate().
The implementation will not attempt to do ANY splitting or coelescing of buffers. Free'd buffers will simply be put on a free list and re-used if they are large enough to satisfy an allocate() request. This is based on the assumption that buffer sizes will not vary wildly for any given allocator. When an allocator is first created the backing file is created of size initialCapacity and the entire file is mapped. The header is written.
When the first allocation request comes in the allocation size is rounded up to be a multiple of the block size. A slice is created from the MappedByteBuffer and the record header is written with a state of ALLOCATED. A VRecord is created and added to a list of allocated records and returned.
When a record is free'd it's state is updated to FREE and it is moved from the allocated list to the free list that is sorted by capacity.
When subsequent allocation requests come in the free list is searched first for a buffer that is of sufficient capacity. If one is found it is marked as ALLOCATED, moved to the allocated list and returned. If one is not found a new one is allocated from the backing file.
If the file is not large enough we grow it using the growth factor and map in a new segment. A list of mapped segments must be maintained.
When compact() is called a new empty file is created and all records whose state is not free are written to it. The file is then grown to be of at least initialCapacity if need be. The old file is renamed to back it up, and the new file is renamed to become the new backing store. The free list is disgarded and the new file is loaded.
When a file is loaded it is mapped and the header is read to verify file type and version. Then the file is scanned and records are placed on the free and allocated lists as dictated by their state. Notes: - Creating many MappedByteBuffers is expensive (creating a slice() of one should be cheap). That's why we want a relatively large initial file size and growth factor. Also MappedByteBuffers mappings are not closed until the objects are finalized. @see VRFile.java
| |