File format not really optimal for converters

[petterroea]

This is nitpicking, but the current file format version(150) isn't really optimal for converters, as the chunk data has to be cached until the entire file is read, before it can be converted, which means a small memory overhead. This, or the file has to be read multiple times.

This is due to the fact that(at least the file format description specifies) that the palette and materials can be at the end of the file. To handle this correctly, one would have to read all the different chunks, store them in memory until everything is read, then scan through and convert each model. Another method is to read through the file multiple times, doing a pass for metadata, and a pass for chunks.

My suggestion is seperating metadata into their own chunk, always at the beginning of the file.

This would let converters have all required metadata at hand for when chunks appear in the file, allowing for direct conversion in a lot of cases.

My suggestion for a more "easy to convert" chunk setup:

Chunk 'MAIN'
{
    //Always the first
    Chunk 'META'
    {
        // pack of models
        Chunk 'PACK'    : optional

        // palette
        Chunk 'RGBA'    : optional

        // materials
        Chunk 'MATT'    : optional
        Chunk 'MATT'
        ...
        Chunk 'MATT'
    }
    

    // models
    Chunk 'SIZE'
    Chunk 'XYZI'

    Chunk 'SIZE'
    Chunk 'XYZI'

    ...

    Chunk 'SIZE'
    Chunk 'XYZI'
}

Advantages of suggested format:

• Color palette and materials are all loaded, ready for creation. A converter could read and dispose of a voxel immediately after converting it, and there is no reason to keep XYZI-chunks in memory.

this could also be solved by specifying that material and palette chunks ALWAYS have to appear before XYZI chunks, but putting them all in a metadata chunk would use the child chunk feature of the format, which is only used once