Group of objects in PRWM

[ashl1]

Hello again.
What do you suggest to bundled load several geometries (group of geometries)?

[kchapelier]

Loading several files separately, this will take advantage of the parallel download in the browser (assuming you are using this in a browser). It should be faster to download X models separately than a bundle containing X models and with HTTP2 slowly becoming the norm, it should get even better.

I initially planned to design a bundle format further down the line, but given the above it probably wouldn't make sense.

[ashl1]

It might be slower for downloading also in HTTP/2 for example 1000 geometries each for 2-5 kBytes. Have you ever seen benchmarks for such the case?

[ashl1]

I've found this one: https://medium.com/@asyncmax/the-right-way-to-bundle-your-assets-for-faster-sites-over-http-2-437c37efe3ff
Seems the best solution is like golden ratio between several bundles.

Therefore I suggest to make bundle syntax.

What do you think about v2 syntax with concatenated v1 files inside? For example, header might be the following:
2 (version in byte), 3 bytes represented number of v1 files inside, whole v1 file1, whole v2 file2, ...

[kchapelier]

I've found this one: https://medium.com/@asyncmax/the-right-way-to-bundle-your-assets-for-faster-sites-over-http-2-437c37efe3ff
Seems the best solution is like golden ratio between several bundles.

Yes, it makes sense...

I can't give you any ETA on this though. My idea would be to consider the bundle format as different format (PRWB), not a version of PRWM. This would allow the format to be a generic container for any types of files, which could for example be useful in the future to include shaders / material descriptions.

Its format would be something like :

Header

Version : 1 byte (0 to 255)
Number of files : 2 bytes (0 to 65535)

// should probably add endianness in there

File block

Identifier : An ASCII encoded C-string (one byte per character, terminated with a NUL character). Typically the name of the source file, eventually sanitized for ASCII.
Type : 1 byte (0 to 255), 0 being a PRWM file.
Byte Offset : 4 bytes (0 to 4294967295), start of the content of the file.
Byte Length : 4 bytes (0 to 4294967295), length of the content of the file.

4 bytes may seem like a lot since it allows up to 4GB files, but 3 bytes would only allow up to 16MB.

Rest of the file

The files' data.

---

One of the design challenge will be to figure out how to differentiate the PRWB files from the PRWM files, since there's no magic string in the header of the file to identify them as such (I dropped it in the spec phase when I decided not to make a bundle format). Maybe make a V2 of the PRWM spec just to include said magic string, so PRWM files would always start with "PRWM" and PRWB with "PRWB" ? This would be a breaking change, so I'm not sure that's the best solution.

[kchapelier]

One alternative to the PRWM / PRWB differentiation issue would be to start the file with 0x00 (which by the current spec of PRWM should be considered as an error because there is no version 0) or 0xFF (because there will never be 255 versions of the PRWM specs).

[kchapelier]

Additionally, there should be some padding before each file data to align its start to a multiple of 4.

[ashl1]

Agree with padding without a doubt.

I don't recommend to use "specific" versions like 0x00 or 0xFF. Do you know how IP packets is being made? Same header with type and payload where payload might contain another type of payload. This provide infinite combinations of versions. Thus using next version 0x02 is preferrable over special 0x00 or 0xFF.

Above, I recommended to use 3 bytes for number of files inside. And I agree with offset + size. It might be usefull when we need concrete part.

The only question is do we need separated formats? I think there is no difference on data but on usage. What is better to use different loaders/exporters with different names or one single, separated into several concrete? What is more readable and understandable by community?

Think we should first understand interface of Loaders/Exporters than make decision on PRWM v2 or another PRWB v1.

[kchapelier]

It's been a long time coming but here is some update on this.

I went on with a separate file format, totally independent for PRWM. The big advantage of making it its own format is that it is not limited to the use case of loading PWRM files (allowing other model file format but also other type of file altogether) and can be made flexible enough to allow anyone to implement their own take on the bundling issue :

• Bundling 3D models which are not PRWM files.
• Bundling matcaps or cubemaps along with the models.
• Storing a full 3D scene in a bundle with a scene.json defining each object scale, position, rotation, material, etc. and a prwm file each model.
• Whatever application-specific feature one may need.

It's roughly based on the tentative spec I posted above on the 22th Oct. with the following changes :

• Added a static type signature "BNDL" at the beginning of the file.
• Used 3 bytes for the number of files instead of 2 as you recommended.
• Changed the type identifier per file to be a NULL-terminated ASCII string instead of a boolean in an effort to make the format easier to debug and code with.

Example with 2 PRWM files

Example with images only

Specifications and implementation not pushed on NPM yet.

[kchapelier]

The few issues of the implementations have been fixed and the reference encoder/decoder (along with the CLI tool and an example web loader) is now on NPM.

[kchapelier]

@ashl1 I forgot. I named you in the main readme of the repository in the paragraph explaining how this format came to be. Let me know if this is an issue for you.