Shall Amethyst adopt a standard "native" asset format (gltf, usd, xbuf?) instead of implementing a custom one in YAML?

[diegor8]

One of the design goals of Amethyst is having the tools be independent processes , with just some information and assets being transferred from them to the engine , which means that the choice of interchange files is specially important ,

The current plan to parse simple formats like .obj , (which can used directly , but can only represent geometry ) for assets , and store scene information ( game maps) as yaml files

In the other hand , if a more complete exchange format is used to interface betweben the tools and runtime (eg fbx) , then the data would have to converted from the dcc format to the exchange format and finally to amethysts format ( eg .blend to fbx to .amethyst) which slows down loading time , but has advantage of letting people export entire scenes , so we could for example use a general porpouse level editor like the one made by sony or Blender as a game engine dcc as panda 3d already can (https://github.com/Moguri/BlenderPanda) , think of how the movie industry can export entire projects back and forth from one program to another ( blender to 3d max to blender is posible ) , whereas unity level editor to ue4 is not , because they utilize a propietary scene representation

Finally , there are glTF and usd which are json based , which means that could be very similar to what Amethyst would end up representing in yaml anywhay and like it they are useable natively (at runtime), but with the advantage of supported standard , so the dcc format would be converted to the one used by amethyst directly ( eg .blend to gltf) , instead of .blend to fbx to yaml .

(here is a demo of usd being used to achive the afromentioned goal of this engine )
https://blenderartists.org/forum/showthread.php?406080-USD-and-Hydra-open-sourced-demo-video-will-it-be-used-for-the-Viewport

the usd is transferred from the pixar tools to the hydra render ( which is a separate executable) and continuously updated as changes are made

[anderejd]

+1 for using existing standards. If existing standard formats does not provide enough opportunity for optimization (fast loading) then a custom format should be designed.

[diegor8]

GlTF was designed for the ground up for speed , the geometry information can be loaded directly into opengl vertex buffers , but because it is generally low level , it has some problems working with vulkan but they will be resolved in the version 1.1 which will be released in spring and will also feature extras such as a unified pbr system ( again think of how until now the same objects cant used in ue4 and unity without modification , as you need to asign the materials manually as well as the normal maps , displacement , etc after exporting them to the target engine )

There lies the problem with the standard, it will change signifcantly very soon . In one hand we could wait , but in the other , people have already have started to implement map and mesh prefab loaders which could conflict this or lead to duplicated effort , also because most of the new features are not essential , they could be left for later and the core could be implemented before is officially released, as i think it has already been finished.

Because of this , i think is extremely important to discuss this issue now , especially about how feasible is to have two run time formats
https://github.com/KhronosGroup/glTF

[Emilgardis]

This sounds amazing, I would love for this to be implemented for Amethyst. There is currently an available gltf crate, so we could use that, but I think that we should make our own to make things easier.

[diegor8]

do you mean this library ? ( https://github.com/Alteous/gltf/tree/master/src ) .

[msiglreith]

Our world/scene/entity (de)serizalizer in combination with asset loader should allow to load various file formats if the corresponding loaders are implemented (fbx, gltf, etc.)

Nonetheless a larger disadvantage I see with regards to gltf is the node based structuring, which doesn't suite our entity component system very well (loading each entity indivdually vs load each component storage).

My current vision: Being able to load arbitrary formats in the engine for quick iterations without the need of an asset pipeline, but also provide support for serialize the whole world including resources into an optimized format designed for our engine.

[diegor8]

(loading each entity indivdually vs load each component storage)

I didnt get it , could you explain the disavantage a little more

[msiglreith]

Oh sorry, trying to clarify:
When loading a new level or scene in the engine, we need to deserialize all the entities and instantiate them in the world. The straightforward approach would be to build each entity like this:

loop {
    world.create_now().
       .with(Comp1)
       .with(Transform)
       .build();
}

For larger numbers of entities and components this might be not optimal as actually we store each component in dedicated storages (Vec, HashMap, etc.)
A more data orientated approach might be to serialize and deserialize the storages:

world.create_entities();
world.deserialize_storage::<Comp1>();
world.deserialize_storage::<Transform>();

This may give a similar advantage as the direct vertex buffer loading you mentioned above but for entities,

the geometry information can be loaded directly into opengl vertex buffers

(I discussed it with @Aceeri on gitter some time ago, probably should also propose this in the corresponding issue)

I'm hoping this makes it more understandable!

[diegor8]

So you mean that nodes as way of representing scenes go against the idea of data oriented programing/ component programing , and because of that this can't be the sole format used by amethyst

[Aceeri]

It can be used and I don't mind it, the idea is that it would be nicer to have all components of a certain type in a block information.

[diegor8]

So , in conclusion , Amethyst will be able to parse glTF and display the contents directly
( not even the supposedly low level and flexible ogre graphics library can do this , if you want to load anything , you need to convert it to their proprietary 3d format first ) , so the advantage of it being fast and low level isnt undermined by having to perform additional proccesing .

If that is everything ( unless someone else has something else to say about the matter ) , i will end this recommending to check the khronos repo for the standard , to get useful ideas for the engine if you are working with the world/scene/entity system , and if not , it also has presentations about how next gen content pipelines will work.

I´ll be re starting this issue again when gltf 1,1 is released (we would still benefit from having it implemented as early as posible , because that it would enable anethyst to use tools that support it) , then i I´ll be able to get contribute to the project of implementing it here ( i have been working with games engines and have been researching about them , but im still a novice programmer ) .

[kvark]

I'm against treating glTF any special, given that it's GL-centric. Sharing the opinion with @msiglreith .

[diegor8]

I used to be , because it was developed at the same time as next generation apis , but as more companies like microsoft started backing it , the main goal of version 1.1 became to make it compatible with direct x , vulkan and metal

Also a developer i recently talked to confirmed that is has already been finished, even thought is not officially released KhronosGroup/glTF#801 (comment)

[kvark]

@diegor8 just an idea that all enums are passed in as GLenum rubs in the wrong way already. It's not a big problem to become compatible with Vulkan/Metal/etc for a model format - this should be given by default, and the very fact glTF needed to address some issues related to that makes me worry.

[diegor8]

In the begining ( before being adopted by microsoft ,autodesk and oculus ) it was just compatible with opengl , there was only one intended target ( the web) , and the standard tries is used by apps that such as cesium , that load thousands on models and on the fly (they cant be cached to a serialized representation ) .

These characteristics make it suitable for being used in Amethyst , an engine witthout an asset pipeline that needs to tramsmit asset on the fly betweben processes

[kvark]

I don't believe their enum choice have much to do with speed. After all, who ends up storing the models with GLenums? Only the pure GL renderers. Anything more complex, any sort of an engine, will have its own types that abstract over GL, so it needs to do a stupid conversion from GL (into internal types, which may then be converted back to GL). And the problem here is that GLenum is insanely large, so one has to always think of the "valid" values in each particular context when converting. I know, I've been there.

load thousands on models and on the fly

As long as you load the buffers directly into VRAM, there is your speed. You aren't going to gain much for meta-data.

at first the only intended target was the web

Yes, and then it was too late to turn back.

[diegor8]

I will tell you that i am not that into low level stuff yet , so i cant really argue with you about the specifics of the glTF , the only thing that i am sure is that it benefits from being a hybrid binary / human readable format , but if it being low level is otherwise pointless , then how can so many big , companies , open source projects like the blender foundation and the 3d qt project ( a data driven render built upon an ecs that can be customized with configuration files , which doesnt uses ogl directly ) and even renowned programmers like John Carmack talk so much about it being a good runtime format , have all of them just accepted gltf inneficiencies , because even then , it is still better than the competence ? ,i will tell you that others are definitely worse just take take into account that .obj is old , doesnt support scenes or animation and is wholly text based (much slower and takes up more space) and fbx is proppietary , contains data unecesary to a game engine , and will stop being supported by blender , because each new version is limited to adding features on top of the old ones without care for how cluttered the standard gets

Also Khronos couldnt get away with promising that the current version will fix the imcompatibililties with other apis if its not possible , they could lose the trust of their members , and if that happens , they would leave .

In the end if gltf is trully not that good or plain bad , another standard format is needed to be supported an encouraged to be used , because as small project Amethyst should make to its users be as easy as possible to get into it and because that way already existing tools can be used much more easily , why write a level editor , when blender or an existing one can be used , there is already a framework for doing that : https://github.com/Kupoman/BlenderRealtimeEngineAddon ( btw this only supports gltf , for the reasons i mentioned before , it being suitable to be shared across proceses, but alternative formats may be added later ) ; or material editor ( other formats may also support an standard pbr system , think of it like an simplifyied version of substance designer , but defined by the asset format ) , again disney's universal scene description is worth taking a look.

[diegor8]

Turns out that some jmonkeyengine developers also reviewed gltf , and were inspired by its capabilities to develop their own format ( this was before version 1.0 of gltf had been oficially released, before the focus had expanded from just webgl to being a general format ) ( https://github.com/xbuf/xbuf ) , and a framework compatible with any engine that implements xbuff , that demonstrates what it can be used for ( https://github.com/xbuf/blender_io_xbuf , https://www.youtube.com/watch?v=3pQd65_dkeM )

In the words of the project

The project would like to be an Community Format shared by several engines, tools. It's Open to any contributions (code, discussion, issues, tools, ...).>

this is what im talking about when i say it would be very beneficial to use standard formats

[kvark]

@diegor8 thanks for the links!
There must have been reasons why jmonkeyengine didn't go with glTF and instead went to create their own stuff. xbuf actually looks pretty neat ;) Would be great to have a loader for it.
I welcome the standard formats by all means! Currently, we don't even have to choose the one. We can support many, and then see what's getting most required.

[diegor8]

According to the faq

Why not glTF ?

Moving and heavy spec ;-)
Json based format requires to write “semantic” parser, converter.

ironically the situation situation has reversed , as of 1.1 gltf is stable , while xbuff will have to move away from protocol buffers or risk being based in obsolete tecnology , because alternaties to it are already mature, potentially faster ( even the human readable json is only slightly slower )and better in different ways ( newer alterantives also consume less memory , which would be specially beneficial for over the network transmissions). If amethyst ever has a say on the spec , doing that is the first thing that should be asked .

[diegor8]

Yeah , so many render engines are wary of doing doing that , i thought supporting multiple formats natively ( without performing a conversion ) wasnt a viable option . In reality teams such as the ogre 3d developers dismiss any format other than their own with comtempt , situations such as the discussion about ogre linked in the xbuff github have happened many times , they have even critized people using the ogre format in their own projects ( why? , It would benefit them ) , telling them they are ripping their format . Athough limiting the reach to not include certain formats can be advantageous to avoid bloat or difficult to use broken ones like fbx .

But i digress , the real reason we should be worried about this now( when only amethyst cli has been developed ) is because it will allow amethyst to use the afromentioned frameworks ( the xbuff project aims to have more tools than just the blender as editor plugin , and there surely are/will be more general tools that work on any engine using gltf) it will not only allow us to contribute to them instead of making our own software that will only be used by this engine but also allow artits to change betweben tools if they want to use an specific engine , or keep the software they are using but change the engine with little workflow change .

[minecrawler]

My 2 cents:

I think it makes a lot of sense to at least support the import of a variety of formats, like .obj, .dae, .3ds, .gltf, etc. (just talking mesh formats here, there are also image and sound formats to consider) even if that is very suboptimal and slow. That would enable people to use any tool during development without the constraints of plugins or converters. It would also make devops really easy. During development, assets are often passed around in a suboptimal format, which contains a lot of additional information not needed by Amethyst, but useful for the artists (for example a .psd file will contain image-layer information). When the game is shipped, however, an optimized format can be chosen, which only includes the least information required, so that the game can be shipped without bloat. Games these days tend to be distributed via the web, where size still matters.

This conclusion also leads to the following thought: If Amethyst only defines one optimized format, based on YAML, then it is nice for humans, but absolutely not optimal for a machine, which would have to interpret text. Text is big and slow. So, if there is a format for Amethyst which is nice for development, there should be another Amethyst format which is nice for release.

[torkleyy]

I also think that both slow but common formats (obj, psd) and binary formats should be supported, which is allowed by my PR. But what do you mean by

one optimized format, based on YAML

? What is optimized on reading a YAML file?

If you're talking about scene files, I planned to parse them at compile time.

[minecrawler]

@torkleyy optimized for the way Amethyst handles scenes (ECS-y, as opposed to node-y (scene graph))

[torkleyy]

@minecrawler

So would you be happy with my approach (parsing them at compile time)? I'll work on that once my asset management PR is merged.

[torkleyy]

Also, if you want to be able to change the scene after compile, you can still load the scene from a binary file, but edit a yaml file which can be converted with amethyst cli / a build.rs (automatically, therefore convenient for artists).

[minecrawler]

I just read through the PR comments and some bits of the source code. I am really looking forward to your reworked asset loader 👍

As for compile-time vs. run-time: In my opinion, something like that depends. Performance-critical stuff which is used frequently, but changed very rarely, should be compiled-in, but other stuff might be better stored in an archive, where it is compressed and easy to be updated if needed. Also new assets can be added over the life-cycle of the game. Think of an MMO, in which a server dictates the assets used for a certain character model; the game might not even know about any assets or scenes at all, just load whatever it is told to and fetch anything missing from the server.

That's why I think that it is totally reasonable to allow for (explicit) compile-time parsing, but leave the option for dynamic loading/changing scenes and other assets.

[diegor8]

@minecrawler The general agreement ( people unaware of this thread have suggested it too ) is that to best and easiest way to make an editor for amethyst is to develop an extension for that connects the engine to blender or other 3d content autoring aplications ( preferably using an already existing editor framework such as the linked in this thread , so we dont have to develop a blender side plugin ) , in that case gltf would be used behind the scenes , the user would be dealing with .blend files ( or
. mb if a plugin for maya is made too ) and be able to import any format that the choosen software can us, but amethyst would only receive gltf assets . ( how this would work https://www.youtube.com/watch?v=3pQd65_dkeM )

[bwasty]

FYI: glTF 2.0 has been released a few days ago and the gltf crate is updated:
https://www.khronos.org/news/press/khronos-releases-gltf-2.0-specification
https://github.com/alteous/gltf

[diegor8]

Now the format will be stable for several years , be fit to be used with vulkan and will have other benefits i mentioned ( engine agnostic material system )

[Xaeroxe]

@ebkalderon Since this issue seems to primarily have to do with 3D rendering assets and you just recently overhauled our renderer I feel it'd probably be appropriate for you to make a decision on what to do with this issue. A lot has changed since this issue was opened, I wonder if glTF is still relevant for our purposes, and can we make it cooperate with specs?

[gaudecker]

Recent post by a Godot engine developer: Why we should all support GLTF 2.0 as THE standard asset exchange format for game engines..

[erlend-sh]

Hi. I was the PM of jMonkeyEngine from 2009-2015.

The main reason we never went with glTF was simply because the spec was younger back then, and as a purely open source & flat-structure project any feature development came down to who had free time & interest. Coincidentally (but not really, because glTF is heating up right now 🔥 ) one of the lead devs @Nehon is now working to support it.

There's a good reason why the team behind Godot is excited about glTF: We (open source game engine developers) have been waiting for a solid open format like this for decades. I'm sure the "asset pipeline" is at the top of just about any OSS engine developer's list of grievances. Trying to build compatibility for a range of 3D formats was one of the biggest challenges we faced with jME.

Industry giants like Unity and Unreal had a major advantage because they had a direct line to the companies that they were making importers for. For us, importing proprietary formats from the likes of Maya and 3ds Max ranged from painstakingly difficult to outright impossible because the formats were closed down and in constant flux.

glTF is a major equalizer in this space, and the indie game developer community would be wise to support it, both technically and otherwise. It may not be supported by the proprietary 3D modeling apps yet, but the more widely it's supported by the rest of the industry, the harder it will be to ignore for the most entrenched players.

Furthermore, the fact that the Khronos Group itself is officially maintaining the Blender Exporter is huge. Blender is a fantastic modeling tool, but the .blend file format is not at all optimised for games (nor should it be) which makes it difficult to work with. With the glTF export this is no longer a problem, and asset creators working with OSS engines no longer need to play the utterly depressing game of "Asset Pipeline Compromise!" to go from idea to production.

@kvark: I welcome the standard formats by all means! Currently, we don't even have to choose the one. We can support many, and then see what's getting most required.

You will be doing yourselves a huge favour if every Amethyst tutorial & example project assumes the use of glTF by default. You really don't want a little bit of .obj here, some .dae there, a bit of .fbx in some places... Sure, you can aim to eventually support some or even all of these (.obj for example is pretty easy to support natively because there are no pesky animations involved), but by treating one as the default, you set a clear expectation for which format and asset pipeline will Just Work ™️.

Lastly, because of the quickly growing ecosystem around glTF you get a lot of other file formats for free, such as .obj, dae and .fbx (FBX in particular is a huge one, as it gives you access to most of the assets in the Unity Asset Store).

exhales -- this was just gonna be a quick "let's see what Amethyst is up to these days" but there you go.

[Xaeroxe]

So, we're going to add gltf support. Before we do that some refactoring is necessary but it's on @Rhuagh's task list so, I think we can close this and if @Rhuagh feels it's necessary we can create another issue regarding purely how it's going to be implemented and lacking the baggage this issue has obtained of justifying the addition.