3D artwork system

[andrew-gardner]

Some artwork is most effectively displayed as 3d geometry, sometimes using mirrors to create pepper's ghost effects, etc.

[felipesanches]

I fully support this idea. It is something I've been dreaming of for a while already and I will be really glad to see happen.

[stilett0]

Let's not forget the smf idea for the fully virtual 3D MESS frontend. Need a graphics card? Slot it in. Type on your virtual keyboard and mouse. Plug in peripherals, then unplug them.
...
I could never quite figure out if he was yanking our chain :-)

[felipesanches]

Can you help figuring out the best 3d scanning solutions for that purpose? We'll need to streamline the process so that we can get good scans of a large number of devices, but without a large monetary burden on the contributors.

Of course, something is better than nothing. So if we can hookup an initial simple 3d layout system, even if it is just a half-baked prototype, it is already a move in the direction of having something better.

[stilett0]

Well the problem with such a proposal is traditionally we've attempted to stay out of the "fancy fancy frontend" business, so a lot of the "3D MESS frontend" idea is more at home in a spinoff or side project. So I don't want to spend long in this thread about it. What Andrew proposed however is fine.

I don't know a lot about 3D scanning but I think Lord Nightmare or balrog know a smattering about it IIRC.

[felipesanches]

Yeah, lets focus on simple things first.
And it is not only about the MESS side of the project. It is also about arcade games. Here some simple examples:

• Wacky Gator
• Golly Ghost

And for non-arcade, I would start with the several Apple II clones, which are mostly the same in terms of circuitry, but have a much greater variety in terms of shape and colours of their external cases (an historically relevant aspect that we currently do not pay much attention to in MAME).

[felipesanches]

Here's a video of Wacky Gator running on MAME.
There's an animated 3D scene also in the video, which I created using OpenSCAD.

https://youtu.be/P48hQkLiNxM

Consider this a mockup of what I would like to see running nativelly in MAME. It still needs textures, for sure. And the alligators should actually be 3d scanned from original parts.

Also the dimensions of the cabinet are guessed, but the overall 3d model is a parametric design, so it should be easy to fix it with the same ease of patching software :-)

That's why I consider OpenSCAD (the algorithmic and parametric 3d modelling tool that was used to create this) to be a good candidate for the documentation of mechanical arcade hardware.

[felipesanches]

The SCAD script that generates this 3d scene is available at:
https://github.com/felipesanches/MAME_3D_models

[balr0g]

My only issue with SCAD is that it's difficult to export it from a parametric CAD program (e.g. Maya or Solidworks). It's a great source format, but I'm afraid not everyone will want to write out designs in OpenSCAD. Unfortunately it doesn't seem like OpenSCAD can export to COLLADA or WaveFront OBJ formats... so I'm not sure what's best here.

[andrew-gardner]

A 3d MAME artwork system needs a number of things, but the data for each of these things should be generic enough to be loaded in as many editor programs as possible. So, while OpenSCAD is a fine way to generate some of the data, the SCAD format isn't very portable. Instead it's the results of the SCAD file that are interesting.

MAME needs:

• A way to represent surfaces in 3d - traditionally this is polygons, and unless someone would push for subdivision surfaces, there really isn't much of a reason to do differently.
• A way to parameterize the surface of these objects - whether it be UVs or PTex'es, MAME will need to texture the objects (and potentially do other things with the parameterization like normal mapping).
• A way to address each object individually - likely by name / tag. This means that a single OBJ file for an entire artwork definition isn't sufficient since OBJ files don't really allow multiple, named objects in a single file.
• A way to define an animation for each object. This can be done on a per-vertex basis or rigidly. It could be done with one of many curve types or just by storing keyframes.
• Lighting information - note that there really is no standard file format for this.
• Other stuff that hasn't been imagined yet (physics parameters? smell? taste? shader types?).

Since each game artwork is a collection of all of this information, a file format that can store collections of things is essential. A common, effective way to do this is by using a tagged bbucket-style file format such as alembic (http://alembic.io/ - supported by Autodesk) where, like XML data files, any type of data you can imagine can be stuffed into the file and tagged appropriately. Unfortunately, insuring the tags and data contained within are identical across different authoring programs is challenging. Collada and FBX (Filmbox) are both attempts at files that can contain everything you could dream of, but really, are quite poor at adding things that the creators of the file format didn't already come up with - not to mention the version hell each have gone through.

Archives containing all the data in single a file (ZIP), using traditional file formats, enhanced by our own meta information are likely the way to go to be the most general. This way, any program that models can modify the meshes and parameterizations (likely OBJ files), any program that animates can create the animations, and whatever else is needed can be done by whatever program can do it.

This is, of course, open to further discussion - I'm just putting out the things I know to work well in situations like this.

[felipesanches]

Let me make a requirements list here as well :-)

Here are some things we need:

• a way to identify surfaces in the 3d mesh to which we'll apply specific emulated video buffers such as:
  • The surface of the 3d model of a CRT
  • The rectangular area where a dot-matrix LCD is located in the panel of an equipment
• 3dimensional positioning of classic elements that are already supported by our current artwork system such as LEDs, 7seg displays, etc...
• methods similar to set_output_ but for informing mechanical movement. This could be something like:
  • set_rotation("element-name", x, y, z, angle) //the x,y,z provides the rotation axis, but this could also be declared in the artwork XML file and then the method would only pass an angle value.
  • set_translate("element-name", x, y, z) //alternatively, a direction could be predetermined in the XML and the translation method would only pass a distance value along that direction

[felipesanches]

Also, we may still need the ability for the driver to algorithmically generate 3d content in addition to the shapes defined by the artwork and to control positioning and coordinate transformation of the objects arbitrarily if needed (perhaps this would be acchieved with a generic API and a scene_update callback, similar in spirit to the custom screen_update callbacks we already use in some places). This is specially the case when we think of 3d scenes that depend on chaotic interactions determined by physics, such as:

• an arduino controlled drone
• a radio-controlled toy racing car or a toy boat in a swimming pool
• plush-toys being grabbed and dropped by a gantry

[felipesanches]

We also need to make sure that coordinate transformations can be composed. That is useful for things like:

• the X/Y/Z CNC axis of several machines such as 3d printers, pen plotters, pick and place machinery, plush-toy gantries, etc
• also useful for stuff we already support in MAME such as the composed movements of the Galaxy Force II Super Deluxe cabinet

[felipesanches]

Type design software has the concept of "anchor-points". These are used for determining the coordinate in the canvas where a glyph can have an additional shape appended. As an exemple, you can have an upper-case A glyph with an anchor point defined a bit above it. That anchor could then be used to attach diacritical marks such as acute, circunflex, or many other outlines for optional marks.

I think we may use the same concept here, by having anchor points in the 3D space, so that there are predefined places where you can attach optional peripherals in a scene. So we would have an Apple II 3d model with anchor points for a Monitor behind it, another anchor point for a printer to the right of it and for floppy drives to its left. An Apple II floppy drive 3d model could have an anchor point for stacking another drive on top of itself. Similarly, a SEGA Genesis would have an anchor point in the cartridge slot (which makes us figure out we actually need an anchor transform as well and not only a point!).

I'm obviously pushing the boundaries here, but that's only for the purpose of brainstorming. We naturally can start with something much simpler than all of this and then gradually improve this artwork subsystem capabilities.

[balr0g]

Let's also not forget pinball, and the concerns with simulating those. (I don't consider the current solutions all that viable in the long-term.)

[felipesanches]

And templates for camera positioning and motion. One device can have a few different views that are most interesting for the user, so these could be available for easy navigation. Examples:

Galaxy Force II Super Deluxe cabinet:

• a view from outside of the cabinet (like people observing someone play on an arcade room)
• a view form inside the cabinet looking directly to the screen

Replicator 1 3D printer:

• zoom in to see the text messages being displayed in the LCD panel
• zoom out to see the whole machine operating
• focus on the 3d printer bed to see what is being printed

Pinball machines:

• playfield views from 3 or 4 different fixed angles
• an interactive camera that smoothly follows overall region of the playfield where the ball is during play
• a view of the dot-matrix display and the backglass

Any device could, by default, have a generic "showroom" view that constantly rotates around the device.

[felipesanches]

For the record, I do have a python+OpenSCAD pinball 3d modelling project at github here:
https://github.com/felipesanches/SolidPinball/

And this is one example of a pinball playfield built using it:
https://github.com/felipesanches/PARTYLandRemake

[felipesanches]

One more consideration:

We should plan to always have at least a basic 3d scene natively available in MAME, with an unbranded overall geometry of the device (just like the Wacky Gator animation in this video https://www.youtube.com/watch?v=P48hQkLiNxM with the alligators being replesented by a simple green brick), while textures (backglass and sticker artwork) and actual 3d scanned models of parts (such as the alligators in Wacky Gator) would be loaded from an optionally available 3d artwork pack.

[felipesanches]

I am busy with my day-time work now, but I'd like to suggest that we start using the wiki pages at this GitHub repository to draft a spec of the 3d artwork system. Can someone with administrative rights enable the wiki feature and then create a "3D Artwork Specification" wiki page?

I'll be back tonight after my work-hours to help start drafting this.

First thoughts/questions are:

• which macro markup keyword to use on a driver's MACHINE_CONFIG for declaring a 3d layout ?
• how to make it compatible with a pre-existing 2D layout?
• which directory will hold the in-tree basic (unbranded) 3d layouts ?
• what will be the structure of each ZIP file containing 3d models and scene/layout description ?
• etc... Please add you own ideas

I'll be back later this night.

[ghost]

Specification is getting pretty heavy.
For minimum requirements, how about adding an 8-core CPU ;-)
In any case, certainly current-gen multi-core CPUs are required.
Maybe one core can be dedicated to the 3D artwork task?
Most multi-core CPU's are under-utilized.....

[felipesanches]

We can start small and increase gradually.
And things will only get slow if we really abuse the system.

Having said that, I appreciate the idea of delegating 3D stuff to a thread in a secondary CPU, if available. Not sure if that's really so necessary and effective performance-wise though. We'll see as we try :-)

[james-wallace-ghub]

If I've understood correctly, the 3D artwork is effectively a part of MAME itself, albeit a background task, as opposed to passing off to an external agent for rendering and input handling i.e. the synergy between Visual Pinball and the VPinMAME DLL build, or even the MAMEHooker DLL's connection to the output system. Anything that can facilitate the display of these non-video features to a good standard is a good idea IMHO, provided that the people who are good at making these kind of things aren't put off by the complexity of the system we end up using.

[james-wallace-ghub]

I'm not so much talking about the system, as the tools - if collaborating with someone like that is going to make the actual process of modelling easier, that's got to help, surely?

[felipesanches]

I have been improving my python-based prototype here:
https://github.com/felipesanches/MAME_3D_models

You'll need to have OpenSCAD installed in order to compile the 3d models.
The end users wiill only deal with the final files, so no worries. And obviously one does not need to use OpenSCAD in order to create 3d models. It is just what I am used to. Naturally we'll have scanned meshes as well, directly from the real parts when 3d scanners get more affordable.

I'm converging towards a certain XML schema for describing the 3d scene and I am starting to like it. The 2 devices I'm working on while developing this system are Wacky Gator and Galaxy Force II Super Deluxe. Their XML 3d artwork description files can be seen here:

https://github.com/felipesanches/MAME_3D_models/blob/master/wackygtr/wackygtr.3dlay
https://github.com/felipesanches/MAME_3D_models/blob/master/gforce2/gforce2.3dlay

[felipesanches]

Also, here are a few previews of the system running:

[felipesanches]

Be sure to read the instructions in the README.md file here: https://github.com/felipesanches/MAME_3D_models

I just added the most important stuff there so that it is easier for everybody to try running this and to provide some feedback.

[felipesanches]

It looks like SEGA UFO is another nice driver that could benefit from this :-)

http://git.redump.net/mame/tree/src/mame/drivers/segaufo.cpp

https://seoulsounds.files.wordpress.com/2010/11/img_0587.jpg

[retro27]

Im interested in your 3d Mame Project, Great Idea, how do I get this work? what build of mame do you use? Ive been looking around and found this (https://3dwarehouse.sketchup.com/user.html?id=0098879480321870168359978) Gozer has created hunderds of 3d cabinets using Sketch Up, They are all free to use. Are they any help?

Cheers

Retro27

[felipesanches]

I wish there was more people interested in this...

[stilett0]

Prior work in this area is mostly closed-source, with a few exceptions:
http://www.digitalcybercherries.com/
http://www.mameworld.info/3darcade/index.html
http://www.futurepinball.com/
https://sourceforge.net/projects/vpinball/
http://www.unit3dpinball.net/

[james-wallace-ghub]

Am I being massively naive here, or does our OSI compliant status give us any pull with regard to the Blender toolbase. It's not easy to use by any stretch, but at least that is open.

[rb6502]

Not really. There's no need to use an F/OSS program to create models for another F/OSS program.

Also, things are happening about this, but they cannot currently be discussed.

[MooglyGuy]

I am already laying the groundwork for a modern rendering system via BGFX. The first step is to re-create the D3D HLSL shader system in a data-driven way in BGFX. For this, I'm using my own custom JSON format. You can view its current form in the "bgfx_shaders" branch, I believe it's a public branch.

From there it's pretty much trivial to add 3D model support to the BGFX renderer. OpenSCAD seems esoteric, but if it's the format used by a lot of existing models, it's a good choice for an initial format. I would hope there would be a more open format that we could leverage. I plan on making it easy for people to add code-side loaders for various formats.

I have to be honest, I think the posted screenshots look pretty ugly without textures. It gets the point across, but it would need better integration with a proper artwork library system which I also intend to write.

[MooglyGuy]

However, don't let me stop anyone from writing a capable, extensible system.

[felipesanches]

As a technology reference, here's a good usage example of 3d scans and photogammetry that may inspire future efforts for MAME 3D artwork: http://phys.org/news/2016-03-archaeologists-d-interactive-digital-reconstruction.html

[happppp]

It's less on-topic and inspirational to this issue than the Utah Teapot.
Please don't clutter here, this is not a forum.

On Wed, Mar 23, 2016 at 9:58 PM, Felipe Corrêa da Silva Sanches <
notifications@github.com> wrote:

[Not sure who did it but...] I think it was a bit rude to simply delete my
previous posting in this thread. Anyway...

As a technology reference, here's a good usage example of 3d scans and
photogammetry that may inspire future efforts for MAME 3D artwork:
http://phys.org/news/2016-03-archaeologists-d-interactive-digital-reconstruction.html

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
#388 (comment)

[felipesanches]

@MooglyGuy, last March (as seen above on #388 (comment)) you mentioned you were working on something related to this issue. Could you perhaps tell us a bit more about the current status of its development, please?

[felipesanches]

@MooglyGuy any update on this?

[galibert]

MG is currently overloaded with real-life work, don't expect an answer from him until he can breathe again. OG.

…

On Wed, Mar 15, 2017 at 6:12 PM, Felipe Corrêa da Silva Sanches < ***@***.***> wrote: @MooglyGuy <https://github.com/MooglyGuy> any update on this? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#388 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AI0i8f6rnN_ReGpi7RNZtiKmV-n0F5nTks5rmBv4gaJpZM4GO8D8> .

[felipesanches]

I do not intend to put pressure on anyone here, but it would be nice though, to at least make public whatever work has been done in the past on this so that other people can jump in. Otherwise, I have the impression that the past statements in this thread end up discouraging other people from doing anything related to this due to fear of unnecessarily duplicating efforts.

[felipesanches]

Some (but not all) of the games mentioned in this article would benefit from 3d models of their cabinets in MAME:

https://arstechnica.com/gaming/2014/08/unemulated-eleven-classic-arcade-games-you-cant-play-at-home/2/

Funny and insightful that Waku-Waku Sonic Patrol Car is indeed emulated in MAME nowadays, but the article author explicitely mentions the lack of a "complete experience" when you miss the cabinet geometry and movements. :-)

[cuavas]

3D artwork doesn't replace the experience of a motion cabinet, though. We're never going to be able to replicate those things without a full-motion simulation cockpit.

[felipesanches]

sure! :-)

[rb6502]

VR potentially could replace part of that experience though.

[JoakimLarsson]

An alternative could be to use the SVG path, as I did in the Prodigy driver, and render WebGL instead, I have read a few tutorials on the subject including a VR renderer written completely in WebGL.

[felipesanches]

Regarding the SVG layout system, how does it behave in an emscripten build? Is there anything special enabling browsers to actually render native SVG (from an emscripten-build C program) or will it necessarily run the transpiled C code from nanosvg for rasterizing SVG by ploting pixels on SDL and then that gets sent to the browser as a pixel buffer ?

I'm asking this because you mentioned both WebGL and SVG in a single comment. These are both web technologies and we've seen great work being done by projects like the Archive.org Internet Arcade hosting emscripten builds of selections of specific MAME drivers. It would be great if there was some way of employing 3d technologies that works well both in the native builds as well as in these special-purpose web-browser builds.

Any idea if there's a way to integreate a piece of emscripten-built C-code with web APIs and technologies such as WebGL and SVG?

[JoakimLarsson]

I have no exprience with emscripten development, SVG is supported by all major browsers and webGL too but would require adequate hardware to be useful. The work I did with SVG is complentary with the internal UI and all is included in the latest build, just try 'mame prodigy -http' snd connect with a browser to http://127.0.0.1:8080 and you'll get basically the same UI layout as with the builtin layout engine, but using SVG. Nothing fancy yet though.

[felipesanches]

This page describes emscripten support for OpenGL / WebGL:
https://kripken.github.io/emscripten-site/docs/porting/multimedia_and_graphics/OpenGL-support.html

[aurium]

My two cents about MAME on a webpage through Emscripten:

OpenGL Display

You can have MAME with WebGL in a webpage compiling the same native code. Emscripten implements the OpenGL, SDL, and some other libs using web technologies out the box. If you write a C/C++ code that requires OpenGL, emscripten will compile it to Javascript or WebAssembly (+ optional HTML with preset <canvas> to do the magick)

SVG Display

If MAME implements a viewer, with nanosvg, based on SDL, yeah, the result will be a raster inside a <canvas> with the same power and limitation of nanosvg.

My proposal is to write an adapter that calls nanosvg on native compilation, or calls a javascript function when compiled for the web. This js function could simply create an iframe in the page (as it open a display window) and send the SVG into it as data url.

[felipesanches]

@MooglyGuy, roughly 5 years ago you said "I am already laying the groundwork for a modern rendering system via BGFX." Has that evolved since then? What's the current status of this?

[MooglyGuy]

If it evolved since then, you'd know. I know you're on the MAME Discord, so if you're actually a graphics programmer and have some input to give, then discuss it there. If not, then the same advice as always applies, be patient and wait for results.

[felipesanches]

MAME is too big, even though I sometimes read commit logs and I also participate in the Discord channel you mentioned, I am not fully aware of everything that's going on. Also your message from 5 years ago made it sound like you were working in private on the new stuff, so I would have no way of knowing the current state of it without asking ;-)

And, sorry, I did not mean to sound rude here. I only posted the question because the topic surfaced on a conversation with a friend a few minutes ago which made me re-read this thread and then I naturally asked about it here.

[MooglyGuy]

I am working on it in both private and public, it just depends on what your definition of "working on it" is. This is the same situation as when any given user asks about support for this or that driver: It will eventually happen, and unless you're volunteering to directly work on it, just be patient and wait for results.