3D artwork system #388

Open
andrew-gardner opened this Issue Oct 14, 2015 · 39 comments

Projects

None yet

10 participants

@andrew-gardner
Member

Some artwork is most effectively displayed as 3d geometry, sometimes using mirrors to create pepper's ghost effects, etc.

@felipesanches
Contributor

I fully support this idea. It is something I've been dreaming of for a while already and I will be really glad to see happen.

@stilett0
Contributor

Let's not forget the smf idea for the fully virtual 3D MESS frontend. Need a graphics card? Slot it in. Type on your virtual keyboard and mouse. Plug in peripherals, then unplug them.
...
I could never quite figure out if he was yanking our chain :-)

@felipesanches
Contributor

Can you help figuring out the best 3d scanning solutions for that purpose? We'll need to streamline the process so that we can get good scans of a large number of devices, but without a large monetary burden on the contributors.

Of course, something is better than nothing. So if we can hookup an initial simple 3d layout system, even if it is just a half-baked prototype, it is already a move in the direction of having something better.

@stilett0
Contributor

Well the problem with such a proposal is traditionally we've attempted to stay out of the "fancy fancy frontend" business, so a lot of the "3D MESS frontend" idea is more at home in a spinoff or side project. So I don't want to spend long in this thread about it. What Andrew proposed however is fine.

I don't know a lot about 3D scanning but I think Lord Nightmare or balrog know a smattering about it IIRC.

@felipesanches
Contributor

Yeah, lets focus on simple things first.
And it is not only about the MESS side of the project. It is also about arcade games. Here some simple examples:

  • Wacky Gator
  • Golly Ghost

And for non-arcade, I would start with the several Apple II clones, which are mostly the same in terms of circuitry, but have a much greater variety in terms of shape and colours of their external cases (an historically relevant aspect that we currently do not pay much attention to in MAME).

@felipesanches
Contributor

Here's a video of Wacky Gator running on MAME.
There's an animated 3D scene also in the video, which I created using OpenSCAD.

https://youtu.be/P48hQkLiNxM

Consider this a mockup of what I would like to see running nativelly in MAME. It still needs textures, for sure. And the alligators should actually be 3d scanned from original parts.

Also the dimensions of the cabinet are guessed, but the overall 3d model is a parametric design, so it should be easy to fix it with the same ease of patching software :-)

That's why I consider OpenSCAD (the algorithmic and parametric 3d modelling tool that was used to create this) to be a good candidate for the documentation of mechanical arcade hardware.

@felipesanches
Contributor

The SCAD script that generates this 3d scene is available at:
https://github.com/felipesanches/MAME_3D_models

@balr0g
Contributor
balr0g commented Oct 28, 2015

My only issue with SCAD is that it's difficult to export it from a parametric CAD program (e.g. Maya or Solidworks). It's a great source format, but I'm afraid not everyone will want to write out designs in OpenSCAD. Unfortunately it doesn't seem like OpenSCAD can export to COLLADA or WaveFront OBJ formats... so I'm not sure what's best here.

@andrew-gardner
Member

A 3d MAME artwork system needs a number of things, but the data for each of these things should be generic enough to be loaded in as many editor programs as possible. So, while OpenSCAD is a fine way to generate some of the data, the SCAD format isn't very portable. Instead it's the results of the SCAD file that are interesting.

MAME needs:

  • A way to represent surfaces in 3d - traditionally this is polygons, and unless someone would push for subdivision surfaces, there really isn't much of a reason to do differently.
  • A way to parameterize the surface of these objects - whether it be UVs or PTex'es, MAME will need to texture the objects (and potentially do other things with the parameterization like normal mapping).
  • A way to address each object individually - likely by name / tag. This means that a single OBJ file for an entire artwork definition isn't sufficient since OBJ files don't really allow multiple, named objects in a single file.
  • A way to define an animation for each object. This can be done on a per-vertex basis or rigidly. It could be done with one of many curve types or just by storing keyframes.
  • Lighting information - note that there really is no standard file format for this.
  • Other stuff that hasn't been imagined yet (physics parameters? smell? taste? shader types?).

Since each game artwork is a collection of all of this information, a file format that can store collections of things is essential. A common, effective way to do this is by using a tagged bbucket-style file format such as alembic (http://alembic.io/ - supported by Autodesk) where, like XML data files, any type of data you can imagine can be stuffed into the file and tagged appropriately. Unfortunately, insuring the tags and data contained within are identical across different authoring programs is challenging. Collada and FBX (Filmbox) are both attempts at files that can contain everything you could dream of, but really, are quite poor at adding things that the creators of the file format didn't already come up with - not to mention the version hell each have gone through.

Archives containing all the data in single a file (ZIP), using traditional file formats, enhanced by our own meta information are likely the way to go to be the most general. This way, any program that models can modify the meshes and parameterizations (likely OBJ files), any program that animates can create the animations, and whatever else is needed can be done by whatever program can do it.

This is, of course, open to further discussion - I'm just putting out the things I know to work well in situations like this.

@felipesanches
Contributor

Let me make a requirements list here as well :-)

Here are some things we need:

  • a way to identify surfaces in the 3d mesh to which we'll apply specific emulated video buffers such as:
    • The surface of the 3d model of a CRT
    • The rectangular area where a dot-matrix LCD is located in the panel of an equipment
  • 3dimensional positioning of classic elements that are already supported by our current artwork system such as LEDs, 7seg displays, etc...
  • methods similar to set_output_ but for informing mechanical movement. This could be something like:
    • set_rotation("element-name", x, y, z, angle) //the x,y,z provides the rotation axis, but this could also be declared in the artwork XML file and then the method would only pass an angle value.
    • set_translate("element-name", x, y, z) //alternatively, a direction could be predetermined in the XML and the translation method would only pass a distance value along that direction
@felipesanches
Contributor

Also, we may still need the ability for the driver to algorithmically generate 3d content in addition to the shapes defined by the artwork and to control positioning and coordinate transformation of the objects arbitrarily if needed (perhaps this would be acchieved with a generic API and a scene_update callback, similar in spirit to the custom screen_update callbacks we already use in some places). This is specially the case when we think of 3d scenes that depend on chaotic interactions determined by physics, such as:

  • an arduino controlled drone
  • a radio-controlled toy racing car or a toy boat in a swimming pool
  • plush-toys being grabbed and dropped by a gantry
@felipesanches
Contributor

We also need to make sure that coordinate transformations can be composed. That is useful for things like:

  • the X/Y/Z CNC axis of several machines such as 3d printers, pen plotters, pick and place machinery, plush-toy gantries, etc
  • also useful for stuff we already support in MAME such as the composed movements of the Galaxy Force II Super Deluxe cabinet
@felipesanches
Contributor

Type design software has the concept of "anchor-points". These are used for determining the coordinate in the canvas where a glyph can have an additional shape appended. As an exemple, you can have an upper-case A glyph with an anchor point defined a bit above it. That anchor could then be used to attach diacritical marks such as acute, circunflex, or many other outlines for optional marks.

I think we may use the same concept here, by having anchor points in the 3D space, so that there are predefined places where you can attach optional peripherals in a scene. So we would have an Apple II 3d model with anchor points for a Monitor behind it, another anchor point for a printer to the right of it and for floppy drives to its left. An Apple II floppy drive 3d model could have an anchor point for stacking another drive on top of itself. Similarly, a SEGA Genesis would have an anchor point in the cartridge slot (which makes us figure out we actually need an anchor transform as well and not only a point!).

I'm obviously pushing the boundaries here, but that's only for the purpose of brainstorming. We naturally can start with something much simpler than all of this and then gradually improve this artwork subsystem capabilities.

@balr0g
Contributor
balr0g commented Oct 28, 2015

Let's also not forget pinball, and the concerns with simulating those. (I don't consider the current solutions all that viable in the long-term.)

@felipesanches
Contributor

And templates for camera positioning and motion. One device can have a few different views that are most interesting for the user, so these could be available for easy navigation. Examples:

Galaxy Force II Super Deluxe cabinet:

  • a view from outside of the cabinet (like people observing someone play on an arcade room)
  • a view form inside the cabinet looking directly to the screen

Replicator 1 3D printer:

  • zoom in to see the text messages being displayed in the LCD panel
  • zoom out to see the whole machine operating
  • focus on the 3d printer bed to see what is being printed

Pinball machines:

  • playfield views from 3 or 4 different fixed angles
  • an interactive camera that smoothly follows overall region of the playfield where the ball is during play
  • a view of the dot-matrix display and the backglass

Any device could, by default, have a generic "showroom" view that constantly rotates around the device.

@felipesanches
Contributor

For the record, I do have a python+OpenSCAD pinball 3d modelling project at github here:
https://github.com/felipesanches/SolidPinball/

And this is one example of a pinball playfield built using it:
https://github.com/felipesanches/PARTYLandRemake

@felipesanches
Contributor

One more consideration:

We should plan to always have at least a basic 3d scene natively available in MAME, with an unbranded overall geometry of the device (just like the Wacky Gator animation in this video https://www.youtube.com/watch?v=P48hQkLiNxM with the alligators being replesented by a simple green brick), while textures (backglass and sticker artwork) and actual 3d scanned models of parts (such as the alligators in Wacky Gator) would be loaded from an optionally available 3d artwork pack.

@felipesanches
Contributor

I am busy with my day-time work now, but I'd like to suggest that we start using the wiki pages at this GitHub repository to draft a spec of the 3d artwork system. Can someone with administrative rights enable the wiki feature and then create a "3D Artwork Specification" wiki page?

I'll be back tonight after my work-hours to help start drafting this.

First thoughts/questions are:

  • which macro markup keyword to use on a driver's MACHINE_CONFIG for declaring a 3d layout ?
  • how to make it compatible with a pre-existing 2D layout?
  • which directory will hold the in-tree basic (unbranded) 3d layouts ?
  • what will be the structure of each ZIP file containing 3d models and scene/layout description ?
  • etc... Please add you own ideas

I'll be back later this night.

@ghost
ghost commented Oct 28, 2015

Specification is getting pretty heavy.
For minimum requirements, how about adding an 8-core CPU ;-)
In any case, certainly current-gen multi-core CPUs are required.
Maybe one core can be dedicated to the 3D artwork task?
Most multi-core CPU's are under-utilized.....

@felipesanches
Contributor

We can start small and increase gradually.
And things will only get slow if we really abuse the system.

Having said that, I appreciate the idea of delegating 3D stuff to a thread in a secondary CPU, if available. Not sure if that's really so necessary and effective performance-wise though. We'll see as we try :-)

@u-man74
u-man74 commented Oct 29, 2015

OK ... just some questions, because it is not clear enough for me:
the 3D artwork would be a visual 3D presentation of mechanical devices, correct?
If so, I totally agree with all aspects andrew-gardner wrote. Especially the used 3D format (.obj, .dxf etc.) needs to be choosen wisely. The GameEx front-end for example, has a wide range of 3D cabinets (over 400 cabs) which is fine, but the supported 3D .x file is a pain, if changes are needed you are basically screwed.

Also why starting from scratch? Why not looking for a good cooperation/synergy with allready existing projects like http://www.digitalcybercherries.com/ ???

In my eyes, the most advanced project to that matter. This would not only be just a presentation, you would actually even play the machines and looking into the future, we would also support VR/AR.

The same goes for V-pinballs, because there are allready hundreds of pinball-machines emulated.

@james-wallace-ghub
Member

If I've understood correctly, the 3D artwork is effectively a part of MAME itself, albeit a background task, as opposed to passing off to an external agent for rendering and input handling i.e. the synergy between Visual Pinball and the VPinMAME DLL build, or even the MAMEHooker DLL's connection to the output system. Anything that can facilitate the display of these non-video features to a good standard is a good idea IMHO, provided that the people who are good at making these kind of things aren't put off by the complexity of the system we end up using.

@u-man74
u-man74 commented Oct 29, 2015

but thats the whole point.... why would be someone skilled in 3D modeling be interested to create "dead" things, if he can do the same for a vivid system, where the model has a higher meaning.

Dont get me wrong, i like the idea too and you are right with anything that can facilitate the display of these non-video features to a good standard is a good idea, but teamworking with other people that do exactly this thing for years, cant be wrong. At least we should look, to support a standard, that others are also doing, so we can benefit of each other and i think they are also maked up their minds about which 3D format is the best. Even if we have our own, it can be considered that it would not be bad, to ask the others, how they would approach a task like ours ;) .

@james-wallace-ghub
Member

I'm not so much talking about the system, as the tools - if collaborating with someone like that is going to make the actual process of modelling easier, that's got to help, surely?

@felipesanches
Contributor

I have been improving my python-based prototype here:
https://github.com/felipesanches/MAME_3D_models

You'll need to have OpenSCAD installed in order to compile the 3d models.
The end users wiill only deal with the final files, so no worries. And obviously one does not need to use OpenSCAD in order to create 3d models. It is just what I am used to. Naturally we'll have scanned meshes as well, directly from the real parts when 3d scanners get more affordable.

I'm converging towards a certain XML schema for describing the 3d scene and I am starting to like it. The 2 devices I'm working on while developing this system are Wacky Gator and Galaxy Force II Super Deluxe. Their XML 3d artwork description files can be seen here:

https://github.com/felipesanches/MAME_3D_models/blob/master/wackygtr/wackygtr.3dlay
https://github.com/felipesanches/MAME_3D_models/blob/master/gforce2/gforce2.3dlay

@felipesanches
Contributor

Also, here are a few previews of the system running:
mame3d_gforce2_observer
mame3d_gforce2_orbiting_camera
mame3d_gforce2_player_viewpoint
mame3d_wackygtr_alligators_camera
mame3d_wackygtr_orbiting_camera
mame3d_wackygtr_scorepanel_camera

@felipesanches
Contributor

Be sure to read the instructions in the README.md file here: https://github.com/felipesanches/MAME_3D_models

I just added the most important stuff there so that it is easier for everybody to try running this and to provide some feedback.

@felipesanches
Contributor

It looks like SEGA UFO is another nice driver that could benefit from this :-)

http://git.redump.net/mame/tree/src/mame/drivers/segaufo.cpp

https://seoulsounds.files.wordpress.com/2010/11/img_0587.jpg

@retro27
retro27 commented Nov 27, 2015

Im interested in your 3d Mame Project, Great Idea, how do I get this work? what build of mame do you use? Ive been looking around and found this (https://3dwarehouse.sketchup.com/user.html?id=0098879480321870168359978) Gozer has created hunderds of 3d cabinets using Sketch Up, They are all free to use. Are they any help?

Cheers

Retro27

@felipesanches
Contributor

@retro27 This proof-of-concept is not (yet?) integrated into the official MAME codebase. It works with the latest MAME code running side-by-side with python scripts which are available in my separate personal repository at https://github.com/felipesanches/MAME_3D_models

Maybe someone might be interested in porting this proof-of-concept into MAME codebase in an experimental branch?

@felipesanches
Contributor

I wish there was more people interested in this...

@james-wallace-ghub
Member

Am I being massively naive here, or does our OSI compliant status give us any pull with regard to the Blender toolbase. It's not easy to use by any stretch, but at least that is open.

@rb6502
Contributor
rb6502 commented Mar 14, 2016

Not really. There's no need to use an F/OSS program to create models for another F/OSS program.

Also, things are happening about this, but they cannot currently be discussed.

@MooglyGuy
Contributor

I am already laying the groundwork for a modern rendering system via BGFX. The first step is to re-create the D3D HLSL shader system in a data-driven way in BGFX. For this, I'm using my own custom JSON format. You can view its current form in the "bgfx_shaders" branch, I believe it's a public branch.

From there it's pretty much trivial to add 3D model support to the BGFX renderer. OpenSCAD seems esoteric, but if it's the format used by a lot of existing models, it's a good choice for an initial format. I would hope there would be a more open format that we could leverage. I plan on making it easy for people to add code-side loaders for various formats.

I have to be honest, I think the posted screenshots look pretty ugly without textures. It gets the point across, but it would need better integration with a proper artwork library system which I also intend to write.

@MooglyGuy
Contributor

However, don't let me stop anyone from writing a capable, extensible system.

@felipesanches
Contributor

As a technology reference, here's a good usage example of 3d scans and photogammetry that may inspire future efforts for MAME 3D artwork: http://phys.org/news/2016-03-archaeologists-d-interactive-digital-reconstruction.html

@happppp
Member
happppp commented Mar 23, 2016

It's less on-topic and inspirational to this issue than the Utah Teapot.
Please don't clutter here, this is not a forum.

On Wed, Mar 23, 2016 at 9:58 PM, Felipe Corrêa da Silva Sanches <
notifications@github.com> wrote:

[Not sure who did it but...] I think it was a bit rude to simply delete my
previous posting in this thread. Anyway...

As a technology reference, here's a good usage example of 3d scans and
photogammetry that may inspire future efforts for MAME 3D artwork:
http://phys.org/news/2016-03-archaeologists-d-interactive-digital-reconstruction.html


You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
#388 (comment)

@felipesanches
Contributor

@MooglyGuy, last March (as seen above on #388 (comment)) you mentioned you were working on something related to this issue. Could you perhaps tell us a bit more about the current status of its development, please?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment