Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[$470 Bounty] Support for MAME backdrop, screen, bezel, and overlay "artwork" #7419

Closed
markwkidd opened this issue Oct 12, 2018 · 70 comments

Comments

Projects
None yet
@markwkidd
Copy link
Contributor

commented Oct 12, 2018

Bounty link - Contribute today!

Context

As I understand it, RetoArch's Overlay system has roots in providing a touch interface rather than reproducing integral game artwork. Even though folks successfully use the existing RA overlay system for applications like displaying arcade game bezels or Vectrex cover overlay sheets, it has limitations that prevent some games with such artwork from being fully emulated.

It is possible to emulate this integral artwork within cores, but doing so requires cores to pre-scale video in order to make use of high-resolution artwork files, negating many of the video-related benefits of libretro-based emulation.

Proposal

Add an artwork system compatible with the MAME artwork format, inspired by but distinct from the existing RetroArch Overlay/Control system.

MAME Layout file documentation: https://docs.mamedev.org/techspecs/layout_files.html

Some, but not all of, the layers in this schema:

  • backdrop Intended for use in situations were the screen image is projected over a backdrop using a semi-reflective mirror (Pepper’s ghost). This arrangement is famously used in the Space Invaders deluxe cabinet.
  • screen This is the emulated video. It is drawn using additive blending.
  • overlay This layer is intended for use translucent overlays used to add colour to games using monochrome monitors like Circus, Gee Bee, and of course Space Invaders. It is drawn using RGB multiplication.
  • bezel This layer is for elements that surround and potentially obscure the screen image. It is drawn with standard alpha blending.

By default, layers are drawn in this order (from back to front):

  1. screen (add)
  2. overlay (multiply)
  3. backdrop (add)
  4. bezel (alpha)

If a view has multiple backdrop elements and no overlay elements, a different order is used (from back to front):

  1. backdrop (alpha)
  2. screen (add)
  3. bezel (alpha)

@markwkidd markwkidd changed the title Add support for "integral artwork" - backdrop/bezel/overlay [$55 Bounty] Add support for "integral artwork" - backdrop/bezel/overlay Oct 12, 2018

@markwkidd

This comment has been minimized.

Copy link
Contributor Author

commented Oct 12, 2018

Bounty up to $55 - thanks @UDb23!

@twinaphex twinaphex added the bounty label Oct 12, 2018

@markwkidd markwkidd changed the title [$55 Bounty] Add support for "integral artwork" - backdrop/bezel/overlay [$65 Bounty] Robust support for backdrop, bezel, and overlay "integral artwork" Oct 12, 2018

@markwkidd markwkidd changed the title [$65 Bounty] Robust support for backdrop, bezel, and overlay "integral artwork" [$150 Bounty] Robust support for backdrop, bezel, and overlay "integral artwork" Oct 13, 2018

@markwkidd

This comment has been minimized.

Copy link
Contributor Author

commented Oct 13, 2018

Bounty up to $150 thanks to Thatman84!

@cosmo0

This comment has been minimized.

Copy link

commented Oct 17, 2018

Hey! I saw your posts on various forums and though maybe I could help a little.

I tried to convert MAME overlays to Libretro, and failed, but maybe my experience can provide a starting point. Let's take OrionsAngel's 1942 as an example.

OrionsAngel, as he distributes his overlays, provides 2 main files for each game: Artwork\1942.zip, containing artworks and overlay layout, and Cfg\1942.cfg, containing various position stuff.

Inside the zip file, there are several things:

  • a lay file (usually called default.lay but not always) describing the various possible layout configurations (because a MAME overlay file can have multiple variants inside) and the positioning of elements (screen, overlay, backdrop, bezel)
  • multiple image files, changing as needed by the lay file

As far as I can tell, the elements positions in the lay file never match the expected output, at least in OrionsAngel's overlays.
To fix this, I assume one has to look in the cfg file. Unfortunately, this file has a completely different logic, which still evades me.

This cfg file is a simple XML file, but it contains relative coordinates:

<screen index="0" hoffset="-0.004000" hstretch="0.982000" />

These coordinates, applied either to the output resolution, or the coordinates in the lay file, do not result in anything meaningful.

Understanding the relationship between the lay and the cfg files values will be mandatory to apply the overlays.


I have already posted my findings on the Libretro forums: https://forums.libretro.com/t/understanding-mame-overlay-config-to-port-them-to-libretro-config/14539

I also have a conversion script started, but as I said, I didn't find how to position the screen correctly, so after the files have been created, I have to measure everything in Photoshop and report the values in the config files: https://github.com/cosmo0/retropie-overlays-arcade-realistic/blob/master/src/import-mame.js


I hope this helps even a little bit :)

@markwkidd

This comment has been minimized.

Copy link
Contributor Author

commented Oct 17, 2018

Thank you @cosmo0 that seems incredibly useful!

@cosmo0

This comment has been minimized.

Copy link

commented Oct 17, 2018

I'll also note that the MAME artworks are composed of 4 layers (screen, backdrop, overlay, bezel) so it doesn't match your 3 layers proposal, the system will have to merge at least 2 of them (probably overlay and bezel).
I have never encountered an artwork with "real" backdrop (only black) so I don't know what it's used for.

I will also note that all the arcade overlays I know have to be displayed on top of the emulated video, and not under, as in your proposal. Many arcade overlays mask a part of the screen, for instance:

  • avalnche (and many others) reproduces real arcade cabinet that used the cabinet artwork to improve the very basic graphics
  • aztarac displays labels in the overlay
  • bzone separates screen areas

In addition, OrionsAngel's artwork adds "screen glare and scratches" to provide more "realism" to his overlays, which has to be put on top of the emulated video.

I could not find any documentation about all of this, nor the MAME code which handles it, but I wasn't sure where to look, so maybe it will be obvious to someone more knowledgeable.

@markwkidd

This comment has been minimized.

Copy link
Contributor Author

commented Oct 17, 2018

@cosmo0 "Backdrops" which display behind the video are an important part of the MAME artwork system. Check the OP, I link to a video demonstrating the romset warrior which requires backdrop artwork. I also link to a blog with illustrations of some handheld games with backdrop art.

This proposal is to allow display of backdrops "under" the emulated video, along with all of the other layers "on top" of video.

Based on what you are saying maybe the proposal needs to be amended to have four layers.

@cosmo0 do you know what the screen layer is? I wonder how it is different than MAME's overlay layer

@cosmo0

This comment has been minimized.

Copy link

commented Oct 17, 2018

@markwkidd ok, all overlays I found simulate the backdrop with a semi-transparent overlay instead, but I understand how it's important.

The screen layer is just the emulated video, and where it's located.

Sample layout file from OrionsAngel's 1942:

<!-- 1942.lay -->
<mamelayout version="2">
  <element name="bezel">
    <image file="1942_bezel.png" />
  </element>
  <element name="bezel_alt1">
    <image file="1942_bezel_alt1.png" />
  </element>
  <element name="bezel_alt2">
    <image file="1942_bezel_alt2.png" />
  </element>
  <element name="screen_bezel">
    <image file="vert_screen_bezel.png" />
  </element>
  <element name="screen_mask">
    <image file="vert_screen_mask.png" />
  </element>
  <view name="Cab Artwork">
    <screen index="0">
      <bounds x="555" y="0" width="810" height="1080" />
    </screen>
    <overlay element="screen_mask">
      <bounds x="554" y="0" width="812" height="1080" />
    </overlay>
    <backdrop element="screen_bezel">
      <bounds x="518" y="0" width="884" height="1080" />
    </backdrop>
    <bezel element="bezel">
      <bounds x="0" y="0" width="1920" height="1080" />
    </bezel>
  </view>
  <view name="Cab Artwork (alt1)">
    <screen index="0">
      <bounds x="555" y="0" width="810" height="1080" />
    </screen>
    <overlay element="screen_mask">
      <bounds x="554" y="0" width="812" height="1080" />
    </overlay>
    <backdrop element="screen_bezel">
      <bounds x="518" y="0" width="884" height="1080" />
    </backdrop>
    <bezel element="bezel_alt1">
      <bounds x="0" y="0" width="1920" height="1080" />
    </bezel>
  </view>
  <view name="Unused (alt2)">
    <screen index="0">
      <bounds x="555" y="0" width="810" height="1080" />
    </screen>
    <overlay element="screen_mask">
      <bounds x="554" y="0" width="812" height="1080" />
    </overlay>
    <backdrop element="screen_bezel">
      <bounds x="518" y="0" width="884" height="1080" />
    </backdrop>
    <bezel element="bezel_alt2">
      <bounds x="0" y="0" width="1920" height="1080" />
    </bezel>
  </view>
</mamelayout>

The element tags declare the various images that can be applied on the final output and give them a name.

Next are the various view, because a MAME artwork can have multiple configurations: for example, you might want to provide various cabinet artwork variations. OrionsAngel often provides multiple views; often it's just a color somewhere that changes, but sometimes it's much more than that.

Inside a view, you use the elements you declared.
Because an arcade cabinet can have multiple screens (like Darius or Punch Out), you have to declare the position of each individual screen (thus the index attribute of the screen node).
Over the screen (or rather, I assume it's over) comes the overlay, using one of the element you previously declared. You next do the same for the backdrop (under the screen) and the bezel (I assume it's sandwiched between the screen and the overlay but I'm not sure).

I'm not sure how bezels and backdrops work with multiple screen; once, I talked about it with OrionsAngel, and he said that MAME didn't handle it well, so he didn't do any multi-screen arcade machine.

@markwkidd

This comment has been minimized.

Copy link
Contributor Author

commented Oct 17, 2018

You are definitely an asset to this thread @cosmo0 !

The first draft of this proposal and bounty were not to implement the MAME format, but because that's the direction it has gone I will update the OP to reflect the need for four layers if we are indeed to have compatibility with that format.

@markwkidd markwkidd changed the title [$150 Bounty] Robust support for backdrop, bezel, and overlay "integral artwork" [$150 Bounty] Robust support for backdrop, screen, bezel, and overlay "integral artwork" Oct 17, 2018

@markwkidd

This comment has been minimized.

Copy link
Contributor Author

commented Oct 17, 2018

This recent discussion has led me to read the MAME artwork documentation, now that we know there is interest in supporting that format specifically. I have updated the OP with some information from their docs.

I'll note that there are two layers, the cpanel (control panel) and marquee that are part of MAME's artwork spec which we have not discussed. I'm not personally as interested in those kinds of artwork so I'm fine with leaving them out of this feature request.

However if someone wants to fully implement the artwork format, details are in the OP's link to the MAME documents.

@cosmo0

This comment has been minimized.

Copy link

commented Oct 17, 2018

You are definitely an asset to this thread @cosmo0 !

Thanks but I'm not sure I deserve the praise 😄

The first draft of this proposal and bounty were not to implement the MAME format, but because that's the direction it has gone I will update the OP to reflect the need for four layers if we are indeed to have compatibility with that format.

Well, I don't want to steer the direction of the project, but in the forums where you posted, I thought you wanted Retroarch to be compatible with MAME overlays? Did I not understand correctly?

That would help many people I think, to be able to just use existing MAME overlays without any conversion hassle.

You might want to check with the people who have contributed to the bounty? Maybe that's not what they signed/paid for?

Thank you so much for finding the layout documentation! 😍 I'll check it out when I get some time.

@markwkidd

This comment has been minimized.

Copy link
Contributor Author

commented Oct 17, 2018

Thanks again! You are definitely an asset @cosmo0 ! :)

Thank you also for your concern about the bounty staying on target: don't worry, all of the contributions to this bounty have arrived after it was changed to specifically indicate compatibility with the MAME layout format. Your comments and these excerpts from the MAME docs are only helping be more specific about the plan everyone agreed to.

@UDb23

This comment has been minimized.

Copy link

commented Oct 18, 2018

Actual Mame artwork supports also lamps or LEDs.
These were used in some arcade classics (e.g. skydiver and Afterburner).

Games that support lamps or LEDs require additional programming in the driver for that game.
They were implemented only after mame .107; so lr-mame 2003 would not benefit from this feature.
I don't think this is necessary for this bounty.

One thing that may be worth underlining is that coding for artwork should consider also lower processing power devices like the Pi to make sure it does not impact actual game performance.
If you got nice artwork and the game is unplayable it makes no sense ;-)

@UDb23

This comment has been minimized.

Copy link

commented Oct 18, 2018

@cosmo0 "ok, all overlays I found simulate the backdrop with a semi-transparent overlay instead, but I understand how it's important."

Having an actual backdrop (as the real arcade games had) below the actual game area ("screen") is totally different from trying to emulate this thru semi transparent areas of the overlay.
Mr do's mame artwork site provides various artwork that include backdrops.

Playing classics like Space Invaders, Omega Race, Battle Zone and Asteroids de Luxe with actual backdrops makes a huge difference :-))

@cosmo0

This comment has been minimized.

Copy link

commented Oct 18, 2018

Mr do's mame artwork site provides various artwork that include backdrops.

I tried to download artwork for Warriors on MrDo's website, but it gave me an error 403 each time...

I assume that when a backdrop is used, MAME only draws where the sprites are, it doesn't "draw" the black area? Like, in Space Invaders, it draws only the ships, barricade, missiles etc, and leaves the empty space alone, like a TV screen would do?

Is Libretro doing the same thing? It would be pretty hard to re-code the video emulation layer just to display a backdrop...

@UDb23

This comment has been minimized.

Copy link

commented Oct 19, 2018

@cosmo0 About the 403 error, just edit the download url: it should be http:// not https://
Yes Mame draws only the sprites afaik. Not sure if libretro cores behave differently. Lr-mame2010 does support new mame artwork (.lay) but, for example, Space Invaders becomes too slow to play with artwork activated on a Pi3 b+.

@nayslayer

This comment has been minimized.

Copy link
Contributor

commented Nov 6, 2018

@markwkidd Helped you to hype up the issue some more ;) Maybe I'll have a go at it myself, though it won't be till the start of December.

@markwkidd

This comment has been minimized.

Copy link
Contributor Author

commented Nov 6, 2018

Love it! Thanks @nayslayer !

@markwkidd markwkidd changed the title [$150 Bounty] Robust support for backdrop, screen, bezel, and overlay "integral artwork" [$400 Bounty] Robust support for backdrop, screen, bezel, and overlay "integral artwork" Nov 7, 2018

@markwkidd

This comment has been minimized.

Copy link
Contributor Author

commented Nov 7, 2018

Bounty more than doubled to $400 thanks to a donation from @nayslayer, thank you!

@Wilstorm

This comment has been minimized.

Copy link

commented Nov 7, 2018

@nayslayer - That's incredibly kind and generous of you! :)

@markwkidd markwkidd changed the title [$400 Bounty] Robust support for backdrop, screen, bezel, and overlay "integral artwork" [$400 Bounty] Support for MAME-style backdrop, screen, bezel, and overlay "artwork" Jan 6, 2019

@markwkidd markwkidd changed the title [$400 Bounty] Support for MAME-style backdrop, screen, bezel, and overlay "artwork" [$405 Bounty] Support for MAME-style backdrop, screen, bezel, and overlay "artwork" Jan 6, 2019

@markwkidd

This comment has been minimized.

Copy link
Contributor Author

commented Jan 6, 2019

bounty up to $405. thanks @188pilas!

@huwpascoe

This comment has been minimized.

Copy link
Contributor

commented Mar 15, 2019

@cosmo0 Each container (element, group, view), has it's own local coordinates for child elements, normalized from the union of all the children. Then you can multiply that to fit whatever container you want.

I made a picture that hopefully explains it better!
image

@huwpascoe

This comment has been minimized.

Copy link
Contributor

commented Mar 20, 2019

Status: Parameter expansion done. Still waiting on the rxml PR.

@huwpascoe

This comment has been minimized.

Copy link
Contributor

commented Mar 22, 2019

Repeating elements done.
(Still waiting on rxml PR to be merged)
image

@twinaphex

This comment has been minimized.

Copy link
Member

commented Mar 23, 2019

OK, the RXML PR has been merged.

@markwkidd

This comment has been minimized.

Copy link
Contributor Author

commented Mar 24, 2019

hi @yesfish ! I wanted to make sure you noticed that there are a few C89-compliance buildfixes that went in earlier today to the RXML code: cec06a0#diff-81b71231d907f8341dcffc8b7082d3c7

@huwpascoe

This comment has been minimized.

Copy link
Contributor

commented Mar 25, 2019

Hooray

Now we can only blame my glaring C violations for overflows! PR coming soon

@huwpascoe

This comment has been minimized.

Copy link
Contributor

commented Mar 28, 2019

About the settings for this, should they share the overlay submenu or get their own layout submenu?

@markwkidd

This comment has been minimized.

Copy link
Contributor Author

commented Mar 28, 2019

I would suggest that the settings go in Settings -> Onscreen Display.

In that location the new artwork settings (perhaps called "Onscreen Artwork") would be alongside Onscreen Overlay and Onscreen Notifications, but not within the Onscreen Overlays.

@huwpascoe

This comment has been minimized.

Copy link
Contributor

commented Mar 29, 2019

Progress report:

The menu system is terrible and I hate it very much. This will take longer than I thought.

@huwpascoe

This comment has been minimized.

Copy link
Contributor

commented Mar 29, 2019

How to get retroarch file browser to filter custom file extensions.

Or how to filebrowser.filter = "*.etc";


Step 1: Have your menu item set up as in this guide https://docs.libretro.com/development/retroarch/new-menu-options/ and copy paste from other file path properties to get the right setting.

Step 2: Add a new entry in menu_displaylist.h DISPLAYLIST_ETC,

Step 3: Copy paste one of the case DISPLAYLIST_...: that has the line info->exts = strdup("cfg"); to match your label in menu_displaylist.h. That cfg is what specifies the file extension and are separated by the | pipe. e.g. "png|jpeg|jpg". In this case though, just set it to "etc".

Step 4: in menu_cbs_deferred_push.c add a new entry in the generic_deferred_push list(deferred_push_etc, DISPLAYLIST_ETC)

Step 5: scroll down to the switch statement and copy paste one of the items to reference case DISPLAYLIST_ETC: BIND_ACTION_DEFERRED_PUSH(cbs, deferred_push_etc);

Step 6: Start the program with a debugger. Navigate to your menu item.

Step 7: set a breakpoint in menu_cbs_deferred_push.c at function int menu_cbs_init_bind_deferred_push()

Step 8: Select your menu item to trigger the breakpoint.

Step 9: Copy the 8 character hex value in label_hash for the next step and stop debugging.

Step 10: Open msg_hash.h and scroll to the bottom to the #define MENU_LABEL_ 0x.... section to create the new #define with the value you coped while debugging.

If it's worked, you will now only see .etc files when you choose your menu item.

Edit: forgot a step

@markwkidd

This comment has been minimized.

Copy link
Contributor Author

commented Mar 29, 2019

@huwpascoe

This comment has been minimized.

Copy link
Contributor

commented Mar 30, 2019

@markwkidd Once all this is done, maybe. At least it's written down somewhere for now.

@huwpascoe

This comment has been minimized.

Copy link
Contributor

commented Mar 31, 2019

Config and menu seem to work.

Now for the main task. There are two issues that need addressing:

Mame layouts require control over where screens are positioned. Retroarch's video drivers are inconsistent with their implementation of set_viewport(). Some video drivers don't implement it at all, and others frequently change and reset the viewport multiple times during a frame, sometimes even internally reloading from the global config. As it stands, it's not a reliable interface.

One of the mame layout modes relies on the screen being drawn over the background with additive blending. The space invaders example (that I've been rendering with SDL up to now), will not display correctly without this feature.

The current plan is to implement everything needed in opengl, then contributors can port it to other drivers if they like.

@huwpascoe

This comment has been minimized.

Copy link
Contributor

commented Mar 31, 2019

Second issue solved, I've been stupid for not noticing this: The background layer is drawn over black. Don't need to change the viewport rendering at all, can just draw the background over with additive blend.

@hizzlekizzle

This comment has been minimized.

Copy link
Member

commented Mar 31, 2019

The current plan is to implement everything needed in opengl, then contributors can port it to other drivers if they like.

Good plan.

@huwpascoe

This comment has been minimized.

Copy link
Contributor

commented Apr 2, 2019

Hacky attempt so not in PR yet, but yay I can get it to show up mostly fine. Actual emulation depicted.
image

@UDb23

This comment has been minimized.

Copy link

commented Apr 2, 2019

Absolutely gorgeous !! Well done.

@UDb23

This comment has been minimized.

Copy link

commented Apr 2, 2019

How's game performance ? Do you notice any slowdown?

@huwpascoe

This comment has been minimized.

Copy link
Contributor

commented Apr 3, 2019

Doesn't seem to slow down. Here's my somewhat working PR. Didn't manage to get .zip loading done but you can extract the contents of a layout.zip somewhere and load default.lay directly.

The gl driver is a terrible mess right now, I literally crammed opengl calls wherever until I can figure out what that render chain stuff does. So, maybe read up on restarting your display server if you don't trust your graphics driver!

Going away for a couple weeks. I'll do some updates from remote desktop if I get the time.

@huwpascoe

This comment has been minimized.

Copy link
Contributor

commented Apr 17, 2019

Finished. #8529

Notes:
some zip will not load correctly. This is due to a bug in retroarch's file_archive_compressed_read() where it fails to dynamically load files which have used the STORE method. If your layout doesn't load, try extracting it to a sub-directory and load from the default.lay file instead.

I've only implemented the essentials with the gl renderer. It can draw images, rectangles, fit the retoarch frame ("screen" in layout terms) and blend the layers correctly with ALPHA, ADD, MOD.

The layout I/O mechanism is available for future changes if someone wants to add emulation feedback or interface with input overlays.

image

@huwpascoe

This comment has been minimized.

Copy link
Contributor

commented Apr 21, 2019

@markwkidd I believe the bounty is now satisfied, please could you close this issue so I may collect?
I'll continue to rebase the PR until it's merged and provide ongoing layout-related support in my spare time. Thank you.

@grant2258

This comment has been minimized.

Copy link
Contributor

commented Apr 27, 2019

I have no idea why your getting no response on this. I have tested seems to be working fine for my tests. I dont know who takes care of the bounty's here or the review process. Job well done my good man or woman :)

@markwkidd

This comment has been minimized.

Copy link
Contributor Author

commented Apr 27, 2019

@huwpascoe

This comment has been minimized.

Copy link
Contributor

commented May 13, 2019

Merged! #8529

@markwkidd markwkidd closed this May 13, 2019

@markwkidd

This comment has been minimized.

Copy link
Contributor Author

commented May 13, 2019

This feature has been merged, and for now it works for people with GL video driver support. I'm closing the issue so that the bounty process can move forward.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.