Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Amiberry - preparation for next release #2870

Open
midwan opened this issue Oct 13, 2019 · 37 comments

Comments

@midwan
Copy link

@midwan midwan commented Oct 13, 2019

Hi guys! The Amiberry BETA testing is going really well, and I'm planning a code freeze at this point. We should be able to get a new stable release really soon now - haven't set a fixed date yet, but I'll update you once that is done also. For now, it's a good opportunity to sync with the various package / distro managers regarding the upcoming changes, to ensure a smooth upgrade.

The version history (https://github.com/midwan/amiberry/wiki/Version-history) is fully updated with any new features and changes, of course. But here are some details that are more specific regarding building it from source, which might have an impact in the distro:

  1. The Makefile has some targets updated, mostly for the RPI platforms. It will also now abort with an error if an invalid target is specified, with the PLATFORM= option.
  2. All Amiberry versions use SDL2 now, the old SDL1 version is gone. The RPI1-RPI2-RPI3 platforms can even use SDL2 with the rpi back-end (e.g. the ones provided by the Raspbian repo), but unfortunately the RPI4 cannot. You'll need an SDL2 compiled from source for the RPI4, with the kmsdrm back-end enabled and the rpi back-end disabled (--enable-video-kmsdrm --disable-video-rpi).
  3. The old SDL1+Dispmanx version is replaced with the new SDL2+Dispmanx version (the PLATFORM name is the same). This one uses Dispmanx for all graphical operations, GUI and emulation screen, and uses SDL2 for everything else (events, sound, game controller handling, etc.). You will need the libsdl2-dev libsdl2-ttf-dev libsdl2-image-dev libraries installed.
  4. Some new options have been added in the amiberry.conf file, to control global defaults of the emulator. You can check the full documentation regarding that file here: https://github.com/midwan/amiberry/wiki/Amiberry.conf-options
  5. The whdboot directory has been updated with a new WHDLoad booter (compared to the previous stable), so make sure you update that as well, when you do an "update from source". Not just the binary. Also, I noticed you had a whdboot-dist directory in there, which is not really used (not sure how it ended up in RetroPie).
  6. Regarding amiberry.conf, a default file is now included in the git repo, although this is regenerated when you click on Rescan ROMs from the Paths panel. It's mostly for people who haven't clicked on that and are wondering where the file is.
  7. The compiled binary is now always named amiberry, regardless of what PLATFORM option was specified. It's up to the package / distro maintainers to rename it if necessary.
  8. Since RetroPie uses 2 pre-configured .uae files, I'd recommend you regenerate those by using the default options from the Quickstart Panel, for ensured compatibility. Some options in the config files have changed over time, to better match WinUAE - you may have old options in there that could cause compatibility issues. The easiest way to do this, would be to start the emulator, set the Quickstart config you want (e.g. A1200) and any other options, then save that as the config name you want. The default config for the emulator is an A500, so perhaps you don't even need a config file for that, unless you're making changes to the default options of course.

I think that is all for now, but I've forgotten anything I'll come back with more.

It would be great if you would start testing the integration of the BETA (currently sitting in the dev branch) with the distro, to catch any issues early. Ping me if you run into problems or have any questions, please!

@cmitu

This comment has been minimized.

Copy link
Contributor

@cmitu cmitu commented Oct 16, 2019

I've been testing some of the changes required in the amiberry module and it seems it's not going to be so complicated, the modifications are not so significant. To the point (where any clarification is needed):

  1. The SDL2 version that will support the PI4 has KMSDRM support. The other PI platforms already include a SDL2 with Dispmanx support.

4 + 6 The RetroPie install doesn't ship an amiberry.conf, but a minimal conf can be added.

  1. The whdboot folder is actually copied from the source build to the binary whdboot-dist and then added to the configuration folder as whdboot.

  2. This will simplify the RetroPie's scriptmodule, the resulting binary was symlink-ed to an amiberry binary anyway.

  3. Can't say if we need to change those uae files, but they're pretty minimal - you can see here and here.

I tested a modified scriptmodule for RetroPie on a Pi4 and it seems ok, I'll have to test on a Pi3 also to see if everything is ok.

EDIT: I could have sworn I've seen somewhere a word about including capsimg.so with amiberry, should that be distributed in the same folder as the amuberry binary ?

@midwan

This comment has been minimized.

Copy link
Author

@midwan midwan commented Oct 16, 2019

@cmitu
Regarding the amiberry.conf, just a clarification. You don't need to ship one, as this file is generated by Amiberry itself when it scans for Kickstart ROMs anyway. The default file is there just for people reading through the wiki and wondering where the file is, if they haven't actually used the emulator yet (specifically, haven't clicked on "Rescan ROMs"). In other words, it's OK to skip it also.

  1. Why keep the whdboot-dist directory after that step however? It's not used anywhere after that step, right?
@joolswills

This comment has been minimized.

Copy link
Member

@joolswills joolswills commented Oct 16, 2019

so that we don't overwrite hostprefs.conf - eg if it has been customised. If it was named whdboot everything would be overwritten on update.

@midwan

This comment has been minimized.

Copy link
Author

@midwan midwan commented Oct 16, 2019

@joolswills I understand, but after copying the rest of the contents of the whdboot directory in the right location, why keep the no longer necessary whdboot-dist directory as well?

@cmitu the optional capsimg.so plugin is still valid, and will be loaded by Amiberry at runtime if found in the same directory the binary is at. If found/loaded, Amiberry can use .ipf images as well.

Normally I include a pre-compiled one with each release, but I also have instructions on how you can compile it from source if necessary: https://github.com/midwan/amiberry/wiki/Enabling-IPF-support

The capsimg.so file does not change from one version to another, so it can be re-used.

@joolswills

This comment has been minimized.

Copy link
Member

@joolswills joolswills commented Oct 16, 2019

Can't remember. I may have had a reason. It's only a few mb, but can remove it in the configure stage I guess.

@midwan

This comment has been minimized.

Copy link
Author

@midwan midwan commented Oct 26, 2019

How are you guys doing regarding this? Is everything set, do you need assistance with something, or do you need more time?
Just checking so I can plan for the release accordingly, I was hoping of going for it some time next week.

@joolswills

This comment has been minimized.

Copy link
Member

@joolswills joolswills commented Oct 26, 2019

I will prepare a PR next week. It's minor changes only I think.

@joolswills

This comment has been minimized.

Copy link
Member

@joolswills joolswills commented Oct 26, 2019

You may prefer me to open an issue on your github issue tracker for this - let me know. Problem with the libxml2 external commit ref.

git clone --recursive --depth 1 --branch dev "https://github.com/midwan/amiberry/" "/home/jools/Repos/RetroPie/RetroPie-Setup/tmp/build/amiberry"
Cloning into '/home/jools/Repos/RetroPie/RetroPie-Setup/tmp/build/amiberry'...
remote: Enumerating objects: 694, done.
remote: Counting objects: 100% (694/694), done.
remote: Compressing objects: 100% (529/529), done.
remote: Total 694 (delta 185), reused 338 (delta 142), pack-reused 0
Receiving objects: 100% (694/694), 5.00 MiB | 7.35 MiB/s, done.
Resolving deltas: 100% (185/185), done.
Submodule 'external/capsimg' (https://github.com/FrodeSolheim/capsimg) registered for path 'external/capsimg'
Submodule 'external/libmpeg2' (https://android.googlesource.com/platform/external/libmpeg2) registered for path 'external/libmpeg2'
Submodule 'external/libxml2' (https://android.googlesource.com/platform/external/libxml2) registered for path 'external/libxml2'
Cloning into '/home/jools/Repos/RetroPie/RetroPie-Setup/tmp/build/amiberry/external/capsimg'...
remote: Enumerating objects: 4, done.        
remote: Counting objects: 100% (4/4), done.        
remote: Compressing objects: 100% (4/4), done.        
remote: Total 139 (delta 0), reused 2 (delta 0), pack-reused 135        
Receiving objects: 100% (139/139), 180.98 KiB | 1.19 MiB/s, done.
Resolving deltas: 100% (21/21), done.
Cloning into '/home/jools/Repos/RetroPie/RetroPie-Setup/tmp/build/amiberry/external/libmpeg2'...
remote: Counting objects: 576, done        
remote: Total 2921 (delta 834), reused 2921 (delta 834)        
Receiving objects: 100% (2921/2921), 1.16 MiB | 4.74 MiB/s, done.
Resolving deltas: 100% (834/834), done.
Cloning into '/home/jools/Repos/RetroPie/RetroPie-Setup/tmp/build/amiberry/external/libxml2'...
remote: Sending approximately 21.74 MiB ...        
remote: Counting objects: 781, done        
remote: Total 45093 (delta 35893), reused 45093 (delta 35893)        
Receiving objects: 100% (45093/45093), 21.74 MiB | 22.90 MiB/s, done.
Resolving deltas: 100% (35893/35893), done.
Submodule path 'external/capsimg': checked out '1713452be0c5adf94fcb22bcb083adc5dac45876'
Submodule path 'external/libmpeg2': checked out '6891249c9ca2237b08665380b169d78f96878a51'
fatal: remote error: want d87078a53642773db4ad38019370c475d570451f not valid
fatal: the remote end hung up unexpectedly
Fetched in submodule path 'external/libxml2', but it did not contain d87078a53642773db4ad38019370c475d570451f. Direct fetching of that commit failed.
HEAD is now in branch 'dev' at commit '1940278d75c93b4c33b88ad17c72173245706987'
@joolswills

This comment has been minimized.

Copy link
Member

@joolswills joolswills commented Oct 26, 2019

This may not be your bug btw. Maybe some issue with compatibility with the other repo. I'll look into it.

@joolswills

This comment has been minimized.

Copy link
Member

@joolswills joolswills commented Oct 26, 2019

Does look like you reference a non existent commit for that submodule.

@joolswills

This comment has been minimized.

Copy link
Member

@joolswills joolswills commented Oct 27, 2019

Got a segfault with amiberry (launching with no parameters and no default config also) for sdl2+dispmanx on rpi2. I will need to debug and open a ticket. Could be something I did wrong of course.

@midwan

This comment has been minimized.

Copy link
Author

@midwan midwan commented Oct 27, 2019

@joolswills The libxml2 issue is my bad actually, and I know about it - I was going to fix it today.
But it's only needed for the Android port I'm working on, you shouldn't need it for the other platforms which have libxml2 in the system. Still, I'll get that fixed today.

I'm curious about the segfault however. If you could share more details on that, we can look into it. What steps did you follow?

@midwan

This comment has been minimized.

Copy link
Author

@midwan midwan commented Oct 27, 2019

@joolswills
The libxml2 submodule issue is now fixed. :)

@midwan

This comment has been minimized.

Copy link
Author

@midwan midwan commented Oct 27, 2019

@joolswills
And I think I found the reason for the segfault: My recent change to allow up to 1GB of RAM.
I'll need to fix this implementation because it tries to pre-allocate the maximum amount it can use during startup, which of course would fail on platforms that only have 1GB of RAM in total. Works fine on RPI4 with 4GB, but not on RPI3/2 with 1GB (and I suspect RPI4 models with 1GB).

@joolswills

This comment has been minimized.

Copy link
Member

@joolswills joolswills commented Oct 27, 2019

Ah. That would make sense. I saw some failed allocation before the segfault.

@midwan

This comment has been minimized.

Copy link
Author

@midwan midwan commented Oct 27, 2019

@joolswills I should have a fix deployed for this in a bit also, testing it now :)

@midwan

This comment has been minimized.

Copy link
Author

@midwan midwan commented Oct 27, 2019

@joolswills
Fixed! Please pull the latest and give it another try, let me know if you run into any more problems.

@joolswills

This comment has been minimized.

Copy link
Member

@joolswills joolswills commented Oct 30, 2019

I'm still getting the libxml2 error I'm afraid. Your repo still references the d87078a commit which doesn't exist.

@joolswills

This comment has been minimized.

Copy link
Member

@joolswills joolswills commented Oct 30, 2019

Segfault is sorted thanks.

Having problems with my joypads with the rpi2 dispmanx build I just made. Dpad doesn't seem to work in GUI or in game. Could be due to config changes so I will check further. But the buttons are ok. Just not movement.

@midwan

This comment has been minimized.

Copy link
Author

@midwan midwan commented Oct 30, 2019

@joolswills the libxml2 submodule issue is fixed (I tested a fresh clone on a different directory, I didn't get the error), so maybe you could try cloning it again?

I haven't seen a problem with joypad control so far, but I can only suspect it has to do with the controller mapping. I've been testing in on a few different joypads here, including a Freeplay CM3 setup I have, it seems to work as expected in all of them. I only had to modify the mapping for the CM3 (as mentioned in our wiki here: https://github.com/midwan/amiberry/wiki/Setting-up-Input-Controllers) (look for "If you find that your joystick / D-Pad does not work but the buttons seem to be working OK")

@joolswills

This comment has been minimized.

Copy link
Member

@joolswills joolswills commented Oct 30, 2019

git clone --recursive --depth 1 --branch dev "https://github.com/midwan/amiberry"
Cloning into 'amiberry'...
remote: Enumerating objects: 694, done.
remote: Counting objects: 100% (694/694), done.
remote: Compressing objects: 100% (529/529), done.
remote: Total 694 (delta 185), reused 337 (delta 142), pack-reused 0
Receiving objects: 100% (694/694), 5.01 MiB | 6.69 MiB/s, done.
Resolving deltas: 100% (185/185), done.
Submodule 'external/capsimg' (https://github.com/FrodeSolheim/capsimg) registered for path 'external/capsimg'
Submodule 'external/libmpeg2' (https://android.googlesource.com/platform/external/libmpeg2) registered for path 'external/libmpeg2'
Submodule 'external/libxml2' (https://github.com/midwan/libxml2) registered for path 'external/libxml2'
Cloning into '/home/jools/temp/amiberry/external/capsimg'...
remote: Enumerating objects: 4, done.        
remote: Counting objects: 100% (4/4), done.        
remote: Compressing objects: 100% (4/4), done.        
remote: Total 139 (delta 0), reused 2 (delta 0), pack-reused 135        
Receiving objects: 100% (139/139), 180.98 KiB | 890.00 KiB/s, done.
Resolving deltas: 100% (21/21), done.
Cloning into '/home/jools/temp/amiberry/external/libmpeg2'...
remote: Counting objects: 577, done        
remote: Total 2922 (delta 834), reused 2922 (delta 834)        
Receiving objects: 100% (2922/2922), 1.16 MiB | 9.19 MiB/s, done.
Resolving deltas: 100% (834/834), done.
Cloning into '/home/jools/temp/amiberry/external/libxml2'...
remote: Enumerating objects: 3402, done.        
remote: Counting objects: 100% (3402/3402), done.        
remote: Compressing objects: 100% (2078/2078), done.        
remote: Total 3402 (delta 766), reused 3397 (delta 765), pack-reused 0        
Receiving objects: 100% (3402/3402), 4.72 MiB | 6.97 MiB/s, done.
Resolving deltas: 100% (766/766), done.
Submodule path 'external/capsimg': checked out '1713452be0c5adf94fcb22bcb083adc5dac45876'
Submodule path 'external/libmpeg2': checked out '6891249c9ca2237b08665380b169d78f96878a51'
error: Server does not allow request for unadvertised object d87078a53642773db4ad38019370c475d570451f
Fetched in submodule path 'external/libxml2', but it did not contain d87078a53642773db4ad38019370c475d570451f. Direct fetching of that commit failed.

(You still have d87078a53642773db4ad38019370c referenced in the dev branch)

That looks like the same controller issue thanks. I'll try it out. However this worked with the sdl1 build - just not with sdl2. If I remove the retroarch controller config I can navigate the GUI with the dpad btw. Would be good to not have to mess about if using retroarch controller configs though as it worked before.

@midwan

This comment has been minimized.

Copy link
Author

@midwan midwan commented Oct 31, 2019

@joolswills
You're right about the libxml2 submodule, I missed it somehow since it was working in my local repo.
I'll get that fixed and get back to you.

@midwan

This comment has been minimized.

Copy link
Author

@midwan midwan commented Oct 31, 2019

@joolswills
I've re-added the libxml2 directly this time, instead of a submodule - this should fix the issue for good. They were not needed for most platforms anyway, it was only for Android.

Regarding the controller issues: SDL1 vs SDL2 have differences in what they detect from controllers, and how they are detected as well. I'll see if I can do something from our side regarding this, I agree that it would be good not having to mess around with config files.

@joolswills

This comment has been minimized.

Copy link
Member

@joolswills joolswills commented Nov 1, 2019

Thanks. WIP PR here - #2886

Your capsimg wiki docs are out of date - path change and .fs suffix. I didn't change though as this is using dev branch etc. You may already be aware of this. Thanks.

@midwan

This comment has been minimized.

Copy link
Author

@midwan midwan commented Nov 1, 2019

Thanks @joolswills
I am aware of the wiki difference, but I haven't updated it until dev gets merged into master, to avoid confusing users. It will be updated when the new version gets released. ;)

@joolswills

This comment has been minimized.

Copy link
Member

@joolswills joolswills commented Nov 7, 2019

The controller issue isn't related to SDL2. I noticed it earlier building from master with SDL1.

I bisected it to

99e13859edd307a4778d7a3a28bb4ff998f3f80c is the first bad commit
commit 99e13859edd307a4778d7a3a28bb4ff998f3f80c
Author: Horace And The Spider <horaceandthespider@hotmail.com>
Date:   Wed May 15 07:50:42 2019 +0100

    AXIS / DPAD Controls bugfix (#480)
    
    * Bugfix for 2nd controller selection
    
    * ignore netbeans project
    
    * Add experimental `-autocd=` loading of files (.cue works very well - .iso should also)
    
    * CD Autoloading adapted to include .uae file check and hostconf controller options
    
    * Beginning of Booter Panel implentation
    
    * Booter Panel development .. start on XML reading for picked LHA file
    
    * New WHDLoad booter, included updated boot-data.zip, plus new hostprefs FIXED_HEIGHT= option and bugfixes for XML reading, and symlink ROM scan. Plus updated XML
    
    * Upload of .RTB files that need to accompany the Symlinked Kickstarts, for WHDLoad compatibility.
    
    * Allows for additional libraries from `boot-data.zip` to be linked on load, to aid compatibility (e.g. Dungeon Master), plus new XML
    
    * Fix hardware settings tab issues from XML reading, plus additional write_log reports for debugging WHD booter.
    
    * Code cleanup (non-essential)
    
    * XML Download button added to paths panel including date checking
    
    * WHDLoad local copy
    
    * Updated button to download WHDLoad and .RTB files (if not present) with XML file.
    
    * Creates a 'fall back' for the WHDLoad file if not present - should fix issues with RetroPie install not correctly providing this file.
    
    * Fixes bug in D-pad / AXIS handling

:040000 040000 dda4a9fadbb9a830c69eba8abd9ea578e2f1475a 11760256e0f18f98ad6954f29439c303fef9c0a5 M	src

Happy to open an issue on your bugtracker but putting it here first as I'm short on time.

Before this change the retroarch controller mapping for my buffalo and another generic gamepad work fine. After this commit only the button mapping is ok - the dpads do not work.

@joolswills

This comment has been minimized.

Copy link
Member

@joolswills joolswills commented Nov 7, 2019

I had a brief look at that commit - looks a bit rough / or incomplete as the indentation is all over the place (and commented out code etc).

@midwan

This comment has been minimized.

Copy link
Author

@midwan midwan commented Nov 7, 2019

@joolswills
Thanks for finding this, I'll take a closer look at it!

@midwan

This comment has been minimized.

Copy link
Author

@midwan midwan commented Nov 7, 2019

I've added midwan/amiberry#528 to track this

@midwan

This comment has been minimized.

Copy link
Author

@midwan midwan commented Nov 7, 2019

@joolswills could you send me a config file that I can test with, please?
I'd like to have a few different samples to check the code against, ideally.

@joolswills

This comment has been minimized.

Copy link
Member

@joolswills joolswills commented Nov 7, 2019

I have posted the controller configs in the amiberry ticket.

@joolswills

This comment has been minimized.

Copy link
Member

@joolswills joolswills commented Nov 7, 2019

(in the meantime, I may include a patch to revert this change in RetroPie)

@midwan

This comment has been minimized.

Copy link
Author

@midwan midwan commented Nov 8, 2019

@joolswills
This should be fixed with today's commit, I believe (in both dev and master now) :)

@joolswills

This comment has been minimized.

Copy link
Member

@joolswills joolswills commented Nov 16, 2019

I see the new version is out. Congrats. I will adjust my PR and rebuild binaries/merge this afternoon.

@midwan

This comment has been minimized.

Copy link
Author

@midwan midwan commented Nov 16, 2019

@joolswills
Indeed, v3.0 was released today! :)
I've been updating the wiki articles accordingly as well, didn't get a chance to post here before you did :)

@cesarpuig

This comment has been minimized.

Copy link

@cesarpuig cesarpuig commented Nov 19, 2019

Hi.

I have installed the latest version of amiberry (3.0 and 3.01), and since then when I try to update the file whdload_db.xlm I get an error in which it tells me that there is no internet connection, it did not happen with previous versions (2.25).

It also happens to me that after executing amiberry, if I enter Reteopie setup from emulationstation or on the console by pressing F4, the keyboard does not work, but this also happens to me after executing some port.

Thank you.

@midwan

This comment has been minimized.

Copy link
Author

@midwan midwan commented Nov 19, 2019

@cesarpuig
Are you using an ethernet or wifi connection? I've seen this happen in cases where both interfaces were connected, try experimenting with one or the other and see if that helps?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.