[Bounty] Increase PSX Memory

[barisyild]

Increase PSX Memory feature can help remove some limitations with a patch in heap memory supported games.

Additional memory must be located in dma.

Development Consoles has 8mb dma but retail consoles has 2mb dma

There are even people who have modified their console on the internet to use 8mb dma. (useless for homebrew development or heap memory increased games)

Resources:

SCPH-5502 8MB Modification: http://www.psxdev.net/forum/viewtopic.php?t=660
https://retrocomputing.stackexchange.com/questions/13551/how-much-memory-did-the-playstation-development-kit-have

Note:

Preferably, it would be nice if the amount of RAM could be adjusted.

Bounty Link: https://www.bountysource.com/issues/90799133-feature-request-increase-psx-memory

[simias]

Where should the additional memory be located?

[barisyild]

Additional memory must be located in dma.

Development Consoles has 8mb dma but retail consoles has 2mb dma

There are even people who have modified their console on the internet to use 8mb dma. (useless for homebrew development or heap memory increased games)

Resources:

SCPH-5502 8MB Modification: http://www.psxdev.net/forum/viewtopic.php?t=660
https://retrocomputing.stackexchange.com/questions/13551/how-much-memory-did-the-playstation-development-kit-have

[barisyild]

By modifying the games, the ram limit of the compatible games can be increased. (Will patch remove some limitations)

Example of Retail Crash Team Racing (SCES-02105)

[simias]

Ah, so instead of having 2MB mirrored 4 times we'd have 8MB proper. That's straightforward to implement but it'll break savestates hard.

[simias]

Out of curiosity what does CTR do differently when it has 8MB of RAM?

[barisyild]

For now, I think of just removing 3 limitations.

• The Oxide character cannot be used in adventure mode or some levels due to the ram restriction.
• 2P and 4P modes are very limited, these restrictions can be removed. (We will patch low quality characters and low quality levels)
• It may be possible to use the boss characters in the OnlineCTR project.

[simias]

I did a quick hack to enable 8MB in beetle: https://github.com/simias/beetle-psx-libretro/tree/dev/ram-8m

It's completely hardcoded and almost certainly won't work with the dynarec but it could be a starting point.

I don't know if it works but it doesn't seem not to work (i.e. it doesn't crash when I start a game).

[barisyild]

I couldn't try it because there was no compiler on my computer, according to my observations ram did not increase.

[rapierx]

I did a quick hack to enable 8MB in beetle: https://github.com/simias/beetle-psx-libretro/tree/dev/ram-8m

It's completely hardcoded and almost certainly won't work with the dynarec but it could be a starting point.

I don't know if it works but it doesn't seem not to work (i.e. it doesn't crash when I start a game).

Doesn't work, RAM is still mirrored 4 times. Look at Cheat Engine results:

Dynarec ON: https://i.imgur.com/ffUmm0v.png
Dynarec off (Beetle interpreter): https://i.imgur.com/FT0c4GJ.png

[barisyild]

Doesn't work, RAM is still mirrored 4 times. Look at Cheat Engine results:

I couldn't compile it. which tool do you use?

[rapierx]

I couldn't compile it. which tool do you use?

MSYS2 on Windows 10, following https://docs.libretro.com/development/retroarch/compilation/windows/#building-cores

[ZachCook]

@rapierx when compiling with HAVE_LIGHTREC=1 (which adds the dynarec on/off options you described, and so I am assuming you compiled with it) the ram is 2MB mirrored 4 times still. Try compiling without that and see if the change worked with cheat engine, the dynarec code will need some additional changes to support 8MB.

[rapierx]

Without Lightrec, looks like RAM mirroring is disabled. I can't find any practical difference in performance. For example, I can't put more enemies on stages, game still hangs.

https://i.imgur.com/eF4oWAw.png

Usually Beetle uses memory addresses similar to Gameshark addresses on actual console, like 8007D954 or 6007D954. With this commit, memory addresses are random; in the image 8007D954 changes to 18873994.

[simias]

I'm not sure how to interpret these CheatEngine screenshots, I'm not familiar with this tool.

The patch won't work with dynarec enabled, I haven't touched that (well, I did update a few calls in the main code but I'm sure that it's not enough). It's meant to be a prototype to see if it works as expected using the interpreter.

Without Lightrec, looks like RAM mirroring is disabled. I can't find any practical difference in performance. For example, I can't put more enemies on stages, game still hangs.

The games almost certainly needs to be patched to make use of the additional RAM. As such without patched game this change won't do anything besides breaking games that rely on the RAM being mirrored somehow.

I can only provide 64bit Linux builds, no Windows, sorry.

[rapierx]

I'm not sure how to interpret these CheatEngine screenshots, I'm not familiar with this tool.

If I use Cheat Engine with Beetle to find game addresses, I usually find 4 addresses equally spaced apart, like in the images posted in #634 (comment). https://i.imgur.com/ffUmm0v.png shows 60117ac0, 60317ac0, 60517ac0, 60717ac0. With RAM mirroring disabled I find just one address.

[barisyild]

@simias Is there a way to enable Printf logs, I found a ram test and want to test it.

RAMTEST.zip

[barisyild]

Without Lightrec, looks like RAM mirroring is disabled. I can't find any practical difference in performance. For example, I can't put more enemies on stages, game still hangs.

https://i.imgur.com/eF4oWAw.png

Usually Beetle uses memory addresses similar to Gameshark addresses on actual console, like 8007D954 or 6007D954. With this commit, memory addresses are random; in the image 8007D954 changes to 18873994.

Games are usually programmed to use 2MB ram.
If you find the swap area as in the example I have shown, you can increase the ram rate.

[niko1point0]

I am also in favor of the memory limitation being updated to 8mb. Crash Team Racing has a lot of room for potential if it has a little more memory (removed features, modding potential). Every other PS1 game in existence will have a potential ability for modding if there is extra room to store data

[barisyild]

I couldn't compile it. which tool do you use?

MSYS2 on Windows 10, following https://docs.libretro.com/development/retroarch/compilation/windows/#building-cores

I'm tried on 2 computer, giving following error.

[simias]

It probably means that both these ifdefs are false:

beetle-psx-libretro/libretro-common/cdrom/cdrom.c

Lines 481 to 487 in 84044f1

	#if defined(__linux__) && !defined(ANDROID)
	if (cached_read || !cdrom_send_command_linux(stream, dir, xfer_buf_pos, request_len, cmd, cmd_len, sense, sizeof(sense)))
	#else
	#if defined(_WIN32) && !defined(_XBOX)
	if (cached_read || !cdrom_send_command_win32(stream, dir, xfer_buf_pos, request_len, cmd, cmd_len, sense, sizeof(sense)))
	#endif
	#endif

Judging by the flags, it looks like the 2nd one should pass on windows (I assume WIN32 is not strictly 32bits but just a historical name?).

You could stick a #define _WIN32 at the top of the file and see if that helps, but I suppose the real problem is that your build environment doesn't define the proper macros which will probably create other issues.

[barisyild]

It probably means that both these ifdefs are false:

beetle-psx-libretro/libretro-common/cdrom/cdrom.c

Lines 481 to 487 in 84044f1

	#if defined(__linux__) && !defined(ANDROID)
	if (cached_read || !cdrom_send_command_linux(stream, dir, xfer_buf_pos, request_len, cmd, cmd_len, sense, sizeof(sense)))
	#else
	#if defined(_WIN32) && !defined(_XBOX)
	if (cached_read || !cdrom_send_command_win32(stream, dir, xfer_buf_pos, request_len, cmd, cmd_len, sense, sizeof(sense)))
	#endif
	#endif

Judging by the flags, it looks like the 2nd one should pass on windows (I assume WIN32 is not strictly 32bits but just a historical name?).

You could stick a #define _WIN32 at the top of the file and see if that helps, but I suppose the real problem is that your build environment doesn't define the proper macros which will probably create other issues.

Could there be an error in the command I compile because many similar errors occur. (Compiles the .so library without problems)

make platform=win

[simias]

I am also in favor of the memory limitation being updated to 8mb. Crash Team Racing has a lot of room for potential if it has a little more memory (removed features, modding potential). Every other PS1 game in existence will have a potential ability for modding if there is extra room to store data

The problem is that we can't just increase the memory by default because it would reduce the accuracy of the emulator. On real Playstations what they seem to have done is wire the memory controller the same way as on the 8MB prototypes but with the two most significant bits of the address bus "in the air".

In practice that means that the 2MB of RAM are mirrored 4 times on the first 8MB of the address space. So for instance if you write a value at address 0x00000000 you can read it back at 0x00200000, 0x00400000 and 0x00600000. If we replace this mirrors with actual RAM we change that behavior, and that may introduce compatibility issue with weird/bugged games. I actually don't know if such games exist, but I don't know that they don't exist.

So this chance needs to be configurable, which makes it a bit trickier, especially when it comes to savestates and maybe the various dynarecs.

[barisyild]

I am also in favor of the memory limitation being updated to 8mb. Crash Team Racing has a lot of room for potential if it has a little more memory (removed features, modding potential). Every other PS1 game in existence will have a potential ability for modding if there is extra room to store data

The problem is that we can't just increase the memory by default because it would reduce the accuracy of the emulator. On real Playstations what they seem to have done is wire the memory controller the same way as on the 8MB prototypes but with the two most significant bits of the address bus "in the air".

In practice that means that the 2MB of RAM are mirrored 4 times on the first 8MB of the address space. So for instance if you write a value at address 0x00000000 you can read it back at 0x00200000, 0x00400000 and 0x00600000. If we replace this mirrors with actual RAM we change that behavior, and that may introduce compatibility issue with weird/bugged games. I actually don't know if such games exist, but I don't know that they don't exist.

So this chance needs to be configurable, which makes it a bit trickier, especially when it comes to savestates and maybe the various dynarecs.

This feature can be added to the menu as an option.
We have accomplished this with a PCSXR that has now been modified. (Emulator is slow because it was builded in debug mode)

https://www.youtube.com/watch?v=KO3ayiXBB5w

[simias]

Could there be an error in the command I compile because many similar errors occur. (Compiles the .so library without problems)

It's very much possible but I only ever build for linux so I really don't know anything about your build environment. I know that many devs here use Windows however, hopefully they'll be able to help better.

[barisyild]

Could there be an error in the command I compile because many similar errors occur. (Compiles the .so library without problems)

It's very much possible but I only ever build for linux so I really don't know anything about your build environment. I know that many devs here use Windows however, hopefully they'll be able to help better.

Finally I managed to compile!
After the memory is 8mb, a value repeats 4 times in memory and 8mb space cannot be used.

When I increased the PCSX-R memory to 8MB, I was able to use all the memory and in no way the same value was found in memory 4 times.

[simias]

Could you share the test you're running?

[barisyild]

Could you share the test you're running?

pcsxr-master-8mb-source.zip

*Emulator requires plugin you can download and use the appropriate plugins from the internet.

Also you can verify ram with following ps1 executable.
RAMTEST.zip

And you may patch CTR (SCUS_944.26) for 8MB ram usage (ISO Patch Recommended)

DC 01 0C 00 00 00 00 D0 F9 00 0C 20
DC 01 0C 00 00 00 00 D0 F9 00 0C 80

[ZachCook]

@barisyild See my comment above about compiling without HAVE_LIGHTREC=1, as that uses a different memory allocation method that has not been changed to support 8MB

[barisyild]

@barisyild See my comment above about compiling without HAVE_LIGHTREC=1, as that uses a different memory allocation method that has not been changed to support 8MB

Do you want me to compile with HAVE_LIGHTREC = 0?

[ZachCook]

Yes, or just not setting it at all, as 0 is default in makefile I think, You currently need to if you want to test the 8MB changes

[barisyild]

Yes, or just not setting it at all, as 0 is default in makefile I think, You currently need to if you want to test the 8MB changes

What setting should I make from retroarch?

[ZachCook]

You just have to compile beetle-psx with HAVE_LIGHTREC=0, retroarch settings do not change it

[barisyild]

You just have to compile beetle-psx with HAVE_LIGHTREC=0, retroarch settings do not change it

Does compiling with HAVE_HW = 1 cause trouble?

[ZachCook]

shouldn't make a difference

[barisyild]

shouldn't make a difference

*CTR boots 3 times slower than it should.
*8 MB RAM not working

[barisyild]

shouldn't make a difference

The oxide is selectable when the ram is 8mb in PCSXR, but this emulator crashed directly.

[ZachCook]

@simias your patch seems to have some side-effects, and doesn't seem to work as expected, should probably be reverted in master (was it accidentally pushed there?)

[simias]

Oh boy, I didn't mean to push it on master, not sure how I did that. I just reverted it, sorry.

[simias]

You just have to compile beetle-psx with HAVE_LIGHTREC=0, retroarch settings do not change it

Does compiling with HAVE_HW = 1 cause trouble?

It will change the name of the resulting library file (with an interstitial _hw_ ).

[simias]

I tried to run the MAIN.EXE that's supposed to run the memory test but I see no output, just the black screen. Am I supposed to turn the BIOS console on to see the output in the terminal? And if so, does somebody know how to do that in beetle?

[barisyild]

I tried to run the MAIN.EXE that's supposed to run the memory test but I see no output, just the black screen. Am I supposed to turn the BIOS console on to see the output in the terminal? And if so, does somebody know how to do that in beetle?

did you test with PCSXR?

[barisyild]

Configuration -> CPU

[niko1point0]

Is there any way for us to be given an input box that controls how much RAM we have? That way we can change it to anything we want: 2mb, 4, 8, 128?

Also, from the modding community, we found how CTR manages to load chunks of the adventure hub in, and out of memory. If we had the ability to set any amount of RAM we want, we have the potential to make a completely open-world CTR environment.

All PS1 games will have the same potential for mods

[simias]

@barisyild I don't understand why you want me to test PCSX-R. I'm sure it works there, but that's not really helping me getting beetle to work.

I guess I'll just dig around the source code to figure out out to enable the BIOS output in beetle, I'm fairly sure mednafen supports it so it shouldn't be too hard.

@niko1point0 Well luckily there's 488MB of unused memory space after the RAM region, so technically it's doable. But making it dynamic will be annoying and might hurt performance, I think it would be better to fix a size that we could deem reasonable enough. 496MB of RAM is probably overkill, maybe 32MB or something like that?

[simias]

Oh I just realized that I hadn't tweaked the DMA, it still would only address 2MB. That's probably not what we want.

[niko1point0]

16mb is the largest amount of RAM that PSX would be able to use. In MIPS assembly, only 3 bytes can be used to hold an address (4 bytes with '80' prefix), and 256 to the power of 3 is 16 million, 16mb

The highest amount ever tested on a physical PS1 is 8mb, but 16mb should be possible. 32mb would not be possible on a physical console, so there should be no need to ever emulate more than 16mb with a patch

[ToniBC]

Hello, it's been a long time since this and there doesn't seem to be any change. Swanstation has this same option, so implementing it should not be very complex.

Do you know anything about this function?

There are games like Gran Turismo 2 that use this to show more polygons in the game and it's worth it.