-
Notifications
You must be signed in to change notification settings - Fork 218
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Firmware too large and constant strings #574
Comments
I'm not a programmer, I only read the first ~70 pages of Kernighan's "The C programming language". Stopped reading at pointers. So the whole code base is certainly missing some optimization. I guess the strings are not getting merged by the compiler because they are stored in the program memory instead of the SRAM since we are even lower on SRAM than on ROM. I'll try it out and report back. |
That's still 70 more than I did :) . <off-topic>BTW, thank you very much for this project. I had original N64 batteries which were magically still hanging on, keeping game saves alive after all this time. I could dump the saves, replace the batteries and write them back.</off-topic> I've tried disabling the I realised that avr-gcc is version 5.4.0, which is getting quite old now. I have no idea if constant merging has been improved since, but it makes it unlikely that anyone on the gcc side would be willing to take a look at why it does not work as expected. Such automated de-duplication would be significantly more convenient than having to maintain a "list" (at least in the general sense, if not in the technical "array" sense) of strings in order to save space, so it is unfortunate it does not work. |
It does seem to work, just with the default modules enabled and two strings replaced I already get results: I just replace
with new function
|
Cool ! For reference, here is the updated top-30 most duplicated strings with default modules (I am on HW3, which I believe affects the menu strings, also I am still on the same commit as above):
So 712 bytes saved. >>> 27 * len("Reset ")
162 This is a much larger saving than I would have expected (keeping one copy of the string, the trailing space is just to represent the trailing NUL).
1120 further bytes saved >>> 61 * len("Press Button... ")
976 The gain is again exceeding expectations. Not sure why, but I'm not complaining :) .
4432 bytes saved ! This is great. |
Now 5330 bytes with default modules, see attachment. |
New top-30 with HW3 and default modules:
This looks a lot cleaner: now the false-positives are taking a significant portion of the output. I tried enabling all modules, but it still goes over the limit. And I am a bit puzzled: it goes over the limit by 4k... More than before. Maybe this is just because I was a bit behind the master branch, and something would have increased the footprint since ? |
I have identified another way to gain ~500 bytes of program space (when all modules are enabled): make flashid an integer instead of a 5-bytes string. Still not enough to fit them all, but getting closer. This saves program memory in 2 ways:
This also saves 200 bytes of ram, which I think come from the many 5-bytes strings which used to be ram-based (so they had to be copied from their initialiser value to ram during boot, so they took 200 bytes of global space). I pushed the change to my fork (linking to the branch as I will likely push some more). EDIT: Important note: I am not testing these changes on-hardware yet, rather getting a feeling of what impact they have on memory use. This is applied on top of the changes you posted a few comments above. [edit] rewording |
Quick progress report before logging off: I gained about 1kB of program space. 3kB (at least) to go. My current targets are duplicated code:
I target the largest functions using information from I exclude But again, I have not tested this against real hardware yet, so I probably broke some things along the way. I am worried about how I can test the changes I am accumulating, as I only have N64, GB (+ GBC) and GBA gamecarts available to dump, and no flashcart. |
I can test NES, SMS, MD and have GBA, GBC, SNES and N64 flashcarts. If there is anything I can test before you invest the time changing the code just tell me. Otherwise just do the changes and we test and fix afterwards. |
Today's news (so far): I've focussed on NES.ino, and reduced the size by abou 1450 bytes. 300 bytes to go before a perfect fit with HW3 (but freeing a few more kB would be more than welcome). I pushed everything so far to my branch. I would like you to start reviewing the changes, to at least point out mistakes I am making (breakages, coding style, bad directions, pointers to other places where similar changes would be applicable, ...), and maybe even pick a few commits for inclusion. Should I open a merge request ? I will likely be force-pushing to it (hopefully only the few topmost commits), so this may cause unwanted notification noise. |
I found another 848 bytes by removing all 5 optional features from the display library like explained here: https://github.com/olikraus/u8g2/wiki/u8g2optimization#u8g2-feature-selection You have to edit the library header file as setting the "without" defines in Cart_Reader.ino doesn't have the same effect. I have no clue what these extra features do but I didn't notice anything acting different now. The font I'm using is restricted to characters from 32 to 127 so I think we are fine. Anyway with your trim branch and the lib edit I can enable all modules on HW5: |
HW3 fits as well.
|
I'm gonna be off for a little while but will test your fork later today. Can you enable "Issues" on your repo then we can track them there better. I found one with a short test, the NES database browsing screen is missing the first letter of the name and also doesn't display the mapper data correctly The first letter appears again when browsing right/forward, disappears when browsing left/back. You can test without a NES cart inserted by just selecting any letter in the manual selection screen. |
Done.
Nice catch, looking into this. |
Found it, and very likely fixed it, |
Update: with my current
This means that there is now enough headroom to re-enable these LCD driver options. While the code does not use them, I believe this is hurting ease of distribution. Especially, it makes updating dependencies cumbersome. |
Here is an update on the global ram space usage (.bss for globals without initialiser, .data for globals with initialiser, descending size down to 16 bytes as an arbitrary cutoff, some N64 stuff missing as this is from my development environment where I already got rid of it):
Descriptions grouped by theme. Display:
SD-card:
Clock gen:
Cart IO & cart-type-specific stuff:
|
While I think there is still more that can be done on this general flash-and-ram-usage-optimisation topic, I think this specific bug can be closed: since 11.1, I see that all modules are enabled again by default. |
Note: I am completely new to arduino in general, so I may be missing something.
I cannot get the latest source (as of 9d80d24) to fit with all modules enabled. On its own this may not be a big issue (I guess this is why they are optional and selectable to begin with). The linker complains that the code is 2444 bytes too large.
Then I took a look at the produced binary (after disabling the NES module just to have a file to analyse) to check what is going on. And here is an extract of what I found:
The first obvious finding is that
-fmerge-constants
(which is supposed to be enabled in-Os
, which is used in the build) is somehow doing a terrible job. I tried editing my platform.txt to explicitly add-fmerge-constants
, without any improvement.So, how much space is used by these ? Ignoring the few strings above which are likely accidental finds by
strings
and not "true" strings, I get:Of course, one copy of each strings would still be necessary, and this ignores the (low ?) probability that some of these would actually be non-constant but actually local variable initial values, so the final gain would not be strictly this much. Still, this is 3 times larger than the extra space needed to fit all modules in the available program space. Also, it obviously ignores any possible duplication from the NES module.
I do not have a magic solution. The best idea I can come up with is terrible: put all constant (including inline) strings in some header file and reference them everywhere. But at least this should not leave gcc the chance to mess things up. This will not help if these duplicate strings come from libraries, but the strings visible above seem to be rather specific to this project, so the losses from libraries seems likely to be low.
The text was updated successfully, but these errors were encountered: