Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(Victory) Road to 2.0 #24

Draft
wants to merge 9 commits into
base: main
Choose a base branch
from
Draft

Conversation

kbembedded
Copy link
Collaborator

@kbembedded kbembedded commented Jan 20, 2024

Spinning up to get gen ii support going. Along with that I'm wanting to implement Mystery Gift and if I feel like it, Pokemon TCG Card Pop! support. Along with incorporating gen ii sprites, I'm hoping to see some of the other sprites get some clean up.

This is still a WIP and will be a Draft until its ready. I'm leaving it open here in case anyone wants to test or start playing with this before its fully done.

  • Gen II Support
    • Add sprites (complete, not yet committed)
    • Add movesets
    • Add types Types are not exchanged as part of trade in gen ii
    • Add items
    • Modify gender
    • Modify shininess
    • Modify Unown form
    • Add Mystery Gift (Will implement later)
  • Layout
    • Reorg menus
      • Create main menu to select trade target
        • Gen I
          • Gen I trades currently broken (as of 20240307), patch list is broken.
        • Gen II (resolves feat: Add support for gen 2 #16)
          • Gen II trades currently broken (as of 20240307), jumps from ready to waiting to ready. This was previously working. I think this is an issue with my game file, it appeared after I messed with giving Mail to pokemon to hold. 20240311: Working for me in my test branch, others still potentially have issues.
        • Gen II Mystery Gift (Will implement later)
        • Select Pinout
    • Require confirmation of exit (maybe only do this after a trade has been done? Mostly to prevent accidentally backing out) (resolves Require hold Back to exit #23)
      • Would love to add a sad pikachu icon to this if there is time/space
    • Fix Select view to allow full pokemon sprite to be visible
    • Fix Trade view to allow full pokemon sprite to be visible (this may not work)
  • Code refactor and shrink
    • Reduce clutter of accessing parts of the pokemon trade struct
    • Support dynamic access of pokemon trade structs for either gen i or gen ii
    • Can likely pack structs
    • Investigate string compression possibilities Due to the nature of the strings, i.e. names of pokemon, items with abbreviations, lack of repetition of strings, the savings here is minimal given the overhead involved with decompressing.
    • Optimize the lazily slapped together functions
    • Move large const data structs in to their own file(s)
    • Compress sprite icons if they are not already (See "Sprite Work") Icons are indeed compressed.
    • Rework Select Pokemon alloc to use an anonymous pointer like Trade now does Not going to target it, if I end up having to touch it then I will, but for now leaving this alone.
    • See about reusing some of the Modules to reduce code footprint
      • e.g. pokemon name, trainer name, both have similar restrictions, could collapse those to the same set
    • I believe there is a small memory leak somewhere, fix it.
  • Sprite Work
    • Auto-generated sprites arn't great, clean up and consider some more customized pixel art (66% complete as of 20240507)
    • Move to using compressed bitmaps rather than icons (currently, Icons take up ~45kbyte) Icons are compressed.
    • Can we store sprites on SD card and load them as needed? How to distribute that? Looks like this is possible
      • Need to find a way to nicely dynamically load/unload these sprites. If on-demand is too non-responsive, the consider loading in chunks of 10+, and load a couple of chunks ahead of time so there is a good window of responsiveness. Testing has found that there is no impact to responsiveness when loading each sprite on-demand.
  • Optional Goals (read: if I feel like working on them)
    • Save pokemon to SD card Not interested in doing this ATM.
    • Some logic to "harvest" GB trainer name and number (I think this would require a trade as OT Num isn't transferred. Requiring a trade is a lot of hassle and is more or less already implemented) This would require at least starting a trade, but in a way that wouldn't clobber the current pokemon -OR- completing a trade with a junk pokemon, transforming it in to what the user wants, and it will pull in the OT ID/Name.
    • Sprite animations (doable, but, may require more space than we have available at the moment) Maybe another time
    • Add Pokemon TCG Card Pop! support
    • Add option to select the default pinout that the tool should use Doing this correctly is going to require a bit of a rewrite to the pin selection menu anyway. Default/Original is going to be most common, so that is good enough for now.
  • Fix issue with malveke ext1 clock interrupt breaking interface buttons (flipper-gblink issue)
    • Wrap MALVEKE detect in debug defines to not break debug when detecting MALVEKE's with Original pinout Per conversations, the app will default to Original pinout and MALVEKE will need to be separately selected.

@EstebanFuentealba
Copy link
Owner

EstebanFuentealba commented Jan 20, 2024

Great roadmap, you're awesome @kbembedded .

Regarding the images, we could try testing something like https://github.com/flipperdevices/flipperzero-firmware/tree/dev/applications/examples/example_apps_assets to see if we can free up some memory.

@kbembedded
Copy link
Collaborator Author

Fairly large assorted updates.

Most of this work so far is focused on cleaning up how all aspects of the trade block are accessed to make it more generic. This should make it easier to plug in gen ii data without having to modify the existing scenes/views. There are now accessor functions for getting and setting pretty much all of the data. This may still see some cleanup down the road if I feel like it, but I think this is good enough for now to start focusing on feature adding.

MALVEKE detection might be a bit broken right now, but that will get cleaned up and checked again later as there is still some more work that needs to be done there.

@kbembedded
Copy link
Collaborator Author

Had a different project come up that took priority, however I'm back at this now and am hoping to have gen ii trades functional soon.

If anyone out there has a MALVEKE, and a copy of Gold that is far enough in to have mystery gift, if you could send me the save file for that, that would be great :D I am not far enough in to Gold to get the mystery gift and I'm not sure how long it will take me playing casually.

@EstebanFuentealba
Copy link
Owner

Hi Kris, great job so far! I haven't had a chance to try it out yet as I'm just getting back from vacation, but I might dive back into the projects today.

What do you think about splitting the PR into several smaller ones? For instance, we could separate "Mystery Gift," "TCG Card Pop," "Sprite animations," and "Customized pixel art" to get a playable version sooner.

I could also help with generating images for the second-generation Pokémon, similar to what I did for the first generation.

What do you think?

Best regards,

@kbembedded
Copy link
Collaborator Author

Hi Kris, great job so far! I haven't had a chance to try it out yet as I'm just getting back from vacation, but I might dive back into the projects today.

Thanks!

What do you think about splitting the PR into several smaller ones? For instance, we could separate "Mystery Gift," "TCG Card Pop," "Sprite animations," and "Customized pixel art" to get a playable version sooner.

I left it as one big one for a couple of reasons.

  1. I had to rip up and rewrite a bunch of code because of how specific the app was originally to gen I, notice there are 47 files changed so far, that's basically every file in the whole repo. I also took that opportunity to reel back in a lot of the hacky things I left in place (before introducing more 🙃). And I'm debating also changing up how I handle the data structures in the background just to make the code more manageable as all of the data handling ballooned (though that could wait for a later release, if I can shave off final binary size, thats going to be helpful in getting all of the sprites to fit).
  2. I wouldn't consider gen II support complete until it also supports Mystery Gift as that is a part of that generation.
  3. Sprite animations probably won't make it in to this PR. Not everything I have listed above will, and I was planning on splitting those out. Card Pop! may also not make it in this anyway.
  4. I've had a handful of people reach out and ask for builds which are easy enough to provide, and let them know that things may still be broken. So, while not getting a public release, those who are interested and who have been interacting with me on this are getting test builds and providing good feedback/testing.
  5. Personally, I want the custom sprites to be a part of this release. It might delay it a bit, but, they are being worked on in the background and about 20% of them are complete at this moment. Once I'm done with code and ready to start bringing it all together, I will also step in and help with the sprite art. The auto generated sprites just don't do it for me after seeing what is possible lol.

I could also help with generating images for the second-generation Pokémon, similar to what I did for the first generation.

What do you think?

A friend of mine already offered and is looking in to this, I might accept it as a placeholder, but I don't want to open this PR until the custom sprite art is done.

@kbembedded
Copy link
Collaborator Author

Loading sprites from SD looks like it will be functional with almost no impact on the application responsiveness. Currently a WIP but will make its way here soon.

In the home stretch of getting all of the features implemented. Still need to do some cleanup and tuning, but, this should be ready soon™

@EstebanFuentealba
Copy link
Owner

Perhaps in future versions, it would be beneficial to apply inheritance to avoid having so many conditions for GENI and GENII.

For instance:

trading.parser(new PokemonTradingParserGenerationI(context));
// or
trading.parser(new PokemonTradingParserGenerationII(context));
// ...
trading.tradePending(); // Call the "tradePending" function based on the initialized object.
class PokemonTradingParser {
  PokemonTradingParser(void *context);
  // ...
  void tradeReset();
  void tradeInit();
  void tradeRandom();
  void tradePending();
  void tradePatchHeader();
  void tradePatchData();
  // ... other functions
}

class PokemonTradingParserGenerationI : public PokemonTradingParser {
  // override necessary methods
  void tradePending() override;
}

class PokemonTradingParserGenerationII : public PokemonTradingParser {
  // override necessary methods
  void tradePending() override;
}

@kbembedded
Copy link
Collaborator Author

kbembedded commented Mar 11, 2024

I'm more fluent in C than C++, and when it comes to the trade protocol, gen I and II are extremely similar with a few exceptions. Right now, what I've done is hacky, and won't make it to the final PR without cleanup. By the end of it, there should only be 1, maybe two generation checks in the trade state machine for the points that greatly differ.

What is currently worse than trade when it comes to per generation conditionals is the data structure handling/setup; which I also have plans to rework before the final PR is opened.

If you want to move things to use C++ constructs, you're more than welcome to. I do know that C++ still has some oddities (hopefully those are being fixed with the new toolchain setup in the works) that makes debugging them very painful in some instances on the Flipper.

@kbembedded
Copy link
Collaborator Author

Latest round of updates brings a bunch of things together.

Gen i and ii trades fully functional again (there were some breakages over time)
Assets now loading from SD card. Will need more changes when final sprite assets come together
Forced synchronization of trade states, i.e. Back disabled for most things, press and hold back to force go back a screen.
All gen ii tunables are set up and working (minus egg, but, I left notes in place to handle eggs later if there is demand for it)
There are a few more things I want to clean up in code before doing a final clean/rebase and moving on to Mystery Gift

Needed for HO-OH default name support eventually

Signed-off-by: Kris Bahnsen <Kris@KBEmbedded.com>
All gen ii pokemon, items, moves, and stats were added. In addition to
the original gen i tunables, the user can specify shininess, gender,
held item, pokerus status, and Unown forms.

Code changes:
Separate out the mass of functions and const structs to standalone files
for interacting with those.

Make the trade and party struct completely opaque and provide accessor
functions to get and set arbitrary members of the struct. *

Create a main menu of sorts to select the desired generation to emulate.
This also moves the EXT pinout selection here so this needs to be set up
before a trade is initiated.

Remove repeated scenes and enforce better reuse, e.g. Level & OT ID use
the same scene code with different scene names; same with Nickname, OT
Name, and Unown Form.

Fix issue with MALVEKE board detection.

Implement exit confirmation from each generations setup. It was really
easy to accidentally back out to the main menu after a trade (e.g. to
edit a pokmeon) and then lose all of the data.

Better synchronize states in trade, if the Flipper wants to go back out
of the trade, act more like a real Gameboy and block until the trade
partner confirms that both sides agree to backing out of the trade menu.

Add gen ii pokemon to data table. This now mixes data between gen i and
ii, but in ways that don't actually conflict. Gen i uses an arbitrary
index to ID the pokemon, gen ii uses the national pokedex number. Gen i
trades types, gen ii does not. Gen ii adds spc_atk and spc_def, gen i
only uses spc; so now every gen i pokemon has spc, spc_atk, and spc_def
and gen ii pokemon only use spc_atk and spc_def. Gender ratio used only
in gen ii. Growth rate is the same for all 251 pokemon between gen i and
ii. Default moves are the only desync'ed entry. The first 151 pokemon in
the table use the default moves from gen i games, the next 100 use
default moves from gen ii.

Add gen ii moves, these are bitmasked to show what moves are valid in
which generations.

Add gen ii items, only used in gen ii.

(*) One of the attempted optimizations was to not use the messy accessor
functions; maintaining a single struct of all possible data needed. When
trade time comes, copy all of that in to the right format for the
selected generation. The problem is that moving to and from the single
common struct to individual trade structs and back added way more code
than what is done now.

Visuals:
Move pokemon icons out of assets/ and set up loading sprites from SD
card. These are held in sprites/, translated to FXBM format (i.e. the
sprite format used by the flipper game engine), and catted together to a
single file.

Rearrange trade and select views to ensure a better view of pokemon
sprites. Sprites will be eventually moved to 56x56 (right now all are
50x50) and this is taken in to account. This also includes proper
layering of drawn assets/sprites/etc. with transparencies to make the
whole interface look more natural.

Add graphics testing macro at compile time

Unused assets, repeated assets, and assets that can be redrawn with
primitives in firmware are removed.

Signed-off-by: Kris Bahnsen <Kris@KBEmbedded.com>
Remove noisy debug, don't need to print out entire patch list

Signed-off-by: Kris Bahnsen <Kris@KBEmbedded.com>
Default pinout to Original, MALVEKE can be manually set, newer
MALVEKE boards are transitioning to the Original pinout, so as
time goes on there will be fewer and fewer MALVEKE pinned boards.

Also save some codespace by re-using the in-firmware pin names

Signed-off-by: Kris Bahnsen <Kris@KBEmbedded.com>
Signed-off-by: Kris Bahnsen <Kris@KBEmbedded.com>
Signed-off-by: Kris Bahnsen <Kris@KBEmbedded.com>
@kbembedded
Copy link
Collaborator Author

Gen II trade mostly locked in at this point. There may need to be some small tweaks, but, for anyone watching this PR; please test and see what you can break 😄

Starting on Mystery Gift support now. Getting the MALVEKE pinout to not break the OK button is on my list when I need a break from IR. And I think I can probably implement the "save default pinout" option pretty easily now that I've worked with loading data from the SD card.

Those, and then sprites are all that remain.

@kbembedded
Copy link
Collaborator Author

Mystery Gift support is going to be kicked down the road. There are complexities with the current IR libraries that would make the code a bit of a nightmare. I either need to spend time to get a better grasp of the IR system, or may need to consider adding some support to the flipper API.

@fexed
Copy link

fexed commented Apr 13, 2024

Testing the Gen II trade with an italian copy of Pokémon Gold, building the fap with the Unleashed firmware, I get stuck on the "waiting" screen just after the trade ended :(

Update: looks like the issue is on the Funny Playing FPGBC side, tested with an original GBC and works flawlessly

@EstebanFuentealba
Copy link
Owner

Testing the Gen II trade with an italian copy of Pokémon Gold, building the fap with the Unleashed firmware, I get stuck on the "waiting" screen just after the trade ended :(

Update: looks like the issue is on the Funny Playing FPGBC side, tested with an original GBC and works flawlessly

Hi @fexed In your FPGBC, what value do you have set for CLKSPD?

@fexed
Copy link

fexed commented Apr 13, 2024

Testing the Gen II trade with an italian copy of Pokémon Gold, building the fap with the Unleashed firmware, I get stuck on the "waiting" screen just after the trade ended :(
Update: looks like the issue is on the Funny Playing FPGBC side, tested with an original GBC and works flawlessly

Hi @fexed In your FPGBC, what value do you have set for CLKSPD?

I don't think that is something I have control on. I did not modify it, and I'm on firmware version 1.08 (which is closed source as far as I know)

Note: after some retrying, I managed to succesfully trade a Pokémon. I was debugging adding some logs when it happened, so indeed a slightly wrong clockspeed seems to be the issue. IIRC the FPGBC doesn't run at exactly 100% but slightly less/more (can't remember rn) and that may be related, too.

@kbembedded
Copy link
Collaborator Author

kbembedded commented Apr 13, 2024

Update: looks like the issue is on the Funny Playing FPGBC side, tested with an original GBC and works flawlessly

Interesting.

In your FPGBC, what value do you have set for CLKSPD?

Note: after some retrying, I managed to succesfully trade a Pokémon. I was debugging adding some logs when it happened, so indeed a slightly wrong clockspeed seems to be the issue. IIRC the FPGBC doesn't run at exactly 100% but slightly less/more (can't remember rn) and that may be related, too.

From what I understand of the FPGBC (I've never used one) and the EXT link interface hardware is that this shouldn't be an issue, even if you crank the CPU clock speed up to max. The short technical answer is its a giant shift register and in the case of the trade tool, it forces the gameboy to drive it. The FPGBC wouldn't (well, shouldn't) exceed what the established clock rates are. The GBC can do different rates, up to 500some kHz, but even this wouldn't overwhelm the flipper side in processing data. Those rates are also dependent on the game, and as far as I can tell Pokemon G/S/C still only ever runs at the base 8192 Hz.

Testing the Gen II trade with an italian copy of Pokémon Gold

Do you have a copy of a Gen I game to test a trade with? Either with the latest trade tool release, this dev branch, etc.

Can you take a video of the trade (Gen II at the very least, and Gen I if you can as well) and upload it somewhere so I can take a look at both the flipper and gameboy while the trade happens?

I might move this to an Issue and have to get a hold of an FPGBC to dig further.

@fexed
Copy link

fexed commented Apr 13, 2024

Sorry for the trouble. As I was recording the Gen II video I think I found out why it didn't work this morning. It looks like the EXT port of the FPGBC is somehow slightly loose compared to (at least my) original GBC. The cable moves a little, and as we all know this kind of serial communication is not very happy to be moved around. By pushing the cable a bit more inside the EXT port of the FPGBC I managed to trade Pokémons without issues a few times.
I performed the test with Gen I (Pokémon Yellow) a couple of days ago and after some issues with the cable (which I handmade by stripping an original trade cable, as I am waiting for the Game Boy tools for the Flipper Zero 👀) I succesfully traded Pokémons. So no issue there, and I thought the problem was on Gen II side.

Anyways, I have all cartridges needed (even some jap and Gen III as well, for perhaps the future 👀) and all kinds of hardware (GB, GBC, GBA and FPGBC). If you need testing just lmk. I imagine that, given this was a false alarm, no video is needed? I can record it nonetheless if you need it.

@kbembedded
Copy link
Collaborator Author

Sorry for the trouble. As I was recording the Gen II video I think I found out why it didn't work this morning. It looks like the EXT port of the FPGBC is somehow slightly loose compared to (at least my) original GBC. The cable moves a little, and as we all know this kind of serial communication is not very happy to be moved around. By pushing the cable a bit more inside the EXT port of the FPGBC I managed to trade Pokémons without issues a few times. I performed the test with Gen I (Pokémon Yellow) a couple of days ago and after some issues with the cable (which I handmade by stripping an original trade cable, as I am waiting for the Game Boy tools for the Flipper Zero 👀) I succesfully traded Pokémons. So no issue there, and I thought the problem was on Gen II side.

No worries! Glad you were able to figure it out.

Anyways, I have all cartridges needed (even some jap and Gen III as well, for perhaps the future 👀) and all kinds of hardware (GB, GBC, GBA and FPGBC). If you need testing just lmk. I imagine that, given this was a false alarm, no video is needed? I can record it nonetheless if you need it.

If you don't already have one of @EstebanFuentealba's MALVEKE boards on order, and are interested in doing some compatibility testing, shoot me an email at kris @ kbembedded.com and I'll get you one of my EXT interface boards for flipper so you don't have to mangle cables anymore 😄

If you want to test different game boy hardware and Pokemon games and just drop notes here on what you tried and if you had any issues; all of that that would be great to get in documentation!

As an aside, JPN carts are currently not compatible with the Flipper trade tool. They use a different data format than all of the other western releases and you will end up with corrupt game data if you attempt a trade with a JPN release of Pokemon. At this time I don't have a JPN cart and am not terribly interested in implementing it. If it is something you are interested in tackling I can help if you have any questions on the current Trade Tool set up. I do have plans for Gen III at some point, however.

Allows old MALVEKE pinouts to work without breaking the OK button

Signed-off-by: Kris Bahnsen <Kris@KBEmbedded.com>
@kbembedded
Copy link
Collaborator Author

With this latest update, older MALVEKE hardware that used to break the OK button after trade, no longer has that issue.

It was a very hacky workaround, but so far appears to be robust in latest OFW release.

If anyone running a CFW would be interested in testing, I'd love to know what you see there.

Signed-off-by: Kris Bahnsen <Kris@KBEmbedded.com>
Signed-off-by: Kris Bahnsen <Kris@KBEmbedded.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Require hold Back to exit feat: Add support for gen 2
3 participants