Generation 8 data

[Naramsim]

PokeAPI has decided to take a temporary shift from https://github.com/veekun/pokedex and thus to allow the community to add the new generation 8 data.

The process to add new data consists as follows:

1. Decide which new data you want to bring in.
2. Look up here how this data is structured in our CSV files
3. Add the new data manually or by using a script and documenting where did you get those pieces of data, read also this document.
4. Open 2 Pull Requests
   • Towards https://github.com/PokeAPI/pokedex with only the CSV data and the scripts you used to gather it
   • Here with the new data and any changes to our API logic to make it work.

---

Sources that might be used for scraping/checking new generation 8 data are:

• Bulbapedia
• Serebii
• PokemonDB
• Pokemon.com
• Pokemon ROMs dump files

[RyanVereque]

I didn't see this specific issue when I posted on Veekun, but I did know that this branch was trying to temporarily add Gen 8 data. Let me link a little write-up I posted there for some data I've gathered from Bulbapedia scrapped with Node(Browser). It's not python, so unless I migrate my code, it'll just be CSV edits for now. Would 2 Pulls consisting of CSV edits be acceptable ? (And of course I'd make sure IDs( and "foreign keys") within the CSV edits are consistent with this fork.
veekun/pokedex#284 (comment)

[Naramsim]

Hi @RyanVereque, surely we are interested in your contribution! We've already updated some csv files.

If you are willing you could modify some other files with your scripts (or extend our modified files/scripts). The important part is that we don't want to change in any way the format of the data, so no columns additions or new file creations.

If you like to discuss further I suggest you open an issue over at our fork so we can discuss it freely.

[CMahk]

I recently ported a section of pkNX dedicated to parsing the dex entry files (zukan_comment_A / B) in the Let's Go! / SwSh game ROM file systems. It at least provides a baseline to work with.

[Naramsim]

@CMahk, when you're done we and the folks at @veekun will be enthusiasts to have your data :)

[git-stitch]

I didn't know you changed your policy for community. I have a complete serebii scraper and started a Poke-simple-api on Rails. I could modify the data to be able to give you serebii's data in a CSV file. Stats, Img, Evo-chain, Ability, Location, Entry have all been scraped.

Edit: you already have all this information. What's missing?

[Naramsim]

Edit: you already have all this information. What's missing?

Basically all gen-8 data :)

[git-stitch]

Edit: you already have all this information. What's missing?

Basically all gen-8 data :)

Ayyyyyy since I built this for no reason I'm still gonna publish it but that's great.

[Artrix9095]

Have you looked at smogon and their repo? they have some pretty trustworthy info there.. its worth looking at.

[zihadmahiuddin]

Looks like both pokemon.csv and pokemon_species.csv have the gen 8 Pokémon added. But pokemon_species_names.csv still doesn't have them.
I was planning to add those so I am looking for "requirements". Like, which languages must be included etc.

Bulbapedia doesn't have the names in a lot of languages. And they only have the genus in English. Should I use Google Translate for the genus names or just keep them empty or something? Asking this because Google Translate is not always accurate.

And thank you very much for the opportunity to contribute.

[Naramsim]

@zihadmahiuddin

I was planning to add those so I am looking for "requirements". Like, which languages must be included etc.

Currently, we have all of these: https://pokeapi.co/api/v2/language

Bulbapedia doesn't have the names in a lot of languages. And they only have the genus in English. Should I use Google Translate for the genus names or just keep them empty or something? Asking this because Google Translate is not always accurate.

😸 Of course, don't use GTranslate :)

Here are the French ones: https://bulbapedia.bulbagarden.net/wiki/List_of_French_Pok%C3%A9mon_names At page end you find an infobox and in the first row there are the links to other languages, such as Japanese

[Naramsim]

@strikingforce909

Have you looked at smogon and their repo? they have some pretty trustworthy info there.. its worth looking at.

We've already looked at Smogon's resources. They have loads of data related to battles and such. So it could be a nice resource to pull data from.

[zihadmahiuddin]

Here are the French ones: https://bulbapedia.bulbagarden.net/wiki/List_of_French_Pok%C3%A9mon_names At page end you find an infobox and in the first row there are the links to other languages, such as Japanese

Nice, looks scrape-able! What about the genus field though? Should I just keep that one empty?

[Naramsim]

Yes, @zihadmahiuddin, you can leave it blank if it's not available

[Wiper-R]

https://github.com/itsjavi/swordshield-data/blob/master/data/json/pokemon.json

Not that much good, but you can fetch abilities, moves from there :)

[RampageRobot]

I'm not sure if this will help, or if this is what you're looking for, but I found a rather large resource for various Pokemon icons. Artwork, sprites, all kinds of stuff. I know that at the very least the official artwork section is complete with all of gen 8. There's also g-maxes on there as well. The link is here.

[Naramsim]

@RampageRobot, hi, thanks for the link. I checked it out and it seems to be a copy of Veekun media

I couldn't find any SWSH sprites apart from the new HOME design. (Which, indeed, we could add to the API)

[RampageRobot]

@Naramsim ahh, yeah, the site I grabbed from must have gotten theirs from Veekun then, sorry about that. They have gen 8 artwork on the page I linked, or at least files for them. Not sure if that's needed at all. It's under the pokemon-artwork folder

[ExperiBass]

How does one dump data from the games?

[Naramsim]

There are various tools on Github, but I think none for gen-8 games.

[ExperiBass]

got a link to one for gens 6 and 7? i think some data was off for those

[cam-ball]

For selfish reasons, I need location and encounter data for Galar, so I decided to try grabbing location information (because it seems like Pokemon data is definitely accounted for)

I wanted to come here and fish for some guidelines, because with just scraped data, I don't have any basis for game_ids or for sub areas within any given location. I've done a very basic add-locations-and-nothing-else first pass for locations, and think it would be trivial to add language data from here, but wanted to confirm that just incrementing the IDs isn't a bad idea

Location commit

TL;DR, how to handle pulling data exclusively from non-official sources?

1. Should locations be in any inherent order?
2. If I have no source for game_index, then it makes sense to not add to that CSV, right?
3. Is having each location's ID just incremented from the previous okay, or should I leave a gap for future-proofing?

any other feedback is fine as well, but I will open a for-realsies PR when some of the above is more solid, if it is a good idea to continue down this track

[Naramsim]

1. Should locations be in any inherent order?

No, I don't think so, the important part is that in the future we will use those ids in the other CSV files for cross-reference.

2. If I have no source for game_index, then it makes sense to not add to that CSV, right?

I haven't tried, maybe our API supports zones without Game IDs. For that you need to open a PR at PokeAPI and I'll try to build. If it doesn't work we can overcome the issue by creating a new unknown game and use it for linking those locations. Anyway, why can't you extract the game from Bulbapedia?

3. Is having each location's ID just incremented from the previous okay, or should I leave a gap for future-proofing?

No gap needed.

[cam-ball]

I haven't tried, maybe our API supports zones without Game IDs. For that you need to open a PR at PokeAPI and I'll try to build. If it doesn't work we can overcome the issue by creating a new unknown game and use it for linking those locations. Anyway, why can't you extract the game from Bulbapedia?

@Naramsim this might be a me problem. I don't understand what game_indicies is referring to in places like this and assumed it was some internal ID that is stored within the game data itself, which is why I wouldn't be able to get it from Bulbapedia (or Serebii or what-have-you), but if I'm totally misunderstanding what that data is (or how to get it) that'd be great news and I can script populating that CSV as well!

[Naramsim]

Well. to be honest @laurenball, I don't know either. To me, it seems like a generation-wise ID. So there can exist multiple game_indexes but only one for each generation. Could it be?

We should checkout Veekun if they provide some documentation about it and we should also check as well in our API.

[cam-ball]

I'll be honest, you not knowing makes me feel a little better. I can poke around and report back what I find :)

[Naramsim]

https://pokeapi.co/docs/v2#locations-section: A list of game indices relevent to this location by generation.

Notice the typo :) Hahha, yeah, we do expose it in the API and I think it's just an index for the location relevant to the generation itself.

Veekun itself seems not to show this piece of information in their website: https://veekun.com/dex/locations/route%2010

[Davidy22]

For the hisuian region spoilers, I presume they're going to be classified under gen 8 like meltan is technically gen 7, so they probably get folded into this issue so adding this comment as an appendix. Also, this may need to move from an issue to just part of official documentation and process since it looks like veekun's not returning to form anytime soon, and we're going to hit gen 9 by the cadence next year.

[christopher-caldwell]

Hello, I'm not sure if this is the proper place, or if it's a "new" issue, but I was wondering how to access the rapid strike Urshifu? https://pokeapi.co/api/v2/pokemon-species/892 -> goes to single strike, and I'm not able to find any references to the rapid strike, water style.

[C-Garza]

@christopher-caldwell That endpoint you have has a varieties property that has an endpoint for rapid strike, is that what you want?
E.x. urshifu-rapid-strike

[christopher-caldwell]

@C-Garza Yes, thank you!

[Rdlenke]

@Naramsim (or any other maintainer, really), is there any requirement to how much data we should have before opening a PR? I have a small bunch of data (gen 8 move flavor texts from pokémon swsh), and was wondering if something like that is worth integrating here.

[Naramsim]

There's no minimum requirement 😄 , even a single line is acceptable. Currently, we are assessing whether to integrate Veekun data back into our repository. So, maybe you could check if your data is the same as Veekun's: https://github.com/veekun/pokedex/tree/master/pokedex/data/csv