Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Is it possible to convert twarc jsnol files to twarc2 jsonl files? #692

Closed
fishfree opened this issue Feb 15, 2023 · 2 comments
Closed

Is it possible to convert twarc jsnol files to twarc2 jsonl files? #692

fishfree opened this issue Feb 15, 2023 · 2 comments

Comments

@fishfree
Copy link

If so, how to do it, please?

@edsu
Copy link
Member

edsu commented Feb 15, 2023

It's theoretically possible, but it's not something that twarc does currently. It also would be a lot or work for little gain. Why do you want to do this?!

@edsu edsu closed this as completed Feb 15, 2023
@igorbrigadir
Copy link
Contributor

The mapping from v1.1 API format to v2 API format is here if you want to take a stab at it: https://developer.twitter.com/en/docs/twitter-api/migrate/data-formats/standard-v1-1-to-v2 and in this interactive thing here too: https://visual-data-format-migration-tool.glitch.me/

However, it's infinitely easier to take a v1.1 dataset, dehydrate it, and hydrate it with twarc2:

twarc dehydrate some_dataset_in_v1.jsonl > ids.txt
twarc2 hydrate ids.txt dataset_in_v2.jsonl

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants