Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CLI Errors fail to show for invalid input file #686

Closed
igorbrigadir opened this issue Jan 16, 2023 · 1 comment
Closed

CLI Errors fail to show for invalid input file #686

igorbrigadir opened this issue Jan 16, 2023 · 1 comment
Labels

Comments

@igorbrigadir
Copy link
Contributor

If an invalid input file like input.txt with quotes is used:

"78021221"
"20452058"
"20434358"

The errors are recorded in twarc.log

2023-01-16 00:02:09,705 WARNING invalid username or user id ""78021221"" on line 1
2023-01-16 00:02:09,705 WARNING invalid username or user id ""20452058"" on line 2
2023-01-16 00:02:09,705 WARNING invalid username or user id ""20434358"" on line 3

But in CLI it looks like it succeeded, which is confusing:

twarc2 timelines --use-search input.txt output.jsonl
100%|█████████████| Processed 3/3 lines of input file [00:00<00:00, 3866.91it/s]

This is also related to #561 - if a line in the input fails, it should show an error, or show a summary at the end showing what failed.

Separately, it may be worth supporting quoted / unquoted input files?

@igorbrigadir
Copy link
Contributor Author

oops, somehow this posted twice #685

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant