Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reading from stdin should use single thread or report an error #62

Open
ehiggs opened this issue May 24, 2017 · 1 comment
Open

Reading from stdin should use single thread or report an error #62

ehiggs opened this issue May 24, 2017 · 1 comment

Comments

@ehiggs
Copy link

ehiggs commented May 24, 2017

When using multiple threads to read from stdin, paratext fails. It works fine with actual files or with a single thread reading stdin.

The code I use is as follows:

#!/usr/bin/env python
import paratext
print sum(map(lambda x: len(x[1]), paratext.load_raw_csv("/dev/stdin",
    no_header=True, allow_quoted_newlines=True)))
$ python2/csvreader-paratext.py < /tmp/hello.csv
Traceback (most recent call last):
  File "python2/csvreader-paratext.py", line 4, in <module>
    allow_quoted_newlines=True)])
  File "/home/ehiggs/.virtualenvs/paratext/lib/python2.7/site-packages/paratext/core.py", line 271, in load_raw_csv
    loader = internal_create_csv_loader(filename, *args, **kwargs)
  File "/home/ehiggs/.virtualenvs/paratext/lib/python2.7/site-packages/paratext/core.py", line 161, in internal_create_csv_loader
    loader.load(_make_posix_filename(filename), params)
  File "/home/ehiggs/.virtualenvs/paratext/lib/python2.7/site-packages/paratext_internal.py", line 414, in load
    return _paratext_internal.ColBasedLoader_load(self, filename, params)
RuntimeError: The file ends with an open quote (4506147)

This could also be an issue if someone were to read in an actual file by name but pass in /dev/stdin.

@ehiggs
Copy link
Author

ehiggs commented May 15, 2019

This is also an issue if you use something like myscript <(grep interesting-line file.csv)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant