Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Q: how to pass a string as stdin to Runners' run()? #7145

Closed
adswa opened this issue Nov 7, 2022 · 5 comments · Fixed by #7155
Closed

Q: how to pass a string as stdin to Runners' run()? #7145

adswa opened this issue Nov 7, 2022 · 5 comments · Fixed by #7155
Labels
severity-normal standard severity team-runner sub-process execution and IO (https://github.com/datalad/datalad/issues/6365)

Comments

@adswa
Copy link
Member

adswa commented Nov 7, 2022

The docstring of the Runner's run method state the following about the stdin= parameter:

stdin : file-like, string, bytes, Queue, or None
If stdin is a file-like, it will be directly used as stdin for the
subprocess. The caller is responsible for writing to it and closing it.
If stdin is a string or bytes, those will be fed to stdin of the
subprocess. If all data is written, stdin will be closed.
If stdin is a Queue, all elements (bytes) put into the Queue will
be passed to stdin until None is read from the queue. If None is read,
stdin of the subprocess is closed.

However, if I provide a string to it (I tried toying around in the context of #6514), I get

Traceback (most recent call last):
  File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/home/adina/repos/datalad/datalad/runner/runnerthreads.py", line 172, in run
    succeeded = self.write(data)
  File "/home/adina/repos/datalad/datalad/runner/runnerthreads.py", line 246, in write
    written += os.write(
TypeError: a bytes-like object is required, not 'str'

I feel like I might be missing something fairly obvious. How can I provide a string as stdin to a runner.run() call?

@adswa adswa changed the title Non-bytelike object as stdin to Runners run() Q: how to pass a string as stdin to Runners' run()? Nov 7, 2022
@mih
Copy link
Member

mih commented Nov 7, 2022

It seems the documentation is wrong, it must be bytes. I can only find usage of this feature where bytes are used specifically.

"somestr".encode('utf-8')

@bpoldrack bpoldrack added severity-normal standard severity team-runner sub-process execution and IO (https://github.com/datalad/datalad/issues/6365) labels Nov 7, 2022
@bpoldrack
Copy link
Member

ATM, I consider this a bug. Runner explicitly checks whether stdin is str or bytes and proceeds to put it in the queue either way. I guess that's where it should be encoded in case of str.

@christian-monch
Copy link
Contributor

Maybe we should change the description and the check and only allow bytes-input. That would force the user to explicitly encode the string (instead of us encoding it with a fixed encoding, or the system-preferred encoding, or the encoding from the protocol keyword arguments)

@bpoldrack
Copy link
Member

instead of us encoding it with a fixed encoding, or the system-preferred encoding, or ...

Seems like a good point. Will change #7155 accordingly.

bpoldrack added a commit to bpoldrack/datalad that referenced this issue Nov 9, 2022
adswa pushed a commit to adswa/datalad that referenced this issue Nov 17, 2022
@yarikoptic-gitmate
Copy link
Collaborator

Issue fixed in 0.17.10

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
severity-normal standard severity team-runner sub-process execution and IO (https://github.com/datalad/datalad/issues/6365)
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants