Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tutorials generate erorrs #3650

Closed
mhpopescu opened this issue Jun 16, 2021 · 1 comment · Fixed by #3668
Closed

Tutorials generate erorrs #3650

mhpopescu opened this issue Jun 16, 2021 · 1 comment · Fixed by #3668

Comments

@mhpopescu
Copy link
Collaborator

mhpopescu commented Jun 16, 2021

Running tutorial_staging.py with Python 3.8.2 I get the following:

	[2021-06-16T14:26:27+0300] [MainThread] [I] [toil.worker] ---TOIL WORKER OUTPUT LOG---
	[2021-06-16T14:26:27+0300] [MainThread] [I] [toil] Running Toil version 5.3.0a1-c4476778aa867caabe7202a84e65697ff7d501eb on host lambda.
	[2021-06-16T14:26:27+0300] [MainThread] [I] [toil.worker] Working on job 'HelloWorld' kind-HelloWorld/instance-1ewqash6
	[2021-06-16T14:26:27+0300] [MainThread] [I] [toil.worker] Loaded body Job('HelloWorld' kind-HelloWorld/instance-1ewqash6) from description 'HelloWorld' kind-HelloWorld/instance-1ewqash6
	Traceback (most recent call last):
	  File "/home/lambda/toil/src/toil/worker.py", line 393, in workerScript
	    job._runner(jobGraph=None, jobStore=jobStore, fileStore=fileStore, defer=defer)
	  File "/home/lambda/toil/src/toil/job.py", line 2360, in _runner
	    returnValues = self._run(jobGraph=None, fileStore=fileStore)
	  File "/home/lambda/toil/src/toil/job.py", line 2281, in _run
	    return self.run(fileStore)
	  File "/home/lambda/toil/examples/tutorial_staging.py", line 13, in run
	    with self.fileStore.readGlobalFileStream(self.inputFileID) as fi:
	AttributeError: 'HelloWorld' object has no attribute 'fileStore'
	[2021-06-16T14:26:27+0300] [MainThread] [E] [toil.worker] Exiting the worker because of a failed job on host lambda

After I replace self.fileStore to fileStore I get:

	[2021-06-16T14:33:04+0300] [MainThread] [I] [toil.worker] ---TOIL WORKER OUTPUT LOG---
	[2021-06-16T14:33:04+0300] [MainThread] [I] [toil] Running Toil version 5.3.0a1-c4476778aa867caabe7202a84e65697ff7d501eb on host lambda.
	[2021-06-16T14:33:04+0300] [MainThread] [I] [toil.worker] Working on job 'HelloWorld' kind-HelloWorld/instance-vdzzyf07
	[2021-06-16T14:33:04+0300] [MainThread] [I] [toil.worker] Loaded body Job('HelloWorld' kind-HelloWorld/instance-vdzzyf07) from description 'HelloWorld' kind-HelloWorld/instance-vdzzyf07
	[2021-06-16T14:33:04+0300] [MainThread] [W] [toil.fileStores.abstractFileStore] Failed job accessed files:
	[2021-06-16T14:33:04+0300] [MainThread] [W] [toil.fileStores.abstractFileStore] Streamed file 'files/no-job/file-eae1da63fd154f34a70f68b2c2d90609/in.txt'
	Traceback (most recent call last):
	  File "/home/lambda/toil/src/toil/worker.py", line 393, in workerScript
	    job._runner(jobGraph=None, jobStore=jobStore, fileStore=fileStore, defer=defer)
	  File "/home/lambda/toil/src/toil/job.py", line 2360, in _runner
	    returnValues = self._run(jobGraph=None, fileStore=fileStore)
	  File "/home/lambda/toil/src/toil/job.py", line 2281, in _run
	    return self.run(fileStore)
	  File "/home/lambda/toil/examples/tutorial_staging.py", line 15, in run
	    fo.write(fi.read() + 'World!')
	TypeError: can't concat str to bytes
	[2021-06-16T14:33:04+0300] [MainThread] [E] [toil.worker] Exiting the worker because of a failed job on host lambda

The problem is that default encoding is binary and here it should have used a string encoding.

Note that self.fileStore is used in more tutorial files that this one.

┆Issue is synchronized with this Jira Task
┆Issue Number: TOIL-924

@DailyDreaming
Copy link
Member

@mhpopescu That's odd. Generally our tutorials are actually under testing: https://github.com/DataBiosphere/toil/blob/master/src/toil/test/docs/scriptsTest.py

But tutorial_staging.py appears to be missing. Will fix it up and add it. Thanks for raising the issue!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants