You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
(I couldn't find anything related after an intensive web search)
I am facing a very confusing issue. I believe the issue started from mrjob v0.6.8 and persists till the latest version. Here is the sample code:
`
from mrjob.job import MRJob
from mrjob.step import MRStep
import sys
`
(Note: don't know why underscores are stripped from above main function declaration)
To run the code, need to create a local directory (e.g., test_input) and put a random small text file in it. When running from a (CentOS) Linux shell in a (Python 3.8.9) virtualenv that has mrjob 0.6.7 installed, it runs fine. However, the code crashes with the following exception when running in a virtualenv that has mrjob 0.6.8 (and beyond) installed (the command line: python test.py test_input):
File "test.bytes.py", line 8, in init_mergedrs
sys.stderr.write('Processing file.\n')
TypeError: a bytes-like object is required, not 'str'
If I comment out the sys.stderr.write (putting a 'pass' statement), the code still crashes locally, but the offending line is now in the 'print(header)' statement (same exception).
The code runs fine remotely on hadoop cluster though (with either mrjob 0.6.7 or 0.6.8 and beyond). Checking v0.6.8 change log doesn't reveal anything that gives any hint. Can anybody help? Thanks.
(The issue also happens in Python 3.7 and 3.9).
The text was updated successfully, but these errors were encountered:
(I couldn't find anything related after an intensive web search)
I am facing a very confusing issue. I believe the issue started from mrjob v0.6.8 and persists till the latest version. Here is the sample code:
`
from mrjob.job import MRJob
from mrjob.step import MRStep
import sys
class MRSp(MRJob):
if name == 'main':
`
(Note: don't know why underscores are stripped from above main function declaration)
To run the code, need to create a local directory (e.g., test_input) and put a random small text file in it. When running from a (CentOS) Linux shell in a (Python 3.8.9) virtualenv that has mrjob 0.6.7 installed, it runs fine. However, the code crashes with the following exception when running in a virtualenv that has mrjob 0.6.8 (and beyond) installed (the command line: python test.py test_input):
File "test.bytes.py", line 8, in init_mergedrs
sys.stderr.write('Processing file.\n')
TypeError: a bytes-like object is required, not 'str'
If I comment out the sys.stderr.write (putting a 'pass' statement), the code still crashes locally, but the offending line is now in the 'print(header)' statement (same exception).
The code runs fine remotely on hadoop cluster though (with either mrjob 0.6.7 or 0.6.8 and beyond). Checking v0.6.8 change log doesn't reveal anything that gives any hint. Can anybody help? Thanks.
(The issue also happens in Python 3.7 and 3.9).
The text was updated successfully, but these errors were encountered: