Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

__init__ fails in Python 3.5 #7

Closed
cidermole opened this issue Feb 2, 2016 · 10 comments
Closed

__init__ fails in Python 3.5 #7

cidermole opened this issue Feb 2, 2016 · 10 comments

Comments

@cidermole
Copy link

/usr/lib/python3.5/site-packages/bashlex/init.py in ()
----> 1 import parser, tokenizer
ImportError: No module named 'tokenizer'
@idank
Copy link
Owner

idank commented Feb 2, 2016

That's disappointing, but unless someone comes along and has the burning desire to make bashlex work with 3.x, I don't see this changing.

@idank idank closed this as completed Feb 2, 2016
@blurrymoi
Copy link
Contributor

Would you mind re-opening this?

@idank
Copy link
Owner

idank commented Feb 20, 2016

Feel free to send a patch. I'm generally opposed to keeping things open if
I have no intention of fixing them.
On Feb 20, 2016 05:54, "blurrymoi" notifications@github.com wrote:

Would you mind re-opening this?

—
Reply to this email directly or view it on GitHub
#7 (comment).

@blurrymoi
Copy link
Contributor

Oh, okay.

@keheliya keheliya mentioned this issue Sep 26, 2016
Closed
idank pushed a commit that referenced this issue Sep 26, 2016
@keheliya @idank
fix python 2/3 compatibility (fixes #7)
0e28c99
@josephfrazier
Copy link
Contributor

Hi @idank, would you mind releasing a new version of bashlex to pypi now that 0e28c99 has landed? I'm trying to use it in another project that requires both python2 and python3 compatibility. Thanks for all your work on this!

@idank
Copy link
Owner

idank commented Nov 17, 2016

Done! :)

@josephfrazier
Copy link
Contributor

Thanks for the quick response! I'm actually having a little trouble getting it to work under Python 3, but I'll make a separate issue for that.

@blurrymoi
Copy link
Contributor

I may have a dubious solution for that.

@josephfrazier
Copy link
Contributor

josephfrazier commented Nov 17, 2016 •
edited

@blurrymoi, the trouble I'm having is with getting the tests to pass. If you check out the code from #14 and run the following, you should see several failures:

python3 tests/test-parser.py
python3 tests/test-tokenizer.py

Is that the problem you're experiencing too, or something else?

EDIT: Here are the outputs:

python3 tests/test-parser.py
tests/test-parser.py:103: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(result, expected, msg)
F.tests/test-parser.py:999: DeprecationWarning: Please use assertRaisesRegex instead.
  parse, 'a "$((2 + 2))"')
.EEEE..EEEE.EEE..EEE...FEE..FE.EE
======================================================================
ERROR: test_command_substitution_dollar_semicolon (__main__.test_parser)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/test-parser.py", line 1076, in test_command_substitution_dollar_semicolon
    commandnode('b', wordnode('b'),),
  File "tests/test-parser.py", line 90, in assertASTEquals
    results = parse(s, **parserargs)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 583, in parse
    parts = [p.parse()]
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 642, in parse
    tree = theparser.parse(lexer=self.tok, context=self)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 277, in parse
    return self.parseopt_notrack(input,lexer,debug,tracking,tokenfunc,context)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 998, in parseopt_notrack
    p.callable(pslice)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 157, in p_simple_command_element
    p[0] = [_expandword(parserobj, p.slice[1])]
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 137, in _expandword
    doublequoted, 0, 0)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 271, in _expandwordinternal
    node, sindex[0] = _paramexpand(parserobj, string, sindex[0])
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 165, in _paramexpand
    return _extractcommandsubst(parserobj, string, zindex + 1)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 55, in _extractcommandsubst
    node, si = _parsedolparen(parserobj, string, sindex)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 42, in _parsedolparen
    node, endp = _recursiveparse(parserobj, base, sindex, tokenizerargs)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 23, in _recursiveparse
    node = p.parse()
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 642, in parse
    tree = theparser.parse(lexer=self.tok, context=self)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 277, in parse
    return self.parseopt_notrack(input,lexer,debug,tracking,tokenfunc,context)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 1079, in parseopt_notrack
    tok = self.errorfunc(errtoken)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 539, in p_error
    p.lexer.source, p.lexpos)
bashlex.errors.ParsingError: unexpected token ')' (position 3)

======================================================================
ERROR: test_compound (__main__.test_parser)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/test-parser.py", line 359, in test_compound
    reservedwordnode(')', ')'),
  File "tests/test-parser.py", line 90, in assertASTEquals
    results = parse(s, **parserargs)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 583, in parse
    parts = [p.parse()]
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 642, in parse
    tree = theparser.parse(lexer=self.tok, context=self)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 277, in parse
    return self.parseopt_notrack(input,lexer,debug,tracking,tokenfunc,context)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 1002, in parseopt_notrack
    state = goto[statestack[-1]][pname]
KeyError: 'simple_list1'

======================================================================
ERROR: test_compound_pipe (__main__.test_parser)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/test-parser.py", line 450, in test_compound_pipe
    wordnode('b'))
  File "tests/test-parser.py", line 90, in assertASTEquals
    results = parse(s, **parserargs)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 583, in parse
    parts = [p.parse()]
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 642, in parse
    tree = theparser.parse(lexer=self.tok, context=self)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 277, in parse
    return self.parseopt_notrack(input,lexer,debug,tracking,tokenfunc,context)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 1002, in parseopt_notrack
    state = goto[statestack[-1]][pname]
KeyError: 'simple_list1'

======================================================================
ERROR: test_compound_redirection (__main__.test_parser)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/test-parser.py", line 435, in test_compound_redirection
    redirects=[redirectnode('> /dev/null', None, '>', wordnode('/dev/null'))]
  File "tests/test-parser.py", line 90, in assertASTEquals
    results = parse(s, **parserargs)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 583, in parse
    parts = [p.parse()]
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 642, in parse
    tree = theparser.parse(lexer=self.tok, context=self)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 277, in parse
    return self.parseopt_notrack(input,lexer,debug,tracking,tokenfunc,context)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 1002, in parseopt_notrack
    state = goto[statestack[-1]][pname]
KeyError: 'simple_list1'

======================================================================
ERROR: test_expansion_limit (__main__.test_parser)
make sure the expansion limit is working by tracking recursive
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/test-parser.py", line 940, in test_expansion_limit
    expansionlimit=1
  File "tests/test-parser.py", line 90, in assertASTEquals
    results = parse(s, **parserargs)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 583, in parse
    parts = [p.parse()]
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 642, in parse
    tree = theparser.parse(lexer=self.tok, context=self)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 277, in parse
    return self.parseopt_notrack(input,lexer,debug,tracking,tokenfunc,context)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 998, in parseopt_notrack
    p.callable(pslice)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 157, in p_simple_command_element
    p[0] = [_expandword(parserobj, p.slice[1])]
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 137, in _expandword
    doublequoted, 0, 0)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 271, in _expandwordinternal
    node, sindex[0] = _paramexpand(parserobj, string, sindex[0])
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 165, in _paramexpand
    return _extractcommandsubst(parserobj, string, zindex + 1)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 55, in _extractcommandsubst
    node, si = _parsedolparen(parserobj, string, sindex)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 42, in _parsedolparen
    node, endp = _recursiveparse(parserobj, base, sindex, tokenizerargs)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 23, in _recursiveparse
    node = p.parse()
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 642, in parse
    tree = theparser.parse(lexer=self.tok, context=self)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 277, in parse
    return self.parseopt_notrack(input,lexer,debug,tracking,tokenfunc,context)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 998, in parseopt_notrack
    p.callable(pslice)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 157, in p_simple_command_element
    p[0] = [_expandword(parserobj, p.slice[1])]
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 137, in _expandword
    doublequoted, 0, 0)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 271, in _expandwordinternal
    node, sindex[0] = _paramexpand(parserobj, string, sindex[0])
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 165, in _paramexpand
    return _extractcommandsubst(parserobj, string, zindex + 1)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 55, in _extractcommandsubst
    node, si = _parsedolparen(parserobj, string, sindex)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 42, in _parsedolparen
    node, endp = _recursiveparse(parserobj, base, sindex, tokenizerargs)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 23, in _recursiveparse
    node = p.parse()
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 642, in parse
    tree = theparser.parse(lexer=self.tok, context=self)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 277, in parse
    return self.parseopt_notrack(input,lexer,debug,tracking,tokenfunc,context)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 1079, in parseopt_notrack
    tok = self.errorfunc(errtoken)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 539, in p_error
    p.lexer.source, p.lexpos)
bashlex.errors.ParsingError: unexpected token ')' (position 11)

======================================================================
ERROR: test_expansion_limit_word (__main__.test_parser)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/test-parser.py", line 982, in test_expansion_limit_word
    parameternode('1', '$1'),
  File "tests/test-parser.py", line 90, in assertASTEquals
    results = parse(s, **parserargs)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 583, in parse
    parts = [p.parse()]
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 642, in parse
    tree = theparser.parse(lexer=self.tok, context=self)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 277, in parse
    return self.parseopt_notrack(input,lexer,debug,tracking,tokenfunc,context)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 998, in parseopt_notrack
    p.callable(pslice)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 157, in p_simple_command_element
    p[0] = [_expandword(parserobj, p.slice[1])]
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 137, in _expandword
    doublequoted, 0, 0)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 271, in _expandwordinternal
    node, sindex[0] = _paramexpand(parserobj, string, sindex[0])
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 165, in _paramexpand
    return _extractcommandsubst(parserobj, string, zindex + 1)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 55, in _extractcommandsubst
    node, si = _parsedolparen(parserobj, string, sindex)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 42, in _parsedolparen
    node, endp = _recursiveparse(parserobj, base, sindex, tokenizerargs)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 23, in _recursiveparse
    node = p.parse()
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 642, in parse
    tree = theparser.parse(lexer=self.tok, context=self)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 277, in parse
    return self.parseopt_notrack(input,lexer,debug,tracking,tokenfunc,context)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 1079, in parseopt_notrack
    tok = self.errorfunc(errtoken)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 539, in p_error
    p.lexer.source, p.lexpos)
bashlex.errors.ParsingError: unexpected token ')' (position 1)

======================================================================
ERROR: test_for (__main__.test_parser)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/test-parser.py", line 825, in test_for
    reservedwordnode('done', 'done'),
  File "tests/test-parser.py", line 90, in assertASTEquals
    results = parse(s, **parserargs)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 583, in parse
    parts = [p.parse()]
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 642, in parse
    tree = theparser.parse(lexer=self.tok, context=self)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 277, in parse
    return self.parseopt_notrack(input,lexer,debug,tracking,tokenfunc,context)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 1079, in parseopt_notrack
    tok = self.errorfunc(errtoken)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 539, in p_error
    p.lexer.source, p.lexpos)
bashlex.errors.ParsingError: unexpected token 'do' (position 7)

======================================================================
ERROR: test_for_expansion (__main__.test_parser)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/test-parser.py", line 809, in test_for_expansion
    reservedwordnode('done', 'done'),
  File "tests/test-parser.py", line 90, in assertASTEquals
    results = parse(s, **parserargs)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 583, in parse
    parts = [p.parse()]
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 642, in parse
    tree = theparser.parse(lexer=self.tok, context=self)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 277, in parse
    return self.parseopt_notrack(input,lexer,debug,tracking,tokenfunc,context)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 998, in parseopt_notrack
    p.callable(pslice)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 157, in p_simple_command_element
    p[0] = [_expandword(parserobj, p.slice[1])]
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 137, in _expandword
    doublequoted, 0, 0)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 271, in _expandwordinternal
    node, sindex[0] = _paramexpand(parserobj, string, sindex[0])
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 165, in _paramexpand
    return _extractcommandsubst(parserobj, string, zindex + 1)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 55, in _extractcommandsubst
    node, si = _parsedolparen(parserobj, string, sindex)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 42, in _parsedolparen
    node, endp = _recursiveparse(parserobj, base, sindex, tokenizerargs)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 23, in _recursiveparse
    node = p.parse()
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 642, in parse
    tree = theparser.parse(lexer=self.tok, context=self)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 277, in parse
    return self.parseopt_notrack(input,lexer,debug,tracking,tokenfunc,context)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 1079, in parseopt_notrack
    tok = self.errorfunc(errtoken)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 539, in p_error
    p.lexer.source, p.lexpos)
bashlex.errors.ParsingError: unexpected token ')' (position 1)

======================================================================
ERROR: test_function_parenthesis_optional (__main__.test_parser)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/test-parser.py", line 1059, in test_function_parenthesis_optional
    body
  File "tests/test-parser.py", line 90, in assertASTEquals
    results = parse(s, **parserargs)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 583, in parse
    parts = [p.parse()]
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 642, in parse
    tree = theparser.parse(lexer=self.tok, context=self)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 277, in parse
    return self.parseopt_notrack(input,lexer,debug,tracking,tokenfunc,context)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 1079, in parseopt_notrack
    tok = self.errorfunc(errtoken)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 539, in p_error
    p.lexer.source, p.lexpos)
bashlex.errors.ParsingError: unexpected token '}' (position 16)

======================================================================
ERROR: test_function_with_keyword (__main__.test_parser)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/test-parser.py", line 1040, in test_function_with_keyword
    body
  File "tests/test-parser.py", line 90, in assertASTEquals
    results = parse(s, **parserargs)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 583, in parse
    parts = [p.parse()]
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 642, in parse
    tree = theparser.parse(lexer=self.tok, context=self)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 277, in parse
    return self.parseopt_notrack(input,lexer,debug,tracking,tokenfunc,context)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 1079, in parseopt_notrack
    tok = self.errorfunc(errtoken)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 539, in p_error
    p.lexer.source, p.lexpos)
bashlex.errors.ParsingError: unexpected token '(' (position 10)

======================================================================
ERROR: test_group (__main__.test_parser)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/test-parser.py", line 476, in test_group
    reservedwordnode('}', '}'),
  File "tests/test-parser.py", line 90, in assertASTEquals
    results = parse(s, **parserargs)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 583, in parse
    parts = [p.parse()]
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 642, in parse
    tree = theparser.parse(lexer=self.tok, context=self)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 277, in parse
    return self.parseopt_notrack(input,lexer,debug,tracking,tokenfunc,context)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 1079, in parseopt_notrack
    tok = self.errorfunc(errtoken)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 539, in p_error
    p.lexer.source, p.lexpos)
bashlex.errors.ParsingError: unexpected token '}' (position 5)

======================================================================
ERROR: test_herestring (__main__.test_parser)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/test-parser.py", line 784, in test_herestring
    wordnode('b'))
  File "tests/test-parser.py", line 90, in assertASTEquals
    results = parse(s, **parserargs)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 583, in parse
    parts = [p.parse()]
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 642, in parse
    tree = theparser.parse(lexer=self.tok, context=self)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 277, in parse
    return self.parseopt_notrack(input,lexer,debug,tracking,tokenfunc,context)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 998, in parseopt_notrack
    p.callable(pslice)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 107, in p_redirection
    output = _expandword(parserobj, p.slice[2])
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 137, in _expandword
    doublequoted, 0, 0)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 271, in _expandwordinternal
    node, sindex[0] = _paramexpand(parserobj, string, sindex[0])
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 165, in _paramexpand
    return _extractcommandsubst(parserobj, string, zindex + 1)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 55, in _extractcommandsubst
    node, si = _parsedolparen(parserobj, string, sindex)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 42, in _parsedolparen
    node, endp = _recursiveparse(parserobj, base, sindex, tokenizerargs)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 23, in _recursiveparse
    node = p.parse()
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 642, in parse
    tree = theparser.parse(lexer=self.tok, context=self)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 277, in parse
    return self.parseopt_notrack(input,lexer,debug,tracking,tokenfunc,context)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 1079, in parseopt_notrack
    tok = self.errorfunc(errtoken)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 539, in p_error
    p.lexer.source, p.lexpos)
bashlex.errors.ParsingError: unexpected token ')' (position 1)

======================================================================
ERROR: test_if (__main__.test_parser)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/test-parser.py", line 574, in test_if
    reservedwordnode('fi', 'fi'),
  File "tests/test-parser.py", line 90, in assertASTEquals
    results = parse(s, **parserargs)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 583, in parse
    parts = [p.parse()]
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 642, in parse
    tree = theparser.parse(lexer=self.tok, context=self)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 277, in parse
    return self.parseopt_notrack(input,lexer,debug,tracking,tokenfunc,context)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 1079, in parseopt_notrack
    tok = self.errorfunc(errtoken)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 539, in p_error
    p.lexer.source, p.lexpos)
bashlex.errors.ParsingError: unexpected token 'then' (position 8)

======================================================================
ERROR: test_if_redirection (__main__.test_parser)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/test-parser.py", line 557, in test_if_redirection
    wordnode('/dev/null'))
  File "tests/test-parser.py", line 90, in assertASTEquals
    results = parse(s, **parserargs)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 583, in parse
    parts = [p.parse()]
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 642, in parse
    tree = theparser.parse(lexer=self.tok, context=self)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 277, in parse
    return self.parseopt_notrack(input,lexer,debug,tracking,tokenfunc,context)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 1079, in parseopt_notrack
    tok = self.errorfunc(errtoken)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 539, in p_error
    p.lexer.source, p.lexpos)
bashlex.errors.ParsingError: unexpected token 'then' (position 8)

======================================================================
ERROR: test_multiline (__main__.test_parser)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/test-parser.py", line 151, in test_multiline
    wordnode('b'))
  File "tests/test-parser.py", line 106, in assertASTsEquals
    results = parse(s, **parserargs)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 583, in parse
    parts = [p.parse()]
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 642, in parse
    tree = theparser.parse(lexer=self.tok, context=self)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 277, in parse
    return self.parseopt_notrack(input,lexer,debug,tracking,tokenfunc,context)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 1079, in parseopt_notrack
    tok = self.errorfunc(errtoken)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 539, in p_error
    p.lexer.source, p.lexpos)
bashlex.errors.ParsingError: unexpected token 'b' (position 2)

======================================================================
ERROR: test_nestedsubs (__main__.test_parser)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/test-parser.py", line 226, in test_nestedsubs
    wordnode('b'),
  File "tests/test-parser.py", line 90, in assertASTEquals
    results = parse(s, **parserargs)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 583, in parse
    parts = [p.parse()]
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 642, in parse
    tree = theparser.parse(lexer=self.tok, context=self)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 277, in parse
    return self.parseopt_notrack(input,lexer,debug,tracking,tokenfunc,context)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 998, in parseopt_notrack
    p.callable(pslice)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 157, in p_simple_command_element
    p[0] = [_expandword(parserobj, p.slice[1])]
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 137, in _expandword
    doublequoted, 0, 0)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 271, in _expandwordinternal
    node, sindex[0] = _paramexpand(parserobj, string, sindex[0])
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 165, in _paramexpand
    return _extractcommandsubst(parserobj, string, zindex + 1)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 55, in _extractcommandsubst
    node, si = _parsedolparen(parserobj, string, sindex)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 42, in _parsedolparen
    node, endp = _recursiveparse(parserobj, base, sindex, tokenizerargs)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 23, in _recursiveparse
    node = p.parse()
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 642, in parse
    tree = theparser.parse(lexer=self.tok, context=self)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 277, in parse
    return self.parseopt_notrack(input,lexer,debug,tracking,tokenfunc,context)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 998, in parseopt_notrack
    p.callable(pslice)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 107, in p_redirection
    output = _expandword(parserobj, p.slice[2])
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 137, in _expandword
    doublequoted, 0, 0)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 271, in _expandwordinternal
    node, sindex[0] = _paramexpand(parserobj, string, sindex[0])
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 165, in _paramexpand
    return _extractcommandsubst(parserobj, string, zindex + 1)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 55, in _extractcommandsubst
    node, si = _parsedolparen(parserobj, string, sindex)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 42, in _parsedolparen
    node, endp = _recursiveparse(parserobj, base, sindex, tokenizerargs)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 23, in _recursiveparse
    node = p.parse()
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 642, in parse
    tree = theparser.parse(lexer=self.tok, context=self)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 277, in parse
    return self.parseopt_notrack(input,lexer,debug,tracking,tokenfunc,context)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 1079, in parseopt_notrack
    tok = self.errorfunc(errtoken)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 539, in p_error
    p.lexer.source, p.lexpos)
bashlex.errors.ParsingError: unexpected token ')' (position 1)

======================================================================
ERROR: test_processsub (__main__.test_parser)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/test-parser.py", line 273, in test_processsub
    wordnode('c'))
  File "tests/test-parser.py", line 90, in assertASTEquals
    results = parse(s, **parserargs)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 583, in parse
    parts = [p.parse()]
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 642, in parse
    tree = theparser.parse(lexer=self.tok, context=self)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 277, in parse
    return self.parseopt_notrack(input,lexer,debug,tracking,tokenfunc,context)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 998, in parseopt_notrack
    p.callable(pslice)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 157, in p_simple_command_element
    p[0] = [_expandword(parserobj, p.slice[1])]
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 137, in _expandword
    doublequoted, 0, 0)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 225, in _expandwordinternal
    node, sindex[0] = _extractprocesssubst(parserobj, string, tindex)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 61, in _extractprocesssubst
    node, si = _parsedolparen(parserobj, string, sindex)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 42, in _parsedolparen
    node, endp = _recursiveparse(parserobj, base, sindex, tokenizerargs)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 23, in _recursiveparse
    node = p.parse()
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 642, in parse
    tree = theparser.parse(lexer=self.tok, context=self)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 277, in parse
    return self.parseopt_notrack(input,lexer,debug,tracking,tokenfunc,context)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 998, in parseopt_notrack
    p.callable(pslice)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 157, in p_simple_command_element
    p[0] = [_expandword(parserobj, p.slice[1])]
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 137, in _expandword
    doublequoted, 0, 0)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 271, in _expandwordinternal
    node, sindex[0] = _paramexpand(parserobj, string, sindex[0])
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 165, in _paramexpand
    return _extractcommandsubst(parserobj, string, zindex + 1)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 55, in _extractcommandsubst
    node, si = _parsedolparen(parserobj, string, sindex)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 42, in _parsedolparen
    node, endp = _recursiveparse(parserobj, base, sindex, tokenizerargs)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 23, in _recursiveparse
    node = p.parse()
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 642, in parse
    tree = theparser.parse(lexer=self.tok, context=self)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 277, in parse
    return self.parseopt_notrack(input,lexer,debug,tracking,tokenfunc,context)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 1079, in parseopt_notrack
    tok = self.errorfunc(errtoken)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 539, in p_error
    p.lexer.source, p.lexpos)
bashlex.errors.ParsingError: unexpected token ')' (position 1)

======================================================================
ERROR: test_while (__main__.test_parser)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/test-parser.py", line 910, in test_while
    reservedwordnode('done', 'done'),
  File "tests/test-parser.py", line 90, in assertASTEquals
    results = parse(s, **parserargs)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 583, in parse
    parts = [p.parse()]
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 642, in parse
    tree = theparser.parse(lexer=self.tok, context=self)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 277, in parse
    return self.parseopt_notrack(input,lexer,debug,tracking,tokenfunc,context)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 1079, in parseopt_notrack
    tok = self.errorfunc(errtoken)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 539, in p_error
    p.lexer.source, p.lexpos)
bashlex.errors.ParsingError: unexpected token 'do' (position 9)

======================================================================
ERROR: test_word_expansion (__main__.test_parser)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/test-parser.py", line 702, in test_word_expansion
    commandnode("b", wordnode("b"))
  File "tests/test-parser.py", line 90, in assertASTEquals
    results = parse(s, **parserargs)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 583, in parse
    parts = [p.parse()]
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 642, in parse
    tree = theparser.parse(lexer=self.tok, context=self)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 277, in parse
    return self.parseopt_notrack(input,lexer,debug,tracking,tokenfunc,context)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 998, in parseopt_notrack
    p.callable(pslice)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 157, in p_simple_command_element
    p[0] = [_expandword(parserobj, p.slice[1])]
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 137, in _expandword
    doublequoted, 0, 0)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 271, in _expandwordinternal
    node, sindex[0] = _paramexpand(parserobj, string, sindex[0])
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 165, in _paramexpand
    return _extractcommandsubst(parserobj, string, zindex + 1)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 55, in _extractcommandsubst
    node, si = _parsedolparen(parserobj, string, sindex)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 42, in _parsedolparen
    node, endp = _recursiveparse(parserobj, base, sindex, tokenizerargs)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/subst.py", line 23, in _recursiveparse
    node = p.parse()
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 642, in parse
    tree = theparser.parse(lexer=self.tok, context=self)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 277, in parse
    return self.parseopt_notrack(input,lexer,debug,tracking,tokenfunc,context)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/yacc.py", line 1079, in parseopt_notrack
    tok = self.errorfunc(errtoken)
  File "/Users/josephfrazier/workspace/bashlex/bashlex/parser.py", line 539, in p_error
    p.lexer.source, p.lexpos)
bashlex.errors.ParsingError: unexpected token ')' (position 1)

======================================================================
FAIL: test_assignments (__main__.test_parser)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/test-parser.py", line 855, in test_assignments
    wordnode('e=d'),
  File "tests/test-parser.py", line 103, in assertASTEquals
    self.assertEquals(result, expected, msg)
AssertionError: CommandNode(parts=[WordNode(parts=[] s='a=b' word='a=b'), Wo[81 chars]e=d') != CommandNode(parts=[AssignmentNode(parts=[] s='a=b' word='a=b[87 chars]e=d') : ASTs not equal for 'a=b c e=d'

result:

CommandNode(s='a=b c e=d', parts=[
  WordNode(s='a=b', word='a=b'),
  WordNode(s='c', word='c'),
  WordNode(s='e=d', word='e=d'),
])

!=

expected:

CommandNode(s='a=b c e=d', parts=[
  AssignmentNode(s='a=b', word='a=b'),
  WordNode(s='c', word='c'),
  WordNode(s='e=d', word='e=d'),
])

======================================================================
FAIL: test_malformed_if (__main__.test_parser)
----------------------------------------------------------------------
bashlex.errors.ParsingError: unexpected token 'then' (position 8)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "tests/test-parser.py", line 673, in test_malformed_if
    self.assertRaisesRegexp(errors.ParsingError, "unexpected EOF.*position 17", parse, s)
AssertionError: "unexpected EOF.*position 17" does not match "unexpected token 'then' (position 8)"

======================================================================
FAIL: test_pipeline (__main__.test_parser)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/test-parser.py", line 168, in test_pipeline
    commandnode('b', wordnode('b'))
  File "tests/test-parser.py", line 103, in assertASTEquals
    self.assertEquals(result, expected, msg)
AssertionError: PipelineNode(parts=[CommandNode(parts=[WordNode(parts=[] s='![153 chars]| b') != PipelineNode(parts=[ReservedwordNode(s='!' word='!'), Command[150 chars]| b') : ASTs not equal for '! a | b'

result:

PipelineNode(s='! a | b', parts=[
  CommandNode(s='! a', parts=[
    WordNode(s='!', word='!'),
    WordNode(s='a', word='a'),
  ]),
  PipeNode(s='|', pipe='|'),
  CommandNode(s='b', parts=[
    WordNode(s='b', word='b'),
  ]),
])

!=

expected:

PipelineNode(s='! a | b', parts=[
  ReservedwordNode(s='!', word='!'),
  CommandNode(s='a', parts=[
    WordNode(s='a', word='a'),
  ]),
  PipeNode(s='|', pipe='|'),
  CommandNode(s='b', parts=[
    WordNode(s='b', word='b'),
  ]),
])

----------------------------------------------------------------------
Ran 35 tests in 0.040s

FAILED (failures=3, errors=19)
python3 tests/test-tokenizer.py
tests/test-tokenizer.py:22: DeprecationWarning: Please use assertEqual instead.
  self.assertEquals(result, tokens)
F......F...tests/test-tokenizer.py:299: DeprecationWarning: Please use assertRaisesRegex instead.
  self.assertRaisesRegexp(errors.ParsingError, msg, tokenize, s)
.......
======================================================================
FAIL: test_assignment (__main__.test_tokenizer)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/test-tokenizer.py", line 246, in test_assignment
    flags=set([flags.word.NOSPLIT, flags.word.ASSIGNMENT]))])
  File "tests/test-tokenizer.py", line 22, in assertTokens
    self.assertEquals(result, tokens)
AssertionError: Lists differ: [<WORD@0:3 'a=b' (ASSIGNMENT)>] != [<ASSIGNMENT_WORD@0:3 'a=b' (ASSIGNMENT NOSPLIT)>]

First differing element 0:
<WORD@0:3 'a=b' (ASSIGNMENT)>
<ASSIGNMENT_WORD@0:3 'a=b' (ASSIGNMENT NOSPLIT)>

- [<WORD@0:3 'a=b' (ASSIGNMENT)>]
+ [<ASSIGNMENT_WORD@0:3 'a=b' (ASSIGNMENT NOSPLIT)>]
?   +++++++++++                          ++++++++


======================================================================
FAIL: test_meta (__main__.test_tokenizer)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "tests/test-tokenizer.py", line 57, in test_meta
    t(tt.BAR_AND, '|&', [44, 46])])
  File "tests/test-tokenizer.py", line 22, in assertTokens
    self.assertEquals(result, tokens)
AssertionError: Lists differ: [<WORD@0:1 '!'>, <AMPERSAND@1:2 '&'>, <LEFT[491 chars]|&'>] != [<BANG@0:1 '!'>, <AMPERSAND@1:2 '&'>, <LEFT[491 chars]|&'>]

First differing element 0:
<WORD@0:1 '!'>
<BANG@0:1 '!'>

Diff is 644 characters long. Set self.maxDiff to None to see it.

----------------------------------------------------------------------
Ran 18 tests in 0.010s

FAILED (failures=2)

blurrymoi added a commit to blurrymoi/bashlex that referenced this issue Nov 18, 2016
@blurrymoi
add full support for both Py2 and Py3 (ref idank#7)
bcb4b29 
* range returns an iterator instead of a list in Py3
* __nonzero__ in Py3 is __bool__
* fix 'hack' table to derive states dynamically
@blurrymoi
Copy link
Contributor

I provided my solution for full compatibility.

idank pushed a commit that referenced this issue Nov 22, 2016
@blurrymoi @idank
parser: fix 'hack' table to derive states dynamically (fixes #7)
da9429e 
the states which need fixing by the 'hack' are different for different Python
versions and even differ on Windows vs Linux. Derive them dynamically
from the state 0, this makes the test suite pass on Python 3.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix python2/3 compatibility keheliya/bashlex
4 participants
@cidermole @idank @josephfrazier @blurrymoi