TypeError in lex

[rmfitzpatrick]

With the 3.6 update, I see the following TypeError where slimit makes a call to ply:

line 893, in lex
    if '.' not in lextab:
TypeError: argument of type 'module' is not iterable

[dabeaz]

Would need more details. The lextab and tabmodule arguments to PLY really are meant to be strings containing the names of the lextab and parsetab modules. I quickly looked at the source code to slimit and don't see where it is doing anything different. Maybe I'm missing something.

[byyo]

This line in slimit ends up passing the module itself instead of the string down to the lex module: https://github.com/rspivak/slimit/blob/master/src/slimit/parser.py#L56

which calls https://github.com/rspivak/slimit/blob/master/src/slimit/lexer.py#L102

[dabeaz]

Looking at it. Hmmm. I'll be honest, I'm not super keen on supporting this passing of the bare module objects like that (although apparently past versions of PLY allowed it for some reason). Let me ponder it further.

[byyo]

IMHO if you don't want to support handling the bare module the version number should bump up to 4.0, since this is technically an API change

[NormalRa]

Pepper Plugin API where its IDL lexer makes a call also exhibits the same a similar issue. This may cause building Android to fail.

[dabeaz]

I'm going to fix this, but I'm going to wait another 4-5 days for a release just to see if other issues come in.

[The-Compiler]

FWIW this also breaks building of Chromium, due to the issue reported by @NormalRa above.

[NormalRa]

Ah yes, bleeding-edge distributions. We are guinea pigs! c:

[dabeaz]

I've pushed a fix to github. Would appreciate some testing.

[The-Compiler]

It's unfortunately still broken when building Chromium:

  File "../../../tools/json_schema_compiler/compiler.py", line 191, in <module>
    opts.namespace, opts.impl_dir, include_rules)
  File "../../../tools/json_schema_compiler/compiler.py", line 128, in GenerateSchema
    code = generator.Generate(namespace).Render()
  File "/tmp/yaourt-tmp-florian/aur-chromium-dev/src/chromium-44.0.2383.0/tools/json_schema_compiler/h_generator.py", line 17, in Generate
    return _Generator(namespace, self._type_generator).Generate()
  File "/tmp/yaourt-tmp-florian/aur-chromium-dev/src/chromium-44.0.2383.0/tools/json_schema_compiler/h_generator.py", line 62, in Generate
    .Cblock(self._type_helper.GenerateIncludes(include_soft=include_soft))
  File "/tmp/yaourt-tmp-florian/aur-chromium-dev/src/chromium-44.0.2383.0/tools/json_schema_compiler/cpp_type_generator.py", line 163, in GenerateIncludes
    for namespace, dependencies in self._NamespaceTypeDependencies().items():
  File "/tmp/yaourt-tmp-florian/aur-chromium-dev/src/chromium-44.0.2383.0/tools/json_schema_compiler/cpp_type_generator.py", line 208, in _NamespaceTypeDependencies
    hard=not prop.optional)
  File "/tmp/yaourt-tmp-florian/aur-chromium-dev/src/chromium-44.0.2383.0/tools/json_schema_compiler/cpp_type_generator.py", line 231, in _TypeDependencies
    deps.add(_TypeDependency(self._FindType(type_.ref_type), hard=hard))
  File "/tmp/yaourt-tmp-florian/aur-chromium-dev/src/chromium-44.0.2383.0/tools/json_schema_compiler/cpp_type_generator.py", line 175, in _FindType
    self._default_namespace)
  File "/tmp/yaourt-tmp-florian/aur-chromium-dev/src/chromium-44.0.2383.0/tools/json_schema_compiler/schema_loader.py", line 76, in ResolveType
    namespace = self.ResolveNamespace(full_namespace)
  File "/tmp/yaourt-tmp-florian/aur-chromium-dev/src/chromium-44.0.2383.0/tools/json_schema_compiler/schema_loader.py", line 64, in ResolveNamespace
    self.LoadSchema(filepath)[0],
  File "/tmp/yaourt-tmp-florian/aur-chromium-dev/src/chromium-44.0.2383.0/tools/json_schema_compiler/schema_loader.py", line 90, in LoadSchema
    api_defs = idl_schema.Load(schema_path)
  File "/tmp/yaourt-tmp-florian/aur-chromium-dev/src/chromium-44.0.2383.0/tools/json_schema_compiler/idl_schema.py", line 525, in Load
    return Process(contents, filename)
  File "/tmp/yaourt-tmp-florian/aur-chromium-dev/src/chromium-44.0.2383.0/tools/json_schema_compiler/idl_schema.py", line 535, in Process
    idl = idl_parser.IDLParser().ParseData(contents, filename)
  File "/tmp/yaourt-tmp-florian/aur-chromium-dev/src/chromium-44.0.2383.0/tools/json_schema_compiler/../../ppapi/generators/idl_parser.py", line 908, in __init__
    optimize=0, write_tables=0)
  File "/usr/lib/python2.7/site-packages/ply/yacc.py", line 3214, in yacc
    if '.' not in tabmodule:
TypeError: argument of type 'NoneType' is not iterable

[dabeaz]

Why is it passing a tabmodule of None? That is not documented nor is it the default value for that argument.

[The-Compiler]

I have no idea, I just know it works when I install ply 3.4 😄

I also opened this Chromium issue but so far nothing happened except of someone adding a NeedsFurtherTriage tag.

[lelit]

Slimit runs ok, with latest patch, thank you.

[dabeaz]

Have made an additional patch to try and fix this.

[protist]

Thanks for this fix. With ply 3.6, I couldn't build qt5 from source, in Arch Linux. The latest git version of ply fixes this.

[gsingh93]

So Ply 3.7 fixes this issue, right? The latest version I can install with pip is 3.6, and that still seems to have the issue. Can you update this on pypi?

[dabeaz]

Download ply-3.7 and see if it fixes the issue.

[gsingh93]

@dabeaz it does. I also see you've updated the pypi version to 3.8, thanks.