KeyError 'file' when using pyplusplus

[chenghuige]

Hi I tried to use py++ to automatic wrap c++ files.
Using gccxml and pygccxml is ok, now I changed to use castxml, and I face below, seems in
def __read_location(self, decl, attrs):
'file' is not in attrs.

{'name': '', 'access': 'public', 'context': '_590', 'offset': '0', 'type': '_3541', 'id': '_1405'}
Traceback (most recent call last):
File "chg.py", line 8, in
,xml_generator_path='/usr/local/bin/castxml')
File "/usr/local/lib/python2.7/dist-packages/pyplusplus/module_builder/boost_python_builder.py", line 104, in init
, indexing_suite_version)
File "/usr/local/lib/python2.7/dist-packages/pyplusplus/module_builder/boost_python_builder.py", line 147, in __parse_declarations
decls = reader.read_files( files, compilation_mode )
File "build/bdist.linux-x86_64/egg/pygccxml/parser/project_reader.py", line 249, in read_files
File "build/bdist.linux-x86_64/egg/pygccxml/parser/project_reader.py", line 278, in __parse_file_by_file
File "build/bdist.linux-x86_64/egg/pygccxml/parser/source_reader.py", line 334, in read_file
File "build/bdist.linux-x86_64/egg/pygccxml/parser/source_reader.py", line 354, in read_cpp_source_file
File "build/bdist.linux-x86_64/egg/pygccxml/parser/source_reader.py", line 443, in __parse_xml_file
File "build/bdist.linux-x86_64/egg/pygccxml/parser/etree_scanner.py", line 56, in read
File "build/bdist.linux-x86_64/egg/pygccxml/parser/scanner.py", line 221, in startElement
File "build/bdist.linux-x86_64/egg/pygccxml/parser/scanner.py", line 274, in __read_location
KeyError: 'file'

I printed attrs
{'context': '_586', 'artificial': '1', 'access': 'public', 'location': 'f28:20', 'file': 'f28', 'inline': '1', 'line': '20', 'id': '_1403', 'name': 'Chg'}
{'file': 'f28', 'name': '=', 'mangled': 'ZN3ChgaSERKS', 'artificial': '1', 'access': 'public', 'returns': '_3982', 'location': 'f28:20', 'context': '_586', 'inline': '1', 'line': '20', 'id': '_1404'}
{'name': '', 'access': 'public', 'context': '_590', 'offset': '0', 'type': '_3541', 'id': '_1405'}
Traceback (most recent call last):
File "chg.py", line 8, in
,xml_generator_path='/usr/local/bin/castxml')

Here 'name' is empty and no 'file',
for a workaround, I modified this py file

    #print attrs
    #if "name" in attrs and attrs["name"] in to_skip:
    if "name" in attrs and attrs["name"] in to_skip or "file" not in attrs:

After that for my simple c++ code, I can wrap correctly just as using gccxml.

[iMichka]

Hi

thank you very much for the feedback.

First, are you on OS X, Linux or Windows, and what compiler do you use ?

As you have seen, there is already a mechanism to skip some fields that have no file attribute (these are defined in the to_skip list). These cases were clearly documented and understood. So there is probably a reason for your case too.

I would prefer to investigate the case more in depth before blindly skipping everything which has not a file attribute. This may be a castxml problem (which is maybe forgetting to write out the file in the xml file). If not, I would like to only skip the specific case you encounter there, to not hide other potential errors in castxml.

For this I will try to create a test to reproduce your case. Would it be possible to reduce you c++ code to a minimalistic example which demonstrate the failures ? This would really help. You can have a look at the first pygccxml example to write a small test case: http://pygccxml.readthedocs.org/en/develop/examples/parsing/example.html

You can also give me the full file if you don't have time, I will trim it down to find the problem. Also, I will be able to look directly at the full xml file, which will greatly help.

[chenghuige]

Well, looks like environment problem? I'm using Linux, ubuntu14.04.

castxml --version
castxml version 0.1-g5cfebb0

CastXML project maintained and supported by Kitware (kitware.com).

clang version 3.9.0 (trunk 259983) (llvm/trunk 259980)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir:

using the latest pygccxml.

clang --version
clang version 3.9.0 (trunk 259983) (llvm/trunk 259980)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/local/bin

gcc --version
gcc (Ubuntu 4.8.4-2ubuntu1~14.04.1) 4.8.4
Copyright (C) 2013 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

I tried the first example as http://pygccxml.readthedocs.org/en/develop/examples/parsing/example.html
It will fail.. with 'file' not find, I printed attrs

$ python example.py
generator_path: /usr/local/bin/castxml
generator_name: castxml
INFO Parsing source file "example.hpp" ...
{'context': '_1', 'location': 'f0:0', 'file': 'f0', 'line': '0', 'type': '_8', 'id': '_2', 'name': '__int128_t'}
{'context': '_1', 'location': 'f0:0', 'file': 'f0', 'line': '0', 'type': '_9', 'id': '_3', 'name': '__uint128_t'}
{'context': '_1', 'location': 'f0:0', 'file': 'f0', 'line': '0', 'type': '_10', 'id': '_4', 'name': '__NSConstantString'}
{'context': '_1', 'location': 'f0:0', 'file': 'f0', 'line': '0', 'type': '_11', 'id': '_5', 'name': '__builtin_ms_va_list'}
{'context': '_1', 'location': 'f0:0', 'file': 'f0', 'line': '0', 'type': '_12', 'id': '_6', 'name': '__builtin_va_list'}
{'context': '_1', 'align': '64', 'members': '_14 _15 _16 _17', 'location': 'f0:0', 'file': 'f0', 'line': '0', 'size': '256', 'id': '_10', 'name': '__NSConstantString_tag'}
{'file': 'f1', 'name': 'a', 'mangled': '_ZN2ns1aE', 'init': '1', 'location': 'f1:2', 'context': '_7', 'line': '2', 'type': '_20', 'id': '_13'}
{'name': '', 'access': 'public', 'context': '_10', 'offset': '0', 'type': '_21', 'id': '_14'}
ERROR error occured, while parsing element with name "Field" and attrs "['name', 'access', 'context', 'offset', 'type', 'id']".
Error: 'file'.
Traceback (most recent call last):
File "example.py", line 20, in
decls = parser.parse([filename], xml_generator_config)
File "build/bdist.linux-x86_64/egg/pygccxml/parser/init.py", line 54, in parse
File "build/bdist.linux-x86_64/egg/pygccxml/parser/project_reader.py", line 249, in read_files
File "build/bdist.linux-x86_64/egg/pygccxml/parser/project_reader.py", line 278, in __parse_file_by_file
File "build/bdist.linux-x86_64/egg/pygccxml/parser/source_reader.py", line 334, in read_file
File "build/bdist.linux-x86_64/egg/pygccxml/parser/source_reader.py", line 354, in read_cpp_source_file
File "build/bdist.linux-x86_64/egg/pygccxml/parser/source_reader.py", line 443, in __parse_xml_file
File "build/bdist.linux-x86_64/egg/pygccxml/parser/etree_scanner.py", line 56, in read
File "build/bdist.linux-x86_64/egg/pygccxml/parser/scanner.py", line 221, in startElement
File "build/bdist.linux-x86_64/egg/pygccxml/parser/scanner.py", line 275, in __read_location
KeyError: 'file'

[iMichka]

Okay I also have a Ubuntu 14.04, and gcc 4.8.4. My castxml was compiled with llvm 3.6.1. I could not reproduce the problems.

I'll need to upgrade to llvm 3.9 then. I need to move stuff around in my root folder because I am starting to have a lot of compilers there. I hope I can do this tomorrow evening, and install the latest compiler. I'll keep you updated of my findings.

[iMichka]

Ok this is weird.

I could reproduce the error and analysed it as much I could.

The problem comes from llvm 3.9 (trunk). I will not write code to support a development version of llvm. This is a moving target ... I am cross-checking this with the CastXML developer (maybe we need to fix this in CastXML; or more probably in LLVM). At least we caught a problem before the llvm release with you help.

For the moment you need to rely on llvm >= 3.8.0. I think this is more reasonable. With this version it should work (I only tested 3.6 and 3.7, but 3.8.0 should be OK).

Of course this bug will stay open while I gather more information and try to get this fixed.

[bradking]

The missing field names on __NSConstantString_tag have been added by upstream LLVM/Clang commit r261887. It is now expected that the builtins will be present but the new Fields will have names.

[iMichka]

Thanks. I still need to tweak things in pygccxml. When I am done I'll push v1.7.3 with the fixes for llvm 3.9.

[iMichka]

All my patches are now on the v1.7.3 branch, which will be released soon. They have also been merged to develop. It should be usable now with the latest llvm. I'll close this bug, please open a new one if you have still problems.