Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Libcanard DSDL compiler fails when source paths ends with a trailing os separator in Windows #70

Closed
aasmune opened this issue Nov 25, 2018 · 1 comment

Comments

@aasmune
Copy link
Member

commented Nov 25, 2018

When generating C source code using the DSDL compiler in libcanard, it fails if the source paths provided from CLI ends with a trailing \\ or /.

$ python libcanard\dsdl_compiler\libcanard_dsdlc -O output dsdl/uavcan/                                                                                             
Internal error                                                                                                                                                      
Traceback (most recent call last):                                                                                                                                  
  File "C:\Python37-32\lib\site-packages\uavcan\dsdl\parser.py", line 708, in parse                                                                                 
    return self.parse_source(filename, source_text)                                                                                                                 
  File "C:\Python37-32\lib\site-packages\uavcan\dsdl\parser.py", line 700, in parse_source                                                                          
    raise ex                                                                                                                                                        
  File "C:\Python37-32\lib\site-packages\uavcan\dsdl\parser.py", line 632, in parse_source                                                                          
    full_typename, version, default_dtid = self._full_typename_version_and_dtid_from_filename(filename)                                                             
  File "C:\Python37-32\lib\site-packages\uavcan\dsdl\parser.py", line 455, in _full_typename_version_and_dtid_from_filename                                         
    full_name = self._namespace_from_filename(filename) + '.' + name                                                                                                
  File "C:\Python37-32\lib\site-packages\uavcan\dsdl\parser.py", line 426, in _namespace_from_filename                                                              
    validate_namespace_name(ns)                                                                                                                                     
  File "C:\Python37-32\lib\site-packages\uavcan\dsdl\parser.py", line 758, in validate_namespace_name                                                               
    enforce(re.match(r'[a-z][a-z0-9_]*$', component), 'Invalid namespace name [%s]', name)                                                                          
  File "C:\Python37-32\lib\site-packages\uavcan\dsdl\parser.py", line 722, in enforce                                                                               
    error(fmt, *args)                                                                                                                                               
  File "C:\Python37-32\lib\site-packages\uavcan\dsdl\parser.py", line 717, in error                                                                                 
    raise DsdlException(fmt % args)                                                                                                                                 
uavcan.dsdl.common.DsdlException: dsdl\uavcan\CoarseOrientation.uavcan: Invalid namespace name []                                                                   
Compiler failure                                                                                                                                                    
Traceback (most recent call last):                                                                                                                                  
  File "C:\Python37-32\lib\site-packages\uavcan\dsdl\parser.py", line 708, in parse                                                                                 
    return self.parse_source(filename, source_text)                                                                                                                 
  File "C:\Python37-32\lib\site-packages\uavcan\dsdl\parser.py", line 700, in parse_source                                                                          
    raise ex                                                                                                                                                        
  File "C:\Python37-32\lib\site-packages\uavcan\dsdl\parser.py", line 632, in parse_source                                                                          
    full_typename, version, default_dtid = self._full_typename_version_and_dtid_from_filename(filename)                                                             
  File "C:\Python37-32\lib\site-packages\uavcan\dsdl\parser.py", line 455, in _full_typename_version_and_dtid_from_filename                                         
    full_name = self._namespace_from_filename(filename) + '.' + name                                                                                                
  File "C:\Python37-32\lib\site-packages\uavcan\dsdl\parser.py", line 426, in _namespace_from_filename                                                              
    validate_namespace_name(ns)                                                                                                                                     
  File "C:\Python37-32\lib\site-packages\uavcan\dsdl\parser.py", line 758, in validate_namespace_name                                                               
    enforce(re.match(r'[a-z][a-z0-9_]*$', component), 'Invalid namespace name [%s]', name)                                                                          
  File "C:\Python37-32\lib\site-packages\uavcan\dsdl\parser.py", line 722, in enforce                                                                               
    error(fmt, *args)                                                                                                                                               
  File "C:\Python37-32\lib\site-packages\uavcan\dsdl\parser.py", line 717, in error                                                                                 
    raise DsdlException(fmt % args)                                                                                                                                 
uavcan.dsdl.common.DsdlException: dsdl\uavcan\CoarseOrientation.uavcan: Invalid namespace name []                                                                   
                                                                                                                                                                    
During handling of the above exception, another exception occurred:                                                                                                 
                                                                                                                                                                    
Traceback (most recent call last):                                                                                                                                  
  File "C:\Users\aasme\Documents\git\uavcan\libcanard\dsdl_compiler\libcanard_dsdl_compiler\__init__.py", line 111, in run_parser                                   
    types = dsdl.parse_namespaces(source_dirs, search_dirs)                                                                                                         
  File "C:\Python37-32\lib\site-packages\uavcan\dsdl\parser.py", line 860, in parse_namespaces                                                                      
    t = parser.parse(filename)                                                                                                                                      
  File "C:\Python37-32\lib\site-packages\uavcan\dsdl\parser.py", line 713, in parse                                                                                 
    raise DsdlException('Internal error: %s' % str(ex), file=filename)                                                                                              
uavcan.dsdl.common.DsdlException: dsdl\uavcan\CoarseOrientation.uavcan: Internal error: dsdl\uavcan\CoarseOrientation.uavcan: Invalid namespace name []             
                                                                                                                                                                    
During handling of the above exception, another exception occurred:                                                                                                 
                                                                                                                                                                    
Traceback (most recent call last):                                                                                                                                  
  File "libcanard\dsdl_compiler\libcanard_dsdlc", line 66, in <module>                                                                                              
    dsdlc_run(args.source_dir, args.incdir, args.outdir, args.header_only)                                                                                          
  File "C:\Users\aasme\Documents\git\uavcan\libcanard\dsdl_compiler\libcanard_dsdl_compiler\__init__.py", line 64, in run                                           
    types = run_parser(source_dirs, include_dirs + source_dirs)                                                                                                     
  File "C:\Users\aasme\Documents\git\uavcan\libcanard\dsdl_compiler\libcanard_dsdl_compiler\__init__.py", line 114, in run_parser                                   
    die(ex)                                                                                                                                                         
  File "C:\Users\aasme\Documents\git\uavcan\libcanard\dsdl_compiler\libcanard_dsdl_compiler\__init__.py", line 107, in die                                          
    raise DsdlCompilerException(str(text))                                                                                                                          
libcanard_dsdl_compiler.DsdlCompilerException: dsdl\uavcan\CoarseOrientation.uavcan: Internal error: dsdl\uavcan\CoarseOrientation.uavcan: Invalid namespace name []
dsdl\uavcan\CoarseOrientation.uavcan: Internal error: dsdl\uavcan\CoarseOrientation.uavcan: Invalid namespace name []                                               

and

$ python libcanard\dsdl_compiler\libcanard_dsdlc -O output dsdl\uavcan\
Internal error
Traceback (most recent call last):
  File "C:\Python37-32\lib\site-packages\uavcan\dsdl\parser.py", line 708, in parse
    return self.parse_source(filename, source_text)
  File "C:\Python37-32\lib\site-packages\uavcan\dsdl\parser.py", line 700, in parse_source
    raise ex
  File "C:\Python37-32\lib\site-packages\uavcan\dsdl\parser.py", line 632, in parse_source
    full_typename, version, default_dtid = self._full_typename_version_and_dtid_from_filename(filename)
  File "C:\Python37-32\lib\site-packages\uavcan\dsdl\parser.py", line 455, in _full_typename_version_and_dtid_from_filename
    full_name = self._namespace_from_filename(filename) + '.' + name
  File "C:\Python37-32\lib\site-packages\uavcan\dsdl\parser.py", line 426, in _namespace_from_filename
    validate_namespace_name(ns)
  File "C:\Python37-32\lib\site-packages\uavcan\dsdl\parser.py", line 758, in validate_namespace_name
    enforce(re.match(r'[a-z][a-z0-9_]*$', component), 'Invalid namespace name [%s]', name)
  File "C:\Python37-32\lib\site-packages\uavcan\dsdl\parser.py", line 722, in enforce
    error(fmt, *args)
  File "C:\Python37-32\lib\site-packages\uavcan\dsdl\parser.py", line 717, in error
    raise DsdlException(fmt % args)
uavcan.dsdl.common.DsdlException: dsdl\uavcan\CoarseOrientation.uavcan: Invalid namespace name []
Compiler failure
Traceback (most recent call last):
  File "C:\Python37-32\lib\site-packages\uavcan\dsdl\parser.py", line 708, in parse
    return self.parse_source(filename, source_text)
  File "C:\Python37-32\lib\site-packages\uavcan\dsdl\parser.py", line 700, in parse_source
    raise ex
  File "C:\Python37-32\lib\site-packages\uavcan\dsdl\parser.py", line 632, in parse_source
    full_typename, version, default_dtid = self._full_typename_version_and_dtid_from_filename(filename)
  File "C:\Python37-32\lib\site-packages\uavcan\dsdl\parser.py", line 455, in _full_typename_version_and_dtid_from_filename
    full_name = self._namespace_from_filename(filename) + '.' + name
  File "C:\Python37-32\lib\site-packages\uavcan\dsdl\parser.py", line 426, in _namespace_from_filename
    validate_namespace_name(ns)
  File "C:\Python37-32\lib\site-packages\uavcan\dsdl\parser.py", line 758, in validate_namespace_name
    enforce(re.match(r'[a-z][a-z0-9_]*$', component), 'Invalid namespace name [%s]', name)
  File "C:\Python37-32\lib\site-packages\uavcan\dsdl\parser.py", line 722, in enforce
    error(fmt, *args)
  File "C:\Python37-32\lib\site-packages\uavcan\dsdl\parser.py", line 717, in error
    raise DsdlException(fmt % args)
uavcan.dsdl.common.DsdlException: dsdl\uavcan\CoarseOrientation.uavcan: Invalid namespace name []

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\aasme\Documents\git\uavcan\libcanard\dsdl_compiler\libcanard_dsdl_compiler\__init__.py", line 111, in run_parser
    types = dsdl.parse_namespaces(source_dirs, search_dirs)
  File "C:\Python37-32\lib\site-packages\uavcan\dsdl\parser.py", line 860, in parse_namespaces
    t = parser.parse(filename)
  File "C:\Python37-32\lib\site-packages\uavcan\dsdl\parser.py", line 713, in parse
    raise DsdlException('Internal error: %s' % str(ex), file=filename)
uavcan.dsdl.common.DsdlException: dsdl\uavcan\CoarseOrientation.uavcan: Internal error: dsdl\uavcan\CoarseOrientation.uavcan: Invalid namespace name []

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "libcanard\dsdl_compiler\libcanard_dsdlc", line 66, in <module>
    dsdlc_run(args.source_dir, args.incdir, args.outdir, args.header_only)
  File "C:\Users\aasme\Documents\git\uavcan\libcanard\dsdl_compiler\libcanard_dsdl_compiler\__init__.py", line 64, in run
    types = run_parser(source_dirs, include_dirs + source_dirs)
  File "C:\Users\aasme\Documents\git\uavcan\libcanard\dsdl_compiler\libcanard_dsdl_compiler\__init__.py", line 114, in run_parser
    die(ex)
  File "C:\Users\aasme\Documents\git\uavcan\libcanard\dsdl_compiler\libcanard_dsdl_compiler\__init__.py", line 107, in die
    raise DsdlCompilerException(str(text))
libcanard_dsdl_compiler.DsdlCompilerException: dsdl\uavcan\CoarseOrientation.uavcan: Internal error: dsdl\uavcan\CoarseOrientation.uavcan: Invalid namespace name []
dsdl\uavcan\CoarseOrientation.uavcan: Internal error: dsdl\uavcan\CoarseOrientation.uavcan: Invalid namespace name []

The reason for this is that the search paths are splitted at the os separators in locate_namespace_directory, and that the last part therefore is empty. This can be fixed by explicitly removing any os separators at the end of the source paths. This can e.g be done in validate_search_directories() in pyuavcan.dsdl.Parser by changing it from

def validate_search_directories(dirnames):
    dirnames = set(dirnames)
    dirnames = list(map(os.path.abspath, dirnames))
    for d1 in dirnames:
        for d2 in dirnames:
            if d1 == d2:
                continue
            enforce(not d1.startswith(d2), 'Nested search directories are not allowed [%s] [%s]', d1, d2)
            enforce(d1.split(os.path.sep)[-1] != d2.split(os.path.sep)[-1],
                    'Namespace roots must be unique [%s] [%s]', d1, d2)
    return dirnames

to e.g

def validate_search_directories(dirnames):
    dirnames = set(dirnames)
    dirnames = list(map(os.path.abspath, dirnames))
    dirnames = list(map(lambda dir: dir.strip(os.sep), dirnames))
    for d1 in dirnames:
        for d2 in dirnames:
            if d1 == d2:
                continue
            enforce(not d1.startswith(d2), 'Nested search directories are not allowed [%s] [%s]', d1, d2)
            enforce(d1.split(os.path.sep)[-1] != d2.split(os.path.sep)[-1],
                    'Namespace roots must be unique [%s] [%s]', d1, d2)
    return dirnames
@pavel-kirienko

This comment has been minimized.

Copy link
Member

commented Nov 27, 2018

Indeed. Thanks for reporting. Any chance to send a pull request?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.