[bug] roles: UNANNOTATED

[EgorBu]

Hi,
I noticed that JS driver produces a lot of roles: UNANNOTATED when extract UAST from https://github.com/babel/babel/blob/master/packages/babel-core/src/transformation/plugin-pass.js.
If you need any other information - I will be glad to answer!

[juanjux]

Generated UAST:

https://gist.github.com/juanjux/7e0e5c6531a821b30df3597398db9ac4

Looks like *Annotation internal types are not being annotated.

Edit: also "ObjectProperty", "ObjectTypeProperty", and "TypeParameterInstantiation".

[EgorBu]

One more file

[EgorBu]

Updated driver:

egor@egor-sourced:~/workspace/style-analyzer/lookout/style/format/research$ docker exec -it bblfshd bblfshctl driver install javascript docker://bblfsh/javascript-driver:v2.1.1
Installing javascript language driver from "docker://bblfsh/javascript-driver:v2.1.1"... Done

It's not fixed yet. Example with the last file (#24 (comment)) still gives a lot of roles: UNANNOTATED.
Code:

declare module.exports: { foo(): number; }

and UAST:

internal_type: "DeclareModuleExports"
properties {
  key: "internalRole"
  value: "body"
}
children {
  internal_type: "TypeAnnotation"
  properties {
    key: "internalRole"
    value: "typeAnnotation"
  }
  children {
    internal_type: "ObjectTypeAnnotation"
    properties {
      key: "exact"
      value: "false"
    }
    properties {
      key: "internalRole"
      value: "typeAnnotation"
    }
    children {
      internal_type: "ObjectTypeProperty"
      properties {
        key: "internalRole"
        value: "properties"
      }
      properties {
        key: "kind"
        value: "init"
      }
      properties {
        key: "optional"
        value: "false"
      }
      properties {
        key: "static"
        value: "false"
      }
      children {
        internal_type: "Identifier"
        properties {
          key: "internalRole"
          value: "key"
        }
        token: "foo"
        start_position {
          offset: 26
          line: 1
          col: 27
        }
        end_position {
          offset: 29
          line: 1
          col: 30
        }
        roles: EXPRESSION
        roles: IDENTIFIER
      }
      children {
        internal_type: "FunctionTypeAnnotation"
        properties {
          key: "internalRole"
          value: "value"
        }
        children {
          internal_type: "NumberTypeAnnotation"
          properties {
            key: "internalRole"
            value: "returnType"
          }
          start_position {
            offset: 33
            line: 1
            col: 34
          }
          end_position {
            offset: 39
            line: 1
            col: 40
          }
          roles: UNANNOTATED
        }
        start_position {
          offset: 26
          line: 1
          col: 27
        }
        end_position {
          offset: 39
          line: 1
          col: 40
        }
        roles: UNANNOTATED
      }
      start_position {
        offset: 26
        line: 1
        col: 27
      }
      end_position {
        offset: 39
        line: 1
        col: 40
      }
      roles: UNANNOTATED
    }
    start_position {
      offset: 24
      line: 1
      col: 25
    }
    end_position {
      offset: 42
      line: 1
      col: 43
    }
    roles: UNANNOTATED
  }
  start_position {
    offset: 22
    line: 1
    col: 23
  }
  end_position {
    offset: 42
    line: 1
    col: 43
  }
  roles: UNANNOTATED
}
start_position {
  line: 1
  col: 1
}
end_position {
  offset: 42
  line: 1
  col: 43
}
roles: UNANNOTATED

[EgorBu]

One more file with broken roles: file.

declare class A mixins B<T>, C {}

[juanjux]

Yes, looks like I missed a few when updating, we've a new PR ready.

#27

[EgorBu]

BTW - good way to check missed roles is to apply bblfsh on this repository:

project = "/path/to/babel/**/*.js"
# collect filenames with full path
files = glob.glob(project, recursive=True)

# extract UASTs
client = bblfsh.BblfshClient("0.0.0.0:9432")
uasts = []
final_files = []  # only files where UAST can be extracted
for file in tqdm(files):
    res = client.parse(file)
    if res.status == 0:
        uasts.append(res.uast)
        final_files.append(file)

# some functionality to check
bad_files = []
for file, uast in zip(final_files, uasts):
    if is_bad_uast(uast):
        bad_files.append(file)

if len(bad_files) == 0:
    print("VICTORY")
else:
    print(len(bad_files))

[juanjux]

Related more general issue: #28

[juanjux]

Forgot to close this after the PRs above fixed the problem, just in case I reparsed that file and could not find any unannotated (also a lot of integration tests were added in those PRs to cover those types).