This is a toolkik for processing code implemented in python. It includes bug-fixed/re-implemented/optimized version of some existing tools like javalang, POSSE, spiral.
Based on these tools, we provide following features:
- AST parsing based on the bug-fixed javalang
- tokenization for identifier based on the optimized spiral
- POS (part-of-speech) annotation for identifier based on the re-implemented POSSE
- phrase chunking for identifier based on the re-implemented POSSE
- abbreviation & synonym relation checking based on heuristics
- token representation learning for code (code2vec) based on FastText
- dependency analysis (CFG and PDG) based on the Java analysis tool -- CodeAnalysis
- action unit parsing based on the POS and chunking.