Python Parser and Tokenizer

This project is a simple tokenizer and parser that parse Java programming language

Features of the application

• Breaks text into tokens with the use of regular expressions
• Retrieve Java identifiers
• Retrieve Java comment tags
• Remove comments
• Retrieve class names
• Retrieves frequency of keywords

Built With

• Python
• PyCharm

Getting Started

To get a local copy up and running follow these steps.

Prerequisites

• Python 3.10.0

Installation

1. Firsly, clone the repo:

   git clone https://github.com/leFos-95/Python-Parser-and-Tokenizer.git
   

2. Open cmd and type:

   pip install nltk
   

3. At cmd type python and then:

   import nltk
   nltk.download('wordnet')
   

4. Run the application

License

Distributed under the MIT License. See LICENSE for more information.

Contact

Lefteris Soulis - lefteris95.soulis@gmail.com

Project Link: https://github.com/leFos-95/Python-Parser-and-Tokenizer