Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tokenizer #15

Closed
5 tasks done
h-sarhan opened this issue May 23, 2023 · 0 comments · Fixed by #19
Closed
5 tasks done

Tokenizer #15

h-sarhan opened this issue May 23, 2023 · 0 comments · Fixed by #19
Assignees
Labels
enhancement New feature or request

Comments

@h-sarhan
Copy link
Owner

h-sarhan commented May 23, 2023

Tokenizer

Write a general purpose tokenizer class that can produce a list of tokens. This list of tokens will help in parsing the configuration file (@h-sarhan)

  • Write a generalized Token class whose type is specified by an enum
  • The Tokenizer class will have rules to tokenize many types of tokens
  • This class is designed to be easily extendible, meaning that we should be able to add new token types easily
  • Add support for tokens that can be found in a config file
  • Fully document any files, classes, and methods written
@h-sarhan h-sarhan self-assigned this May 23, 2023
@h-sarhan h-sarhan mentioned this issue May 23, 2023
5 tasks
@h-sarhan h-sarhan added the enhancement New feature or request label May 23, 2023
@h-sarhan h-sarhan linked a pull request Jul 7, 2023 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant