Files

database
ethical-hacking
general
gui-programming
handling-pdf-files
images
machine-learning
- age-and-gender-detection
- asr
- blur-faces
- cartoonify-images
- clustering-algorithms
- contour-detection
- control-image-generation-with-controlnet
- credit-card-fraud-detection
- customer-churn-detection
- depth2image-stable-diffusion
- dimensionality-reduction-feature-extraction
- dimensionality-reduction-feature-selection
- dropout-in-pytorch
- edge-detection
- edit-images-instruct-pix2pix
- explainable-ai
- face-age-prediction
- face-gender-detection
- face_detection
- facial-recognition-system
- feature-extraction-autoencoders
- feature-selection
- finetuning-vit-image-classification
- handling-inbalance-churn-data
- hog-feature-extraction
- image-captioning
- image-classifier-using-transfer-learning
- image-classifier
- image-segmentation-transformers
- image-transformation
- imbalance-learning
- k-fold-cross-validation-sklearn
- kmeans-image-segmentation
- logistic-regression-in-pytorch
- malaria-classification
- nlp
  - bert-text-classification
  - bleu-score
  - chatbot-transformers
  - fake-news-classification
  - machine-translation
  - named-entity-recognition
  - pretraining-bert
  - rouge-score
  - semantic-textual-similarity
  - sentiment-analysis-vader
  - spam-classifier
  - speech-recognition-transformers
  - text-classification
  - text-generation-transformers
  - text-generator
  - text-paraphrasing
  - text-summarization
  - tokenization-stemming-lemmatization
    - README.md
    - example1_splitting_by_whitespace.py
    - example2_splitting_by_comma.py
    - example3_splitting_by_whitespace.py
    - part_of_speech_tagging.py
    - port_stemmer.py
    - requirements.txt
    - reviews.txt
    - sentence_tokenization_nltk.py
    - snowball_stemmer.py
    - subword_tokenization_bert.py
    - textblob_tokenization.py
    - tokenize_bpe_tokenizer.py
    - training_bpe_tokenizer.py
    - word_tokenization_nltk.py
    - wordnet_lemmatizer.py
    - wordpiece_tokenizer.py
    - xlnet_sentencepiece_tokenizer.py
  - wer-score
- object-detection
- optical-character-recognition
- plotly-visualization
- recommender-system-using-association-rules
- satellite-image-classification
- shape-detection
- sift
- skin-cancer-detection
- speech-emotion-recognition
- speech-recognition
- stable-diffusion-models
- stable-diffusion-upscaler
- stock-prediction
- technical-indicators
- text-to-speech
- trading-with-fxcm
- visual-question-answering
python-for-multimedia
python-standard-library
scapy
web-programming
web-scraping
.gitignore
LICENSE
README.md

tokenization-stemming-lemmatization

x4nth055

add tokenization, stemming & lemmatization tutorial

Jan 5, 2023

c03bf85 · Jan 5, 2023

History

This branch is up to date with x4nth055/pythoncode-tutorials:master.

Name	Name	Last commit message	Last commit date
parent directory ..
README.md	README.md	add tokenization, stemming & lemmatization tutorial	Jan 5, 2023
example1_splitting_by_whitespace.py	example1_splitting_by_whitespace.py	add tokenization, stemming & lemmatization tutorial	Jan 5, 2023
example2_splitting_by_comma.py	example2_splitting_by_comma.py	add tokenization, stemming & lemmatization tutorial	Jan 5, 2023
example3_splitting_by_whitespace.py	example3_splitting_by_whitespace.py	add tokenization, stemming & lemmatization tutorial	Jan 5, 2023
part_of_speech_tagging.py	part_of_speech_tagging.py	add tokenization, stemming & lemmatization tutorial	Jan 5, 2023
port_stemmer.py	port_stemmer.py	add tokenization, stemming & lemmatization tutorial	Jan 5, 2023
requirements.txt	requirements.txt	add tokenization, stemming & lemmatization tutorial	Jan 5, 2023
reviews.txt	reviews.txt	add tokenization, stemming & lemmatization tutorial	Jan 5, 2023
sentence_tokenization_nltk.py	sentence_tokenization_nltk.py	add tokenization, stemming & lemmatization tutorial	Jan 5, 2023
snowball_stemmer.py	snowball_stemmer.py	add tokenization, stemming & lemmatization tutorial	Jan 5, 2023
subword_tokenization_bert.py	subword_tokenization_bert.py	add tokenization, stemming & lemmatization tutorial	Jan 5, 2023
textblob_tokenization.py	textblob_tokenization.py	add tokenization, stemming & lemmatization tutorial	Jan 5, 2023
tokenize_bpe_tokenizer.py	tokenize_bpe_tokenizer.py	add tokenization, stemming & lemmatization tutorial	Jan 5, 2023
training_bpe_tokenizer.py	training_bpe_tokenizer.py	add tokenization, stemming & lemmatization tutorial	Jan 5, 2023
word_tokenization_nltk.py	word_tokenization_nltk.py	add tokenization, stemming & lemmatization tutorial	Jan 5, 2023
wordnet_lemmatizer.py	wordnet_lemmatizer.py	add tokenization, stemming & lemmatization tutorial	Jan 5, 2023
wordpiece_tokenizer.py	wordpiece_tokenizer.py	add tokenization, stemming & lemmatization tutorial	Jan 5, 2023
xlnet_sentencepiece_tokenizer.py	xlnet_sentencepiece_tokenizer.py	add tokenization, stemming & lemmatization tutorial	Jan 5, 2023

README.md

Tokenization, Stemming, and Lemmatization in Python

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Files

tokenization-stemming-lemmatization

tokenization-stemming-lemmatization

README.md

Tokenization, Stemming, and Lemmatization in Python

Collapse file tree

Files

tokenization-stemming-lemmatization

Directory actions

More options

Directory actions

More options

Latest commit

History

tokenization-stemming-lemmatization

Folders and files

parent directory

README.md

Tokenization, Stemming, and Lemmatization in Python