This project is the implementation of Language Modelling. The goal is to model Indian first names using different character-level language models. The models include Feedforward Neural Networks (FNN) and Recurrent Neural Networks (RNN), both of which are trained to generate Indian first names based on character sequences.
This assignment aims to implement language models for generating Indian first names. The models are trained at the character level to predict the next character in a sequence, eventually generating entire names. The assignment uses two primary models:
- Feedforward Neural Networks (FNN)
- Recurrent Neural Networks (RNN)
The dataset used for training consists of a list of Indian first names, broken down into individual characters. The models are trained to predict the next character given a sequence of characters.
- Input: Sequences of characters from Indian names.
- Output: Predicted next character in the sequence, enabling the model to generate entire names.
The FNN model is a simple neural network that takes a sequence of characters as input, flattens it, and predicts the next character in the sequence. It consists of fully connected layers with activation functions to process the input.
- Architecture:
- Input Layer: Character sequence (one-hot encoded or embedded)
- Fully Connected Layers: Process the flattened input
- Output Layer: Predicts the next character
The RNN model is designed for sequence data and processes one character at a time while maintaining a hidden state. It is more suited to modeling sequences like names since it can retain information about previously seen characters.
- Architecture:
- RNN Layer: Processes sequences of characters one at a time
- Hidden State: Maintains context across time steps in the sequence
- Output Layer: Predicts the next character
The models are trained using a character-level language modeling approach. The training process involves:
- Splitting the names into character sequences.
- Feeding the sequences into the model.
- Predicting the next character.
- Computing the loss between the predicted and actual characters.
- Updating the model using backpropagation.
The model is evaluated using an automated process where specific print statements (EVALUATION) are used to generate results. The evaluation includes:
- Accuracy: Measures how often the model predicts the correct next character.
- Loss: Measures the difference between the predicted and actual characters.
You can monitor the evaluation metrics to determine the effectiveness of the FNN and RNN models.
- Validation Perplexity (without smoothing): 13.6315
- Validation Perplexity (with smoothing): 13.6321
- Generated Names:
- Prefix:
<s><s>sh - Names:
peaenm</s>ccdji)nghlgar, <s>jhm)jye,uidf&ya</s>r&&, (slyulpfltrru0,l</s>b&a, skkt</s>chi,rim,aauo<s>i</s>, 0o,yap,g0&cftdiom</s>m0
- Prefix:
- Validation Perplexity (without smoothing): 5.5944
- Validation Perplexity (with smoothing): 6.0882
- Generated Names:
- Prefix:
<s><s>sh - Names:
ha, ha, ha, ha, ha
- Prefix:
- Validation Perplexity (with smoothing): 4.6436
- Generated Names:
- Prefix:
<s><s>sh - Names:
shnanashal, shlanka, shish, sh, sheesakharadha
- Prefix:
- Validation Perplexity: 7.3570
- Generated Names:
- Prefix:
<s><s>sh - Names:
<s><s>shmtmdsfnmrmbjpcjm, <s><s>shbnmnahaskjaansks, <s><s>shykdbfayjansfmrsd, <s><s>shksmmlnrdspsmrsmf, <s><s>shkaspkbtasmsssjan
- Prefix:
- Generated Names (Prefix:
aash):aashb, aashdss, aashi, aashk, aashljmnr
- Validation Perplexity: 13.4957
- Generated Names:
- Prefix:
<s><s>sh - Names:
shiikkss, sheellaa, sheemm, sheennaa, shpprrbb
- Prefix:
- Generated Names (Prefix:
aash):aashii, aashaa, aashaannee, aashaassaa, aash
PALLEKONDA_NAVEEN_KUMAR_22915_assignment2.py: Main Python script for training and evaluating the models.fnn/: Directory containing the FNN model files.rnn/: Directory containing the RNN model files.loss.json: JSON file containing the training loss history.