Character-based word embeddings model based on RNN
Chars2vec library could be very useful if you are dealing with the texts containing abbreviations, slang, typos, or some other specific textual dataset. Chars2vec language model is based on the symbolic representation of words – the model maps each word to a vector of a fixed length. These vector representations are obtained with a custom neural network while the latter is being trained on pairs of similar and non-similar words. This custom neural net includes LSTM, reading sequences of characters in words, as its part. The model maps similarly written words to proximal vectors. This approach enables creation of an embedding in vector space for any sequence of characters. Chars2vec models does not keep any dictionary of embeddings, but generates embedding vectors inplace using pretrained model.
There are pretrained models of dimensions 50, 100, 150, 200 and 300 for the English language. The library provides convenient user API to train a model for an arbitrary set of characters. Read more details about the architecture of Chars2vec: Character-based language model for handling real world texts with spelling errors and human slang in Hacker Noon.
Model available for Python 2.7 and 3.0+.
Download project source and run in your command line 1. Build and install from source
>> python setup.py install
Run in your command line 2. Via pip
>> pip install chars2vec
chars2vec.load_model(str path) initializes the model from directory
There are 5 pretrained English model with dimensions: 50, 100, 150, 200 and 300.
To load this pretrained models:
import chars2vec # Load Inutition Engineering pretrained model # Models names: 'eng_50', 'eng_100', 'eng_150', 'eng_200', 'eng_300' c2v_model = chars2vec.load_model('eng_50')
numpy.ndarray of shape
(n_words, dim) with word embeddings.
words = ['list', 'of', 'words'] # Create word embeddings word_embeddings = c2v_model.vectorize_words(words)
chars2vec.train_model(int emb_dim, X_train, y_train, model_chars)
creates and trains new chars2vec model and returns
emb_dim is a dimension of the model.
X_train is a list or numpy.ndarray of word pairs.
y_train is a list or numpy.ndarray of target values that describe the proximity of words.
Training set (
y_train) consists of pairs of "similar" and "not similar" words;
a pair of "similar" words is labeled with 0 target value, and a pair of "not similar" with 1.
model_chars is a list of chars for the model.
Characters which are not in the
list will be ignored by the model.
Read more about chars2vec training and generation of training dataset in article about chars2vec.
chars2vec.save_model(c2v_model, str path_to_model) saves the trained model
to the directory.
import chars2vec dim = 50 path_to_model = 'path/to/model/directory' X_train = [('mecbanizing', 'mechanizing'), # similar words, target is equal 0 ('dicovery', 'dis7overy'), # similar words, target is equal 0 ('prot$oplasmatic', 'prtoplasmatic'), # similar words, target is equal 0 ('copulateng', 'lzateful'), # not similar words, target is equal 1 ('estry', 'evadin6'), # not similar words, target is equal 1 ('cirrfosis', 'afear') # not similar words, target is equal 1 ] y_train = [0, 0, 0, 1, 1, 1] model_chars = ['!', '"', '#', '$', '%', '&', "'", '(', ')', '*', '+', ',', '-', '.', '/', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', ':', ';', '<', '=', '>', '?', '@', '_', 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z'] # Create and train chars2vec model using given training data my_c2v_model = chars2vec.train_model(dim, X_train, y_train, model_chars) # Save your pretrained model chars2vec.save_model(my_c2v_model, path_to_model) # Load your pretrained model c2v_model = chars2vec.load_model(path_to_model)
Full code examples for usage and training models see in