Skip to content

A small utility for converting Stanford GloVe vectors to HDF5 / NumPy


Notifications You must be signed in to change notification settings


Folders and files

Last commit message
Last commit date

Latest commit



13 Commits

Repository files navigation


A small utility for converting Stanford GloVe vectors to HDF5 / NumPy. The pretrained Stanford vectors are distributed as zipped text files with one line per vector. This is not the most convenient way of interacting with the vectors, this utility converts the zip files into NumPy arrays contained in HDF5 (using h5py) files with a separate sqllite dictionary for the vocabulry.

The GloVe code (in C) is available on github and you can download the pretrained Stanford GloVe vectors from

Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. GloVe: Global Vectors for Word Representation.

Install & Usage


Extract the 50-dimensional word vectors using LZF compression for HDF5

$ git clone
$ cd glove2h5
$ python -m glove2h5 ~/Downloads/ --collection glove.6B.50d.txt --compression lzf

Assuming you've downloaded the GloVe vectors from into ./GloVe

Convert all GloVe vectors in to NumPy and store in an HDF5 file. The call below creates a vocabulary stored in a sqlitedict.SqliteDict and extracts the GloVe vectors into an vectors.h5 file. The results are stored in the current directory under $SOURCEFILE.glove2h5 where $SOURCEFILE is the name of the original source file, without the file extension. A separate vocabulary is created to allow indexing the vectors from HDF5.

glove2h5 = GloVe2H5.create_from('./GloVe/', compression='lzf')

# get the 50 dimensional vector for 'sample'

# get the 100 dimensional vector for 'sample'

Extract only certain dimensional vectors

The file contains vectors in 50, 100, 200 and 300 dimensions. Each of these is stored in a separate file in the zip archive.

-- glove.6B.50d.txt    # 50 dimensional vectors
-- glove.6B.100d.txt   # 100 dimensional vectors
-- glove.6B.200d.txt   # 200 dimensional vectors
-- glove.6B.300d.txt   # 300 dimensional vectors

Extracting all of them into HDF5 is unnecessary (and obivously slow) if you only need some of them. You can provide a keyword to create_from to only extract certain files contained in the zip archive.

# extract only the 100 dimensional vectors
glove2h5_100d = GloVe2H5.create_from('./GloVe/', collections=['glove.6B.100d.txt'], compression='lzf')`

# the collection is defined automatically as 'glove.6B.100D'

Load already extracted vectors

You can load an earlier extracted set of vectors by just calling the constructor

glove2h5_100d = GloVe2H5('./GloVe/', collection='glove.6B.100d.txt')`

# the collection was defined to be 'glove.6B.100D' so we don't need it for __getitem__ anymore


  • Python 3.6
  • h5py
  • sqlitedict


A small utility for converting Stanford GloVe vectors to HDF5 / NumPy







No releases published


No packages published