Beta 7 - Glove Word Vector

DL4J
4

It looks like you have stumbled upon a bug in beta7.

As a workaround, you can load the file in beta6 and then save it as a binary in dl4j format like this:

Word2Vec wordVectors = WordVectorSerializer.readWord2VecModel(new File("glove.6B.50d.txt"));
WordVectorSerializer.writeWord2VecModel(wordVectors, new File("glove.bin"));

And then you can reload the binary in beta7:

Word2Vec wordVectors = WordVectorSerializer.readWord2VecModel(new File("glove.bin"));

We have filed an issue for the underlying problem: ND4J: concat op fails with 3073 or more inputs on CPU · Issue #8961 · eclipse/deeplearning4j · GitHub

Edit: It appears to be a problem with the mkldnn implementation. If you don’t want to go back and forth between versions, you can also disable mkldnn with Nd4j.getEnvironment().allowHelpers(false); before loading and save the file in another format that way.