Deep Learning 4j: using a local model

DL4J
1

So given the following code:

    private Criteria<String, float[]> makeCriteria() {
        return Criteria.builder()
                .setTypes(String.class, float[].class)
                .optModelUrls(MODEL_URL)
                .optEngine("PyTorch")
                .optTranslatorFactory(new TextEmbeddingTranslatorFactory())
                .build();
    }

Works great for dowloading a model and returning sentence embeddings fast. However, given that it needs the model URL:

String MODEL_URL = "djl://ai.djl.huggingface.pytorch/sentence-transformers/all-MiniLM-L12-v2";

It loads this from the website directly, so I’m trying to refactor it to load from the disk instead.

To do this I see I should only need to do the following:

    private Criteria<String, float[]> makeCriteria() {
        return Criteria.builder()
                .setTypes(String.class, float[].class)
                .optModelPath(MODEL_PATH)
                .optEngine("PyTorch")
                .optTranslatorFactory(new TextEmbeddingTranslatorFactory())
                .build();
    }

But I can’t seem to find out how to take a model from my cache and point it to the model above.

I’ve tried pointing it directly to the cache directory:

             .optModelPath(Path.of("/Users/krickert/.djl.ai/cache/repo/model/nlp/text_embedding/ai/djl/huggingface/pytorch/sentence-transformers/\n" +
                        "all-MiniLM-L12-v2/true/0.0.1/"))

and even tried "all-MiniLM-L12-v2/’

Can someone let me know what I can change to have it load from dsk only?

2

Figured it out!

    private Criteria<String, float[]> makeCriteria() {
        return Criteria.builder()
                .setTypes(String.class, float[].class)
                .optModelPath(Path.of("/Users/krickert/.djl.ai/cache/repo/model/nlp/text_embedding/ai/djl/huggingface/pytorch/sentence-transformers/all-MiniLM-L12-v2/true/0.0.1" ))
                .optModelName("all-MiniLM-L12-v2")
                .optEngine("PyTorch")
                .optTranslatorFactory(new TextEmbeddingTranslatorFactory())
                .build();
    }

So I can work with this:

  1. Locally run it with the URL so it’ll download the model manually
  2. Package up those contents
  3. Deploy that

Are there any instructions on how the repo model is structured? I’d still like to do this properly and host my own local repo on the network.

3

I think you’ve asked the question in the wrong place. This is the DL4J community, your question is about DJL, a different library.

But I’m happy that you managed to figure out a solution anyway.