Does anyone know how to load and classify a single image with TransferLearningHelper?

DL4J
1

Hi guys, I am trying to classify a single image using the TransferLearningHelper class. Unfortunately the DL4J examples only show how to evaluate the model using iterators , splitting files in a directory in training and test sets. I am trying to classify an image using this code:

NativeImageLoader nativeImageLoader = new NativeImageLoader(height, width,3);
 DataNormalization scaler = new VGG16ImagePreProcessor();
INDArray img = nativeImageLoader.asMatrix(new File("C:\\sample.jpg"));
 scaler..transform(img);
 transferLearningHelper.outputFromFeaturized(img);

This code is not working , the method "outputFromFeaturized (img) "
throws exception …

thanks

2

@javaberlin could you adapt the example we have publicly to something I can take a look at so I can understand all the assumptions made when running this? Thanks!

3

@agibsonccc I am using the FitFromFeatutized.java example
at deeplearning4j-examples/FitFromFeaturized.java at master · eclipse/deeplearning4j-examples · GitHub

The piece of code below is what I added to that example ( after line 110 ) , basically I am saving the model , and trying to open it. That part works. After that I read a single image from the flowers directory and try to classify it. This is where I get an Exception:

Caused by: org.deeplearning4j.exception.DL4JInvalidInputException: Input that is not a matrix; expected matrix (rank 2), got rank 4 array with shape [1, 3, 224, 224]. Missing preprocessor or wrong input type? (layer name: predictions, layer index: 1, layer type: OutputLayer)

This is the code I added to the “FitFearturized.java” class :


        for (int epoch = 0; epoch < nEpochs; epoch++) {
            if (epoch == 0) {
                Evaluation eval = transferLearningHelper.unfrozenGraph().evaluate(testIter);
                log.info("Eval stats BEFORE fit.....");
                log.info(eval.stats()+"\n");
                testIter.reset();
            }
            int iter = 0;
            while (trainIter.hasNext()) {
                transferLearningHelper.fitFeaturized(trainIter.next());
                if (iter % 10 == 0) {
                    log.info("Evaluate model at iter " + iter + " ....");
                    Evaluation eval = transferLearningHelper.unfrozenGraph().evaluate(testIter);
                    log.info(eval.stats());
                    testIter.reset();
                }
                iter++;
            }
            trainIter.reset();
            log.info("Epoch #"+epoch+" complete");
        }
        log.info("Model build complete");
//  ##########  BEGINN ADDED CODE #########
        ModelSerializer.writeModel(vgg16Transfer, new File("M:\\models\\flowersmodel.zip"), true);
        ComputationGraph model = null;
        log.info("Trying to open the model");
        try {
            model = ModelSerializer.restoreComputationGraph("M:\\models\\flowersmodel.zip");
            log.info("Success: Model opened");
        } catch (Exception e) {
           log.error("Unable to open model: " +e.getMessage());
        }

        NativeImageLoader nativeImageLoader = new NativeImageLoader(224, 224,3);
        INDArray img = nativeImageLoader.asMatrix(new File("C:\\Users\\kh\\dl4j-examples-data\\dl4j-examples\\flower_photos\\daisy\\5547758_eea9edfd54_n.jpg"));
        DataNormalization scaler = new VGG16ImagePreProcessor();
        scaler.transform(img);
        INDArray[] output = transferLearningHelper.unfrozenGraph().output(false, img);
        log.info("Classification: "+output);
    }

My questions would be:

  1. Is this the right way to save the trained model , using the vgg16Transfer object, or do I have to save the “transferLearningHelper.unfrozenGraph()” instead ?
  2. There is something wrong the way I am loading the image, how to do it correctly ?
  3. How to do the classification ? There are other methods like “outputSingle(…)”, “outputFeaturized(…)” etc. …

Thanks