Adding trained neural net to android app

DL4J
1

MultiLayerNetwork model = null;
try {
AssetManager assetManager = context.getAssets();
InputStream modelInputStream = assetManager.open(“small_test.zip”);
model = ModelSerializer.restoreMultiLayerNetwork(modelInputStream);

        if (model != null) {
            Log.i("NeuralNetService", "Model loaded successfully. Number of layers: " + model.getnLayers());
        } else {
            Log.e("NeuralNetService", "Error: Model not loaded");
        }

    } catch (IllegalArgumentException e) {
        Log.e("NeuralNetService", "Illegal argument while loading model", e);
    } catch (IOException e) {
        Log.e("NeuralNetService", "IO error while loading model", e);
    } catch (Exception e) {
        Log.e("NeuralNetService", "Unexpected error while loading model", e);
    }

Above is code I use to load a trained model (zip file) from my assets directory in an android app. I get the following error:

E/NeuralNetService(20162): Unexpected error while loading model
E/NeuralNetService(20162): java.lang.IllegalStateException: Illegal set of indices for array: need at least 1 point/interval/all/specified indices for rank 1 array ([295203]), got indices [Interval(b=0,e=0,s=1,inclusive), Interval(b=0,e=160,s=1)]
E/NeuralNetService(20162): at org.nd4j.base.Preconditions.throwStateEx(Preconditions.java:641)
E/NeuralNetService(20162): at org.nd4j.base.Preconditions.checkState(Preconditions.java:412)
E/NeuralNetService(20162): at org.nd4j.linalg.api.ndarray.BaseNDArray.get(BaseNDArray.java:4119)
E/NeuralNetService(20162): at org.deeplearning4j.nn.multilayer.MultiLayerNetwork.init(MultiLayerNetwork.java:709)…

any assistance will be appreciated

2

This error happens when loading in the model. So it is not an issue related to the input to the model, as it gives this error even without passing any input to the model.

3

@CrossFawn1 print out the shapes. It’s saying your indexing here:
62): java.lang.IllegalStateException: Illegal set of indices for array: need at least 1 point/interval/all/specified indices for rank 1 array ([295203]), got indices [Interval(b=0,e=0,s=1,inclusive), Interval(b=0,e=160,s=1)]

is out of bounds. Double check your neural net sizes and your inputs data/labels.

4

when you say the neural net sizes what exactly are you referring to? I have been looking through the configuration files produced after saving the model looking for errors in there, but everything seems fine.

Also to add context, the models load fine in intellij, using the normal restoring method. The issue really is just in loading into my Android app. Furthermore I have created multiple different models to try and see where the problem could be, and they all give the same error, just with the value of e, (in this line: Interval(b=0,e=160,s=1)) being different.

Is the problem more likely coming from the saved model itself or is it an issue related to the loading process ?

also thank you for the help.

5

@CrossFawn1 I don’t see your model architecture.

The “shapes” I need to see are what the network expects as data, what your input data’s shape is and depending on the layer what the layer’s output is.

Everything works from 1 layer to the next starting with your input data.

6

Shapes:

Layer 0 (ConvolutionLayer) output shape: Rank: 2, DataType: FLOAT, Offset: 0, Order: c, Shape: [1,3], Stride: [1,1]
Layer 1 (SubsamplingLayer) output shape: Rank: 2, DataType: FLOAT, Offset: 0, Order: c, Shape: [1,3], Stride: [1,1]
Layer 2 (ConvolutionLayer) output shape: Rank: 2, DataType: FLOAT, Offset: 0, Order: c, Shape: [1,3], Stride: [1,1]
Layer 3 (ConvolutionLayer) output shape: Rank: 2, DataType: FLOAT, Offset: 0, Order: c, Shape: [1,3], Stride: [1,1]
Layer 4 (SubsamplingLayer) output shape: Rank: 2, DataType: FLOAT, Offset: 0, Order: c, Shape: [1,3], Stride: [1,1]
Layer 5 (ConvolutionLayer) output shape: Rank: 2, DataType: FLOAT, Offset: 0, Order: c, Shape: [1,3], Stride: [1,1]
Layer 6 (ConvolutionLayer) output shape: Rank: 2, DataType: FLOAT, Offset: 0, Order: c, Shape: [1,3], Stride: [1,1]
Layer 7 (SubsamplingLayer) output shape: Rank: 2, DataType: FLOAT, Offset: 0, Order: c, Shape: [1,3], Stride: [1,1]
Layer 8 (ConvolutionLayer) output shape: Rank: 2, DataType: FLOAT, Offset: 0, Order: c, Shape: [1,3], Stride: [1,1]
Layer 9 (ConvolutionLayer) output shape: Rank: 2, DataType: FLOAT, Offset: 0, Order: c, Shape: [1,3], Stride: [1,1]
Layer 10 (SubsamplingLayer) output shape: Rank: 2, DataType: FLOAT, Offset: 0, Order: c, Shape: [1,3], Stride: [1,1]
Layer 11 (ConvolutionLayer) output shape: Rank: 2, DataType: FLOAT, Offset: 0, Order: c, Shape: [1,3], Stride: [1,1]
Layer 12 (ConvolutionLayer) output shape: Rank: 2, DataType: FLOAT, Offset: 0, Order: c, Shape: [1,3], Stride: [1,1]
Layer 13 (DenseLayer) output shape: Rank: 2, DataType: FLOAT, Offset: 0, Order: c, Shape: [1,3], Stride: [1,1]
Layer 14 (DenseLayer) output shape: Rank: 2, DataType: FLOAT, Offset: 0, Order: c, Shape: [1,3], Stride: [1,1]
Layer 15 (OutputLayer) output shape: Rank: 2, DataType: FLOAT, Offset: 0, Order: c, Shape: [1,3], Stride: [1,1]

Architecture:

MultiLayerConfiguration conf = new NeuralNetConfiguration.Builder()
.seed(seed)
.optimizationAlgo(OptimizationAlgorithm.STOCHASTIC_GRADIENT_DESCENT)
.updater(new Adam(0.0001))
.l2(0.0001)
.list()
.layer(0, new ConvolutionLayer.Builder()
.kernelSize(3,3) // dimensions of filters
.stride(1,1)
.padding(1,1)
.nOut(64) // number of filters
.activation(Activation.RELU)
.weightInit(WeightInit.RELU)
.build())
.layer(1, new SubsamplingLayer.Builder(SubsamplingLayer.PoolingType.MAX)
.kernelSize(2,2)
.stride(2,2)
.build())
.layer(2, new ConvolutionLayer.Builder()
.kernelSize(3,3)
.stride(1,1)
.padding(1,1)
.nOut(64)
.activation(Activation.RELU)
.weightInit(WeightInit.RELU)
.build())
.layer(3, new ConvolutionLayer.Builder()
.kernelSize(3,3)
.stride(1,1)
.padding(1,1)
.nOut(128)
.activation(Activation.RELU)
.weightInit(WeightInit.RELU)
.build())
.layer(4, new SubsamplingLayer.Builder(SubsamplingLayer.PoolingType.MAX)
.kernelSize(2,2)
.stride(2,2)
.build())
.layer(5, new ConvolutionLayer.Builder()
.kernelSize(3,3)
.stride(1,1)
.padding(1,1)
.nOut(256)
.activation(Activation.RELU)
.weightInit(WeightInit.RELU)
.build())
.layer(6, new ConvolutionLayer.Builder()
.kernelSize(3,3)
.stride(1,1)
.padding(1,1)
.nOut(256)
.activation(Activation.RELU)
.weightInit(WeightInit.RELU)
.build())
.layer(7, new SubsamplingLayer.Builder(SubsamplingLayer.PoolingType.MAX)
.kernelSize(2,2)
.stride(2,2)
.build())
.layer(8, new ConvolutionLayer.Builder()
.kernelSize(3,3)
.stride(1,1)
.padding(1,1)
.nOut(256)
.activation(Activation.RELU)
.weightInit(WeightInit.RELU)
.build())
.layer(9, new ConvolutionLayer.Builder()
.kernelSize(3,3)
.stride(1,1)
.padding(1,1)
.nOut(256)
.activation(Activation.RELU)
.weightInit(WeightInit.RELU)
.build())
.layer(10, new SubsamplingLayer.Builder(SubsamplingLayer.PoolingType.MAX)
.kernelSize(2,2)
.stride(2,2)
.build())
.layer(11, new ConvolutionLayer.Builder()
.kernelSize(3,3)
.stride(1,1)
.padding(1,1)
.nOut(256)
.activation(Activation.RELU)
.weightInit(WeightInit.RELU)
.build())
.layer(12, new ConvolutionLayer.Builder()
.kernelSize(3,3)
.stride(1,1)
.padding(1,1)
.nOut(512)
.activation(Activation.RELU)
.weightInit(WeightInit.RELU)
.build())
.layer(13, new DenseLayer.Builder()
.activation(Activation.RELU)
.nOut(128)
.weightInit(WeightInit.RELU)
.dropOut(0.3)
.build())
.layer(14, new DenseLayer.Builder()
.activation(Activation.RELU)
.nOut(32)
.weightInit(WeightInit.RELU)
.dropOut(0.3)
.build())
.layer(15, new OutputLayer.Builder(LossFunctions.LossFunction.NEGATIVELOGLIKELIHOOD)
.nIn(32)
.nOut(3)
.activation(Activation.SOFTMAX)
.build())
.setInputType(InputType.convolutional(image_height, image_width, color_channels))
.build();

model is trained on 48x48 greyscale facial images, from the fer-2013 dataset, but uses only happy, angry and sad emotions