Cannot read the array length because "inputsForOp" is null

SidneyLann · August 16, 2022, 2:41pm

This issue mark fixed at #9145 and # 9744, but the exception still exists in M2.1

exception in M2:
java.lang.NullPointerException: Cannot load from object array because “inputsForOp” is null
at org.nd4j.linalg.api.ops.impl.layers.recurrent.LSTMLayer.configureWithSameDiff(LSTMLayer.java:210)
at org.nd4j.autodiff.samediff.SameDiff.fromFlatBuffers(SameDiff.java:5503)
at org.nd4j.autodiff.samediff.SameDiff.fromFlatFile(SameDiff.java:5359)
at org.nd4j.autodiff.samediff.SameDiff.load(SameDiff.java:5250)
at org.nd4j.autodiff.listeners.checkpoint.CheckpointListener.loadCheckpoint(CheckpointListener.java:407)

exception in M2.1:
java.lang.NullPointerException: Cannot read the array length because “inputsForOp” is null
at org.nd4j.linalg.api.ops.impl.layers.recurrent.LSTMLayer.configureWithSameDiff(LSTMLayer.java:210)
at org.nd4j.autodiff.samediff.SameDiff.fromFlatBuffers(SameDiff.java:6058)
at org.nd4j.autodiff.samediff.SameDiff.fromFlatFile(SameDiff.java:5914)
at org.nd4j.autodiff.samediff.SameDiff.load(SameDiff.java:5805)
at org.nd4j.autodiff.listeners.checkpoint.CheckpointListener.loadCheckpoint(CheckpointListener.java:407)
at org.nd4j.autodiff.listeners.checkpoint.CheckpointListener.loadLastCheckpoint(CheckpointListener.java:417)

agibsonccc · August 25, 2022, 1:16am

@SidneyLann do you have the model?

SidneyLann · August 25, 2022, 2:48pm

I sent you my project before, it contains the model. Or you can remove the old model and change it to use M2.1 and train several minutes and then get a model. thanks.

agibsonccc · August 30, 2022, 4:54am

@SidneyLann let me see if this does it: Fixes #9756 by agibsonccc · Pull Request #9757 · deeplearning4j/deeplearning4j · GitHub In my tests everything ran. Not sure what the difference was. Part of it was also just some changes not being committed properly (likely a botched pull request merge)

Sorry for the confusion. I’ll do another point release to address this.

SidneyLann · August 30, 2022, 4:57am

OK. Thank you for your help!

SidneyLann · September 13, 2022, 2:42am

This issue still exists.
java.lang.NullPointerException: Cannot load from object array because “inputsForOp” is null
at org.nd4j.linalg.api.ops.impl.layers.recurrent.LSTMLayer.configureWithSameDiff(LSTMLayer.java:210)
at org.nd4j.autodiff.samediff.SameDiff.fromFlatBuffers(SameDiff.java:5503)
at org.nd4j.autodiff.samediff.SameDiff.fromFlatFile(SameDiff.java:5359)
at org.nd4j.autodiff.samediff.SameDiff.load(SameDiff.java:5250)
at org.nd4j.autodiff.listeners.checkpoint.CheckpointListener.loadCheckpoint(CheckpointListener.java:407)
at org.nd4j.autodiff.listeners.checkpoint.CheckpointListener.loadLastCheckpoint(CheckpointListener.java:417)

agibsonccc · September 13, 2022, 2:52am

@SidneyLann what version are you using? If you look straight at the source this is impossible. We do a null check right here:

github.com

deeplearning4j/deeplearning4j/blob/29fd9c70c0dfa690ccedba130b43dc3e36c37b24/nd4j/nd4j-backends/nd4j-api-parent/nd4j-api/src/main/java/org/nd4j/linalg/api/ops/impl/layers/recurrent/LSTMLayer.java#L210


      
          boolean  hasBiases = bArguments.get(0);   // indicates whether biases array is provided
          boolean  hasSeqLen = bArguments.get(1);   // indicates whether seqLen array is provided
          boolean  hasInitH = bArguments.get(2);    // indicates whether initial output is provided
          boolean  hasInitC =bArguments.get(3);    // indicates whether initial cell state is provided
          boolean  hasPH = bArguments.get(4);       // indicates whether peephole connections are present
          boolean  retFullSeq = bArguments.get(5);  // indicates whether gradient vs. outputs is given for whole time sequence dLdh
          // {dLdh_0, dLdh_1, ... , dLdh_sL-1}
          boolean  retLastH = bArguments.get(6);    // indicates whether gradient vs. output at last time step (dLdhL) is given
          boolean  retLastC = bArguments.get(7);    // indicates whether gradient vs. cell state at last time step (dLdcL) is given
          
          
if(inputsForOp != null && inputsForOp.length > 1)
              builder.weights(sameDiff.getVariable(inputsForOp[1]));
          if(inputsForOp != null && inputsForOp.length > 2)
              builder.rWeights(sameDiff.getVariable(inputsForOp[2]));
          
          

          
if(hasBiases) {
              builder.bias(sameDiff.getVariable(inputsForOp[3]));
          }
          
          
if(hasPH) {

@SidneyLann edit: I downloaded the source for M2.1. It appears that’s still in there. It’s already fixed in master so I guess just use snapshots. I’ll be doing another release soon anyways after checking one more issue.

SidneyLann · September 13, 2022, 7:45am

I can’t download new jar, the pom setting is not correct?

agibsonccc · September 13, 2022, 12:28pm

@SidneyLann that’s the old sonatype host. You need to use s01.oss.sonatype.org now.

SidneyLann · September 13, 2022, 12:37pm

Thanks. But NPE at this line now:
builder.peepholeWeights(sameDiff.getVariable(inputsForOp[inputsForOp.length - 1]));

java.lang.NullPointerException: Cannot read the array length because “inputsForOp” is null
at org.nd4j.linalg.api.ops.impl.layers.recurrent.LSTMLayer.configureWithSameDiff(LSTMLayer.java:205)

Need to check all references to inputsForOp.

agibsonccc · September 13, 2022, 1:22pm

@SidneyLann I wonder if this is actually burying a deeper problem somewhere…has peephole and those booleans are either wrong or the state configured there mismatches your parameters. Before we go out of our way adding null checks for everything I want to make sure that we’re not trying to just work around a problem somewhere else.

SidneyLann · September 14, 2022, 12:08am

    if(inputsForOp != null && hasBiases) {
        builder.bias(sameDiff.getVariable(inputsForOp[3]));
    }

    if(inputsForOp != null && hasPH) {
        builder.peepholeWeights(sameDiff.getVariable(inputsForOp[inputsForOp.length - 1]));
    }

When I check like this, it can be trained and reloaded to train.

agibsonccc · September 14, 2022, 3:57am

@SidneyLann I haven’t looked at this yet but the main thing I wanted to look at was to see if the neural network was imported correctly. There seems to be something off with variables being missing. I haven’t seen that a lot and want to make sure everything is correct.

Topic		Replies	Views
"memoryManagerClazz" is null SameDiff	3	192	September 30, 2022
Parallel Inference NullPointerException DL4J	4	46	April 24, 2024
Nd4j throwing errors during maven goal execution ND4J	7	413	December 9, 2021
JsonMappingException ND4J	35	1446	December 4, 2020
Exception in thread "ADSI prefetch thread" Exception in thread "main" java.lang.RuntimeException: java.lang.IllegalArgumentException: Invalid indices: cannot get [1,5] from a [20, 5] NDArray DL4J	7	787	September 24, 2020

Cannot read the array length because "inputsForOp" is null

Related Topics