1D CNN+LSTM Configuration Exception

DL4J
1

the net config:
‘’’
protected int cnnStrides = {1, 2};// Strides for each CNN layer

protected int[] cnnNeurons = {72, 36}; //cnn各层的神经元数量
protected int[] rnnNeurons={64,32};//rnn各层的神经元数量
int[] cnnKernelSizes = {3, 3}; // Kernel sizes for each CNN layer  
int[] cnnPaddings = {1,1}; // Paddings for each CNN layer

public MultiLayerConfiguration getNetConf() {
    DataType dataType = DataType.FLOAT;
    NeuralNetConfiguration.Builder nncBuilder = new NeuralNetConfiguration.Builder()
            .seed(System.currentTimeMillis())
            .weightInit(WeightInit.XAVIER)
            .optimizationAlgo(OptimizationAlgorithm.STOCHASTIC_GRADIENT_DESCENT)
            .updater(new Adam(lrSchedule))//(lrSchedule))
            //                .gradientNormalization(GradientNormalization.RenormalizeL2PerLayer)
            .dataType(dataType);

    nncBuilder.l1(l1);
    nncBuilder.l2(l2);

    NeuralNetConfiguration.ListBuilder listBuilder = nncBuilder.list();
    int nIn = featuresCount;//36
    int layerIndex = 0;
    final int cnnLayerCount = cnnNeurons.length;
   
    // Add CNN layers
    for (int i = 0; i < cnnLayerCount; i++) {
        listBuilder.layer(layerIndex, new Convolution1D.Builder()
                .kernelSize(cnnKernelSizes[i])
                .stride(cnnStrides[i])
                .padding(cnnPaddings[i])
                .nIn(nIn)
                .nOut(cnnNeurons[i])
                .activation(Activation.RELU)
                .build());
        
        nIn = cnnNeurons[i];
        ++layerIndex;
    }
    
    // Add RNN layers
    for (int i = 0; i < this.rnnNeurons.length; ++i) {
        listBuilder.layer(layerIndex, new LSTM.Builder()
                .dropOut(dropOut)
                .activation(Activation.SOFTSIGN)
                .nIn(nIn)
                .nOut(rnnNeurons[i])
                .build());

        nIn = rnnNeurons[i];
        ++layerIndex;

    }

    listBuilder.layer(layerIndex, new RnnOutputLayer.Builder(new LossMSE()).updater(new Adam(outLrSchedule))//
            .activation(Activation.IDENTITY).nIn(nIn).nOut(1).build());

    MultiLayerConfiguration conf = listBuilder.build();
    return conf;
}

‘’’
Exception in thread “main” java.lang.IllegalStateException: Sequence lengths do not match for RnnOutputLayer input and labels:Arrays should be rank 3 with shape [minibatch, size, sequenceLength] - mismatch on dimension 2 (sequence length) - input=[256, 32, 30] vs. label=[256, 1, 30]
at org.nd4j.common.base.Preconditions.throwStateEx(Preconditions.java:639)
at org.nd4j.common.base.Preconditions.checkState(Preconditions.java:337)
at org.deeplearning4j.nn.layers.recurrent.RnnOutputLayer.backpropGradient(RnnOutputLayer.java:59)
at org.deeplearning4j.nn.multilayer.MultiLayerNetwork.calcBackpropGradients(MultiLayerNetwork.java:1998)
at org.deeplearning4j.nn.multilayer.MultiLayerNetwork.computeGradientAndScore(MultiLayerNetwork.java:2813)
at org.deeplearning4j.nn.multilayer.MultiLayerNetwork.computeGradientAndScore(MultiLayerNetwork.java:2756)
at org.deeplearning4j.optimize.solvers.BaseOptimizer.gradientAndScore(BaseOptimizer.java:174)
at org.deeplearning4j.optimize.solvers.StochasticGradientDescent.optimize(StochasticGradientDescent.java:61)
at org.deeplearning4j.optimize.Solver.optimize(Solver.java:52)
at org.deeplearning4j.nn.multilayer.MultiLayerNetwork.fitHelper(MultiLayerNetwork.java:1767)
at org.deeplearning4j.nn.multilayer.MultiLayerNetwork.fit(MultiLayerNetwork.java:1688)
at com.cq.aifocusstocks.train.RnnPredictModel.train(RnnPredictModel.java:175)
at com.cq.aifocusstocks.train.CnnLstmRegPredictor.trainModel(CnnLstmRegPredictor.java:209)
at com.cq.aifocusstocks.train.TrainCnnLstmModel.main(TrainCnnLstmModel.java:15)

the pom.xml:
‘’’

4.0.0
com.mycompany
AIFocusStocks-train
1.0-SNAPSHOT
jar

<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<maven.compiler.source>17</maven.compiler.source>
<maven.compiler.target>17</maven.compiler.target>
<nd4j.version>1.0.0-M2.1</nd4j.version>
<dl4j.version>1.0.0-M2.1</dl4j.version>
<nd4j-cuda.version>1.0.0-M2.1</nd4j-cuda.version>


<dl4j.ui.version>1.0.0-M2.1</dl4j.ui.version>

<dependencies>        
    <dependency>
        <groupId>org.nd4j</groupId>
        <artifactId>nd4j-cuda-11.6</artifactId>
        <version>${nd4j-cuda.version}</version>
    </dependency>
    <dependency>
        <groupId>org.nd4j</groupId>
        <artifactId>nd4j-cuda-11.6</artifactId>            
        <version>${nd4j-cuda.version}</version>
        <classifier>linux-x86_64</classifier>
    </dependency>
    <dependency>
        <groupId>org.deeplearning4j</groupId>
        <artifactId>deeplearning4j-core</artifactId>
        <version>${dl4j.version}</version>
    </dependency>

    <dependency>
        <groupId>org.deeplearning4j</groupId>
        <artifactId>deeplearning4j-ui</artifactId>
        <version>${dl4j.ui.version}</version>
    </dependency>

    <dependency>
        <groupId>org.deeplearning4j</groupId>
        <artifactId>deeplearning4j-parallel-wrapper</artifactId>
        <version>${dl4j.version}</version>
        <type>jar</type>
    </dependency>
</dependencies>
'''

cuda: cuda-repo-ubuntu2204-11-8

cudnn: cudnn-linux-x86_64-8.9.7.29_cuda11-archive

OS： ubuntu2024.04

2

@agibsonccc The exception message indicates that the sequence lengths between the output layer’s input and the labels do not match, but both are 30. The issue seems to be with the first dimension (input=[256, 32, 30] vs. label=[256, 1, 30]). The output layer’s nIn is 32, and shouldn’t the labels match the shape of the output? Why should it match the input shape?

3

@AlexBlack Is it necessary to add a preprocessor between the 1D CNN layer and the LSTM layer? The data shapes are consistent between the two, so there’s no need to add one, right?

4

‘’’
listBuilder.layer(layerIndex, new RnnOutputLayer.Builder(new LossMSE()).updater(new Adam(outLrSchedule)).activation(Activation.IDENTITY).nIn(nIn).nOut(1).dataFormat(RNNFormat.NCW).build());

‘’’
This exception has been resolved.