# Multi Label Sequence Classification

**URL:** https://community.konduit.ai/t/multi-label-sequence-classification/1079
**Category:** Tuning Help
**Created:** [December 25, 2020, 4:28pm UTC](https://community.konduit.ai/t/multi-label-sequence-classification/1079 "2020-12-25T16:28:17Z")
**Posts on this page:** 5
**Page:** 1

<div class="post-metadata">

### Author: ![DarkWingMcQuack](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/darkwingmcquack/32/411_2.png) [@DarkWingMcQuack](https://community.konduit.ai/u/DarkWingMcQuack)
#### Post date: [December 25, 2020, 4:28pm UTC](https://community.konduit.ai/t/multi-label-sequence-classification/1079/1 "2020-12-25T16:28:17Z")

</div>

Hello Everybody,  
I am trying to do a multi label classification for sequences.  
I started with a simple LSTM and used this as configuration:

```java
    new NeuralNetConfiguration.Builder()
      .seed(seed)
      .updater(new Adam())
      .l2(1e-5)
      .weightInit(WeightInit.XAVIER)
      .biasInit(0)
      .gradientNormalization(GradientNormalization.ClipElementWiseAbsoluteValue)
      .gradientNormalizationThreshold(10.0)
      .trainingWorkspaceMode(WorkspaceMode.ENABLED)
      .inferenceWorkspaceMode(WorkspaceMode.ENABLED)
      .list()
      .layer(
        0,
        new LSTM.Builder()
          .nIn(numberOfFeatures)
          .nOut(100)
          .activation(Activation.TANH)
          .build()
      )
      .layer(
        1,
        new RnnOutputLayer.Builder()
          .activation(Activation.SIGMOID)
          .weightInit(WeightInit.XAVIER)
          .lossFunction(new LossMultiLabel())
          .nIn(100)
          .nOut(numberOfLabels)
          .build()
      )
      .build()

```

but after the training the evaluation of the network always looks like this:  
 ![grafik](https://canada1.discourse-cdn.com/flex035/uploads/konduit/original/1X/5bd88d1e2e3290574bdc8520ddb228136cb3f959.png)

To evaluate this i used a `RocBinary`.  
Is there something obviously wrong with what I am doing here?  
If I use `LossBinaryXENT` instead of `LossMultiLabel`, the same thing happens.  
I hope someone can help me with this.  
Greetings Quack

---

<div class="post-metadata">

### Author: ![DarkWingMcQuack](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/darkwingmcquack/32/411_2.png) [@DarkWingMcQuack](https://community.konduit.ai/u/DarkWingMcQuack)
#### Post date: [December 28, 2020, 3:00pm UTC](https://community.konduit.ai/t/multi-label-sequence-classification/1079/2 "2020-12-28T15:00:10Z")

</div>

To give a little bit more information, here are my train and test iterators:

```nohighlight
    val trainFeatures = new CSVSequenceRecordReader();
    trainFeatures.initialize(
      new NumberedFileInputSplit(
        trainFolder + "/%d.csv",
        0,
        numberOfTrainFiles - 1
      )
    );

    val trainLabels = new CSVSequenceRecordReader();
    trainLabels.initialize(
      new NumberedFileInputSplit(
        trainLabelFolder + "/%d.csv",
        0,
        numberOfTrainFiles - 1
      )
    );

    val trainIter = new SequenceRecordReaderDataSetIterator(
      trainFeatures,
      trainLabels,
      batchSize,
      numberOfLabels,
      false,
      SequenceRecordReaderDataSetIterator.AlignmentMode.ALIGN_END
    );

    val testFeatures = new CSVSequenceRecordReader()
    testFeatures.initialize(
      new NumberedFileInputSplit(
        testFolder + "/%d.csv",
        trainMinIdx,
        trainMaxIdx
      )
    )

    val testLabels = new CSVSequenceRecordReader()
    testLabels.initialize(
      new NumberedFileInputSplit(
        testLabelFolder + "/%d.csv",
        testMinIdx,
        testMaxIdx
      )
    )

    val testIter = new SequenceRecordReaderDataSetIterator(
      testFeatures,
      testLabels,
      batchSize,
      numberOfLabels,
      false,
      SequenceRecordReaderDataSetIterator.AlignmentMode.ALIGN_END
    )

```

I realy need help with this issue, there dont seem to be any examples for multi label classification, and i dont know if i am doing something wrong here.  
Greetings Duck

---

<div class="post-metadata">

### Author: ![cagneymoreau](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/cagneymoreau/32/296_2.png) [@cagneymoreau](https://community.konduit.ai/u/cagneymoreau)
#### Post date: [December 29, 2020, 2:29am UTC](https://community.konduit.ai/t/multi-label-sequence-classification/1079/3 "2020-12-29T02:29:15Z")

</div>

have you tried a practice project where you use a lstm net to predict a sine wave first?

---

<div class="post-metadata">

### Author: ![DarkWingMcQuack](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/darkwingmcquack/32/411_2.png) [@DarkWingMcQuack](https://community.konduit.ai/u/DarkWingMcQuack)
#### Post date: [December 29, 2020, 10:21am UTC](https://community.konduit.ai/t/multi-label-sequence-classification/1079/4 "2020-12-29T10:21:56Z")

</div>

Yes, I have build LSTM networks for classification and regression before, i just cannot seem to make the multi label classification work.

---

<div class="post-metadata">

### Author: ![thomas](https://avatars.discourse-cdn.com/v4/letter/t/e36b37/32.png) [@thomas](https://community.konduit.ai/u/thomas)
#### Post date: [June 24, 2021, 2:32pm UTC](https://community.konduit.ai/t/multi-label-sequence-classification/1079/5 "2021-06-24T14:32:09Z")

</div>

I am not sure how exactly multi-label works. but have you trid some other activation function for the output layer like SOFTMAX which is often used for classification problems?