# Label and features in the same file for time series data (SequenceRecordReader.DataSetIterator) **URL:** \ **Category:** DataVec\ **Created:** [February 16, 2021, 5:00pm UTC](https://community.konduit.ai/t/label-and-features-in-the-same-file-for-time-series-data-sequencerecordreader-datasetiterator/1186 "2021-02-16T17:00:35Z")\ **Posts on this page:** 15\ **Page:** 1
**Author:** ![absurd](https://avatars.discourse-cdn.com/v4/letter/a/59ef9b/32.png) [@absurd](https://community.konduit.ai/u/absurd)\ **Post date:** [February 16, 2021, 5:00pm UTC](https://community.konduit.ai/t/label-and-features-in-the-same-file-for-time-series-data-sequencerecordreader-datasetiterator/1186/1 "2021-02-16T17:00:35Z")
The SequenceRecordReaderDataSetIterator has a form where the features and the labels come from the same reader ``` /** Constructor where features and labels come from the SAME RecordReader (i.e., target/label is a column in the * same data as the features). Defaults to regression = false - i.e., for classification * @param reader SequenceRecordReader with data * @param miniBatchSize size of each minibatch * @param numPossibleLabels number of labels/classes for classification * @param labelIndex index in input of the label index. If in regression mode and numPossibleLabels > 1, labelIndex denotes the * first index for labels. Everything before that index will be treated as input(s) and * everything from that index (inclusive) to the end will be treated as output(s) */ public SequenceRecordReaderDataSetIterator(SequenceRecordReader reader, int miniBatchSize, int numPossibleLabels, int labelIndex) { this(reader, miniBatchSize, numPossibleLabels, labelIndex, false); } ``` Im a bit confused about how this data should be shaped. Does that mean the label should be repeated for every time step? Meaning each step in the time series record would have the same value in the label column? t|features|label 1|FFFFF|L 2|FFFFF|L 3|FFFFF|L Or is there some other shape that I’m not seeing? Like perhaps the label should be formatted as the very last time step? 1FFFFF 2FFFFF 3FFFFF L ---
**Author:** ![agibsonccc](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/agibsonccc/32/697_2.png) [@agibsonccc](https://community.konduit.ai/u/agibsonccc)\ **Post date:** [February 19, 2021, 12:34pm UTC](https://community.konduit.ai/t/label-and-features-in-the-same-file-for-time-series-data-sequencerecordreader-datasetiterator/1186/2 "2021-02-19T12:34:28Z")
@absurd Take a look at a sample problem: [https://github.com/eclipse/deeplearning4j/blob/c523c4f0c728b2f78748f94e067413143d4f6967/deeplearning4j/deeplearning4j-core/src/test/java/org/deeplearning4j/datasets/datavec/RecordReaderDataSetiteratorTest.java#L155](https://github.com/eclipse/deeplearning4j/blob/c523c4f0c728b2f78748f94e067413143d4f6967/deeplearning4j/deeplearning4j-core/src/test/java/org/deeplearning4j/datasets/datavec/RecordReaderDataSetiteratorTest.java#L155) Here isthe associated test files repo: [https://github.com/KonduitAI/dl4j-test-resources/tree/master/src/main/resources](https://github.com/KonduitAI/dl4j-test-resources/tree/master/src/main/resources) Input Data: [https://github.com/KonduitAI/dl4j-test-resources/blob/master/src/main/resources/csvsequence\_0.txt](https://github.com/KonduitAI/dl4j-test-resources/blob/master/src/main/resources/csvsequence_0.txt) [https://github.com/KonduitAI/dl4j-test-resources/blob/master/src/main/resources/csvsequence\_1.txt](https://github.com/KonduitAI/dl4j-test-resources/blob/master/src/main/resources/csvsequence_1.txt) [https://github.com/KonduitAI/dl4j-test-resources/blob/master/src/main/resources/csvsequence\_2.txt](https://github.com/KonduitAI/dl4j-test-resources/blob/master/src/main/resources/csvsequence_2.txt) Labels: [https://github.com/KonduitAI/dl4j-test-resources/blob/master/src/main/resources/csvsequencelabels\_0.txt](https://github.com/KonduitAI/dl4j-test-resources/blob/master/src/main/resources/csvsequencelabels_0.txt) [https://github.com/KonduitAI/dl4j-test-resources/blob/master/src/main/resources/csvsequencelabels\_1.txt](https://github.com/KonduitAI/dl4j-test-resources/blob/master/src/main/resources/csvsequencelabels_1.txt) [https://github.com/KonduitAI/dl4j-test-resources/blob/master/src/main/resources/csvsequencelabels\_2.txt](https://github.com/KonduitAI/dl4j-test-resources/blob/master/src/main/resources/csvsequencelabels_2.txt) It basically boils down to 1 per step. Hopefully that helps. ---
**Author:** ![absurd](https://avatars.discourse-cdn.com/v4/letter/a/59ef9b/32.png) [@absurd](https://community.konduit.ai/u/absurd)\ **Post date:** [February 20, 2021, 8:58pm UTC](https://community.konduit.ai/t/label-and-features-in-the-same-file-for-time-series-data-sequencerecordreader-datasetiterator/1186/3 "2021-02-20T20:58:26Z")
Thanks. Thats not quite the solution i was looking for. I think it just further reinforces my conclusion that its impossible to encode what I needed in a single reader. Here is my experience (code follows in the next post). Case 1: Separate readers, one for features one for labels Features hardcoded are in 3x4 grid, 3 features, 4 timesteps Labels are hardcoded to be 1. After passing throught the iterator > ``` > Features Labels > 3x4 1x1 > [f,f,f > f,f,f --> [l] > f,f,f > f,f,f] > > ``` This makes sense the whole time sequence, (3x4) received a single label (1x1) Case 2: Here im trying to reproduce the above behavior (3x4) → (1x1) with a single reader. The features remain in a hardcoded 3x4 matrix, but now the hardcoded label is appended to each row making it a 4x4 matrix When passed through the sequence iterator the features are (3x4), as before, but now labels are in a (1x4) ``` SingleSrc Features Labels 4x4 3x4 1x4 [f,f,f,l, [f,f,f [l, f,f,f,l, f,f,f --> l, f,f,f,l f,f,f l f,f,f,l] f,f,f] l] ``` So my conclusion is its impossible to achieve the reduction [C x R] → [1x1] with a single reader, instead you are limited to a reduction of [C x R] → [1 x R] ---
**Author:** ![absurd](https://avatars.discourse-cdn.com/v4/letter/a/59ef9b/32.png) [@absurd](https://community.konduit.ai/u/absurd)\ **Post date:** [February 20, 2021, 9:09pm UTC](https://community.konduit.ai/t/label-and-features-in-the-same-file-for-time-series-data-sequencerecordreader-datasetiterator/1186/4 "2021-02-20T21:09:59Z")
Here is my code (Groovy) to test for the above conclusion. ``` def w = { //convert int to a writable return new org.datavec.api.writable.IntWritable(it) } ``` Case 1: Two readers one for features (3x4) one for labels (1x1) ``` List>> lb = [[[new org.datavec.api.writable.IntWritable(1)]]] def labelReader = new CollectionSequenceRecordReader(lb) List>> ft = [[[w(101),w(102),w(103)], [w(104),w(105),w(106)], [w(107),w(108),w(109)], [w(110),w(111),w(112)]]] def featureReader = new CollectionSequenceRecordReader(ft) SequenceRecordReaderDataSetIterator iterator = new SequenceRecordReaderDataSetIterator(featureReader, labelReader, 10, -1, true) def seq = iterator.next() println "features: "+seq.features.shapeInfoToString() println "labels: "+seq.labels.shapeInfoToString() ``` The output on this one is: _features: Rank: 3,Offset: 0_ _Order: f Shape: [1,3,4], stride: [1,1,3]_ _labels: Rank: 3,Offset: 0_ _Order: f Shape: [1,1,1], stride: [1,1,1]_ This makes sense, the features are 3x4 and the labels are 1x1 Case 2: single reader 4x4, label are at index 3 ``` List>> ft2 = [[[w(101),w(102),w(103), w(1)], [w(104),w(105),w(106), w(1)], [w(107),w(108),w(109), w(1)], [w(110),w(111),w(112), w(1)]]] def singleReader = new CollectionSequenceRecordReader(ft2) SequenceRecordReaderDataSetIterator iterator2 = new SequenceRecordReaderDataSetIterator(singleReader, 10, -1, 3, true) def seq2 = iterator2.next() println "features: "+seq2.features.shapeInfoToString() println "labels: "+seq2.labels.shapeInfoToString() ``` The output here is : _features: Rank: 3,Offset: 0_ _Order: f Shape: [1, **3,4**], stride: [1,1,3]_ _labels: Rank: 3,Offset: 0_ _Order: f Shape: [1, **1,4**], stride: [1,1,1]_ Here the features are 3x4 matrix as before, but the labels are now forced into 1x4 matrix. ---
**Author:** ![agibsonccc](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/agibsonccc/32/697_2.png) [@agibsonccc](https://community.konduit.ai/u/agibsonccc)\ **Post date:** [February 22, 2021, 12:30pm UTC](https://community.konduit.ai/t/label-and-features-in-the-same-file-for-time-series-data-sequencerecordreader-datasetiterator/1186/5 "2021-02-22T12:30:47Z")
> [@absurd](#): > > step? Meaning each step in the time series record would have the same value in the label column? @absurd sorry your question isn’t clear to me then. If you need your data in a specific shape, could you describe what your direct goal is? Like “data pipeline that produces ndarrays in this format with labels laid out like xyz” would help. Maybe I can help with the solution rather than trying to dissect the interpretation of the problem. From my guess you need the data in a specific layout and are trying to get it there. Sorry for not quite understanding what you want to do here. ---
**Author:** ![treo](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/treo/32/47_2.png) [@treo](https://community.konduit.ai/u/treo)\ **Post date:** [February 22, 2021, 1:04pm UTC](https://community.konduit.ai/t/label-and-features-in-the-same-file-for-time-series-data-sequencerecordreader-datasetiterator/1186/6 "2021-02-22T13:04:18Z")
As @agibsonccc said, I think we need an idea for what you are trying to do. My best guess is that you have some kind of sequence classification task, where you want to consume the entire sequence and produce one output. The form where features and labels come from the same sequence, is meant for the case were you have both a sequence of features and a sequence of labels. Usually, when you have only a sequence of features and a single label, they are saved differently, e.g. in two files. > [@absurd](#): > > Here the features are 3x4 matrix as before, but the labels are now forced into 1x4 matrix. The labels you provide already are a 1x4 matrix. There is nothing forced about it. If you really have your data saved in a format where you just want to use the last label in your sequence as your overall label (what my best guess is what you are trying to do), then the solution for that is fairly trivial: Set [LabelLastTimeStepPreProcessor](https://github.com/eclipse/deeplearning4j/blob/master/nd4j/nd4j-backends/nd4j-api-parent/nd4j-api/src/main/java/org/nd4j/linalg/dataset/api/preprocessor/LabelLastTimeStepPreProcessor.java) ([javadoc](https://javadoc.io/doc/org.nd4j/nd4j-api/1.0.0-beta7/org/nd4j/linalg/dataset/api/preprocessor/LabelLastTimeStepPreProcessor.html)) as a preprocessor on your dataset iterator. It will work with sequences of differing lengths and will always use only the very last label in your sequence as the label. ---
**Author:** ![absurd](https://avatars.discourse-cdn.com/v4/letter/a/59ef9b/32.png) [@absurd](https://community.konduit.ai/u/absurd)\ **Post date:** [February 22, 2021, 3:15pm UTC](https://community.konduit.ai/t/label-and-features-in-the-same-file-for-time-series-data-sequencerecordreader-datasetiterator/1186/7 "2021-02-22T15:15:10Z")
> [@treo](#): > > My best guess is that you have some kind of sequence classification task, where you want to consume the entire sequence and produce one output… > > [LabelLastTimeStepPreProcessor](https://github.com/eclipse/deeplearning4j/blob/master/nd4j/nd4j-backends/nd4j-api-parent/nd4j-api/src/main/java/org/nd4j/linalg/dataset/api/preprocessor/LabelLastTimeStepPreProcessor.java) Aaaaaah. Nice. That’s exactly what I was trying to do; classify a sequence. And that’s exactly the solution I needed. So basically add the label to each time step, and add the LabelLastTimeStepPreProcessor to only keep the very last label. 👍 ---
**Author:** ![absurd](https://avatars.discourse-cdn.com/v4/letter/a/59ef9b/32.png) [@absurd](https://community.konduit.ai/u/absurd)\ **Post date:** [February 22, 2021, 3:23pm UTC](https://community.konduit.ai/t/label-and-features-in-the-same-file-for-time-series-data-sequencerecordreader-datasetiterator/1186/8 "2021-02-22T15:23:06Z")
> [@agibsonccc](#): > > Like “data pipeline that produces ndarrays in this format with labels laid out like xyz” would help. I find this extremely difficult as well, the nomenclature used is varied from source to source. So sometimes its hard to know how to correctly phrase a question. I already have the solution above from @treo. But for posterity perhaps I should have phrased it as : _I’m building a data pipeline to classify sequences. There are 9 features, with 30 timesteps. (9x30), and a single label (1x1). The caveat for me is that the features and the label have to come from the same reader/file. Is that possible with DL4J? I already know how to do it, with two separate readers, but what about a single reader?_ ---
**Author:** ![absurd](https://avatars.discourse-cdn.com/v4/letter/a/59ef9b/32.png) [@absurd](https://community.konduit.ai/u/absurd)\ **Post date:** [April 15, 2021, 5:14pm UTC](https://community.konduit.ai/t/label-and-features-in-the-same-file-for-time-series-data-sequencerecordreader-datasetiterator/1186/9 "2021-04-15T17:14:31Z")
Err, hate to resurrect this thread, but perhaps I’m not clear on how to use the LabelLastTimeStepPreProcessor. So, I added the preprocessor to my DataSetIterator and everything looks ok, the INDArray shapes, I sampled look as expected. iterator.setPreProcessor(new LabelLastTimeStepPreProcessor()) * * * ``` features: Rank: 3, DataType: FLOAT, Offset: 0, Order: f, Shape: [5,11,30], Stride: [1,5,55] labels: Rank: 2, DataType: FLOAT, Offset: 0, Order: c, Shape: [5,2], Stride: [2,1] ``` * * * Above reflects what I imagined this preprocessor does: 5 samples per batch; 11 features X 30 timesteps per sample 1 label per sample-\> 5 labels per batch; 2 classes per label ( one-hot-for-1or0) However when I try to fit/train this data on a RNN; I get the following exception; ``` Expected rank 3 labels array, got label array with shape [5, 2] ``` So is there something else that needs to be done on the RNN side so make it work with the preprocessor? Or did I misunderstand where the preprocessor should be applied? ---
**Author:** ![treo](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/treo/32/47_2.png) [@treo](https://community.konduit.ai/u/treo)\ **Post date:** [April 15, 2021, 6:12pm UTC](https://community.konduit.ai/t/label-and-features-in-the-same-file-for-time-series-data-sequencerecordreader-datasetiterator/1186/10 "2021-04-15T18:12:29Z")
In that case you need to use a regular `OutputLayer` instead of an `RnnOutputLayer`. ---
**Author:** ![absurd](https://avatars.discourse-cdn.com/v4/letter/a/59ef9b/32.png) [@absurd](https://community.konduit.ai/u/absurd)\ **Post date:** [April 15, 2021, 8:13pm UTC](https://community.konduit.ai/t/label-and-features-in-the-same-file-for-time-series-data-sequencerecordreader-datasetiterator/1186/11 "2021-04-15T20:13:24Z")
Thanks for your suggestion. I tried a regular OutputLayer instead of an RnnOutputLayer. But get a different exception now. ``` java.lang.IllegalArgumentException: Labels and preOutput must have equal shapes: got shapes [5, 2] vs [150, 2] at org.nd4j.common.base.Preconditions.throwEx(Preconditions.java:636) at org.nd4j.linalg.lossfunctions.impl.LossMCXENT.computeGradient(LossMCXENT.java:149) .... ``` ---
**Author:** ![treo](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/treo/32/47_2.png) [@treo](https://community.konduit.ai/u/treo)\ **Post date:** [April 16, 2021, 11:20am UTC](https://community.konduit.ai/t/label-and-features-in-the-same-file-for-time-series-data-sequencerecordreader-datasetiterator/1186/12 "2021-04-16T11:20:17Z")
In that case you also need to use either a LastTimeStepLayer or a LastTimeStepVertex right before your output. ---
**Author:** ![absurd](https://avatars.discourse-cdn.com/v4/letter/a/59ef9b/32.png) [@absurd](https://community.konduit.ai/u/absurd)\ **Post date:** [April 16, 2021, 5:05pm UTC](https://community.konduit.ai/t/label-and-features-in-the-same-file-for-time-series-data-sequencerecordreader-datasetiterator/1186/13 "2021-04-16T17:05:53Z")
Ok, not sure if I’m missing some intuition about the code base, but I cant figure how to add this _LastTimeStepLayer_ to my nnconf. I searched for some code examples and didnt find any either. So from what I can see, the _LastTimeStepLayer_ constructor expects an _nn.api.Layer_ ``` public LastTimeStepLayer(@NonNull @NotNull org.deeplearning4j.nn.api.Layer underlying){ .. } ``` However when we are building the nn configuration, using _NeuralNetConfiguration.Builder_ and _NeuralNetConfiguration.ListBuilder_ we are working with _nn.conf.layers.Layer_ not the _nn.api.Layer_ that _LastTimeStepLayer_ expects. i tried something like… ``` LSTM lstm1 = new LSTM.Builder()...build() LSTM lstm2 = new LSTM.Builder()...build() OutputLayer outputLayer = new OutputLayer.Builder()..build() new NeuralNetConfiguration.Builder() ...... .list() .layer(0, lstm1) .layer(1, new LastTimeStepLayer(lstm2)) //<---This is the part that errors and I cant figure out .layer(2, outputLayer) ``` Any thoughts? There is some kind package peculiarity here that I cant seem to grasp. _ListBuilder_ expects a _nn.conf.layers.Layer_, but _LastTimeStepLayer_ is an _nn.api.Layer_, so I cant add it to the configuration. Similarly _LastTimeStepLayer_’s construtor expects and _nn.api.Layer_, but _LSTM_ is an _nn.con.layers.Layer_, so I cant even wrap the _LSTM_ in a _LastTimeStepLayer_ in the first place. ---
**Author:** ![treo](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/treo/32/47_2.png) [@treo](https://community.konduit.ai/u/treo)\ **Post date:** [April 16, 2021, 5:28pm UTC](https://community.konduit.ai/t/label-and-features-in-the-same-file-for-time-series-data-sequencerecordreader-datasetiterator/1186/14 "2021-04-16T17:28:34Z")
Ugh, for some reason there is a naming inconsistency. The thing you’d use in the configuration is called just `LastTimeStep`. @agibsonccc maybe we should fix that in the next release? ---
**Author:** ![absurd](https://avatars.discourse-cdn.com/v4/letter/a/59ef9b/32.png) [@absurd](https://community.konduit.ai/u/absurd)\ **Post date:** [April 16, 2021, 6:20pm UTC](https://community.konduit.ai/t/label-and-features-in-the-same-file-for-time-series-data-sequencerecordreader-datasetiterator/1186/15 "2021-04-16T18:20:57Z")
Perfect, thanks, _LastTimeStep_ worked as expected.