Getting output prediction for LSTM

DL4J
1

Hello,

i didn’t understand how to get output for a LSTM network (probably more specifically any network)

i have a set of CSV file with that format, every file has 200 rows. there are 10 “attributes” and label can assume only 0/1 values, so i guess there are 2 labels :slight_smile:

0/1 (label),double,double,double…
0/1 (label),double,double,double…

and so on…

so i splitted the file arbitrary from 0-50 are used for train and 51-60 for test

datasetiterator are created in that way

     SequenceRecordReader reader = new CSVSequenceRecordReader(0, ",");
     reader.initialize(new NumberedFileInputSplit("data/csv/%d.csv", 0, 50));
     DataSetIterator trainData = new SequenceRecordReaderDataSetIterator(reader, 100, 2, 0, false);

     SequenceRecordReader reader = new CSVSequenceRecordReader(0, ",");
     reader.initialize(new NumberedFileInputSplit("data/csv/%d.csv", 51, 60));
     DataSetIterator testData= new SequenceRecordReaderDataSetIterator(reader, 100, 2, 0, false);

same apply for test.

net is created with

MultiLayerConfiguration conf = new NeuralNetConfiguration.Builder()
            .seed(123)    //Random number generator seed for improved repeatability. Optional.
            .weightInit(WeightInit.XAVIER)
            .updater(new Adam(0.005))
            .list()
            .layer(0, new LSTM.Builder().activation(Activation.TANH).nIn(numFeatures).nOut(HID_LAY).build())

            .layer(1, new LSTM.Builder().activation(Activation.TANH).nIn(HID_LAY).nOut(HID_LAY).build())

            .layer(2, new RnnOutputLayer.Builder(LossFunctions.LossFunction.MCXENT)
                    .activation(Activation.SOFTMAX).nIn(HID_LAY).nOut(2).build())
            .build();

    MultiLayerNetwork net = new MultiLayerNetwork(conf);

hid_lay it’s 200.
now, i train for some times with (eventually i also use net.evaluate(test) to see the progress, but it doesn’t matter at this point.

            net.fit(trainData);
            trainData.reset();

and then i am trying to extract data. my idea was to do something like

            INDArray output = net.output(testData);

but i don’t understand at this point, what information i can extract from output.

how to get the “next” prediction for testData?

how to get the prediction for testData at position N? (suppose again that testData is loaded from a set of csv (9x) that have 200 rows each, how can i get the prediction for index YYYY)

thanks for helping :slight_smile: hope my english is clear enough :slight_smile:

2

@ramarro123 take a look at rnn time step: Recurrent Neural Network - Deeplearning4j if you have something more specific after reading this feel free to post here.

3

@agibsonccc can you please elaborate “more specific”?

i review the link, and from scetch (unfortunatley there aren’t many examples, just one for ucisequenceclassification, that doesn’t extract result, jsut validate with eval)

having said so, let me re-ask the specific question, in a more specific way.

specific part (not necessary a clever question)

i want to use this

As with other types of neural networks, predictions can be generated for RNNs using the MultiLayerNetwork.output() and MultiLayerNetwork.feedForward() methods. These methods can be useful in many circumstances;

so, using output() to classify my testSet.

having executed

            INDArray output = net.output(testData);

i can’t understand how to get the “classified” value of a certain index of my testData (containing let’s say 2000 entry)

i hope that’s specific enough :slight_smile:

as for the quality of the question, could be a dumb question, so feel free to tell me if silly questions aren’t gonna get covered :slight_smile:

4

@ramarro123 so if you read the RNN section you’ll see there’s different ways of calling “output” on the network. Try to understand which type you need.

That could be the full series or that could be the next time step. It doesn’t really look like you read the docs or I would have saw some additional questions.

If you don’t quite get what’s going on there please try to ask more questions rather than ignore it.

“More specific” should be additional questions whether it be on the structure of the input data or how time series works.

Beyond that…try looking at this example:

Beyond that, evaluate actually does use output underneath. You should want to know how well your network does.

The output call for the full time series will just give you an output with the classifications. Those are going to usually be standard softmax outputs with probabilities.

For the labels when you’re using output you use argMax. We actually do have more examples of this take a look here: