Demand:
I want to create a DataSetIterator.
Question:
Unable to understand the data structure of the MnistDataSetIterator.
@TRSK-CONMI then go ahead and just use the RecordReaderDataSetIterator. Beyond that, you can implement your own interface. Docs can be found here:
https://deeplearning4j.konduit.ai/models/iterators
A datasetiterator just allows you to iterate through a dataset. A DataSet is a pair of inputs and labels.
Thanks fo helping, agibsonccc.
This is the code That I modified.
`
DataSetIterator mnistTest = new MnistDataSetIterator(batchSize, false, 12345);
while (mnistTest.hasNext()) {
DataSet dataSet = mnistTest.next();
System.out.println("===================================");
System.out.println(dataSet.getLabels());
break;
}
`
This is my printout.
`
===================================
[[ 0, 0, 0, 1.0000, 0, 0, 0, 0, 0, 0],
[ 0, 0, 1.0000, 0, 0, 0, 0, 0, 0, 0],
[ 0, 1.0000, 0, 0, 0, 0, 0, 0, 0, 0],
[ 0, 0, 1.0000, 0, 0, 0, 0, 0, 0, 0],
[ 0, 0, 0, 0, 0, 1.0000, 0, 0, 0, 0],
[ 0, 1.0000, 0, 0, 0, 0, 0, 0, 0, 0],
[ 0, 0, 0, 1.0000, 0, 0, 0, 0, 0, 0],
[ 0, 0, 0, 0, 1.0000, 0, 0, 0, 0, 0],
[ 0, 1.0000, 0, 0, 0, 0, 0, 0, 0, 0],
[ 0, 0, 0, 0, 0, 0, 1.0000, 0, 0, 0],
[ 0, 0, 0, 1.0000, 0, 0, 0, 0, 0, 0],
[ 0, 1.0000, 0, 0, 0, 0, 0, 0, 0, 0],
[ 0, 0, 0, 1.0000, 0, 0, 0, 0, 0, 0],
[ 0, 0, 0, 0, 1.0000, 0, 0, 0, 0, 0],
[ 0, 1.0000, 0, 0, 0, 0, 0, 0, 0, 0],
[ 0, 0, 0, 0, 0, 0, 1.0000, 0, 0, 0],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 1.0000],
[ 0, 0, 0, 0, 0, 0, 1.0000, 0, 0, 0],
[ 0, 0, 0, 0, 0, 1.0000, 0, 0, 0, 0],
[ 0, 0, 0, 1.0000, 0, 0, 0, 0, 0, 0],
[ 0, 0, 0, 0, 0, 0, 0, 1.0000, 0, 0],
[ 0, 0, 0, 0, 0, 0, 0, 1.0000, 0, 0],
[ 0, 0, 0, 0, 0, 1.0000, 0, 0, 0, 0],
[ 0, 0, 1.0000, 0, 0, 0, 0, 0, 0, 0],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 1.0000],
[ 1.0000, 0, 0, 0, 0, 0, 0, 0, 0, 0],
[ 0, 0, 0, 0, 0, 1.0000, 0, 0, 0, 0],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 1.0000],
[ 0, 0, 0, 0, 0, 0, 0, 1.0000, 0, 0],
[ 0, 0, 0, 0, 0, 1.0000, 0, 0, 0, 0],
[ 0, 1.0000, 0, 0, 0, 0, 0, 0, 0, 0],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 1.0000],
[ 0, 0, 0, 0, 1.0000, 0, 0, 0, 0, 0],
[ 0, 0, 0, 1.0000, 0, 0, 0, 0, 0, 0],
[ 0, 0, 0, 0, 1.0000, 0, 0, 0, 0, 0],
[ 0, 0, 0, 0, 0, 0, 1.0000, 0, 0, 0],
[ 0, 0, 0, 0, 0, 0, 0, 1.0000, 0, 0],
[ 1.0000, 0, 0, 0, 0, 0, 0, 0, 0, 0],
[ 0, 0, 1.0000, 0, 0, 0, 0, 0, 0, 0],
[ 0, 0, 0, 0, 0, 1.0000, 0, 0, 0, 0],
[ 1.0000, 0, 0, 0, 0, 0, 0, 0, 0, 0],
[ 0, 0, 0, 0, 0, 0, 0, 1.0000, 0, 0],
[ 0, 0, 0, 0, 0, 0, 0, 0, 1.0000, 0],
[ 0, 0, 0, 0, 0, 0, 0, 0, 1.0000, 0],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 1.0000],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 1.0000],
[ 0, 1.0000, 0, 0, 0, 0, 0, 0, 0, 0],
[ 0, 0, 0, 0, 0, 0, 0, 1.0000, 0, 0],
[ 0, 0, 0, 0, 0, 1.0000, 0, 0, 0, 0],
[ 0, 1.0000, 0, 0, 0, 0, 0, 0, 0, 0],
[ 0, 0, 0, 0, 0, 0, 1.0000, 0, 0, 0],
[ 1.0000, 0, 0, 0, 0, 0, 0, 0, 0, 0],
[ 0, 0, 0, 0, 0, 1.0000, 0, 0, 0, 0],
[ 0, 0, 0, 0, 1.0000, 0, 0, 0, 0, 0],
[ 0, 0, 0, 0, 0, 0, 0, 0, 1.0000, 0],
[ 0, 0, 0, 0, 0, 0, 1.0000, 0, 0, 0],
[ 1.0000, 0, 0, 0, 0, 0, 0, 0, 0, 0],
[ 0, 0, 0, 0, 0, 1.0000, 0, 0, 0, 0],
[ 1.0000, 0, 0, 0, 0, 0, 0, 0, 0, 0],
[ 0, 0, 0, 0, 0, 0, 0, 0, 1.0000, 0],
[ 0, 0, 0, 0, 0, 0, 0, 0, 0, 1.0000],
[ 0, 1.0000, 0, 0, 0, 0, 0, 0, 0, 0],
[ 0, 0, 0, 0, 0, 0, 0, 1.0000, 0, 0],
[ 0, 0, 1.0000, 0, 0, 0, 0, 0, 0, 0]]
------------------------------------------------------------------------
`
Can you tell me what information this is?
@TRSK-CONMI it’d help if you read the docs a bit 
I’m going to insist you don’t ignore the page I gave you. You really need to understand what’s available here. We explain all this in depth.
If you’re new to machine learning in general, please also make sure you spend time on the basics. My explanation below is going to assume you have the basics down:
That’s what a dataset is. A dataset is a class containing 2 ndarrays containing the inputs and the labels.
The inputs from mnist are 1 x 784. The labels are 1 x 10 representing 10 labels.
The mnistdatasetiterator downloads the dataset which is a binary file (it’s not an image) and converts it to the ndarray format you’re seeing there. Again, going to insist you spend some time on the fundamentals.
Thanks.
My foundation is not solid enough, my English is not very good.
At the beginning, I only read some principles of TensorFlow, but unfortunately many of those books were wrong. Compared with the TensorFlow on the official website, I found a lot of problems. Later, I stopped going into TensorFlow because many aspects of Python were not rigorous enough.
I like Java, so I want to keep working on it.The address above is not detailed enough.When it comes to concepts, I won’t, so I ask.
Thanks.
I have two last questions. The document of 0.9.1 seems to be written.
1.Is the label the file name?
2.How do I use a bunch of images to create a new DataSetIterator?(Looks like an error with FileDataSetIterator)
Thanks.
My foundation is not solid enough, my English is not very good.
At the beginning, I only read some principles of TensorFlow, but unfortunately many of those books were wrong. Compared with the TensorFlow on the official website, I found a lot of problems. Later, I stopped going into TensorFlow because many aspects of Python were not rigorous enough.
I like Java, so I want to keep working on it.The address above is not detailed enough.When it comes to concepts, I won’t, so I ask.
Thanks.
I have two last questions. The document of 0.9.1 seems to be written.
1.Is the label the file name?
2.How do I use a bunch of images to create a new DataSetIterator?(Looks like an error with FileDataSetIterator)
Maybe my translation software is not very good, if the tone is not good, please forgive me, that is not my intention.I am a modest learner.
@TRSK-CONMI got it thanks.
-
The label is not the file name. A “label” is the actual class. In machine learning and supervised learning a class represents the true label. The output from the iterator is the input data and a label with a 1 where the index represents the class. I would highly recommend looking in to the basics of machine learning just covering classification and softmax at a minimum.
-
I"m not really sure what you mean by “error” is, but here’s an example with images/mnist:
https://github.com/eclipse/deeplearning4j-examples/blob/9799225aa9edec0af7985b07510bfbd02bc80df6/dl4j-examples/src/main/java/org/deeplearning4j/examples/quickstart/modeling/convolution/LeNetMNISTReLu.java#L73
Thank you for your help,agibsonccc.
That’s exactly the kind of complete code I need.
