# How To feed model with data from memory

**URL:** <https://community.konduit.ai/t/how-to-feed-model-with-data-from-memory/1568>\
**Category:** DataVec\
**Created:** [August 17, 2021, 9:26am UTC](https://community.konduit.ai/t/how-to-feed-model-with-data-from-memory/1568 "2021-08-17T09:26:32Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![AmrRahmy](https://avatars.discourse-cdn.com/v4/letter/a/cc9497/32.png) [@AmrRahmy](https://community.konduit.ai/u/AmrRahmy)\
**Post date:** [August 17, 2021, 9:26am UTC](https://community.konduit.ai/t/how-to-feed-model-with-data-from-memory/1568/1 "2021-08-17T09:26:32Z")

</div>

Hi,

I can see some examples using csv as the source for the data, but I would rather use a database that i already prepared.

I can see that (MultiLayerNetwork)model.fit() accepts INDArray, and Dataset among other things.

given I have a database with data, I can load the data in batches into the program,

for example,  
i have 10 inputs(int and double or float) in an object  
i am expecting the output to be 2 float numbers

MultiLayerConfiguration conf = new NeuralNetConfiguration.Builder()  
…  
.build();

MultiLayerNetwork model = new MultiLayerNetwork(conf);  
model.init();  
…

model.fit(data);

my questions are,

- how to use INDArray or Dataset or the iterator versions, with ten values per row/entry? can i see a simple example if you don’t mind

- can I use model.fit() multiple times in a loop?

for(var data : dataList) {  
model.fit(data)  
}

---

<div class="post-metadata">

**Author:** ![treo](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/treo/32/47_2.png) [@treo](https://community.konduit.ai/u/treo)\
**Post date:** [August 17, 2021, 11:46am UTC](https://community.konduit.ai/t/how-to-feed-model-with-data-from-memory/1568/2 "2021-08-17T11:46:56Z")

</div>

It isn’t documented very well, but there is a JDBC Record reader.

You can see it being used here: [https://github.com/eclipse/deeplearning4j/blob/master/datavec/datavec-jdbc/src/test/java/org/datavec/api/records/reader/impl/JDBCRecordReaderTest.java](https://github.com/eclipse/deeplearning4j/blob/master/datavec/datavec-jdbc/src/test/java/org/datavec/api/records/reader/impl/JDBCRecordReaderTest.java)

Once you have set it up, you can use it like any other record reader (e.g. like a csv record reader).

---

<div class="post-metadata">

**Author:** ![AmrRahmy](https://avatars.discourse-cdn.com/v4/letter/a/cc9497/32.png) [@AmrRahmy](https://community.konduit.ai/u/AmrRahmy)\
**Post date:** [August 17, 2021, 1:04pm UTC](https://community.konduit.ai/t/how-to-feed-model-with-data-from-memory/1568/3 "2021-08-17T13:04:29Z")

</div>

Thanks for the reply.

I don’t want to use the data from the database directly, the data is raw in the database, and the database tables are designed, not a table of input data.

I want to fetch the data from the database, transform it into the input which is integers and double(can change to float) in memory, queue it in, and pass it to the model.

But i am not familiar with the Dataset and INDArray creation,  
if the input is 10 values and the output is 2, where or how to map the data to the those objects/classes

is the first parameter expected an input array with a size of the 10 input values? then the output as an array of 2 values? how to add multiple objects? should it be array[sizeOfEntriesToTrainAtOneTime][10][2] for example, or put INDArray into a dataset? as in the dataset is a collection of INDArrays

INDArray nd = Nd4j.create(new float{1,2,3,4,5,6,7,8,9,10},new float{2,2});

---

<div class="post-metadata">

**Author:** ![treo](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/treo/32/47_2.png) [@treo](https://community.konduit.ai/u/treo)\
**Post date:** [August 17, 2021, 1:11pm UTC](https://community.konduit.ai/t/how-to-feed-model-with-data-from-memory/1568/4 "2021-08-17T13:11:51Z")

</div>

A Dataset object is at least two INDArrays:

- one for _a batch of features_
- one for _a batch of labels_

So if you create it like in your example, you will need to `Nd4j.vstack` them into a batch first before you pass it to the model.

But if you are building it like that you can also directly pass it to [`model.fit`](https://javadoc.io/static/org.deeplearning4j/deeplearning4j-nn/1.0.0-M1.1/org/deeplearning4j/nn/multilayer/MultiLayerNetwork.html#fit-org.nd4j.linalg.api.ndarray.INDArray-org.nd4j.linalg.api.ndarray.INDArray-)

---

<div class="post-metadata">

**Author:** ![AmrRahmy](https://avatars.discourse-cdn.com/v4/letter/a/cc9497/32.png) [@AmrRahmy](https://community.konduit.ai/u/AmrRahmy)\
**Post date:** [August 18, 2021, 4:34am UTC](https://community.konduit.ai/t/how-to-feed-model-with-data-from-memory/1568/5 "2021-08-18T04:34:44Z")

</div>

simplifying the question,

if I have a,  
List customInput; //3 features  
customInput.size() = 1000;

List customExpectedOutput; ////1 output  
customExpectedOutput.size() = 1000;

public class CustomInput {  
public float val1;  
public float val2;  
public float val3;  
}

public class CustomExpectedOutput {  
public float val1;  
}

and I want to use model.fit(dataSet),

how to create dataSet of type DataSet?