# Recommended way to create INDArray for prediction?

**URL:** <https://community.konduit.ai/t/recommended-way-to-create-indarray-for-prediction/560>\
**Category:** DL4J\
**Created:** [May 28, 2020, 9:56am UTC](https://community.konduit.ai/t/recommended-way-to-create-indarray-for-prediction/560 "2020-05-28T09:56:14Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![kullanici0606](https://avatars.discourse-cdn.com/v4/letter/k/f07891/32.png) [@kullanici0606](https://community.konduit.ai/u/kullanici0606)\
**Post date:** [May 28, 2020, 9:56am UTC](https://community.konduit.ai/t/recommended-way-to-create-indarray-for-prediction/560/1 "2020-05-28T09:56:14Z")

</div>

Hi,

We are using Deeplearning4j for making predictions where model is trained by Keras and imported to Deeplearning4j. Version is 1.0.0-beta6 [1]. We are using CPU backend, our CPU supports AVX2 and AVX512 instructions [2].

Our code looks similar to this:

```
private static final int BATCH_SIZE = 4096;
private static final int INPUT_SIZE = 512;
private static final int [] SHAPE = { BATCH_SIZE, 1, INPUT_SIZE };

private void predict(ComputationGraph graph, float[] input1, float[] input2) {
    try(INDArray firstInput = Nd4j.create(input1, SHAPE);
        INDArray secondInput = Nd4j.create(input2, SHAPE)) {
        INDArray result = graph.outputSingle(firstInput, secondInput);
        process(result);
        result.close();
    }
}

```

After some profiling and logging, we saw than Nd4j.create part is taking a lot longer than graph.outputSingle. Therefore, I thought we must be doing something wrong.

float arrays (input1, input2) are re-used, meaning we alllocate them once in application’s lifetime and fill data (override) in them for each batch.

So, is the recommended way of feeding data to graph for prediction and INDArray creation is to create them every time needed as we did in the code above? Or can we re-use them since the underlying arrays are already re-used? Or what is the best and most efficient way for this?

Could you please help and guide us in this issue?

[1] 1.0.0-beta7 gave an error during prections for imported model, therefore we delayed migration.  
[2] We have declared nd4j-native avx2 and avx512 dependendencies in our pom.xml

Thanks in advance.

---

<div class="post-metadata">

**Author:** ![treo](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/treo/32/47_2.png) [@treo](https://community.konduit.ai/u/treo)\
**Post date:** [May 28, 2020, 10:31am UTC](https://community.konduit.ai/t/recommended-way-to-create-indarray-for-prediction/560/2 "2020-05-28T10:31:38Z")

</div>

How exactly are you measuring the time? The first Nd4j call takes some time to initialize the whole system, so on cold starts, what ever you call will have the initialization overhead.

Other than that, you will very likely benefit from using workspaces, as that allows the system to reuse the memory for the both input arrays instead of having a constant alloc/dealloc going on.

For examples how to use workspaces see [https://github.com/eclipse/deeplearning4j-examples/blob/master/nd4j-examples/src/main/java/org/nd4j/examples/Nd4jEx15\_Workspaces.java](https://github.com/eclipse/deeplearning4j-examples/blob/master/nd4j-examples/src/main/java/org/nd4j/examples/Nd4jEx15_Workspaces.java)

---

<div class="post-metadata">

**Author:** ![kullanici0606](https://avatars.discourse-cdn.com/v4/letter/k/f07891/32.png) [@kullanici0606](https://community.konduit.ai/u/kullanici0606)\
**Post date:** [May 28, 2020, 11:37am UTC](https://community.konduit.ai/t/recommended-way-to-create-indarray-for-prediction/560/3 "2020-05-28T11:37:42Z")

</div>

You are right, after double-checking and running app longer, I realized that I must have been measuring time incorrectly. Most probably, as you said, I saw first initialization time. After running some time, there is no noticable overhead of nd4j call compared to prediction.

Thanks a lot for helping and also pointing workspaces out. I will look at workspaces, it looks like it is going to help us.

---

<div class="post-metadata">

**Author:** ![kullanici0606](https://avatars.discourse-cdn.com/v4/letter/k/f07891/32.png) [@kullanici0606](https://community.konduit.ai/u/kullanici0606)\
**Post date:** [May 28, 2020, 12:55pm UTC](https://community.konduit.ai/t/recommended-way-to-create-indarray-for-prediction/560/4 "2020-05-28T12:55:52Z")

</div>

When using workspaces, should I close created INDArray or not? In the examples, they are not closed.

---

<div class="post-metadata">

**Author:** ![treo](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/treo/32/47_2.png) [@treo](https://community.konduit.ai/u/treo)\
**Post date:** [May 29, 2020, 6:49am UTC](https://community.konduit.ai/t/recommended-way-to-create-indarray-for-prediction/560/5 "2020-05-29T06:49:43Z")

</div>

You don’t shouldn’t need to close them.

---

<div class="post-metadata">

**Author:** ![kullanici0606](https://avatars.discourse-cdn.com/v4/letter/k/f07891/32.png) [@kullanici0606](https://community.konduit.ai/u/kullanici0606)\
**Post date:** [May 29, 2020, 7:22am UTC](https://community.konduit.ai/t/recommended-way-to-create-indarray-for-prediction/560/6 "2020-05-29T07:22:00Z")

</div>

Thank you very much, you helped a lot in this topic.