# How do I use \`loader.asMatrix\` with multiple threads?

**URL:** <https://community.konduit.ai/t/how-do-i-use-loader-asmatrix-with-multiple-threads/2594>\
**Category:** DL4J\
**Created:** [June 9, 2023, 3:29am UTC](https://community.konduit.ai/t/how-do-i-use-loader-asmatrix-with-multiple-threads/2594 "2023-06-09T03:29:56Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![visionki](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/visionki/32/1157_2.png) [@visionki](https://community.konduit.ai/u/visionki)\
**Post date:** [June 9, 2023, 3:29am UTC](https://community.konduit.ai/t/how-do-i-use-loader-asmatrix-with-multiple-threads/2594/1 "2023-06-09T03:29:56Z")

</div>

Question linkage [Using model predictions with multiple threads can cause results to be overwritten](https://github.com/deeplearning4j/deeplearning4j/issues/9993)。

I tried to use the model prediction under multi-threaded and found that some results were overwritten, then I tested and found that it seems that the results of `loader.asMatrix` are overwritten under multi-threaded, he is not thread safe right, how can I load the image under multi-threaded.

---

<div class="post-metadata">

**Author:** ![visionki](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/visionki/32/1157_2.png) [@visionki](https://community.konduit.ai/u/visionki)\
**Post date:** [June 9, 2023, 8:20am UTC](https://community.konduit.ai/t/how-do-i-use-loader-asmatrix-with-multiple-threads/2594/2 "2023-06-09T08:20:08Z")

</div>

I was using `NativeImageLoader ` and it was messing up with multiple threads, but then I tried using `ImageLoader ` and I don’t have a problem. I don’t know if this is a solution.

---

<div class="post-metadata">

**Author:** ![agibsonccc](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/agibsonccc/32/697_2.png) [@agibsonccc](https://community.konduit.ai/u/agibsonccc)\
**Post date:** [June 9, 2023, 10:50am UTC](https://community.konduit.ai/t/how-do-i-use-loader-asmatrix-with-multiple-threads/2594/3 "2023-06-09T10:50:44Z")

</div>

@visionki could you clarify? Is the input data overridden? Models themselves are not thread safe. You need to use parallelinference.

---

<div class="post-metadata">

**Author:** ![visionki](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/visionki/32/1157_2.png) [@visionki](https://community.konduit.ai/u/visionki)\
**Post date:** [June 9, 2023, 11:36am UTC](https://community.konduit.ai/t/how-do-i-use-loader-asmatrix-with-multiple-threads/2594/4 "2023-06-09T11:36:37Z")

</div>

@agibsonccc Yes, I used `parallelinference`, Here is my code snippet:

```java
@Slf4j
@Component
public class NsfwUtil {
    ComputationGraph model;
    private final ParallelInference parallelInference;
    private final ImageLoader imgLoader;
    NativeImageLoader loader;

    public NsfwUtil() throws IOException, UnsupportedKerasConfigurationException, InvalidKerasConfigurationException {
        model = KerasModelImport.importKerasModelAndWeights(new ClassPathResource("model/nsfw.299x299.h5").getInputStream());
        parallelInference = new ParallelInference.Builder(model)
                .inferenceMode(InferenceMode.BATCHED)
                .batchLimit(32)
                .workers(5)
                .build();
        loader = new NativeImageLoader(299, 299, 3, new ColorConversionTransform(COLOR_BGR2RGB));
        imgLoader = new ImageLoader(299, 299, 3);
    }

    @Async("loadExecutor")
    public CompletableFuture<AnalysisResult> getResultAsync(String url){
        try (InputStream inputStream = Unirest.get(url)
                .asObjectAsync(RawResponse::getContent)
                .get()
                .getBody()){
            INDArray matrix = loader.asMatrix(inputStream, false);
            log.info("{}", matrix.hashCode());
            INDArray output = parallelInference.output(matrix.div(255));
            return CompletableFuture.completedFuture(new AnalysisResult(output.data().asDouble()));
        }catch (Exception e){
            throw new RuntimeException();
        }
    }
}

```

The above example uses `NativeImageLoader` to load images, and the output of `matrix.hashCode()` is occasionally duplicated (common when first started), and the log is as follows:

```java
735038670
735038670
988874855
988874855
2103345588
-822422519
920533466
-1450966807
-1952245710
-2122104256
-82726789

```

When I replace it with `ImageLoader` to load the image, this does not happen.

---

<div class="post-metadata">

**Author:** ![agibsonccc](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/agibsonccc/32/697_2.png) [@agibsonccc](https://community.konduit.ai/u/agibsonccc)\
**Post date:** [June 9, 2023, 11:38am UTC](https://community.konduit.ai/t/how-do-i-use-loader-asmatrix-with-multiple-threads/2594/5 "2023-06-09T11:38:06Z")

</div>

@visionki I would recommend a threadlocal for the NativeImageLoader. That should help your issue.

---

<div class="post-metadata">

**Author:** ![visionki](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/visionki/32/1157_2.png) [@visionki](https://community.konduit.ai/u/visionki)\
**Post date:** [June 11, 2023, 4:40am UTC](https://community.konduit.ai/t/how-do-i-use-loader-asmatrix-with-multiple-threads/2594/6 "2023-06-11T04:40:13Z")

</div>

@agibsonccc I want to know what is the difference between `NativeImageLoader` and `ImageLoader`, I can’t find the relevant documentation. looking at the comments it seems `NativeImageLoader` is to load images with opencv, `ImageLoader` is to load images with BufferedImage, do you mean you don’t recommend using `ImageLoader`?

---

<div class="post-metadata">

**Author:** ![agibsonccc](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/agibsonccc/32/697_2.png) [@agibsonccc](https://community.konduit.ai/u/agibsonccc)\
**Post date:** [June 11, 2023, 6:07am UTC](https://community.konduit.ai/t/how-do-i-use-loader-asmatrix-with-multiple-threads/2594/7 "2023-06-11T06:07:02Z")

</div>

@visionki NativeImageLoader uses opencv and is the recommended way of loading images. There’s not really much to document there.

---

<div class="post-metadata">

**Author:** ![visionki](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/visionki/32/1157_2.png) [@visionki](https://community.konduit.ai/u/visionki)\
**Post date:** [June 11, 2023, 6:28am UTC](https://community.konduit.ai/t/how-do-i-use-loader-asmatrix-with-multiple-threads/2594/8 "2023-06-11T06:28:23Z")

</div>

@agibsonccc Ok, thanks for your reply, I’ll try what you said.

---

<div class="post-metadata">

**Author:** ![visionki](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/visionki/32/1157_2.png) [@visionki](https://community.konduit.ai/u/visionki)\
**Post date:** [June 15, 2023, 8:21am UTC](https://community.konduit.ai/t/how-do-i-use-loader-asmatrix-with-multiple-threads/2594/9 "2023-06-15T08:21:58Z")

</div>

@agibsonccc Hello, still similar code as above, I found that after running for a while the memory footprint will keep increasing, memory footprint is:

 ![image](https://canada1.discourse-cdn.com/flex035/uploads/konduit/original/2X/d/ddae9ceaaf7b77b581f311f959aa70016a2dfc4e.png)

When I periodically execute `system.gc()` manually, the memory footprint seems to become normal:

 ![image](https://canada1.discourse-cdn.com/flex035/uploads/konduit/original/2X/b/bb4857a6c009a9d04084077ee9fbc210fa153734.png)

What do I need to release so that memory doesn’t keep growing?

---

<div class="post-metadata">

**Author:** ![agibsonccc](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/agibsonccc/32/697_2.png) [@agibsonccc](https://community.konduit.ai/u/agibsonccc)\
**Post date:** [June 15, 2023, 10:36am UTC](https://community.konduit.ai/t/how-do-i-use-loader-asmatrix-with-multiple-threads/2594/10 "2023-06-15T10:36:17Z")

</div>

@visionki that’s pretty common with threadlocals if not managed properly. Are you creating new threads every time?