# Memory leak in ND4J if using the ZGC garbage collector

**URL:** <https://community.konduit.ai/t/memory-leak-in-nd4j-if-using-the-zgc-garbage-collector/1684>\
**Category:** ND4J\
**Created:** [November 20, 2021, 2:36pm UTC](https://community.konduit.ai/t/memory-leak-in-nd4j-if-using-the-zgc-garbage-collector/1684 "2021-11-20T14:36:16Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![MankaranSingh](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/mankaransingh/32/607_2.png) [@MankaranSingh](https://community.konduit.ai/u/MankaranSingh)\
**Post date:** [November 20, 2021, 2:36pm UTC](https://community.konduit.ai/t/memory-leak-in-nd4j-if-using-the-zgc-garbage-collector/1684/1 "2021-11-20T14:36:16Z")

</div>

I know that ZGC is still in an experimental phase, but recently I tried it out and it caused a memory leak.  
here are the logs:

```auto
java.lang.OutOfMemoryError: Cannot allocate new LongPointer(1): totalBytes = 512, physicalBytes = 4146M
        at com.badlogic.gdx.backends.lwjgl3.Lwjgl3Application.<init>(Lwjgl3Application.java:122)
        at com.flow.app.lwjgl3.Lwjgl3Launcher.createApplication(Lwjgl3Launcher.java:33)
        at com.flow.app.lwjgl3.Lwjgl3Launcher.main(Lwjgl3Launcher.java:19)
Caused by: java.lang.OutOfMemoryError: Cannot allocate new LongPointer(1): totalBytes = 512, physicalBytes = 4146M
        at org.bytedeco.javacpp.LongPointer.<init>(LongPointer.java:88)
        at org.bytedeco.javacpp.LongPointer.<init>(LongPointer.java:53)
        at org.nd4j.linalg.cpu.nativecpu.ops.CpuOpContext.setIArguments(CpuOpContext.java:60)
        at org.nd4j.linalg.cpu.nativecpu.ops.NativeOpExecutioner.exec(NativeOpExecutioner.java:1558)
        at org.nd4j.linalg.factory.Nd4j.exec(Nd4j.java:6566)
        at org.nd4j.linalg.cpu.nativecpu.CpuNDArrayFactory.concat(CpuNDArrayFactory.java:579)
        at org.nd4j.linalg.factory.Nd4j.concat(Nd4j.java:4857)
        at org.nd4j.linalg.factory.BaseNDArrayFactory.vstack(BaseNDArrayFactory.java:983)
        at org.nd4j.linalg.factory.Nd4j.vstack(Nd4j.java:4641)
        at com.flow.vision.Preprocess.eulerAnglesToRotationMatrix(Preprocess.java:161)
        at com.flow.app.StreamScreen.updateLanes(StreamScreen.java:129)
        at com.flow.app.StreamScreen.render(StreamScreen.java:183)
        at com.badlogic.gdx.Game.render(Game.java:46)
        at com.badlogic.gdx.backends.lwjgl3.Lwjgl3Window.update(Lwjgl3Window.java:403)
        at com.badlogic.gdx.backends.lwjgl3.Lwjgl3Application.loop(Lwjgl3Application.java:143)
        at com.badlogic.gdx.backends.lwjgl3.Lwjgl3Application.<init>(Lwjgl3Application.java:116)
        ... 2 more
Caused by: java.lang.OutOfMemoryError: Physical memory usage is too high: physicalBytes (4146M) > maxPhysicalBytes (4040M)
        at org.bytedeco.javacpp.Pointer.deallocator(Pointer.java:700)
        at org.bytedeco.javacpp.Pointer.init(Pointer.java:126)
        at org.bytedeco.javacpp.LongPointer.allocateArray(Native Method)
        at org.bytedeco.javacpp.LongPointer.<init>(LongPointer.java:80)
        ... 17 more

```

specs: windows 10, nd4j 1.0.0-M1

thanks!

---

<div class="post-metadata">

**Author:** ![saudet](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/saudet/32/69_2.png) [@saudet](https://community.konduit.ai/u/saudet)\
**Post date:** [November 20, 2021, 2:38pm UTC](https://community.konduit.ai/t/memory-leak-in-nd4j-if-using-the-zgc-garbage-collector/1684/2 "2021-11-20T14:38:35Z")

</div>

It’s not a memory leak. Using JavaCPP 1.5.6 should help with this: [OutOfMemoryError: Physical memory usage is too high: physicalBytes (5300M) \> maxPhysicalBytes (5300M) · Issue #468 · bytedeco/javacpp · GitHub](https://github.com/bytedeco/javacpp/issues/468)

---

<div class="post-metadata">

**Author:** ![MankaranSingh](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/mankaransingh/32/607_2.png) [@MankaranSingh](https://community.konduit.ai/u/MankaranSingh)\
**Post date:** [November 21, 2021, 5:59am UTC](https://community.konduit.ai/t/memory-leak-in-nd4j-if-using-the-zgc-garbage-collector/1684/3 "2021-11-21T05:59:04Z")

</div>

Thanks for the reply ! This error showed up after running the app for 3+ mins with ZGC. It works fine if not using ZGC with almost constant memory usage. The other thing I noticed was extremely high RAM usage in the profiler when ZGC was used. I will post screenshots soon.

btw how do I update javaCPP ? I only have nd4j as a dependency. Do I need to edit the nd4j pom files for this ?

---

<div class="post-metadata">

**Author:** ![MankaranSingh](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/mankaransingh/32/607_2.png) [@MankaranSingh](https://community.konduit.ai/u/MankaranSingh)\
**Post date:** [November 21, 2021, 6:13am UTC](https://community.konduit.ai/t/memory-leak-in-nd4j-if-using-the-zgc-garbage-collector/1684/4 "2021-11-21T06:13:23Z")

</div>

No ZGC:

 ![no_zgc](https://canada1.discourse-cdn.com/flex035/uploads/konduit/original/1X/49f0d3a51031a972cc44a13db2cccaa1c27b5d9a.jpeg)

---

<div class="post-metadata">

**Author:** ![MankaranSingh](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/mankaransingh/32/607_2.png) [@MankaranSingh](https://community.konduit.ai/u/MankaranSingh)\
**Post date:** [November 21, 2021, 6:14am UTC](https://community.konduit.ai/t/memory-leak-in-nd4j-if-using-the-zgc-garbage-collector/1684/5 "2021-11-21T06:14:02Z")

</div>

With ZGC:

 ![zgc2](https://canada1.discourse-cdn.com/flex035/uploads/konduit/original/1X/3e86b59385359654500c2124470aa0bb031f127e.jpeg)

In this case the heap shoots to 2GB. and the app finally crrashes at last.

Let me know if you need any heap dumps or other things!  
thx

---

<div class="post-metadata">

**Author:** ![saudet](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/saudet/32/69_2.png) [@saudet](https://community.konduit.ai/u/saudet)\
**Post date:** [November 21, 2021, 6:48am UTC](https://community.konduit.ai/t/memory-leak-in-nd4j-if-using-the-zgc-garbage-collector/1684/6 "2021-11-21T06:48:47Z")

</div>

Yes, in the pom.xml file like this: [https://github.com/bytedeco/javacpp#downloads](https://github.com/bytedeco/javacpp#downloads)

---

<div class="post-metadata">

**Author:** ![partarstu](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/partarstu/32/276_2.png) [@partarstu](https://community.konduit.ai/u/partarstu)\
**Post date:** [November 21, 2021, 9:20pm UTC](https://community.konduit.ai/t/memory-leak-in-nd4j-if-using-the-zgc-garbage-collector/1684/7 "2021-11-21T21:20:33Z")

</div>

> [@MankaranSingh](#):
>
> The other thing I noticed was extremely high RAM usage in the profiler when ZGC was used.

It’s actually the nature of ZGC. It provides low-latency garbage collection, but at a price - by “eating” more RAM. It depends on many factors of course, but I’ve noticed in practice, that when the heap is quite volatile and a lot of stuff needs to be collected by GC very often, ZGC starts taking more memory for own purposes in order to guarantee the low-latency (it needs memory space in order keep all the “household” info). There’s one thing though - the actual array memory allocations for DL4J occur in the native memory, not heap. So what you see on your heap usage graphs is the part which is being used basically by your app directly (like by generated/loaded test/train data, service data etc.). Those graphs won’t show you how much exactly the whole application takes - this thing can be done by running `org.bytedeco.javacpp.Pointer#physicalBytes()`
