Deeplearning4j-core .jar file size

1

Hi,

i worked on an image classifier (CNN) the last weeks and packaged it to an executable .jar file the very first time today. While doing so (using “mvn package”) i realized that my .jar file has an enormous size of 1.3GB. After playing around with the pom.xml i found out, that after packaging, the deeplearning4j-core dependency needs over 800MB.

So my questions:

  • Is there a possibility to reduce the size of the .jar file? (without dl4j .jar file only needs 60MB)
  • is there a workaround, to use the pre-trained network (which i saved as .zip file) without dl4j dependencies, so i can use it in my application to make predictions?

maybe for a little bit more context:

  • i am trying to classify incoming images with my pre-trained network. so the only file which i would need in that application is the saved network (.zip file). however to load this file i think i need deeplearning4j-core?!

Thanks :slight_smile:

2

@jakob_404 mind posting your exact pom? Some aspects can be explained here:
https://deeplearning4j.konduit.ai/v/en-1.0.0-m1/multi-project/how-to-guides/developer-docs/javacpp#dl-4-j-and-javacpp-overview

3 
 <dependency>
            <groupId>org.nd4j</groupId>
            <artifactId>nd4j-native</artifactId>
            <version>${dl4j.version}</version>
        </dependency>



        <dependency>
            <groupId>org.deeplearning4j</groupId>
            <artifactId>deeplearning4j-core</artifactId>
            <version>${dl4j.version}</version>
        </dependency>

These are the two dependencies related to dl4j. I already removed “nd4j-native-platform” and am now using “nd4j-native” which already saves 400MB. Thanks for the link, hopefully i find something useful to save even more space.

4

The above javacpp link is no longer working.

When examining the jar file, it looks like multiple libraries for 3 platforms: windows, mac, linux and some different processor types.
(e.g. bytedeco/tensorflow/macosx_x86_64/libtensorflow_cc.so,linux_x86_64/libtensorflow_cc.so)
Perhaps the size could be reduced by targeting a specific platform?

5

@craig88 Let’s reframe this a bit. You’re on the right track.

Deeplearlning4j-core itself is a fairly decent sized jar that you can break down in to different components:

  1. Computer vision modules
  2. deeplearning4j-nn: the core configuration api
  3. Nd4j dependencies

Computer vision and nd4j both use javacpp. Those generally have an org.bytedeco. namespace or a -platform suffix.

Using that context you can read this:

In essence it’s telling you to use -Djavacpp.platform= in your maven builds. This will allow you to use the platform you want.