Limit native dependencies to one platform

ND4J
1

I would like to use DL4J and ND4J in Java EE app but adding them as dependencies inflates WAR size to the whooping 1.5 Gb. The culprits are the unnecessary platform-specific jar files. And the problem is not only the ND4J itself but also indirect dependencies:

  • WEB-INF\lib\hdf5-1.12.1-1.5.7-windows-x86_64.jar
  • WEB-INF\lib\javacpp-1.5.7-windows-x86_64.jar
  • WEB-INF\lib\leptonica-1.82.0-1.5.7-windows-x86_64.jar
  • WEB-INF\lib\mkl-2022.0-1.5.7-windows-x86_64.jar
  • WEB-INF\lib\javacpp-1.5.7-windows-x86_64.jar
  • WEB-INF\lib\openblas-0.3.19-1.5.7-windows-x86_64.jar
  • WEB-INF\lib\ffmpeg-5.0-1.5.7-windows-x86_64.jar

I tried couple of suggestions from the net, but nothing helped.

I added “classifier” in pom:

         <dependency>
            <groupId>org.nd4j</groupId>
            <artifactId>nd4j-native-platform</artifactId>
            <version>1.0.0-M2.1</version>
            <classifier>windows-x86_64</classifier>
        </dependency>

But it resulted in the error:

Could not find artifact org.nd4j:nd4j-native-platform:jar:windows-x86_64:1.0.0-M2.1

I tried to use “-Djavacpp.platform=windows-x86_64” option:

mvn -Djavacpp.platform=windows-x86_64 install

But got an error:

Unknown lifecycle phase ".platform=windows-x86_64".

Could somebody please advise a working solution that will help with inherited dependencies as well?

2

@sogawa-sps don’t specify a classifier for -platform. Just specify the system property on the command line. nd4j-native-platform does not itself have a classifier. It’s a way of letting the user specify the dependencies they want using system properties.

If you want to do that manually (that’s not usually the norm) you would use the nd4j-native dependency:

https://repo1.maven.org/maven2/org/nd4j/nd4j-native/1.0.0-M2.1/

Change to:

 <dependency>
            <groupId>org.nd4j</groupId>
            <artifactId>nd4j-native-platform</artifactId>
            <version>1.0.0-M2.1</version>
        </dependency>

then specify -Djavacpp.platform=windows-x86_64

Your other error there is a side effect of trying to specify a classifier with the nd4j-native-platform.
Try that and it should work.

3

Thank you for explanation about classifier, it’s clear now.

I returned to the " -Djavacpp.platform=windows-x86_64" option and found out why it’s not worked. The issue was in Powershell that I used to to run mvn command (it looks like it swallows “-D”). I switched to cmd and it worked like a charm for all dependencies both direct and inderect. WAR size is 290 MB now.