Compiling deeplearning4j with -Pcuda does not produce cuda artifacts


I am working on compiling deeplearning4j for my specific architecture. In the top level directory I am running the command:

mvn clean install -Pcuda -DskipTests -PskipTestResourceEnforcement=true -Djavacpp.platform=linux-x86_64

But the following dependencies that I thought should exist are not produced:

  • org.nd4j:nd4j-cuda-11.2-platform:jar:chriscustom1.0
  • org.deeplearning4j:deeplearning4j-cuda-11.2:jar:chriscustom1.0

Other artifacts are produced, but not this one. I am branched off the 1.0.0-M1.1 tag.

What am I doing wrong?

I can successfully produce a -Pcpu build, FYI, and run on the CPU with it.

@chris2 you can just download cuda 11.2 support from maven central now. This was fixed last week. Note that going forward we’ll mainly support cuda 11.6 and 11.7 though.

Amazing! Can I get cuda 11.6 support from maven central or your baseline or something? I don’t see it yet: Maven Central Repository Search I’m actually only compiling because I must use 11.6. I was trying wit h 11.2 just to learn how to build it from a simpler test case. @agibsonccc

Also, part of why I have had to recompile is that the linux_x86 *.so files in the nd4j native packages didn’t work on my older operating system/architecture. They worked after I recompiled them. I have run into the same problem with your new cuda artifacts. Do you have some instructions floating around for compiling the -Pcuda project?

Hello! I have discovered a couple of critical facts that I think have taken me 90% of the way to a solution here. For the record:

  1. I discovered the shell script and have updated it for the proper versions of cuda/cudann/etc
  2. You gave me a clue when you said you were all changing the build to build the nd4j backends from the root. When I moved to the nd4j-backends directory and started building there, it started creating the artifacts.

It will be some time before the build is complete, but I think this is going to work.

@chris2 yes that’s all you need to do is change the base versions. Basically cuda version support just consists of changing the versions in the pom and ensuring that libnd4j c++ compilation matches that. Cuda 11.6 is on the way with the next release.