Illegal reference to restricted type var error while building Deeplearning4j

DL4J
1

Hello All!
I am trying to build deeplearning4j from sources (GitHub - deeplearning4j/deeplearning4j: Suite of tools for deploying and training deep learning models using the JVM. Highlights include model import for keras, tensorflow, and onnx/pytorch, a modular and tiny c++ library for running math code and a java based math library on top of the core c++ library. Also includes samediff: a pytorch/tensorflow like library for running deep learning using automatic differentiation. - master branch) but I get the following error during the compilation.

[ERROR] COMPILATION ERROR : 
[INFO] -------------------------------------------------------------
[ERROR] /home/mary/Projects/deeplearning4j/nd4j/nd4j-backends/nd4j-backend-impls/nd4j-cuda/src/main/java/org/nd4j/linalg/jcublas/ops/executioner/CudaExecutioner.java:[28,14] illegal reference to restricted type 'var'
[INFO] 1 error
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for deeplearning4j 1.0.0-SNAPSHOT:
[INFO] 
[INFO] deeplearning4j ..................................... SUCCESS [  0.556 s]
[INFO] libnd4j ............................................ SUCCESS [46:58 min]
[INFO] nd4j ............................................... SUCCESS [  0.889 s]
[INFO] nd4j-shade ......................................... SUCCESS [  0.088 s]
[INFO] jackson ............................................ SUCCESS [  9.456 s]
[INFO] protobuf ........................................... SUCCESS [  7.804 s]
[INFO] guava .............................................. SUCCESS [  6.839 s]
[INFO] netty-common ....................................... SUCCESS [  5.678 s]
[INFO] nd4j-common ........................................ SUCCESS [  2.910 s]
[INFO] nd4j-backends ...................................... SUCCESS [  0.052 s]
[INFO] nd4j-api-parent .................................... SUCCESS [  0.050 s]
[INFO] nd4j-api ........................................... SUCCESS [ 33.784 s]
[INFO] nd4j-common-tests .................................. SUCCESS [  0.565 s]
[INFO] nd4j-serde ......................................... SUCCESS [  0.045 s]
[INFO] nd4j-aeron ......................................... SUCCESS [  0.801 s]
[INFO] nd4j-arrow ......................................... SUCCESS [  0.578 s]
[INFO] resources .......................................... SUCCESS [  0.600 s]
[INFO] nd4j-kryo .......................................... SUCCESS [  0.453 s]
[INFO] nd4j-backend-impls ................................. SUCCESS [  0.094 s]
[INFO] nd4j-presets-common ................................ SUCCESS [  0.422 s]
[INFO] nd4j-native-api .................................... SUCCESS [  0.827 s]
[INFO] nd4j-cuda .......................................... FAILURE [ 10.547 s]
[INFO] nd4j-cuda-platform ................................. SKIPPED
[INFO] nd4j-parameter-server-parent ....................... SKIPPED
[INFO] nd4j-parameter-server-model ........................ SKIPPED
[INFO] nd4j-parameter-server .............................. SKIPPED
[INFO] nd4j-parameter-server-client ....................... SKIPPED
[INFO] nd4j-parameter-server-rocksdb-storage .............. SKIPPED
[INFO] nd4j-parameter-server-node ......................... SKIPPED
[INFO] nd4j-tensorflow .................................... SKIPPED
[INFO] nd4j-tensorflow-lite ............................... SKIPPED
[INFO] nd4j-onnxruntime ................................... SKIPPED
[INFO] nd4j-tvm ........................................... SKIPPED
[INFO] samediff-import .................................... SKIPPED
[INFO] samediff-import-api ................................ SKIPPED
[INFO] samediff-import-onnx ............................... SKIPPED
[INFO] samediff-import-tensorflow ......................... SKIPPED
[INFO] DataVec ............................................ SKIPPED
[INFO] datavec-api ........................................ SKIPPED
[INFO] datavec-data ....................................... SKIPPED
[INFO] datavec-data-image ................................. SKIPPED
[INFO] datavec-arrow ...................................... SKIPPED
[INFO] python4j-parent .................................... SKIPPED
[INFO] python4j-core ...................................... SKIPPED
[INFO] python4j-numpy ..................................... SKIPPED
[INFO] datavec-local ...................................... SKIPPED
[INFO] datavec-spark_2.12 ................................. SKIPPED
[INFO] datavec-jdbc ....................................... SKIPPED
[INFO] datavec-excel ...................................... SKIPPED
[INFO] DeepLearning4j ..................................... SKIPPED
[INFO] deeplearning4j-data ................................ SKIPPED
[INFO] deeplearning4j-datavec-iterators ................... SKIPPED
[INFO] deeplearning4j-datasets ............................ SKIPPED
[INFO] deeplearning4j-utility-iterators ................... SKIPPED
[INFO] deeplearning4j-common-tests ........................ SKIPPED
[INFO] deeplearning4j-nn .................................. SKIPPED
[INFO] deeplearning4j-modelimport ......................... SKIPPED
[INFO] deeplearning4j-ui-parent ........................... SKIPPED
[INFO] deeplearning4j-ui-components ....................... SKIPPED
[INFO] deeplearning4j-core ................................ SKIPPED
[INFO] deeplearning4j-ui-model ............................ SKIPPED
[INFO] deeplearning4j-vertx ............................... SKIPPED
[INFO] deeplearning4j-nlp-parent .......................... SKIPPED
[INFO] deeplearning4j-nlp ................................. SKIPPED
[INFO] deeplearning4j-ui .................................. SKIPPED
[INFO] DeepLearning4j-scaleout-parent ..................... SKIPPED
[INFO] Spark parent ....................................... SKIPPED
[INFO] dl4j-spark ......................................... SKIPPED
[INFO] deeplearning4j-parallel-wrapper .................... SKIPPED
[INFO] dl4j-spark-parameterserver ......................... SKIPPED
[INFO] deeplearning4j-scaleout-parallelwrapper-parameter-server SKIPPED
[INFO] deeplearning4j-graph ............................... SKIPPED
[INFO] deeplearning4j-zoo ................................. SKIPPED
[INFO] omnihub ............................................ SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  48:22 min
[INFO] Finished at: 2022-11-01T15:31:29+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.1:compile (compile) on project nd4j-cuda-11.6: Compilation failure
[ERROR] /home/mary/Projects/deeplearning4j/nd4j/nd4j-backends/nd4j-backend-impls/nd4j-cuda/src/main/java/org/nd4j/linalg/jcublas/ops/executioner/CudaExecutioner.java:[28,14] illegal reference to restricted type 'var'
[ERROR] 
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <args> -rf :nd4j-cuda-11.6

This is the full log with the maven debug messages:

I am using JDK 11.0.16, CMake 3.25, CUDA 11.6, and maven 3.6.3. My OS is Ubuntu 20.04 and I have an NVIDIA GTX 1650 Ti GPU on my machine.

conf
conf889×360 67.5 KB

I built with the following commands:

./change-cuda-versions.sh 11.6
 mvn clean install -X -Dmaven.test.skip -Dlibnd4j.cuda=11.6 -Dlibnd4j.compute=75

I see that the issue is caused by lombok.var being imported, could it be that I am missing a dependency?

Thank you!

2

@MaryXek usual question - any reason? Are you looking to contribute? If not, snapshots should be enough for you if you need recent features.

Regarding that version this was a JDK 11 upgrade issue with lombok, see fix here: Fix cuda build, add missing declarations for cuda, fix cuda backend f…or java 11 upgrade by agibsonccc · Pull Request #9838 · deeplearning4j/deeplearning4j · GitHub

if you want feel free to pull this branch for the build.

3

Hi @agibsonccc! Thank you for your reply, this solved the issue!
I am a member of the TornadoVM team and I am trying to do a proof of concept.

4

Great to hear! Happy to help if you need anything else. The next release will be out in a bit. Our main way of using cuda (javacpp) just did its release today so I can upgrade the associated cuda library for c++ as well.

1 Like
5

@MaryXek if you would like I pushed a branch for the latest cuda version here: Upgrades javacpp and associated deps to 1.5.8 by agibsonccc · Pull Request #9840 · deeplearning4j/deeplearning4j · GitHub
all you should need to do is use the change-cuda-versions.sh script as before and it should update everything to the cuda version you would like to run.

Make sure to try out our cudnn bindings if you are doing anything specific.

1 Like
6

That’s great @agibsonccc, I will definitely try it out! Thank you so much!