Hey All,
I’m trying to build the DL4J stack and was following the Build from Source Page. I made it to the section that needs me to use the install script to build. However that install script link doesn’t actually exist anymore. Is there a more up to date guide to build from source?
Thanks for flagging this.
We will be updating the build from source page very soon. Actually building is pretty simple after you’ve set up all the requirements:
mvn -B -V -U clean install -pl '!jumpy,!pydatavec,!pydl4j' -DskipTests=true -Dmaven.test.skip=true
Thank you for the quick reply! Are the requirements still the same as described in the build page?
The requirements should be still the same. If you can’t compile with the maven invocation above, then please tell us.
Hey treo, I’m getting this error once I start the maven command.
[ERROR] Failed to execute goal org.bytedeco:javacpp:1.5.3-SNAPSHOT:build (javacpp-cppbuild-validate) on project libnd4j: Execution javacpp-cppbuild-validate of goal org.bytedeco:javacpp:1.5.3-SNAPSHOT:build failed: Plugin org.bytedeco:javacpp:1.5.3-SNAPSHOT or one of its dependencies could not be resolved: The following artifacts could not be resolved: org.bytedeco:openblas:jar:android-arm:0.3.7-1.5.3-SNAPSHOT, org.bytedeco:openblas:jar:android-arm64:0.3.7-1.5.3-SNAPSHOT, org.bytedeco:openblas:jar:android-x86:0.3.7-1.5.3-SNAPSHOT, org.bytedeco:openblas:jar:android-x86_64:0.3.7-1.5.3-SNAPSHOT, org.bytedeco:openblas:jar:ios-arm64:0.3.7-1.5.3-SNAPSHOT, org.bytedeco:openblas:jar:ios-x86_64:0.3.7-1.5.3-SNAPSHOT, org.bytedeco:openblas:jar:linux-armhf:0.3.7-1.5.3-SNAPSHOT, org.bytedeco:openblas:jar:linux-arm64:0.3.7-1.5.3-SNAPSHOT, org.bytedeco:openblas:jar:linux-ppc64le:0.3.7-1.5.3-SNAPSHOT, org.bytedeco:openblas:jar:windows-x86:0.3.7-1.5.3-SNAPSHOT: Could not find artifact org.bytedeco:openblas:jar:android-arm:0.3.7-1.5.3-SNAPSHOT in sonatype-nexus-snapshots (https://oss.sonatype.org/content/repositories/snapshots) → [Help 1]
Looks like the fix for this hasn’t been pushed to the eclipse repository yet. You can try building from https://github.com/KonduitAI/deeplearning4j but that fork is a bit more unstable, as most active development happens there.
Had some issues initialy but it turns out that was because of an old cmake version. Just needed to upgrade it. Seems like its building and is probably gonna take a while. Thanks for all the help. I’ll let you know if anything else comes up!
Hey Treo, getting another during the build process. Any idea what my issue would be?
[WARNING] Rule 1: org.apache.maven.plugins.enforcer.RequireFilesExist failed with message:
!!! You have to compile libnd4j with cpu support first!
Some required files are missing:
/home/zamlz/src/java/dl4j_build/libnd4j/blas/NativeOps.h
/home/zamlz/src/java/dl4j_build/libnd4j/blasbuild/cpu/blas
Then later down the line I see,
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-enforcer-plugin:1.4.1:enforce (libnd4j-checks) on project nd4j-native: Some Enforcer rules have failed. Look above for specific messages explaining why the rule failed. -> [Help 1]
You should have another error message further up the logs.
I see these other errors before the erro r messages I sent. (just searched for the word ERROR)
[ERROR] Error fetching link: /home/zamlz/src/java/deeplearning4j-konduit/nd4j/nd4j-shade/jackson/target/apidocs/package-list. Ignored it.
[ERROR] Error fetching link: /home/zamlz/src/java/deeplearning4j-konduit/nd4j/nd4j-shade/guava/target/apidocs/package-list. Ignored it.
Should I output my result with the -e or -X flag?
That’s odd. Are you building only within the nd4j folder? The command I posted is meant to be run from the root folder.
Nope, I’m building within the root folder.
That’s weird then. Can you collect the full output and post the log here as an attachment, or on gist.github.com?
https://gist.github.com/zamlz/29c4f24185ff6546ee63406cf011c7de
Its quite a big log but here it is.
Oh, you did enable full debug on it. The normal output level is usually enough to diagnose what is wrong.
Anyway, the files that it says are missing, are they actually missing?
From what I see in the logs, libnd4j did compile successfully.
Yup appears that its actually missing.
`--> ls -la ./nd4j/nd4j-shade/{guava,jackson}/target/apidocs
./nd4j/nd4j-shade/guava/target/apidocs:
total 8.0K
drwxr-xr-x 2 zamlz zamlz 4.0K Feb 26 11:55 ./
drwxr-xr-x 8 zamlz zamlz 4.0K Feb 26 11:55 ../
./nd4j/nd4j-shade/jackson/target/apidocs:
total 8.0K
drwxr-xr-x 2 zamlz zamlz 4.0K Feb 26 11:54 ./
drwxr-xr-x 8 zamlz zamlz 4.0K Feb 26 11:54 ../
I didn’t mean the package-list files, those are red herrings. You can ignore them. It fails because of what you’ve posted earlier:
Those files must exist.
Wait hold on. I just noticed something about that path. Its not in the folder I’m currently building in. So in the doc page I linked way earlier, they say to set the environment variable $LIBND4J_HOME and I had it set to /home/zamlz/src/java/dl4j_build/libnd4j. Could this be what is causing the issues? Because there are no files in there, thats for sure. The location of my root directory that I’m building from is /home/zamlz/src/java/deeplearning4j-konduit.
Yes that is exactly the reason.
Alright! Rebuilding now and hopefully thats the last of the errors. Thanks again for the help Treo!