ND4J on OS/X execution error and building concerns

ND4J
1

Short question. I have managed to get to this error a couple times trying to build ND4J and then run it standalone class path.

Exception in thread “main” java.lang.NoClassDefFoundError: org/bytedeco/javacpp/openblas
at org.nd4j.linalg.cpu.nativecpu.blas.CpuBlas.setMaxThreads(CpuBlas.java:120)

Where should this class be found?

I opened an incident on this…
deeplearning4j/nd4j/issues/2947

Searching my machine or Google I have been unable to determine where a org.bytedeco/javacpp/openblas class might be. I did find javacpp-presets gives you a couple locations for the class but in different packages.

Generally on building OS/X as mentioned in the issue it appeared changing to this in the nd4j pom.xml
seemed needed
<lombok.version>1.18.20</lombok.version>

Today I tried it again first running into this error. (Nd4jbackend) NoAvailableBackendException with…
Please see: http://nd4j.org/getstarted.html
hardcoded directly in the exception. This is a busted link.

I tried to follow this for today’s attempt
https://github.com/eclipse/deeplearning4j/blob/master/libnd4j/macOSx10%20(CPU%20only).md

The gcc-5 recommendation seems a little dated. Unfortunately I didn’t save all the output from my attempts but before getting gcc squared away to run buildnativeoperations.sh…
I had errors that _stdio.h couldn’t be found. I also had…

dyld: lazy symbol binding failed: Symbol not found: ___emutls_get_address
Referenced from: /usr/local/opt/gcc@5/lib/gcc/5/libgomp.1.dylib
Expected in: /usr/lib/libSystem.B.dylib

dyld: Symbol not found: ___emutls_get_address
Referenced from: /usr/local/opt/gcc@5/lib/gcc/5/libgomp.1.dylib
Expected in: /usr/lib/libSystem.B.dylib

I found this issues/8217 suggesting gcc-8 as more current so I tried changing to that. (I’m only allowed 3 links on my first post).
I think for one doing
gcc unlink gcc@5
cmake still seemed sort of stuck expecting gcc-5 so I changed buildnativeoperations.sh to…
eval $CMAKE_COMMAND -D CMAKE_C_COMPILER=gcc-8 -D CMAKE_CXX_COMPILER=g+±8 “$BLAS_ARG” “$ARCH_ARG” “$NAME_ARG” “$SHARED_LIBS_ARG” “$MINIFIER_ARG” “$OPERATIONS_ARG” “$BUILD_TYPE” “$PACKAGING_ARG” “$EXPERIMENTAL_ARG” “$CUDA_COMPUTE” -DDEV=FALSE -DMKL_MULTI_THREADED=TRUE …/…

maybe the OS/X directions should be changed to reflect that gcc-8 be preferred to start with?

But this got me back to the same error above I had gotten just throwing jars and dylibs at my test.

If anyone can tell me where the org/bytedeco/javacpp/openblas class should be found or what I should change so it doesn’t expect it I would appreciate ti.

Thanks.

2

Please try again with a more recent version.

3

A more recent version of what?

4

I thought this morning I tried pulling current versions of nd4j, libnd4j and javacpp from origin using GitHub desktop. I think only one showed an update. Current gcc seemed not to fix anything. So I’m not sure what should be more recent?

5

@mik3hall of note. Please use the proper repository. That repository has long been abandoned. The repository is actually under: https://github.com/eclipse/deeplearning4j. Could you please tell me where you found the link to that repository? It would be handy to make sure other folks don’t get distracted.

nd4j.org/ is also not the proper site. It’s: https://deeplearning4j.konduit.ai/ under the nd4j section.

Your concern about the missing link isn’t a problem with the new site.

So I’m going to ignore that here.

Your building from source docs are also wrong. You can find them here:

The most recent version (as you can see in the docs) is 1.0.0-M1.1 you should be fine using that.

For mac osx, generally if you just follow the github action for mac you can find for example the brew commands:
https://github.com/eclipse/deeplearning4j/blob/master/.github/workflows/build-deploy-mac.yml#L164

If you need any help navigating that let me know.

6

Blockquote
Could you please tell me where you found the link to that repository? It would be handy to make sure other folks don’t get distracted.

That was a little time back. I remembered seeing this a while ago and at this point was interested in a current, maintained, matrix library so probably just searched on ND4J and went from there. If these are not the current sources for any of this it might be an idea to mark the pages as no longer valid. It would of saved me quite a few hours of distraction.

I will attempt to follow the information you have indicated. I see for OS/X it still seems to involve home-brew and cmake. I still have a bit of a concern there in having gone back to try gcc-5 and that being remembered somehow when I try cmake. I might have to figure that out yet to follow this process. Again eliminating or marking obsolete the page that says to use gcc-5 could sometime save someone considerable effort and potential problems.

I will let you know if I run into any issues in this process.

Thanks for the reply.

7

The libndj4 file “macOSx10 (CPU only).md” has the exact directions I tried to follow for that.
Including the gcc-5 which had the issues previously mentioned. gcc-8 seeming to be what was successful for that part.

8

@mik3hall Unfortunately, when we moved the library to the eclipse foundation we lost access to some of those. We’re working on getting that resolved.

In the mean time, libnd4j is about as stock as it gets for a c++ library. You can use maven if you want for the build process. As long as the right tools (compilers etc) are installed, we auto configure many of the flags you need to set for building libnd4j and then invoke the exact same script you would use for the build process.

9

[SOLVED]
I got the full deeplearning4j and build nd4j from that. Got it working well enough to do a simple test. For libnd4j it switched back to clang so the gcc-5 wasn’t a concern.
Now have to try getting beyond the simple test.
Thanks again

10

@mik3hall thanks a lot for following up. Glad you got the build working!