Python4j-numpy exception No module named 'tensorflow'

ND4J
1

/home/sidney/ner/classes:/home/sidney/ner/pcng-idea-ner-1.0-bin.jar -Xms1g -Xmx1g -Dorg.bytedeco.javacpp.maxbytes=1024m -Dorg.bytedeco.javacpp.maxphysicalbytes=1024m com.pcng.idea.ner.Py4jTest
Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.javacpp.Loader
Debug: Loading /home/sidney/.javacpp/cache/pcng-idea-ner-1.0-bin.jar/org/bytedeco/javacpp/linux-x86_64/libjnijavacpp.so
Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.javacpp.Pointer
Debug: Locking /home/sidney/.javacpp/cache before extracting
Debug: Extracting jar:file:/home/sidney/ner/pcng-idea-ner-1.0-bin.jar!/org/bytedeco/cpython/linux-x86_64
Debug: Creating symbolic link /home/sidney/.javacpp/cache/pcng-idea-ner-1.0-bin.jar/org/bytedeco/cpython/linux-x86_64/lib/libgcc_s.so
Debug: Creating symbolic link /home/sidney/.javacpp/cache/pcng-idea-ner-1.0-bin.jar/org/bytedeco/cpython/linux-x86_64/libffi.so
Debug: Creating symbolic link /home/sidney/.javacpp/cache/pcng-idea-ner-1.0-bin.jar/org/bytedeco/cpython/linux-x86_64/lib/libffi.so
Debug: Creating symbolic link /home/sidney/.javacpp/cache/pcng-idea-ner-1.0-bin.jar/org/bytedeco/cpython/linux-x86_64/lib/libpython3.10.so
Debug: Creating symbolic link /home/sidney/.javacpp/cache/pcng-idea-ner-1.0-bin.jar/org/bytedeco/cpython/linux-x86_64/lib/libffi.so
Debug: Creating symbolic link /home/sidney/.javacpp/cache/pcng-idea-ner-1.0-bin.jar/org/bytedeco/cpython/linux-x86_64/lib/libgcc_s.so
Debug: Creating symbolic link /home/sidney/.javacpp/cache/pcng-idea-ner-1.0-bin.jar/org/bytedeco/cpython/linux-x86_64/lib/libquadmath.so
Debug: Creating symbolic link /home/sidney/.javacpp/cache/pcng-idea-ner-1.0-bin.jar/org/bytedeco/cpython/linux-x86_64/lib/libquadmath.so
Debug: Creating symbolic link /home/sidney/.javacpp/cache/pcng-idea-ner-1.0-bin.jar/org/bytedeco/cpython/linux-x86_64/lib/libgfortran.so
Debug: Creating symbolic link /home/sidney/.javacpp/cache/pcng-idea-ner-1.0-bin.jar/org/bytedeco/cpython/linux-x86_64/lib/libpython3.so
Debug: Creating symbolic link /home/sidney/.javacpp/cache/pcng-idea-ner-1.0-bin.jar/org/bytedeco/cpython/linux-x86_64/lib/libpython3.10.so
Debug: Creating symbolic link /home/sidney/.javacpp/cache/pcng-idea-ner-1.0-bin.jar/org/bytedeco/cpython/linux-x86_64/lib/libgfortran.so
Debug: Creating symbolic link /home/sidney/.javacpp/cache/pcng-idea-ner-1.0-bin.jar/org/bytedeco/cpython/linux-x86_64/lib/libopenblas.so
Debug: Creating symbolic link /home/sidney/.javacpp/cache/pcng-idea-ner-1.0-bin.jar/org/bytedeco/cpython/linux-x86_64/lib/libopenblas.so
Debug: Creating symbolic link /home/sidney/.javacpp/cache/pcng-idea-ner-1.0-bin.jar/org/bytedeco/cpython/linux-x86_64/lib/libpython3.so
Debug: Locking /home/sidney/.javacpp/cache before extracting
Debug: Extracting jar:file:/home/sidney/ner/pcng-idea-ner-1.0-bin.jar!/org/bytedeco/numpy/linux-x86_64/python/
Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.cpython.global.python
Debug: Loading /home/sidney/.javacpp/cache/pcng-idea-ner-1.0-bin.jar/org/bytedeco/cpython/linux-x86_64/libffi.so.6
Debug: Loading /home/sidney/.javacpp/cache/pcng-idea-ner-1.0-bin.jar/org/bytedeco/cpython/linux-x86_64/libffi.so
Debug: Loading library libcrypto-1_1
Debug: Failed to load for libcrypto-1_1: java.lang.UnsatisfiedLinkError: no libcrypto-1_1 in java.library.path: /usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib
Debug: Loading library libssl-1_1
Debug: Failed to load for libssl-1_1: java.lang.UnsatisfiedLinkError: no libssl-1_1 in java.library.path: /usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib
Debug: Loading /home/sidney/.javacpp/cache/pcng-idea-ner-1.0-bin.jar/org/bytedeco/cpython/linux-x86_64/libpython3.10.so.1.0
Debug: Loading /home/sidney/.javacpp/cache/pcng-idea-ner-1.0-bin.jar/org/bytedeco/cpython/linux-x86_64/libjnipython.so
Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.openblas.global.openblas_nolapack
Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.openblas.global.openblas_nolapack
Debug: Loading /home/sidney/.javacpp/cache/pcng-idea-ner-1.0-bin.jar/org/bytedeco/openblas/linux-x86_64/libgcc_s.so.1
Debug: Loading /home/sidney/.javacpp/cache/pcng-idea-ner-1.0-bin.jar/org/bytedeco/openblas/linux-x86_64/libquadmath.so.0
Debug: Loading /home/sidney/.javacpp/cache/pcng-idea-ner-1.0-bin.jar/org/bytedeco/openblas/linux-x86_64/libgfortran.so.5
Debug: Loading library gfortran
Debug: Failed to load for gfortran@.4: java.lang.UnsatisfiedLinkError: no gfortran in java.library.path: /usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib
Debug: Loading library gfortran
Debug: Failed to load for gfortran@.3: java.lang.UnsatisfiedLinkError: no gfortran in java.library.path: /usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib
Debug: Loading /home/sidney/.javacpp/cache/pcng-idea-ner-1.0-bin.jar/org/bytedeco/openblas/linux-x86_64/libopenblas_nolapack.so.0
Debug: Loading /home/sidney/.javacpp/cache/pcng-idea-ner-1.0-bin.jar/org/bytedeco/openblas/linux-x86_64/libjniopenblas_nolapack.so
Debug: Loading class org.bytedeco.openblas.global.openblas
Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.openblas.global.openblas_nolapack
Debug: Loading class org.bytedeco.openblas.global.openblas
Debug: Loading /home/sidney/.javacpp/cache/pcng-idea-ner-1.0-bin.jar/org/bytedeco/openblas/linux-x86_64/libopenblas.so.0
Debug: Loading /home/sidney/.javacpp/cache/pcng-idea-ner-1.0-bin.jar/org/bytedeco/openblas/linux-x86_64/libjniopenblas.so
Debug: Loading class org.bytedeco.cpython.global.python
Debug: Loading class org.bytedeco.numpy.global.numpy
Debug: Loading library npymath
Debug: Failed to load for npymath: java.lang.UnsatisfiedLinkError: no npymath in java.library.path: /usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib
Debug: Loading /home/sidney/.javacpp/cache/pcng-idea-ner-1.0-bin.jar/org/bytedeco/numpy/linux-x86_64/libjninumpy.so
Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.javacpp.BytePointer
Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.cpython.global.python
Debug: Loading class org.bytedeco.cpython.PyThreadState
[main] INFO org.nd4j.python4j.PythonGIL - Pre Gil State ensure for thread 1
[main] INFO org.nd4j.python4j.PythonGIL - Thread 1 acquired GIL
Debug: Loading class org.bytedeco.javacpp.presets.javacpp
Debug: Loading class org.bytedeco.cpython.global.python
Debug: Loading class org.bytedeco.cpython.PyObject
No module named ‘tensorflow’
Traceback (most recent call last):
File “”, line 29, in
ModuleNotFoundError: No module named ‘tensorflow’
[main] INFO org.nd4j.python4j.PythonGIL - Releasing GIL on thread 1
org.nd4j.python4j.PythonException: ModuleNotFoundError: No module named ‘tensorflow’
at org.nd4j.python4j.PythonExecutioner.throwIfExecutionFailed(PythonExecutioner.java:201)
at org.nd4j.python4j.PythonExecutioner.exec(PythonExecutioner.java:229)
at org.nd4j.python4j.PythonExecutioner.exec(PythonExecutioner.java:236)
at com.pcng.idea.ner.Py4jTest.main(Py4jTest.java:37)

The app run successfully when there has no dependency python4j-numpy in pom.xml, after add this dependency, below error accors
ModuleNotFoundError: No module named ‘tensorflow’

This happens in centos8 and jdk17, any idea? thanks.

2

@SidneyLann download conda and ensure you set the python path up to include that. Ensure it’s the exact same version. We don’t bundle dependencies automatically.
Set the path using the system property: org.eclipse.python4j.path

See more here:

We won’t support bundling random deps and reinventing the wheel. Conda solves that problem pretty well. Just make sure it’s the exact same version as what python4j runs and the rest will be fine.

3

Still not work after reinstall numpy from 1.22.3 to 1.22.2, now M2+py3.10.2+numpy1.22.2 are used.

4

@SidneyLann I’m a bit confused…what does numpy have to do with a missing tensorflow import? I thought the error was pretty clear there. Ensure tensorflow is setup in your anaconda and just ensure the python version from anaconda matches the one being used from python4j.

We have numpy built in if you include python4j-numpy so just focus on other dependencies.

Make sure to actually read the error and just look for missing imports of things you want to run, install them via conda, and setup the python path.

Beyond that I can’t read your computer so I have no clue what you’ve tried and what you haven’t. Our guide is pretty comprehensive for how to get the path so when you try again and need help make sure to post what you tried first.

5

org.nd4j.python4j.PythonException: Unable to find converter for python object of type <class ‘numpy.ndarray’>
Above exception if no python4j-numpy involved.

ModuleNotFoundError: No module named ‘tensorflow’
Above exception if have python4j-numpy dependency.

public static void main(String args) {
System.setProperty(“org.bytedeco.javacpp.logger.debug”, “true”);
System.setProperty(“org.eclipse.python4j.path”, “/home/sidney/Python3102/lib/python3.10/site-packages”);
try (PythonGIL gil = PythonGIL.lock()) {
try (PythonGC gc = PythonGC.watch()) {
List inputs = new ArrayList<>();
List outputs = new ArrayList<>();
inputs.add(new PythonVariable<>(“x”, PythonTypes.STR, "Hello "));
inputs.add(new PythonVariable<>(“y”, PythonTypes.STR, “World”));

    PythonVariable out = new PythonVariable<>("z", PythonTypes.LIST);
    String code = "import tensorflow as tf\nz = tf.keras.backend.get_value(tf.constant([[0., 0., 0., 2.3], [0., 0., 0., 3.1], [0., 0., 0., 1.2]]))";
    PythonExecutioner.exec(code, inputs,Collections.singletonList(out));
  }
} catch (Throwable e) {
  e.printStackTrace();
}

}

Could you please help to run this code in jdk17?

6

@SidneyLann no because you’re not listening to me. I gave you tons of docs on how to setup the python path. I stated multiple times that’s the source of the problem. Python itself is telling you what the exact problem is.
I’m not helping you any further till you actually read my previous advice. Your new questions indicate you didn’t read anything.

7

I of course begin from the doc, but the org.eclipse.python4j.path does not work. App run successfully after I copy all files under org.eclipse.python4j.path to the built-in py path in ~/.javacpp.

8

@SidneyLann then elaborate on that. I just want you to show me you followed the docs.
Let’s focus on what I asked you about with the python path. How did you set it?
That’s the only thing to focus on here everything else is noise.

Show me how you setup the anaconda install as well.