Single threaded Nd4j operations

Hello,

My use case requires any application to run on a single thread as our hardware accepts multiple submissions from multiple users, and unintended multithreading can cause conflicts.

I am using Nd4j primarily for matrix multiplication calling Nd4j.setNumThreads(1) prior to any operations. Jobs I submit end up with ~170% CPU usage throughout their run however, indicating that multiple threads are still being used.

I am using version 1.0.0-beta4. I am wondering if I am just configuring it incorrectly to meet my requirements or if there is anything fundamental I am missing which would make this infeasible.

1 Like

That setting only sets how many threads ND4J itself uses. But ND4J also uses BLAS libraries, which, unless configured differently, will use a varying number of threads, depending on the input size.

In order to tell the BLAS library (usually either OpenBLAS or MKL) to use just one thread you also need to set the “OMP_NUM_THREADS” environment variable to 1.

1 Like

That looks like its working. Thanks for the quick response.

Edit: I notice you are using beta4. I also cannot access the method .setNumThreads() from the latest version, if anyone wants to chime in.

Can you share where you found the OMP_NUM_THREADS environment variable? My boss is also concerned with being able to run on a single thread.

I looked in org.bytecode.openblas.global.openblas, but see only OPENBLAS_THREAD = 1 and OPENBLAS_GEMM_MULTITHREAD_THRESHOLD=4.

ND4J is using a BLAS library, and most of those are using OpenMP under the hood. On the last few pages of the reference guide you’ve got the environment variables that can influence it:

Ideally you start with OMP_DISPLAY_ENV=true to get idea what version it actually is, and then get the reference guide to that to know what exactly you can do.

Thanks for the response and linked document. I took a look at it and it’s full things that are absolute Greek to me:

"A directive that can specify multiple directive variants,
one of which may be conditionally selected to replace the
metadirective based on the enclosing OpenMP context."

I’m a high level user who is clueless under the hood. Let me rephrase my question to give you the context of my cluelessness:

When you say start with OMP_DISPLAY_ENV=true, what are you functionally telling me to do?

  • Set an OS environment variable (it doesn’t exist on my machine, do I create it?)
  • Alter a config file somewhere?
  • Alter some code in my projects dependencies?
  • Type something into the command line?

Yes, these are environment variables. You don’t necessarily need to set them at OS level, you can set them up in your IDE or in a script that starts your application.

How exactly you should do this, will depend on the environment you are actually running in.

Ah, okay. I’m on Windows 10, in IntelliJ. I’ll do some Googling about how to do that and see what I can find before I bother you more.

Just as a follow-up, I wanted to let you know that your solution worked for me. Thanks.