Nd4j logging bubbling up

ND4J
1

I’m using log4j2 (2.16) and am trying to find a way to disable or filter the logging that occurs in ND4J from my own logging. While it is good info to know, I’d like to be able to turn it off and on:

[INFO]	2022-01-06T14:34:56,250	utils.Data		Reading src/test/resources/cluster_type0.csv
[INFO]	2022-01-06T14:34:56,539	factory.Nd4jBackend		Loaded [CpuBackend] backend
[INFO]	2022-01-06T14:34:57,271	nativeblas.NativeOpsHolder		Number of threads used for linear algebra: 8
[INFO]	2022-01-06T14:34:57,272	nativecpu.CpuNDArrayFactory		Binary level Generic x86 optimization level AVX/AVX2
[INFO]	2022-01-06T14:34:57,293	nativeblas.Nd4jBlas		Number of threads used for OpenMP BLAS: 8
[INFO]	2022-01-06T14:34:57,304	executioner.DefaultOpExecutioner		Backend used: [CPU]; OS: [Windows 10]
[INFO]	2022-01-06T14:34:57,304	executioner.DefaultOpExecutioner		Cores: [16]; Memory: [14.1GB];
[INFO]	2022-01-06T14:34:57,304	executioner.DefaultOpExecutioner		Blas vendor: [OPENBLAS]
[INFO]	2022-01-06T14:34:57,306	nativecpu.CpuBackend		Backend build information:
 GCC: "10.3.0"
STD version: 201103L
DEFAULT_ENGINE: samediff::ENGINE_CPU
HAVE_FLATBUFFERS
HAVE_OPENBLAS

my log4j2.properties file looks like so:

name=LogProperties
status=warn

appender.console.type = Console
appender.console.name = consoleLogger
appender.console.layout.type = PatternLayout
appender.console.layout.pattern = [%p]\t%d{ISO8601}\t%c{2}\t\t%m%n

appender.file.type = File
appender.file.name = fileLogger
appender.file.append = false
appender.file.fileName = dean.log
appender.file.layout.type = PatternLayout
appender.file.layout.pattern = %m%n

rootLogger.level = info
rootLogger.appenderRef.stdout.ref = consoleLogger
rootLogger.appenderRef.file.ref = fileLogger
2

I don’t really have a lot of experience with log4j, but in other logging systems you can specify the log level by class path, so if you want nd4j to shutup, you can probably tell it to use a less verbose log level for everything in org.nd4j.

3

Hi. Based on DL4J’s POM file I’d deduce that SLF4J is used for logging as a facade and Logback as the specific implementation. So the solution would be to create logback.xml file in the resources folder and set the logging level there. If however you want to use log4j, I’d suggest you use SLF4J as well and simply put the corresponding log4j dependency as the first one in your POM (or the corresponding build management framework’s file if you don’t use Maven). I’d also suggest to use the log4j’s version which is fully patched against log4Shell vulnerabilities.

4

@partarstu @wcneill this is correct. It depends on what part of the library you’re using though. I think the only place we specify logback is in tests or in the UI. Beyond that it just uses slf4j.

1 Like
5

Thanks. I was under the impression that slf4j was just an abstraction layer, and picks up on a concrete implementation based on jar files found in your project. Is that not true?

I’ll look into logbacks, thanks.

6

You’ve got that right.

You don’t really have to move to logback, you just have to configure your log4j2 so it uses a lower log level for the nd4j namespace.