Null pointer when running the UIServer

jfarjona · August 17, 2021, 8:18pm

Hello guys,

I am trying to run the UI Server as explained (deeplearning4j-gitbook-sync/visualization.md at en-1.0.0-beta7 · KonduitAI/deeplearning4j-gitbook-sync · GitHub).

But when I try to open the link I get:

java.lang.NullPointerException: cause
	at io.netty.util.internal.ObjectUtil.checkNotNull(ObjectUtil.java:39) ~[netty-common-4.1.66.Final.jar:4.1.66.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:278) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final]
	at io.netty.channel.DefaultChannelPipeline.fireExceptionCaught(DefaultChannelPipeline.java:907) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final]
	at io.vertx.core.net.impl.VertxHandler.fail(VertxHandler.java:97) ~[vertx-core-3.9.0.jar:3.9.0]
	at io.vertx.core.net.impl.ConnectionBase.fail(ConnectionBase.java:86) ~[vertx-core-3.9.0.jar:3.9.0]
	at io.vertx.core.http.impl.Http1xServerConnection.handleError(Http1xServerConnection.java:480) ~[vertx-core-3.9.0.jar:3.9.0]
	at io.vertx.core.http.impl.Http1xServerConnection.handleMessage(Http1xServerConnection.java:121) ~[vertx-core-3.9.0.jar:3.9.0]
	at io.vertx.core.impl.ContextImpl.executeTask(ContextImpl.java:369) ~[vertx-core-3.9.0.jar:3.9.0]
	at io.vertx.core.impl.EventLoopContext.execute(EventLoopContext.java:43) ~[vertx-core-3.9.0.jar:3.9.0]
	at io.vertx.core.impl.ContextImpl.executeFromIO(ContextImpl.java:232) ~[vertx-core-3.9.0.jar:3.9.0]
	at io.vertx.core.net.impl.VertxHandler.channelRead(VertxHandler.java:173) ~[vertx-core-3.9.0.jar:3.9.0]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final]
	at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final]
	at io.netty.handler.codec.http.websocketx.extensions.WebSocketServerExtensionHandler.channelRead(WebSocketServerExtensionHandler.java:99) ~[netty-codec-http-4.1.66.Final.jar:4.1.66.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final]
	at io.vertx.core.http.impl.Http1xUpgradeToH2CHandler.channelRead(Http1xUpgradeToH2CHandler.java:109) ~[vertx-core-3.9.0.jar:3.9.0]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) ~[netty-codec-4.1.66.Final.jar:4.1.66.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) ~[netty-codec-4.1.66.Final.jar:4.1.66.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final]
	at io.vertx.core.http.impl.Http1xOrH2CHandler.end(Http1xOrH2CHandler.java:61) ~[vertx-core-3.9.0.jar:3.9.0]
	at io.vertx.core.http.impl.Http1xOrH2CHandler.channelRead(Http1xOrH2CHandler.java:38) ~[vertx-core-3.9.0.jar:3.9.0]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:719) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:655) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:581) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final]
	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) ~[netty-common-4.1.66.Final.jar:4.1.66.Final]
	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[netty-common-4.1.66.Final.jar:4.1.66.Final]
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[netty-common-4.1.66.Final.jar:4.1.66.Final]
	at java.base/java.lang.Thread.run(Thread.java:834) ~[na:na]

My dependencies:

<dependencies>
    <dependency>
      <groupId>com.google.guava</groupId>
      <artifactId>guava</artifactId>
      <version>30.1.1-jre</version>
    </dependency>
    <dependency>
      <groupId>nz.ac.waikato.cms.weka</groupId>
      <artifactId>weka-dev</artifactId>
      <version>3.9.5</version>
    </dependency>
    <dependency>
      <groupId>org.apache.commons</groupId>
      <artifactId>commons-math3</artifactId>
      <version>3.6.1</version>
    </dependency>
    <dependency>
      <groupId>org.deeplearning4j</groupId>
      <artifactId>deeplearning4j-core</artifactId>
      <version>1.0.0-M1.1</version>
    </dependency>
    <dependency>
      <groupId>org.deeplearning4j</groupId>
      <artifactId>deeplearning4j-ui</artifactId>
      <version>1.0.0-M1.1</version>
      <exclusions>
        <exclusion>
          <groupId>com.google.guava</groupId>
          <artifactId>guava</artifactId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>org.nd4j</groupId>
      <artifactId>nd4j-api</artifactId>
      <version>1.0.0-M1.1</version>
    </dependency>
    <dependency>
      <groupId>org.nd4j</groupId>
      <artifactId>nd4j-native-platform</artifactId>
      <version>1.0.0-M1.1</version>
    </dependency>
    <dependency>
      <groupId>org.slf4j</groupId>
      <artifactId>log4j-over-slf4j</artifactId>
      <version>1.7.32</version>
    </dependency>
    <dependency>
      <groupId>org.slf4j</groupId>
      <artifactId>slf4j-api</artifactId>
      <version>1.7.32</version>
    </dependency>
    <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-configuration-processor</artifactId>
      <optional>true</optional>
    </dependency>
    <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter</artifactId>
    </dependency>
    <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-test</artifactId>
      <scope>test</scope>
      <exclusions>
        <exclusion>
          <groupId>org.junit.vintage</groupId>
          <artifactId>junit-vintage-engine</artifactId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-web-services</artifactId>
    </dependency>
    <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-webflux</artifactId>
    </dependency>
    <dependency>
      <groupId>org.springframework.security</groupId>
      <artifactId>spring-security-crypto</artifactId>
    </dependency>
    <dependency>
      <groupId>tech.tablesaw</groupId>
      <artifactId>tablesaw-core</artifactId>
      <version>0.38.3</version>
      <exclusions>
        <exclusion>
          <groupId>com.google.guava</groupId>
          <artifactId>guava</artifactId>
        </exclusion>
      </exclusions>
    </dependency>
    <dependency>
      <groupId>tech.tablesaw</groupId>
      <artifactId>tablesaw-excel</artifactId>
      <version>0.38.3</version>
    </dependency>
    <dependency>
      <groupId>tech.tablesaw</groupId>
      <artifactId>tablesaw-jsplot</artifactId>
      <version>0.38.3</version>
    </dependency>

I had to upgrade Guava from release 19.0 (from DL4J dependencies), otherwise I get the exception (from jtablesaw, not related with DL4J):

java.lang.NoClassDefFoundError: null
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[na:na]
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[na:na]
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[na:na]
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490) ~[na:na]
	at java.base/java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:603) ~[na:na]
	at java.base/java.util.concurrent.ForkJoinTask.reportException(ForkJoinTask.java:678) ~[na:na]
	at java.base/java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:737) ~[na:na]
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:159) ~[na:na]
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:173) ~[na:na]
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233) ~[na:na]
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497) ~[na:na]
	at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:661) ~[na:na]
	at com.doblones.ai.services.NetBuilder.build(NetBuilder.java:308) ~[classes/:na]
	at com.doblones.ai.Runner.run(Runner.java:93) ~[classes/:na]
	at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:782) ~[spring-boot-2.5.3.jar:2.5.3]
	at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:772) ~[spring-boot-2.5.3.jar:2.5.3]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:345) ~[spring-boot-2.5.3.jar:2.5.3]
	at com.doblones.ai.Runner.main(Runner.java:67) ~[classes/:na]
Caused by: java.lang.NoClassDefFoundError: com/google/common/collect/Streams$FunctionWithIndex
	at tech.tablesaw.io.FileReader.parseRows(FileReader.java:181) ~[tablesaw-core-0.38.3.jar:na]
	at tech.tablesaw.io.csv.CsvReader.read(CsvReader.java:99) ~[tablesaw-core-0.38.3.jar:na]
	at tech.tablesaw.io.csv.CsvReader.read(CsvReader.java:87) ~[tablesaw-core-0.38.3.jar:na]
	at tech.tablesaw.io.DataFrameReader.csv(DataFrameReader.java:195) ~[tablesaw-core-0.38.3.jar:na]
	at com.doblones.ai.services.Downloader.downloadTickerEOD(Downloader.java:246) ~[classes/:na]
	at com.doblones.ai.services.NetBuilder.lambda$build$1(NetBuilder.java:315) ~[classes/:na]
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) ~[na:na]
	at java.base/java.util.AbstractList$RandomAccessSpliterator.forEachRemaining(AbstractList.java:720) ~[na:na]
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[na:na]
	at java.base/java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:290) ~[na:na]
	at java.base/java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:746) ~[na:na]
	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) ~[na:na]
	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020) ~[na:na]
	at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656) ~[na:na]
	at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594) ~[na:na]
	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183) ~[na:na]
Caused by: java.lang.ClassNotFoundException: com.google.common.collect.Streams$FunctionWithIndex
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581) ~[na:na]
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178) ~[na:na]
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521) ~[na:na]
	... 16 common frames omitted

My code:

public NetRunner() {
    super();
    // Set the tunning UI
    uiServer = UIServer.getInstance();
    statsStorage = new InMemoryStatsStorage();
    uiServer.attach(statsStorage);

    int listenerFrequency = 1;

    listener = new StatsListener(statsStorage, listenerFrequency);
  }

...
MultiLayerNetwork net = new MultiLayerNetwork(conf);

net.setListeners(listener);
net.init();

...

Thanks,

Juan

agibsonccc · August 18, 2021, 1:32pm

@jfarjona the root cause is right there. It’s saying it can’t a guava related class. You’ll need to ensure via mvn dependency:tree that all versions are the same for guava.

jfarjona · August 18, 2021, 4:18pm

Thanks @agibsonccc , the second Exception is not related with DL4J, I put it there because I have to override the dependencies from DL4J to use a higher version of Guava.

My question refers to the first exception (Nullpointer) which is coming from netty/vertx; Guava doesn’t show in the stack trace. It seems to be an Exception raised by Vertx on line

io.vertx.core.impl.ContextImpl.executeTask(ContextImpl.java:369)

My 3 cents: I think it may be a problem on the websocket handler, which is throwing a Nullpointer exception. The code in Vertx, is:

try {
      setContext(current, this);
      hTask.handle(arg);
      return true;
    } catch (Throwable t) {
      reportException(t);
      return false;
    } finally {

Which means that hTask is null, or somewhere inside the execution of hTask a Nullpointer Exception takes place. I would assume that hTask is responding to websocket requests, see the trace:

at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final]
	at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final]
-->	at io.netty.handler.codec.http.websocketx.extensions.WebSocketServerExtensionHandler.channelRead(WebSocketServerExtensionHandler.java:99) ~[netty-codec-http-4.1.66.Final.jar:4.1.66.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final]

My guess is that somewhere (I don’t know the DL4J UIServer inner workings) in the websocket task implementation a null pointer is happening, or, hTask is null (perhaps failed to set the task at initialization?). Or perhaps I am missing a class in my dependencies that initializes the websocket task and sets it to null because it doesn’t find the class… I don’t see any ClassNotFoundException in the logs or stdout.

Thanks.

Juan

agibsonccc · August 18, 2021, 9:50pm

@jfarjona thanks for the detailed explanation! Do you mind filing an issue so we can track this? I still think the possible cause could be dependency related.

For now, one suggestion might be to just run the UI in a self contained uber jar in a separate process using the maven shade plugin. That would be better than trying to dependency manage everything.

jfarjona · August 27, 2021, 10:13pm

Thanks @agibsonccc , I have created the issue entry. I did just what you said and the issue persists. I am unable to use the UI Server.

github.com/eclipse/deeplearning4j

Receiving a NullPointerException when accessing the UI Server

opened 11:21PM - 18 Aug 21 UTC

jfarjona

#### Issue Description Getting a Nullpointer Exception from netty/vertx, when… trying to access the running UI Server: `` java.lang.NullPointerException: cause at io.netty.util.internal.ObjectUtil.checkNotNull(ObjectUtil.java:39) ~[netty-common-4.1.66.Final.jar:4.1.66.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:278) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final] at io.netty.channel.DefaultChannelPipeline.fireExceptionCaught(DefaultChannelPipeline.java:907) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final] at io.vertx.core.net.impl.VertxHandler.fail(VertxHandler.java:97) ~[vertx-core-3.9.0.jar:3.9.0] at io.vertx.core.net.impl.ConnectionBase.fail(ConnectionBase.java:86) ~[vertx-core-3.9.0.jar:3.9.0] at io.vertx.core.http.impl.Http1xServerConnection.handleError(Http1xServerConnection.java:480) ~[vertx-core-3.9.0.jar:3.9.0] at io.vertx.core.http.impl.Http1xServerConnection.handleMessage(Http1xServerConnection.java:121) ~[vertx-core-3.9.0.jar:3.9.0] at io.vertx.core.impl.ContextImpl.executeTask(ContextImpl.java:369) ~[vertx-core-3.9.0.jar:3.9.0] at io.vertx.core.impl.EventLoopContext.execute(EventLoopContext.java:43) ~[vertx-core-3.9.0.jar:3.9.0] at io.vertx.core.impl.ContextImpl.executeFromIO(ContextImpl.java:232) ~[vertx-core-3.9.0.jar:3.9.0] at io.vertx.core.net.impl.VertxHandler.channelRead(VertxHandler.java:173) ~[vertx-core-3.9.0.jar:3.9.0] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final] at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final] at io.netty.handler.codec.http.websocketx.extensions.WebSocketServerExtensionHandler.channelRead(WebSocketServerExtensionHandler.java:99) ~[netty-codec-http-4.1.66.Final.jar:4.1.66.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final] at io.vertx.core.http.impl.Http1xUpgradeToH2CHandler.channelRead(Http1xUpgradeToH2CHandler.java:109) ~[vertx-core-3.9.0.jar:3.9.0] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final] at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) ~[netty-codec-4.1.66.Final.jar:4.1.66.Final] at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) ~[netty-codec-4.1.66.Final.jar:4.1.66.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final] at io.vertx.core.http.impl.Http1xOrH2CHandler.end(Http1xOrH2CHandler.java:61) ~[vertx-core-3.9.0.jar:3.9.0] at io.vertx.core.http.impl.Http1xOrH2CHandler.channelRead(Http1xOrH2CHandler.java:38) ~[vertx-core-3.9.0.jar:3.9.0] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final] at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final] at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final] at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:719) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:655) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final] at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:581) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final] at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final] at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) ~[netty-common-4.1.66.Final.jar:4.1.66.Final] at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[netty-common-4.1.66.Final.jar:4.1.66.Final] at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[netty-common-4.1.66.Final.jar:4.1.66.Final] at java.base/java.lang.Thread.run(Thread.java:834) ~[na:na] `` It seems to be an Exception raised by Vertx on line ``` io.vertx.core.impl.ContextImpl.executeTask(ContextImpl.java:369) ``` I think it may be a problem on the websocket handler, which is throwing a Nullpointer exception. The code in Vertx, is: ``` try { setContext(current, this); hTask.handle(arg); return true; } catch (Throwable t) { reportException(t); return false; } finally { ``` Which means that hTask is null, or somewhere inside the execution of hTask a Nullpointer Exception takes place. I would assume that hTask is responding to websocket requests, see the trace: ``` at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final] at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final] --> at io.netty.handler.codec.http.websocketx.extensions.WebSocketServerExtensionHandler.channelRead(WebSocketServerExtensionHandler.java:99) ~[netty-codec-http-4.1.66.Final.jar:4.1.66.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.66.Final.jar:4.1.66.Final] ``` My guess is that somewhere (I don't know the DL4J UIServer inner workings) in the websocket task implementation a null pointer is happening, or, hTask is null (perhaps failed to set the task at initialization?). Or perhaps I am missing a class in my dependencies that initializes the websocket task and sets it to null because it doesn't find the class... I don't see any ClassNotFoundException in the logs or stdout. My code: public NetRunner() { super(); // Set the tunning UI uiServer = UIServer.getInstance(); statsStorage = new InMemoryStatsStorage(); uiServer.attach(statsStorage); int listenerFrequency = 1; listener = new StatsListener(statsStorage, listenerFrequency); } ... MultiLayerNetwork net = new MultiLayerNetwork(conf); net.setListeners(listener); net.init(); ... My dependencies: `` <dependencies> <dependency> <groupId>com.google.guava</groupId> <artifactId>guava</artifactId> <version>30.1.1-jre</version> </dependency> <dependency> <groupId>nz.ac.waikato.cms.weka</groupId> <artifactId>weka-dev</artifactId> <version>3.9.5</version> </dependency> <dependency> <groupId>org.apache.commons</groupId> <artifactId>commons-math3</artifactId> <version>3.6.1</version> </dependency> <dependency> <groupId>org.deeplearning4j</groupId> <artifactId>deeplearning4j-core</artifactId> <version>1.0.0-M1.1</version> </dependency> <dependency> <groupId>org.deeplearning4j</groupId> <artifactId>deeplearning4j-ui</artifactId> <version>1.0.0-M1.1</version> <exclusions> <exclusion> <groupId>com.google.guava</groupId> <artifactId>guava</artifactId> </exclusion> </exclusions> </dependency> <dependency> <groupId>org.nd4j</groupId> <artifactId>nd4j-api</artifactId> <version>1.0.0-M1.1</version> </dependency> <dependency> <groupId>org.nd4j</groupId> <artifactId>nd4j-native-platform</artifactId> <version>1.0.0-M1.1</version> </dependency> <dependency> <groupId>org.slf4j</groupId> <artifactId>log4j-over-slf4j</artifactId> <version>1.7.32</version> </dependency> <dependency> <groupId>org.slf4j</groupId> <artifactId>slf4j-api</artifactId> <version>1.7.32</version> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-configuration-processor</artifactId> <optional>true</optional> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-test</artifactId> <scope>test</scope> <exclusions> <exclusion> <groupId>org.junit.vintage</groupId> <artifactId>junit-vintage-engine</artifactId> </exclusion> </exclusions> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web-services</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-webflux</artifactId> </dependency> <dependency> <groupId>org.springframework.security</groupId> <artifactId>spring-security-crypto</artifactId> </dependency> <dependency> <groupId>tech.tablesaw</groupId> <artifactId>tablesaw-core</artifactId> <version>0.38.3</version> <exclusions> <exclusion> <groupId>com.google.guava</groupId> <artifactId>guava</artifactId> </exclusion> </exclusions> </dependency> <dependency> <groupId>tech.tablesaw</groupId> <artifactId>tablesaw-excel</artifactId> <version>0.38.3</version> </dependency> <dependency> <groupId>tech.tablesaw</groupId> <artifactId>tablesaw-jsplot</artifactId> <version>0.38.3</version> </dependency> </dependencies> `` More info: see https://community.konduit.ai/t/null-pointer-when-running-the-uiserver/1570

agibsonccc · August 27, 2021, 10:15pm

@jfarjona could you clarify a bit? A self contained uber jar with just the dependencies won’t have any sort of a clash like that. Standalone should show you how to do that:

github.com

eclipse/deeplearning4j/blob/master/deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-ui-standalone/pom.xml

<?xml version="1.0" encoding="UTF-8"?>
<!--
  ~ /* ******************************************************************************
  ~  *
  ~  *
  ~  * This program and the accompanying materials are made available under the
  ~  * terms of the Apache License, Version 2.0 which is available at
  ~  * https://www.apache.org/licenses/LICENSE-2.0.
  ~  *
  ~  *  See the NOTICE file distributed with this work for additional
  ~  *  information regarding copyright ownership.
  ~  * Unless required by applicable law or agreed to in writing, software
  ~  * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
  ~  * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
  ~  * License for the specific language governing permissions and limitations
  ~  * under the License.
  ~  *
  ~  * SPDX-License-Identifier: Apache-2.0
  ~  ******************************************************************************/
  -->

This file has been truncated. show original

Edit: to be clear you shouldn’t declare a dependency in your project. It should literally be a separate process you connect to remotely.

jfarjona · August 28, 2021, 5:18pm

Thanks, @agibsonccc, I am a bit lost. If I understand you correctly, I should run the UI Server in one process, and my program that builds and trains the model in another process?
If that is the case, then how do I tell the StatsListener to connect and report data to the other process? (i.e., is there a configuration to tell the port/IP?)

Sorry for my ignorance, I am very new to DL4J.

agibsonccc · August 29, 2021, 6:12am

@jfarjona that is what the remote stats listener is for.
Here’s the example for it:

github.com

eclipse/deeplearning4j-examples/blob/master/dl4j-examples/src/main/java/org/deeplearning4j/examples/quickstart/features/userinterface/RemoteUIExample.java

/*******************************************************************************
 *
 *
 *
 * This program and the accompanying materials are made available under the
 * terms of the Apache License, Version 2.0 which is available at
 * https://www.apache.org/licenses/LICENSE-2.0.
 *  See the NOTICE file distributed with this work for additional
 *  information regarding copyright ownership.
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
 * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
 * License for the specific language governing permissions and limitations
 * under the License.
 *
 * SPDX-License-Identifier: Apache-2.0
 ******************************************************************************/

package org.deeplearning4j.examples.quickstart.features.userinterface;

This file has been truncated. show original

jfarjona · August 29, 2021, 4:52pm

Thanks, will try this.

Topic		Replies	Views
UIServer not starting up (I think) DL4J	4	1824	June 8, 2020
Can't visualize training process DL4J	24	1486	March 2, 2021
Problems using the UI in beta6 DL4J	4	957	February 17, 2020
Cannot access Webserver DL4J	12	769	September 29, 2020
"memoryManagerClazz" is null SameDiff	3	264	September 30, 2022

Null pointer when running the UIServer

Related topics