Training UI from a HPC to local macosx-arm64 machine

DL4J
1

I’m executing my code on a linux-based HPC, but I’ll need training result to pass back as a UI page to my local computer through http. I have the code on my HPC

StatsStorageRouter remoteUIRouter = new RemoteUIStatsStorageRouter("http://192.168.1.214:9000"); - my IP
model.setListeners(new StatsListener(remoteUIRouter)

and the code on my local macosx-arm64 machine.

public class RemoteUI {
public static void main(String[] args) throws Exception{
UIServer uiServer = UIServer.getInstance();
uiServer.enableRemoteListener();
Thread.sleep(99999);}}

I ran the Remote UI main class on my local first, and then run the training code containing remote UI router, but the error arise on the HPC machine, which says
[Thread-2] WARN org.deeplearning4j.core.storage.impl.RemoteUIStatsStorageRouter - Error posting to remote UI at http://192.168.1.214:9000/remoteReceive, consecutive failure count = 1. Waiting 1000 ms before retrying java.net.NoRouteToHostException: No route to host at java.base/sun.nio.ch.Net.connect0(Native Method) at java.base/sun.nio.ch.Net.connect(Net.java:589) at java.base/sun.nio.ch.Net.connect(Net.java:578) at java.base/sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:583) at java.base/java.net.Socket.connect(Socket.java:752) at java.base/java.net.Socket.connect(Socket.java:687) at java.base/sun.net.NetworkClient.doConnect(NetworkClient.java:183) at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:531) at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:636) at java.base/sun.net.www.http.HttpClient.<init>(HttpClient.java:280) at java.base/sun.net.www.http.HttpClient.New(HttpClient.java:386) at java.base/sun.net.www.http.HttpClient.New(HttpClient.java:408) at java.base/sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1310) at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1243) at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1129) at java.base/sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:1058) at java.base/sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1460) at java.base/sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1423) at org.deeplearning4j.core.storage.impl.RemoteUIStatsStorageRouter.tryPost(RemoteUIStatsStorageRouter.java:352) at org.deeplearning4j.core.storage.impl.RemoteUIStatsStorageRouter$PostRunnable.runHelper(RemoteUIStatsStorageRouter.java:267) at org.deeplearning4j.core.storage.impl.RemoteUIStatsStorageRouter$PostRunnable.run(RemoteUIStatsStorageRouter.java:241) at java.base/java.lang.Thread.run(Thread.java:1570)

As for my Remote UI class in local computer, the code is successfully run and the UI instance is started at http://localhost:9000/. I can open the page through http://localhost:9000/train/overview, but what is lacking is the training result passing from the HPC (the page has the model score section but no data presented on graph).

I’ve hence checked my mac’s firewall setting and it says “This computer’s firewall is currently turned off. All incoming connections to this computer is allowed.” Can you advise on how to resolve such a matter? I’m glad to provide more details if needed.

2

@timmy1010697 this is almost certainly an issue with your configuration. Ensure you have the right IP/configuration.

3

I just try to implement the BaseUI example within the intellij remote environment(gateway), and the training ui can successfully open on my local machine. However, when I compile the code with its dependency into a .jar and submit it through slurm through a shell script with the line java -jar target/Project-1.0-SNAPSHOT-jar-with-dependencies.jar, the training UI can’t be opened. What can be the cause of this and are there any suggestion to fix it?

4

@timmy1010697 can you clarify what you mean? There should be some run logs or something. It’s hard to help you without knowing more information unfortunately. All issues have some logs somewhere.