UIServer in multi use mode

DL4J
1

Hi, I am trying to get the UIServer in multi use mode working. I can create it, create multiple InMemoryStatsStorage objects and pass them via the function argument. The problem seems to be that I cannot store the sessionIds within each InMemoryStatsStorage object? The server comes up with “unknown sessionID”. How do you store the sessionId within each InMemoryStatsStorage object?

Many thanks.

2

Have you tried running an example project with a UIServer ? Does it work?

3

Yes, I can get the UIServer to work no problem in single use mode.

4

Hi @Cpt_Whittmann,
Currently, we don’t have example code for multi-session mode UIServer, so you would have to figure out the usage from the unit test cases. This feature is my contribution to the project, so I will give you this tutorial on how to use DL4J UIServer in multi-session mode.

In single-session mode, you can use one StatsStorage to store multiple training sessions. You may want to set a meaningful session ID for each training so that you can use the session selector drop-down that lists training sessions in the StatsStorage.

However, if you want to display only one session at a time, you can use multi-session mode.
In multi-session mode, multiple independent training and/or optimization sessions can be displayed, based on the session ID.
Note: This working mode does not provide user separation, e.g. the loaded sessions can be viewed from different browser sessions, but changing display language in one browser will affect the setting in all browsers visiting that session.

With the following steps, you can set up a multi-session UIServer:

  1. Set session ID of listener: By default, session ID is a random string (UUID), but you can to set your own ID on the listener (StatsListener or ArbiterStatusListener), before training/optimization.
  2. Start training/optimization: The listener will then store the following events (training/optimization) with that session ID.
  3. Start UIServer with a function that maps StatStorage to session ID (Function<String, StatsStorage> statsStorageProvider).
  4. Visit the session URL to auto-attach the stats storage of the session ID.
    • training session: http://localhost:9000/train/sessionID
    • optimization session: http://localhost:9000/arbiter/sessionID

During auto-attach, UIServer checks if the provided StatsStorage (which can contain multiple sessions) actually contain the given session ID. However, the session ID will be written to the storage only after the first stats storage event, so you have to start training/optimization before trying to attach the stats storage.

You will see a list of attached session ID-s
- training sessions: http://localhost:9000/train/
- optimization sessions: http://localhost:9000/arbiter/

See this training test case with StatsListener:

https://github.com/eclipse/deeplearning4j/blob/master/deeplearning4j/deeplearning4j-ui-parent/deeplearning4j-vertx/src/test/java/org/deeplearning4j/ui/TestVertxUIMultiSession.java#L158

For optimization, session ID can be set in ArbiterStatusListener constructor and it cannot be changed later. See this optimization test case:

Feel free to ask if something is not clear.

5

Many thanks for this. I was looking as to how the sessionId was stored with relation to the StatsStorage Object and this is done with a StatsListener object. I will test this out.