UI, visualization, LearningSchedule, TrainingListener

Showcase
1

I made a TrainingListener with a JavaFX UI. Like UIServer, it shows score, gradients, activations, weights, and log mean ratios of updates to parameters, per layer.

It has an associated LearningSchedule which you can control from the UI. You can adjust the LearningRate, as well as the decay factor of the learning rate. It’s nice to see the ratios change when you do that.

LearningUI
LearningUI920×651 80.9 KB

If I knew an easy way to adjust L2 or WeightDecay, I’d control them from the UI too.

I use StatsStorage to get the ratios of updates to parameters. Is there an cheaper way to get them? Can I get them directly from the Model?

The score chart uses JavaFX LineChart, so it has built-in functionality and prettiness.

I’ll push a branch if you want.

3 Likes
2

I see that I can add a schedule for WeightDecay. I modified the UI to let me zoom into the line graph.

3

Hi Donald,
I’m trying to change the learning rate while training without using a learning rate schedule.
Apparently you have done that in your UI. Can you please share the way you did this or your github repository?
Thank you in advance.

4

I used a learning rate schedule! The learning rate schedule has a “public static double learningRate;” declaration, and the UI modifies that. The core of the learning rate scheduler class is

public class UILinkedLearningSchedule implements ISchedule, Serializable, Cloneable {
   private static UILinkedLearningSchedule instance = new UILinkedLearningSchedule();
   public static double learningRate = 0.001;
 
   @Override
    public double valueAt(int iteration, int epoch) {
        return learningRate;
    }
}

The UI class is a TrainingListener which modifies the static variable above. Does that explain enough?

1 Like
5

Thank you Donald. This is really helpful.

6

I made a git repository, so I can update bugs: https://github.com/DonaldAlan/javafx_ui_for_deeplearning4j

1 Like
7

Thank you again Donald. Really appriciate your help.

8

https://github.com/DonaldAlan/javafx_ui_for_deeplearning4j is a github repository with the code, a README.md and a better screenshot.

9

I added support for visualizing weights params (via images stacked in 3d). It lets you navigate in 3d, by clicking and dragging, and via the arrow keys. And it lets you ctrl-click and drag an image. See screenshots and code at https://github.com/DonaldAlan/javafx_ui_for_deeplearning4j .