Regression to find the center of a rectangle/oval

Tuning Help
1

Hi,

i am trying to predict the center of a black rectangle/oval in an image.
My training data contains 10000 randomly generated 100x100 images with either a rectangle or oval with random size somewhere on the image. 17

The problem is that it seems like the network has some troubles to find the optimal solution and the score spikes up sometimes during training (learning rate to high?).

My Code:

public static void main(String[] args) throws Exception {
        int height = 100;    // height of the picture in px
        int width = 100;     // width of the picture in px
        int channels = 1;   // single channel for grayscale images
        int outputNum = 2; 
        int batchSize = 54; // number of samples that will be propagated through the network in each iteration
        int nEpochs = 1;    // number of training epochs

        int seed = 1234;    // number used to initialize a pseudorandom number generator.
        Random randNumGen = new Random(seed);


        File trainData = new File("D:\\Deeplearning4j\\squares\\pics");
        FileSplit trainSplit = new FileSplit(trainData, NativeImageLoader.ALLOWED_FORMATS, randNumGen);
        MySquareMultiLabelGen labelMaker = new MySquareMultiLabelGen(); // use parent directory name as the image label
        ImageRecordReader trainRR = new ImageRecordReader(height, width, channels, labelMaker);
        trainRR.initialize(trainSplit);
        DataSetIterator trainIter = new RecordReaderDataSetIterator.Builder(trainRR, batchSize)
            .regression(1,2)
            .build();

        DataNormalization imageScaler = new ImagePreProcessingScaler();
        imageScaler.fit(trainIter);
        trainIter.setPreProcessor(imageScaler);

        MultiLayerConfiguration conf = new NeuralNetConfiguration.Builder()
            .seed(seed)
            .l2(0.0005) // ridge regression value
            .updater(new AdaDelta())
            .weightInit(WeightInit.XAVIER)
            .list()
            .layer(new ConvolutionLayer.Builder(5, 5)
                .nIn(channels)
                .stride(1, 1)
                .nOut(10)
                .activation(Activation.IDENTITY)
                .build())
            .layer(new DenseLayer.Builder().activation(Activation.RELU)
                .nOut(200)
                .build())
            .layer(new DenseLayer.Builder().activation(Activation.RELU)
                .nOut(100)
                .build())
            .layer(new OutputLayer.Builder(LossFunctions.LossFunction.MSE)
                .nOut(outputNum)
                .activation(Activation.IDENTITY)
                .build())
            .setInputType(InputType.convolutional(height, width, channels)) // InputType.convolutional for normal image
            .build();

        MultiLayerNetwork net = new MultiLayerNetwork(conf);
        net.init();
        net.setListeners(new ScoreIterationListener(10));

        System.out.println("Total num of params: " + net.numParams());

        net.fit(trainIter, 4);
    }

What i tried so far:

  • (not)normalizing the labels
  • (not)normalizing the inputs
  • decrease/increase complexity of the neural network
  • different updaters
  • more/less epochs for training
  • increase/decrease batchsize

Edit: make code readable

2

Your labels should be between 0 and 1, that way you will get a result relative to your input size.

The network architecture however looks very suspicious. Because you have only a single convolution of 5x5 pixels, it can at best find: edges, background and foreground. Take a look at the model zoo how the more complex models are built, and maybe read up on how CNNs are usually structured (esp. the progression of layers)

1 Like