Tuning Image Classifier

Hi everyone,

From the help of @treo and various others in this community, I’ve been able to get my CNN up and running very quickly! I was just wondering if there’s anything I can change to increase the accuracy of the model. I’m currently running a setup on the following data:

  • 180 classes/labels/birds
  • ~25,000 training images (224x224, 3 channels)
  • 900 testing images
  • 900 validation images

I’m using a CNN with the following characteristics:

7 layers (incl. output layer):

  • Convulutional (1,1 strides, weightInit Xavier, activation Relu)
  • Subsampling layer (2,2 strides, pooling type MAX)
  • Convulutional (1,1 strides, weightInit Xavier, activation Relu)
  • Subsampling layer (2,2 strides, pooling type MAX)
  • Convulutional (1,1 strides, weightInit Xavier, activation Relu)
  • Subsampling layer (2,2 strides, pooling type MAX)
  • Output layer (LossFN negative-log-likelihood, activation softmax, weightinit xavier)

Misc parameters:

  • Optimization using stochastic gradient descent
  • l2 of 0.004
  • Learning rate of 0.001
  • Momentum of 0.9
  • Early Stopping Trainer terminating after 50 epochs or 60 mins, evaluating on validation set every 2 epochs

With this setup, I’m currently getting an average of approximately 45-55% accuracy across the testing set after producing the model from the earlyStoppingTrainer. It might also be important to note that I’m running this network using CUDA 10.2 with a GTX 1080.

The full source code is uploaded to Github here.

Thanks for your help!

The dataset you are working with is significantly more complex than MNIST, so I’d start with an architecture that is more suited for the task.

Take a look at the LeNet based model in the Animal Classification example, or the architecture from the Cifar Example.

1 Like

I’ve implemented an architecture similar to the one from the CIFAR animal classification example with much better results now, thank you for your help!

There’s a resnet model in the model zoo. Am I right to say we can use that pretrained model and use transfer learning to get better scores? Unfortunately there isn’t an example that I know of.

@phong-phuong
https://deeplearning4j.konduit.ai/tuning-and-training/transfer-learning

This shows how to use transfer learning.

So yes, you can use a model from the model zoo and apply transfer learning to it to make it fit your actual task better.

1 Like

Hi @camhammel,

I recommend using DenseNet example for image classification :

I had very good results with that arhitecture. Also please check my latest PR on that example:

In fixed some mistakes that I made initially when created that example.

Also i think that for your dataset this model might be overwhelming, so I recommend to start with only 2 or 3 dense blocks (In example it has 4 dense bloks), and with growRate like 16…

If you want to read more about DenseNet please check this:

Have a nice day