Hi all,
I’m looking to make an implementation on EMNIST letters. My actual model predict good answer 90%.
I see this paper :
In paragraph 4.4 i try to implement this architecture but doesn’t work (for 1 epoch) :
My implementation :
MultiLayerConfiguration conf = new NeuralNetConfiguration.Builder().seed(seed).updater(new RmsProp())
.weightInit(WeightInit.XAVIER).list()
// premier bloc
.layer(new ConvolutionLayer.Builder(3, 3).nIn(channels).nOut(473).build())
.layer(new SubsamplingLayer.Builder(SubsamplingLayer.PoolingType.AVG).kernelSize(2, 2).build())
.layer(new BatchNormalization.Builder().activation(Activation.RELU).dropOut(0.15).build())
// Deuxième bloc
.layer(new ConvolutionLayer.Builder(3, 3).nOut(238).build())
.layer(new BatchNormalization.Builder().activation(Activation.LEAKYRELU).dropOut(0.20).build())
// Troisième bloc
.layer(new ConvolutionLayer.Builder(3, 3).nOut(133).build())
.layer(new BatchNormalization.Builder().activation(Activation.RELU).dropOut(0.10).build())
// quatrième bloc
.layer(new ConvolutionLayer.Builder(3, 3).nOut(387).build())
.layer(new BatchNormalization.Builder().activation(Activation.THRESHOLDEDRELU).dropOut(0.10).build())
// cinquième bloc
.layer(new ConvolutionLayer.Builder(5, 5).nOut(187).build())
.layer(new BatchNormalization.Builder().activation(Activation.ELU).dropOut(0.50).build())
// Première couche fully
.layer(new DenseLayer.Builder().activation(Activation.RELU).nOut(313).build())
.layer(new BatchNormalization.Builder().dropOut(0.20).build())
// Deuxième couche fully
// .layer(new DenseLayer.Builder().activation(Activation.ELU).nOut(252).build())
.layer(new DenseLayer.Builder().activation(Activation.SIGMOID).nOut(252).build())
.layer(new BatchNormalization.Builder().dropOut(0.20).build())
// couche d'activation
.layer(new OutputLayer.Builder().nOut(outputNum).lossFunction(new LossNegativeLogLikelihood(weights))
.activation(Activation.SOFTMAX).build())
.setInputType(InputType.convolutionalFlat(height, width, channels)).build();
I put LossNegativeLogLikelihood because in need to get weigths on my unbalanced data.
Some ideas ?
