Any idea why this hubert loss function would fail? Same network is working correctly for MSE loss function.
public class HuberLoss extends SameDiffLoss{
@Override
public SDVariable defineLoss(SameDiff sd, SDVariable yPred, SDVariable yTrue) {
return sd.loss.huberLoss(yTrue, yPred, null, 1.0);
}
}
NDArray::applyScalarArr BoolOps: this dtype: [5]; scalar dtype: [6]
Exception in thread "main" 14:24:54.952 [main] ERROR o.n.l.c.n.ops.NativeOpExecutioner - Failed to execute op huber_loss_grad. Attempted to execute with 3 inputs, 3 outputs, 1 targs,0 bargs and 1 iargs. Inputs: [(FLOAT,[1,3],c), (FLOAT,[],c), (FLOAT,[1,3],c)]. Outputs: [(FLOAT,[1,3],c), (FLOAT,[],c), (FLOAT,[1,3],c)]. tArgs: [1.0]. iArgs: [3]. bArgs: -. Input var names: [layerInput, sd_var, labels]. Output var names: [layerInput-grad, sd_var-grad, labels-grad] - Please see above message (printed out from c++) for a possible cause of error.
java.lang.RuntimeException: NDArray::applyScalarArr bool method: this and scalar arrays must have the same type!
at org.nd4j.linalg.cpu.nativecpu.ops.NativeOpExecutioner.exec(NativeOpExecutioner.java:1918)
at org.nd4j.linalg.factory.Nd4j.exec(Nd4j.java:6575)
at org.nd4j.autodiff.samediff.internal.InferenceSession.doExec(InferenceSession.java:487)
at org.nd4j.autodiff.samediff.internal.InferenceSession.getOutputs(InferenceSession.java:214)
at org.nd4j.autodiff.samediff.internal.InferenceSession.getOutputs(InferenceSession.java:60)
at org.nd4j.autodiff.samediff.internal.AbstractSession.output(AbstractSession.java:386)
at org.nd4j.autodiff.samediff.SameDiff.directExecHelper(SameDiff.java:2579)
at org.nd4j.autodiff.samediff.SameDiff.batchOutputHelper(SameDiff.java:2547)
@DeepLearner first question to look in to…this is a data type error. We always throw an exception when the 2 aren’t the same data type. Could you double check the data types of all the variables being passed to the operation?
yPred is dType float, yTrue is dType float. A simple example to demonstrate the error is below. This was tested on 1.0.0-M1
public class Example {
public static final int INPUT_SIZE=2;
public static final int OUTPUT_SIZE=2;
public static void main (String av[]) {
MultiLayerConfiguration conf =new NeuralNetConfiguration.Builder()
.seed(12345)
.optimizationAlgo(OptimizationAlgorithm.STOCHASTIC_GRADIENT_DESCENT)
.list()
.layer(0, new DenseLayer.Builder().nIn(INPUT_SIZE).nOut(OUTPUT_SIZE).weightInit(WeightInit.RELU).activation(Activation.RELU)
.build())
.layer(1, new OutputLayer.Builder(new HuberLoss()).nIn(OUTPUT_SIZE).nOut(OUTPUT_SIZE).weightInit(WeightInit.IDENTITY).activation(Activation.IDENTITY)
.build())
.build();
INDArray input = Nd4j.create(new float[] {-1f,1f}, new int[] {1,2});
INDArray expected = Nd4j.create(new float[] {-0.5f,0.5f}, new int[] {1,2});
MultiLayerNetwork network = new MultiLayerNetwork(conf);
network.init();
network.fit(input, expected);
}
}
@DeepLearner so I just tried this… the HuberLoss() actually doesn’t extend ILossFunction that the OutputLayer is expecting, normally you would have to wrap that with samediff loss or something.
Could you actually give me something that runs out of the box and compiles? Thanks!
public class Example {
public static final int INPUT_SIZE=2;
public static final int OUTPUT_SIZE=2;
public static void main (String av[]) {
MultiLayerConfiguration conf =new NeuralNetConfiguration.Builder()
.seed(12345)
.optimizationAlgo(OptimizationAlgorithm.STOCHASTIC_GRADIENT_DESCENT)
.list()
.layer(0, new DenseLayer.Builder().nIn(INPUT_SIZE).nOut(OUTPUT_SIZE).weightInit(WeightInit.RELU).activation(Activation.RELU)
.build())
.layer(1, new OutputLayer.Builder(new HuberLoss()).nIn(OUTPUT_SIZE).nOut(OUTPUT_SIZE).weightInit(WeightInit.IDENTITY).activation(Activation.IDENTITY)
.build())
.build();
INDArray input = Nd4j.create(new float[] {-1f,1f}, new int[] {1,2});
INDArray expected = Nd4j.create(new float[] {-0.5f,0.5f}, new int[] {1,2});
MultiLayerNetwork network = new MultiLayerNetwork(conf);
network.init();
network.fit(input, expected);
}
}
class HuberLoss extends SameDiffLoss{
@Override
public SDVariable defineLoss(SameDiff sd, SDVariable yPred, SDVariable yTrue) {
return sd.loss.huberLoss(yTrue, yPred, null, 1.0);
}
}