Per-sample weights or label fractions

cacophany53 · July 9, 2020, 2:42pm

Hi there.

Just wondering if per-sample weighting is currently possible? For our project we need to apply fractional weights to each sample.

Alternatively can DL4J handle having a fraction specified for label values for each sample? This example would indicate that it can: deeplearning4j-examples/ImageDrawer.java at master · eclipse/deeplearning4j-examples · GitHub

However, the Constructor for DataSet specifies that labels must be binarized: https://deeplearning4j.org/api/latest/org/nd4j/linalg/dataset/DataSet.html#DataSet-org.nd4j.linalg.api.ndarray.INDArray-org.nd4j.linalg.api.ndarray.INDArray-org.nd4j.linalg.api.ndarray.INDArray-org.nd4j.linalg.api.ndarray.INDArray-

I’m confused as this seems to be contradictory.

Any advice is greatly appreciated.

arthur-dai-618 · October 21, 2021, 8:46am

I’m facing the same problem, did you solve it?

agibsonccc · October 21, 2021, 9:01am

@arthur-dai-618 depending on whether you’re using samediff or dl4j yes both are possible. Can you clarify which API you’re using?

arthur-dai-618 · October 21, 2021, 9:42am

I’m using dl4j, how to do it?

agibsonccc · October 21, 2021, 10:28am

@arthur-dai-618 here’s weighted labels:

github.com

eclipse/deeplearning4j-examples/blob/master/dl4j-examples/src/main/java/org/deeplearning4j/examples/quickstart/features/classimbalance/WeightedLossFunctionExample.java#L73


      
                  MultiLayerConfiguration conf = new NeuralNetConfiguration.Builder()
                      .activation(Activation.RELU)
                      .weightInit(WeightInit.XAVIER)
                      .updater(new Sgd(0.1))
                      .list()
                      .layer(new DenseLayer.Builder().nIn(numInputs).nOut(5)
                          .build())
                      .layer(new DenseLayer.Builder().nIn(5).nOut(5)
                          .build())
                      .layer(new OutputLayer.Builder()
                          .lossFunction(new LossMCXENT(weightsArray))     // *** Weighted loss function configured here ***
                          .activation(Activation.SOFTMAX)
                          .nIn(5).nOut(numClasses).build())
                      .build();
          
          
        //Initialize and use the model as before
                  MultiLayerNetwork model = new MultiLayerNetwork(conf);
                  model.init();
              }
          }

For per example weights, you’d have to implement custom loss:

github.com

eclipse/deeplearning4j-examples/blob/master/dl4j-examples/src/main/java/org/deeplearning4j/examples/advanced/features/customizingdl4j/lossfunctions/CustomLossUsageEx.java

/*******************************************************************************
 *
 *
 *
 * This program and the accompanying materials are made available under the
 * terms of the Apache License, Version 2.0 which is available at
 * https://www.apache.org/licenses/LICENSE-2.0.
 *  See the NOTICE file distributed with this work for additional
 *  information regarding copyright ownership.
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
 * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
 * License for the specific language governing permissions and limitations
 * under the License.
 *
 * SPDX-License-Identifier: Apache-2.0
 ******************************************************************************/

package org.deeplearning4j.examples.advanced.features.customizingdl4j.lossfunctions;

This file has been truncated. show original

You could also look in to different sampling techniques as well.

arthur-dai-618 · October 21, 2021, 12:18pm

That helps me a lot, thank you very much!

Topic		Replies	Views
Weighted learning based on input data / feature DL4J	4	970	April 8, 2020
Problem with Conv1D example DL4J	0	193	December 8, 2022
Basic deeplearning4j classification example DL4J	4	901	February 3, 2020
Advice on proper use of Dataset labels DL4J	4	347	March 19, 2020
Issue in getting 2d DataSet DL4J	3	662	June 15, 2020

Per-sample weights or label fractions

Related Topics