# Weighted learning based on input data / feature

**URL:** <https://community.konduit.ai/t/weighted-learning-based-on-input-data-feature/400>\
**Category:** DL4J\
**Created:** [April 8, 2020, 5:40pm UTC](https://community.konduit.ai/t/weighted-learning-based-on-input-data-feature/400 "2020-04-08T17:40:25Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![JayWoild](https://avatars.discourse-cdn.com/v4/letter/j/dc4da7/32.png) [@JayWoild](https://community.konduit.ai/u/JayWoild)\
**Post date:** [April 8, 2020, 5:40pm UTC](https://community.konduit.ai/t/weighted-learning-based-on-input-data-feature/400/1 "2020-04-08T17:40:25Z")

</div>

Hello,

I’m looking to modify the training process by giving weight to an input data set based on a feature, i.e. making all female inputs 1.7 times as important.  
So far I’ve come across the weighted loss function, which doesn’t help me since it determines weight based on the class/label rather than the feature.  
A custom loss function has the same fault, as I can only access the labels and preOutput Arrays (as far as I can see).

Is there a way to weigh input, other than by transforming the input data (i.e. by multiplying the female inputs)?

Thanks in advance.

---

<div class="post-metadata">

**Author:** ![treo](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/treo/32/47_2.png) [@treo](https://community.konduit.ai/u/treo)\
**Post date:** [April 8, 2020, 6:23pm UTC](https://community.konduit.ai/t/weighted-learning-based-on-input-data-feature/400/2 "2020-04-08T18:23:32Z")

</div>

I’m not quite sure I can follow what you are trying to do. Making a feature more important isn’t something that you can properly do with any kind of machine learning, as all of them pretty much by definition try to use all the features you give them in order to match to your data the best they can.

If you just want to make sure that examples with a specific feature have a better (training) accuracy, then you can make them appear more often during the training - debalancing your training set effectively, if it was balanced beforehand.

What you are asking for has been asked for before: [Per-example weights · Issue #2616 · eclipse/deeplearning4j · GitHub](https://github.com/eclipse/deeplearning4j/issues/2616)

But we currently don’t have any specific ways to do it other than manually.

---

<div class="post-metadata">

**Author:** ![JayWoild](https://avatars.discourse-cdn.com/v4/letter/j/dc4da7/32.png) [@JayWoild](https://community.konduit.ai/u/JayWoild)\
**Post date:** [April 8, 2020, 6:36pm UTC](https://community.konduit.ai/t/weighted-learning-based-on-input-data-feature/400/3 "2020-04-08T18:36:48Z")

</div>

Sorry for the lack of clarity, I was looking for the second option - giving more weight to specific examples, without modifying the dataset.

By “doing it manually” you mean debalancing the training set, correct?

Thank you for your speedy reply.

---

<div class="post-metadata">

**Author:** ![treo](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/treo/32/47_2.png) [@treo](https://community.konduit.ai/u/treo)\
**Post date:** [April 8, 2020, 6:51pm UTC](https://community.konduit.ai/t/weighted-learning-based-on-input-data-feature/400/4 "2020-04-08T18:51:06Z")

</div>

> [@JayWoild](#):
>
> By “doing it manually” you mean debalancing the training set, correct?

Yes.

You have multiple options of how you can attack that:

- You can just copy the data multiple times in your input data
- You can wrap the record reader that reads your data and just emit your examples multiple times
- You can wrap the DataSetIterator and create unbalanced DataSets based on the incoming DataSet

The first two options are likely to be easier to implement correctly.

But you’ve also got to remember that you might run into overfitting problems if you oversample data like that.

If you’ve got enough data, you can also try it the other way around: Discard examples that you’d like to weaken, thereby making the examples to want to strengthen appear more often relatively.

---

<div class="post-metadata">

**Author:** ![JayWoild](https://avatars.discourse-cdn.com/v4/letter/j/dc4da7/32.png) [@JayWoild](https://community.konduit.ai/u/JayWoild)\
**Post date:** [April 8, 2020, 6:58pm UTC](https://community.konduit.ai/t/weighted-learning-based-on-input-data-feature/400/5 "2020-04-08T18:58:11Z")

</div>

Thanks, I hadn’t thought of manipulating the record reader. I’ll give that a try.