# Incremental training **URL:** \ **Category:** DL4J\ **Created:** [February 27, 2020, 6:37pm UTC](https://community.konduit.ai/t/incremental-training/189 "2020-02-27T18:37:12Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![poorni19](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/poorni19/32/67_2.png) [@poorni19](https://community.konduit.ai/u/poorni19)\ **Post date:** [February 27, 2020, 6:37pm UTC](https://community.konduit.ai/t/incremental-training/189/1 "2020-02-27T18:37:12Z")
Does Dl4j supports incremental training ? ---
**Author:** ![treo](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/treo/32/47_2.png) [@treo](https://community.konduit.ai/u/treo)\ **Post date:** [February 27, 2020, 6:39pm UTC](https://community.konduit.ai/t/incremental-training/189/2 "2020-02-27T18:39:31Z")
It depends on how you mean incremental training. In principle all SGD based training is incremental, as you train your model with different batches. You can always reload your model and continue training. ---
**Author:** ![poorni19](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/poorni19/32/67_2.png) [@poorni19](https://community.konduit.ai/u/poorni19)\ **Post date:** [February 27, 2020, 6:43pm UTC](https://community.konduit.ai/t/incremental-training/189/3 "2020-02-27T18:43:19Z")
Yes , exactly load a trained model and add new training data and train on top of it . Wondering if the model may encounter “catastrophic forgetting” and forget what it’s being learned previously. ---
**Author:** ![treo](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/treo/32/47_2.png) [@treo](https://community.konduit.ai/u/treo)\ **Post date:** [February 27, 2020, 6:47pm UTC](https://community.konduit.ai/t/incremental-training/189/4 "2020-02-27T18:47:59Z")
That is always something that may happen when training a neural network - it doesn’t really matter which framework you are using for that. In some cases it might be even a feature - for example if you have data that changes over time, and you need the trained network to forget about previous training. If you want to ensure that it doesn’t do that, all you have to do is to use a union of old and new data when training. ---
**Author:** ![poorni19](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/poorni19/32/67_2.png) [@poorni19](https://community.konduit.ai/u/poorni19)\ **Post date:** [February 28, 2020, 5:31pm UTC](https://community.konduit.ai/t/incremental-training/189/5 "2020-02-28T17:31:51Z")
Is there an option to combine multiple models and make an inference ? I am just trying out various options to avoid re training when there is more data added to the system . ---
**Author:** ![treo](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/treo/32/47_2.png) [@treo](https://community.konduit.ai/u/treo)\ **Post date:** [February 28, 2020, 5:34pm UTC](https://community.konduit.ai/t/incremental-training/189/6 "2020-02-28T17:34:09Z")
There is a lot of research literature about that around. But all of that depends on your actual use case. ---
**Author:** ![poorni19](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/poorni19/32/67_2.png) [@poorni19](https://community.konduit.ai/u/poorni19)\ **Post date:** [February 28, 2020, 5:38pm UTC](https://community.konduit.ai/t/incremental-training/189/7 "2020-02-28T17:38:40Z")
I am looking for any specific option available in dl4j ? Like loading computation graph from models m1 and m2 ,and combine them to make a inference. I understand its quiet tricky …just exploring options ---
**Author:** ![treo](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/treo/32/47_2.png) [@treo](https://community.konduit.ai/u/treo)\ **Post date:** [February 28, 2020, 5:43pm UTC](https://community.konduit.ai/t/incremental-training/189/8 "2020-02-28T17:43:59Z")
If you mean ensemble models, then no, there is no out of the box way that I’d be aware of in DL4J. But it isn’t something that would be hard to implement on your own. You basically load the models you want to load, and run a (weighted) majority vote. The answer with the most votes wins. You can make this as complicated as you want, e.g. you can train a model to decide if which model should get an input and decide alone on the output, or even a whole hierarchy of things like that, or even combine that with other libraries. The sky is the limit - you are using java after all.