# How to "Add" multiple layers in a CNN architecture?

**URL:** <https://community.konduit.ai/t/how-to-add-multiple-layers-in-a-cnn-architecture/1603>\
**Category:** Uncategorized\
**Created:** [September 15, 2021, 7:57pm UTC](https://community.konduit.ai/t/how-to-add-multiple-layers-in-a-cnn-architecture/1603 "2021-09-15T19:57:46Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![FanDev](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/fandev/32/231_2.png) [@FanDev](https://community.konduit.ai/u/FanDev)\
**Post date:** [September 15, 2021, 7:57pm UTC](https://community.konduit.ai/t/how-to-add-multiple-layers-in-a-cnn-architecture/1603/1 "2021-09-15T19:57:46Z")

</div>

Hi, I have a question to implement “Add” in DL4J:

In Python, in my CNN model, I use “Add” to add several convolution layers together (same output form), such as:

`layer_all_added = add([layer1, layer2, layer3])`

then `layer_all_added` will be used as input for the next convolutional layer.

How can I implement it in DL4J?

Thanks.

---

<div class="post-metadata">

**Author:** ![agibsonccc](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/agibsonccc/32/697_2.png) [@agibsonccc](https://community.konduit.ai/u/agibsonccc)\
**Post date:** [September 16, 2021, 12:07am UTC](https://community.konduit.ai/t/how-to-add-multiple-layers-in-a-cnn-architecture/1603/2 "2021-09-16T00:07:28Z")

</div>

@FanDev could you flesh this out a bit? Which API are you using? The ComputationGraph or Samediff? What is the add supposed to do? is that supposed to be an automatic concatneation? An element wise add?

---

<div class="post-metadata">

**Author:** ![FanDev](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/fandev/32/231_2.png) [@FanDev](https://community.konduit.ai/u/FanDev)\
**Post date:** [September 16, 2021, 6:46pm UTC](https://community.konduit.ai/t/how-to-add-multiple-layers-in-a-cnn-architecture/1603/3 "2021-09-16T18:46:01Z")

</div>

Sure, let me explain it further.

In my neural network, I applied multiple dilation convolutions with different dilation rates, such as:

```auto
layer0 = Conv3D(neuron_size, (3,3,3), activation= 'relu', padding = 'same')(inputs) //regular conv layer

layer1 = Conv3D(neuron_size, (3,3,3), activation='relu', padding='same', dilation_rate=1)(layer0) //dilation conv layer with dilation rate = 1
layer2 = Conv3D(neuron_size, (3,3,3), activation='relu', padding='same', dilation_rate=2)(layer1) //dilation conv layer
layer3 = Conv3D(neuron_size, (3,3,3), activation='relu', padding='same', dilation_rate=4)(layer2) //dilation conv layer
Add_All_Dilation = add([layer1, layer2, layer3])

merge1 = concatenate([UpSampling3D(size=(2,2,2))(Add_All_Dilation),layer0], axis = 4)
...

```

Since `layer1`, `layer2`, `layer3` have the same output format, I am able to add their weights together to create a more strong feature map.

SO my questions are:

1. How to perform “Add” in DL4J? Do I need to consider this as an additional layer?
2. (extra question) How can I assign the concatenate (in DL4J called “MergeVetax”?) with `axis=4` in DL4J?

Thanks.

---

<div class="post-metadata">

**Author:** ![agibsonccc](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/agibsonccc/32/697_2.png) [@agibsonccc](https://community.konduit.ai/u/agibsonccc)\
**Post date:** [September 17, 2021, 12:40am UTC](https://community.konduit.ai/t/how-to-add-multiple-layers-in-a-cnn-architecture/1603/4 "2021-09-17T00:40:12Z")

</div>

@FanDev thanks a lot… You’ll want to use the element wise vertex with Op.add:

> <https://github.com/eclipse/deeplearning4j/blob/master/deeplearning4j/deeplearning4j-nn/src/main/java/org/deeplearning4j/nn/graph/vertex/impl/ElementWiseVertex.java>

You would take the output names of those 3 layers as inputs.