Multi Input Model

DL4J
1

How can I implement a model with two inputs? One input would be for a convolution layer and another would be for a dense layers?

I would like to do something as the following (from Tensorflow code in Python):

inputs1 = tf.keras.layers.Input(shape=shape1)
inputs2 = tf.keras.layers.Input(shape=shape2)

x1 = tf.keras.layers.Conv2D(filters=32, kernel_size=(3, 3), padding='same', activation='swish')(inputs1)
x1 = tf.keras.layers.GlobalAvgPool2D()(x1)
x2 = tf.keras.layers.Dense(20, use_bias=False, activation='swish')(inputs2)
together = tf.keras.layers.Concatenate()([x1, x2])

outputs = tf.keras.layers.Dense(num_output, activation='softmax')(together)
model = tf.keras.Model([inputs1, inputs2], outputs)

How can I do something similar in DL4J? So far I have this:

MultiLayerConfiguration config = new NeuralNetConfiguration.Builder()
            .updater(new Adam())
            .optimizationAlgo(OptimizationAlgorithm.STOCHASTIC_GRADIENT_DESCENT)
            .weightInit(WeightInit.XAVIER)
            .list()
            .layer(new ConvolutionLayer.Builder().kernelSize(3, 3).stride(2, 2).padding(2, 2).activation(Activation.SWISH)
                    .nIn(shape1).nOut(32).build())
            .layer(new GlobalPoolingLayer(PoolingType.AVG))
            .layer(new OutputLayer.Builder(LossFunctions.LossFunction.NEGATIVELOGLIKELIHOOD)
                    .name("output")
                    .nOut(num_output)
                    .activation(Activation.IDENTITY)
                    .build())
            .setInputType(InputType.convolutional(shape1)
            .build();

But this does not incorporate the input2 nor the concatenation.

2

@brandon don’t use multi layer networks for this. Use the computation graph API. Please check our examples: Sign in to GitHub · GitHub