# Temporal Convolutional Network

**URL:** <https://community.konduit.ai/t/temporal-convolutional-network/118>\
**Category:** DL4J\
**Created:** [February 5, 2020, 3:15pm UTC](https://community.konduit.ai/t/temporal-convolutional-network/118 "2020-02-05T15:15:24Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![mdavis95](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/mdavis95/32/46_2.png) [@mdavis95](https://community.konduit.ai/u/mdavis95)\
**Post date:** [February 5, 2020, 3:15pm UTC](https://community.konduit.ai/t/temporal-convolutional-network/118/1 "2020-02-05T15:15:24Z")

</div>

Is there an example of wiring up a network similar to [GitHub - philipperemy/keras-tcn: Keras Temporal Convolutional Network.](https://github.com/philipperemy/keras-tcn) in DL4J. I am trying to use this style of network on a sequence of word GloVe embeddings using a setup like below:

```java
ComputationGraphConfiguration conf = new NeuralNetConfiguration.Builder()
    				.seed(12345)
    				.optimizationAlgo(OptimizationAlgorithm.STOCHASTIC_GRADIENT_DESCENT)
    				.updater(new Nesterovs()).weightInit(XAVIER).activation(Activation.TANH)
    				.graphBuilder()
    				.addInputs("in")
    				.addLayer("c0", new Convolution1D.Builder(3, 1).convolutionMode(ConvolutionMode.Causal).dilation(1).nIn(300).nOut(256).build(), "in")
    				.addLayer("c1", new Convolution1D.Builder(3, 1).convolutionMode(ConvolutionMode.Causal).dilation(2).nIn(256).nOut(256).build(), "c0")
    				.addLayer("c2", new Convolution1D.Builder(3, 1).convolutionMode(ConvolutionMode.Causal).dilation(4).nIn(256).nOut(256).build(), "c1")
    				.addLayer("c3", new Convolution1D.Builder(3, 1).convolutionMode(ConvolutionMode.Causal).dilation(8).nIn(256).nOut(256).build(), "c2")
    				.addLayer("p2", new GlobalPoolingLayer.Builder(PoolingType.MAX).build(), "c3")
    				.addLayer("batchNorm", new BatchNormalization.Builder().nIn(256).nOut(256).build(), "p2")
    				.addLayer("l0", new DenseLayer.Builder().nIn(256).nOut(256).dropOut(0.5).build(), "batchNorm")
    				.addLayer("l1", new DenseLayer.Builder().nIn(256).nOut(128).build(), "l0")
    				.layer("out",	new OutputLayer.Builder().nIn(DENSE_L1).nOut(1).lossFunction(LossFunctions.LossFunction.XENT).activation(Activation.SIGMOID).build(),"l1")
    				.setOutputs("out")
    				.build();

```

But I keep getting errors like the following:

```auto
 Failed to execute op conv1d. Attempted to execute with 3 inputs, 1 outputs, 0 targs,0 bargs and 6 iargs. Inputs: [(FLOAT,[200,256,300],c), (FLOAT,[3,256,256],f), (FLOAT,[256],c)]. Outputs: [(FLOAT,[200,256,300],f)]. tArgs: -. iArgs: [3, 1, 0, 4, 2, 0]. bArgs: -. Op own name: "f428232c-01be-4f85-a02a-91a72a8adb6f" - Please see above message (printed out from c++) for a possible cause of error.

java.lang.RuntimeException: Op [conv1d] execution failed
	at org.nd4j.linalg.cpu.nativecpu.ops.NativeOpExecutioner.exec(NativeOpExecutioner.java:1723) ~[nd4j-native-1.0.0-beta6.jar:?]
	at org.nd4j.linalg.factory.Nd4j.exec(Nd4j.java:6599) ~[nd4j-api-1.0.0-beta6.jar:1.0.0-beta6]
	at org.deeplearning4j.nn.layers.convolution.Convolution1DLayer.causalConv1dForward(Convolution1DLayer.java:206) ~[deeplearning4j-nn-1.0.0-beta6.jar:?]
	at org.deeplearning4j.nn.layers.convolution.Convolution1DLayer.preOutput(Convolution1DLayer.java:161) ~[deeplearning4j-nn-1.0.0-beta6.jar:?]
	at org.deeplearning4j.nn.layers.convolution.ConvolutionLayer.activate(ConvolutionLayer.java:446) ~[deeplearning4j-nn-1.0.0-beta6.jar:?]
	at org.deeplearning4j.nn.layers.convolution.Convolution1DLayer.activate(Convolution1DLayer.java:212) ~[deeplearning4j-nn-1.0.0-beta6.jar:?]
	at org.deeplearning4j.nn.graph.vertex.impl.LayerVertex.doForward(LayerVertex.java:111) ~[deeplearning4j-nn-1.0.0-beta6.jar:?]
	at org.deeplearning4j.nn.graph.ComputationGraph.ffToLayerActivationsInWS(ComputationGraph.java:2136) ~[deeplearning4j-nn-1.0.0-beta6.jar:?]
	at org.deeplearning4j.nn.graph.ComputationGraph.computeGradientAndScore(ComputationGraph.java:1373) ~[deeplearning4j-nn-1.0.0-beta6.jar:?]
	at org.deeplearning4j.nn.graph.ComputationGraph.computeGradientAndScore(ComputationGraph.java:1342) ~[deeplearning4j-nn-1.0.0-beta6.jar:?]
	at org.deeplearning4j.optimize.solvers.BaseOptimizer.gradientAndScore(BaseOptimizer.java:170) ~[deeplearning4j-nn-1.0.0-beta6.jar:?]
	at org.deeplearning4j.optimize.solvers.StochasticGradientDescent.optimize(StochasticGradientDescent.java:63) ~[deeplearning4j-nn-1.0.0-beta6.jar:?]
	at org.deeplearning4j.optimize.Solver.optimize(Solver.java:52) ~[deeplearning4j-nn-1.0.0-beta6.jar:?]
	at org.deeplearning4j.nn.graph.ComputationGraph.fitHelper(ComputationGraph.java:1166) ~[deeplearning4j-nn-1.0.0-beta6.jar:?]
	at org.deeplearning4j.nn.graph.ComputationGraph.fit(ComputationGraph.java:1116) ~[deeplearning4j-nn-1.0.0-beta6.jar:?]
	at org.deeplearning4j.nn.graph.ComputationGraph.fit(ComputationGraph.java:1083) ~[deeplearning4j-nn-1.0.0-beta6.jar:?]
	at org.deeplearning4j.nn.graph.ComputationGraph.fit(ComputationGraph.java:1019) ~[deeplearning4j-nn-1.0.0-beta6.jar:?]
	at com.masked.aiproject.GloveDilatedCNN.trainEpoch(GloveDilatedCNN.java:264) ~[main/:?]
	at com.masked.aiproject.GloveDilatedCNN.handleFile(GloveDilatedCNN.java:205) ~[main/:?]
	at com.masked.aiproject.GloveDilatedCNN.lambda$main$0(GloveDilatedCNN.java:117) [main/:?]
	at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125) [guava-26.0-jre.jar:?]
	at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57) [guava-26.0-jre.jar:?]
	at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78) [guava-26.0-jre.jar:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: java.lang.RuntimeException: could not create a dilated convolution forward descriptor
	at org.nd4j.linalg.cpu.nativecpu.ops.NativeOpExecutioner.exec(NativeOpExecutioner.java:2019) ~[nd4j-native-1.0.0-beta6.jar:?]
	at org.nd4j.linalg.cpu.nativecpu.ops.NativeOpExecutioner.exec(NativeOpExecutioner.java:1713) ~[nd4j-native-1.0.0-beta6.jar:?]
	... 25 more

```

Does anyone have an example or any tips on how to debug this?

Edit: Added formatting so it is easier to read the post.

---

<div class="post-metadata">

**Author:** ![treo](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/treo/32/47_2.png) [@treo](https://community.konduit.ai/u/treo)\
**Post date:** [February 5, 2020, 5:55pm UTC](https://community.konduit.ai/t/temporal-convolutional-network/118/2 "2020-02-05T17:55:46Z")

</div>

I think the example you are looking for is [CNN Sentence Classification](https://github.com/eclipse/deeplearning4j-examples/blob/master/dl4j-examples/src/main/java/org/deeplearning4j/examples/convolution/sentenceclassification/CnnSentenceClassificationExample.java)

It iterates over a sequence of W2V embeddings.

Also, was there anything else printed above this exception? The C++ error messages go straight to STDERR if I remember correctly and usually have more information about what exactly may be wrong in your setup.

---

<div class="post-metadata">

**Author:** ![mdavis95](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/mdavis95/32/46_2.png) [@mdavis95](https://community.konduit.ai/u/mdavis95)\
**Post date:** [February 5, 2020, 6:35pm UTC](https://community.konduit.ai/t/temporal-convolutional-network/118/3 "2020-02-05T18:35:33Z")

</div>

Thanks for the example I will take a look.

As a side note, if I remove dilation(x) this setup works fine.

I am rerunning again to look closer for the C++ error message.

---

<div class="post-metadata">

**Author:** ![mdavis95](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/mdavis95/32/46_2.png) [@mdavis95](https://community.konduit.ai/u/mdavis95)\
**Post date:** [February 5, 2020, 8:23pm UTC](https://community.konduit.ai/t/temporal-convolutional-network/118/4 "2020-02-05T20:23:10Z")

</div>

I am able to reproduce my error on the CnnSentenceClassificationExample. If I change the network setup to

```java
 ComputationGraphConfiguration config = new NeuralNetConfiguration.Builder()
                .weightInit(WeightInit.RELU)
                .activation(Activation.LEAKYRELU)
                .updater(new Adam(0.01))
                .graphBuilder()
                .addInputs("input")
                .addLayer("c0", new Convolution1D.Builder(3, 1 ).convolutionMode(ConvolutionMode.Causal).nIn(300).nOut(128).build(), "input")
                .addLayer("c1", new Convolution1D.Builder(3, 1).convolutionMode(ConvolutionMode.Causal).dilation(2).nIn(128).nOut(128).build(), "c0")
                .addLayer("c2", new Convolution1D.Builder(3, 1).convolutionMode(ConvolutionMode.Causal).dilation(4).nIn(128).nOut(128).build(), "c1")
                .addLayer("c3", new Convolution1D.Builder(3, 1).convolutionMode(ConvolutionMode.Causal).dilation(8).nIn(128).nOut(128).build(), "c2")
                .addLayer("p2", new GlobalPoolingLayer.Builder(PoolingType.MAX).build(), "c3")
                .addLayer("batchNorm", new BatchNormalization.Builder().nIn(128).nOut(128).build(), "p2")
                .addLayer("l0", new DenseLayer.Builder().nIn(128).nOut(128).build(), "batchNorm")
                .addLayer("l1", new DenseLayer.Builder().nIn(128).nOut(128).build(), "l0")
                .addLayer("out", new OutputLayer.Builder()
                        .lossFunction(LossFunctions.LossFunction.MCXENT)
                        .activation(Activation.SOFTMAX)
                        .nIn(128)
                        .nOut(2) //2 classes: positive or negative
                        .build(), "l1")
                .setOutputs("out")
                .build();

```

And change the iterator to CNN1D

```java
CnnSentenceDataSetIterator.Builder(Format.CNN1D)
                .sentenceProvider(sentenceProvider)
                .wordVectors(wordVectors)
                .minibatchSize(minibatchSize)
                .maxSentenceLength(maxSentenceLength)
                .useNormalizedWordVectors(false)
                .build();

```

It explodes. It works fine when I remove the dilation.

I don’t see anything useful on standard err other than what was posted above.

```auto
2020-02-05 15:30:12,996 [main] <INFO> factory.Nd4jBackend: Loaded [CpuBackend] backend
2020-02-05 15:30:13,383 [main] <INFO> nativeblas.NativeOpsHolder: Number of threads used for linear algebra: 12
2020-02-05 15:30:13,419 [main] <INFO> nativeblas.Nd4jBlas: Number of threads used for OpenMP BLAS: 12
2020-02-05 15:30:13,422 [main] <INFO> executioner.DefaultOpExecutioner: Backend used: [CPU]; OS: [Linux]
2020-02-05 15:30:13,422 [main] <INFO> executioner.DefaultOpExecutioner: Cores: [24]; Memory: [30.0GB];
2020-02-05 15:30:13,422 [main] <INFO> executioner.DefaultOpExecutioner: Blas vendor: [OPENBLAS]
2020-02-05 15:30:13,450 [main] <INFO> graph.ComputationGraph: Starting ComputationGraph with WorkspaceModes set to [training: ENABLED; inference: ENABLED], cacheMode set to [NONE]
Number of parameters by layer:
	c0	115328
	c1	49280
	c2	49280
	c3	49280
	p2	0
	batchNorm	512
	l0	16512
	l1	16512
	out	258
Loading word vectors and creating DataSetIterators
Starting training
2020-02-05 15:31:08,962 [main] <ERROR> ops.NativeOpExecutioner: Failed to execute op conv1d. Attempted to execute with 3 inputs, 1 outputs, 0 targs,0 bargs and 6 iargs. Inputs: [(FLOAT,[32,128,256],c), (FLOAT,[3,128,128],f), (FLOAT,[128],c)]. Outputs: [(FLOAT,[32,128,256],f)]. tArgs: -. iArgs: [3, 1, 0, 4, 2, 0]. bArgs: -. Op own name: "a9b9d1b1-bea5-46f8-9b71-4f40f5d07dd6" - Please see above message (printed out from c++) for a possible cause of error.
Exception in thread "main" java.lang.RuntimeException: Op [conv1d] execution failed
	at org.nd4j.linalg.cpu.nativecpu.ops.NativeOpExecutioner.exec(NativeOpExecutioner.java:1723)
	at org.nd4j.linalg.factory.Nd4j.exec(Nd4j.java:6599)
	at org.deeplearning4j.nn.layers.convolution.Convolution1DLayer.causalConv1dForward(Convolution1DLayer.java:206)
	at org.deeplearning4j.nn.layers.convolution.Convolution1DLayer.preOutput(Convolution1DLayer.java:161)
	at org.deeplearning4j.nn.layers.convolution.ConvolutionLayer.activate(ConvolutionLayer.java:446)
	at org.deeplearning4j.nn.layers.convolution.Convolution1DLayer.activate(Convolution1DLayer.java:212)
	at org.deeplearning4j.nn.graph.vertex.impl.LayerVertex.doForward(LayerVertex.java:111)
	at org.deeplearning4j.nn.graph.ComputationGraph.ffToLayerActivationsInWS(ComputationGraph.java:2136)
	at org.deeplearning4j.nn.graph.ComputationGraph.computeGradientAndScore(ComputationGraph.java:1373)
	at org.deeplearning4j.nn.graph.ComputationGraph.computeGradientAndScore(ComputationGraph.java:1342)
	at org.deeplearning4j.optimize.solvers.BaseOptimizer.gradientAndScore(BaseOptimizer.java:170)
	at org.deeplearning4j.optimize.solvers.StochasticGradientDescent.optimize(StochasticGradientDescent.java:63)
	at org.deeplearning4j.optimize.Solver.optimize(Solver.java:52)
	at org.deeplearning4j.nn.graph.ComputationGraph.fitHelper(ComputationGraph.java:1166)
	at org.deeplearning4j.nn.graph.ComputationGraph.fit(ComputationGraph.java:1116)
	at org.deeplearning4j.nn.graph.ComputationGraph.fit(ComputationGraph.java:1083)
	at org.deeplearning4j.nn.graph.ComputationGraph.fit(ComputationGraph.java:1019)
	at org.deeplearning4j.nn.graph.ComputationGraph.fit(ComputationGraph.java:1007)
	at org.deeplearning4j.examples.convolution.sentenceclassification.CnnSentenceClassificationExample.main(CnnSentenceClassificationExample.java:137)
Caused by: java.lang.RuntimeException: could not create a dilated convolution forward descriptor
	at org.nd4j.linalg.cpu.nativecpu.ops.NativeOpExecutioner.exec(NativeOpExecutioner.java:2019)
	at org.nd4j.linalg.cpu.nativecpu.ops.NativeOpExecutioner.exec(NativeOpExecutioner.java:1713)
	... 18 more
Caused by: java.lang.RuntimeException: could not create a dilated convolution forward descriptor

> Task :analyses:CnnSentenceClassificationExample.main() FAILED

Execution failed for task ':analyses:CnnSentenceClassificationExample.main()'.
> Process 'command '/home/mdavis/.local/share/JetBrains/Toolbox/apps/IDEA-U/ch-0/193.5662.53/jbr/bin/java'' finished with non-zero exit value 1

```

---

<div class="post-metadata">

**Author:** ![raver119](https://avatars.discourse-cdn.com/v4/letter/r/9de053/32.png) [@raver119](https://community.konduit.ai/u/raver119)\
**Post date:** [February 6, 2020, 8:17am UTC](https://community.konduit.ai/t/temporal-convolutional-network/118/5 "2020-02-06T08:17:49Z")

</div>

Can you please disable mkldnn use, and run again?

You can do it with this line of code:

```auto
Nd4j.getEnvironment().allowHelpers(false);

```

Just add it at the top of your code

---

<div class="post-metadata">

**Author:** ![AlexBlack](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/alexblack/32/25_2.png) [@AlexBlack](https://community.konduit.ai/u/AlexBlack)\
**Post date:** [February 6, 2020, 9:16am UTC](https://community.konduit.ai/t/temporal-convolutional-network/118/6 "2020-02-06T09:16:38Z")

</div>

@mdavis95 @raver119 Unfortunately Nd4j.getEnvironment() was added after 1.0.0-beta6 was released.

You can use this instead, which works for 1.0.0-beta6 and earlier:  
Nd4jCpu.Environment.getInstance().allowHelpers(false);

---

<div class="post-metadata">

**Author:** ![raver119](https://avatars.discourse-cdn.com/v4/letter/r/9de053/32.png) [@raver119](https://community.konduit.ai/u/raver119)\
**Post date:** [February 6, 2020, 9:17am UTC](https://community.konduit.ai/t/temporal-convolutional-network/118/7 "2020-02-06T09:17:51Z")

</div>

Ouch! 🙂

Well, same idea. Lets disable MKL-DNN and see what happens.

---

<div class="post-metadata">

**Author:** ![mdavis95](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/mdavis95/32/46_2.png) [@mdavis95](https://community.konduit.ai/u/mdavis95)\
**Post date:** [February 6, 2020, 1:22pm UTC](https://community.konduit.ai/t/temporal-convolutional-network/118/8 "2020-02-06T13:22:08Z")

</div>

Yep, that makes it work.

---

<div class="post-metadata">

**Author:** ![raver119](https://avatars.discourse-cdn.com/v4/letter/r/9de053/32.png) [@raver119](https://community.konduit.ai/u/raver119)\
**Post date:** [February 6, 2020, 1:27pm UTC](https://community.konduit.ai/t/temporal-convolutional-network/118/9 "2020-02-06T13:27:54Z")

</div>

I see, thank you. We’ll investigate why MKL-DNN forbids that. It might be their natural limitation.

---

<div class="post-metadata">

**Author:** ![mdavis95](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/mdavis95/32/46_2.png) [@mdavis95](https://community.konduit.ai/u/mdavis95)\
**Post date:** [February 6, 2020, 1:43pm UTC](https://community.konduit.ai/t/temporal-convolutional-network/118/10 "2020-02-06T13:43:05Z")

</div>

Would this be likely be fine on a GPU?

---

<div class="post-metadata">

**Author:** ![treo](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/treo/32/47_2.png) [@treo](https://community.konduit.ai/u/treo)\
**Post date:** [February 6, 2020, 2:41pm UTC](https://community.konduit.ai/t/temporal-convolutional-network/118/11 "2020-02-06T14:41:48Z")

</div>

According to the documentation cuDNN supports dilation, so I expect this to be fine on a GPU.

---

<div class="post-metadata">

**Author:** ![mdavis95](https://yyz1.discourse-cdn.com/flex035/user_avatar/community.konduit.ai/mdavis95/32/46_2.png) [@mdavis95](https://community.konduit.ai/u/mdavis95)\
**Post date:** [February 6, 2020, 10:03pm UTC](https://community.konduit.ai/t/temporal-convolutional-network/118/12 "2020-02-06T22:03:03Z")

</div>

It works great on GPU thanks! I appreciate the help getting this up and running.

I tried to track down MKL-DNN support for features and only found this check:

> <https://github.com/oneapi-src/oneDNN/blob/master/src/common/convolution.cpp#L117>

Debugging that is outside my league but I figured I would pass on the link on the off chance it helps.