Questions on two types of networks

DL4J
1

Hi,

By checking the dl4j documents, I know that dl4j has two types of networks (MultiLayerNetwork and Computational Graph). I have two questions:

(1) Workflow question: In a complete workflow of deeplearning4j, does the deeplearning4j framework build the MultiLayerNetwork first, then translate/convert MultiLayerNetwork to Computational Graph? Or, deeplearning4j just uses these two types of networks separately (developers have to choose either one of them in their dl4j applications).

(2) Optimization question: Does the deeplearning4j framework has any optimizations on MultiLayerNetwork and Computational Graph (for example, layer fusing in MultiLayerNetwork or vertex fusing in Computational Graph)? If there are some optimizations during the training process, does it mean we could have two different MultiLayerNetwork and Computational Graph during training (before and after optimization)?

Thanks.

2

@bolunli11 MultiLayerNetwork is like keras’s sequential network. It’s a simplified API when you don’t need a complex computation graph where you’re doing things like running multiple inputs at the same time.

Computation graphs are for more complex neural networks that typically have more than 1 input or multiple paths where you split, run some computation, then merge again before then having 1 or more outputs.

Keep in mind there’s a 3rd api samediff as well which is lower level and suggested if you need to implement anything more cutting edge.

All of these use the same c++ library internally for the math operations.

Regarding graph optimizations, both of those apis don’t really do anything special in that regard. A lot of those concepts will be more applicable in the lower level samediff api.

3

@agibsonccc Thanks for the reply! So MultiLayerNetwork and Computation graphs are used in different scenarios, does it mean, if I decide to use MultiLayerNetwork, then I don’t need to touch anything related to the Computation graphs?

4

@bolunli11 more or less. If needed though, the multilayernetwork has a toComputationGraph() method if you ever want to do anything with it.

1 Like