Encapsulate TF c/c++ interface

Too many basic ops which TF has such as tf.einsum(…) missing in DL4J, is it possible to encapsulate TF c/c++ interface to have such ops?

@SidneyLann Many ops aren’t needed for inference which is the main target. I am evaluating doing something like this by allowing dynamic wrapping of other ops. If you want to contribute something, feel free to open an issue to discuss it, otherwise I’ll try to introduce something like that. As it stands, a short term goal I have is to add all ops that are both in TF and defined in onnx: https://github.com/onnx/onnx/blob/master/docs/Operators.md
There’s actually not that many missing. It’s a fair point though and I’ll keep that in mind when we’re doing the next release.

DL4J to me is for building model and trainning, not only inference. I want to port GAT from py and some basic ops not found in DL4J like tf.einsum etc. In fact, I had never used TF and no the py skill, just have skill reading py source.

Up to now, DL4J has all featrues I want except GAT.

einsum is a pretty complex op though, and unless you need dynamic mapping, you should be able to implement whatever underlying computation you need directly.

Yes. So I think if it is possible to implement nd4j=libnd4j+tf c++?

What I’m saying is that you can implement what einsum is doing manually, because it is usually doing one of the things that is actually implemented in ND4J.

Mixing tf and nd4j isn’t something you can do.

attn_for_self = tf.einsum(“…NHI , IHO → …NHO”, x, self.attn_kernel_self)

What SameDiff ops in ND4J should I use then? Or just combine INDArray methods?

Does DL4J have most seperate OPs to implement tf.einsum()?

hi. we believe you can replicate many cases with tensormmul, transpose and sum.

Ok. I’ll try. Thanks.

you are welcome.
I would also note that we have really kool slicing dicing.
so when einsum does not have a direct equivalent operation or if its complicated, then you can use slicing-dicing.

here are some examples:


double[][] array1 = new double[][] {{0.28021965, 0.64048402, 0.65738616},

                {0.01936167, 0.57834777, 0.87508878},

                {0.21689266, 0.25800494, 0.76577352},

                {0.98532048, 0.25336705, 0.76421956},

                {0.37685337, 0.76396022, 0.59882881}};

double[][] array2 = new double[][]{{0.8374383 , 0.98424262, 0.56054711, 0.90322204},

                {0.06912883, 0.42909746, 0.90220309, 0.1547967 },

                {0.92493407, 0.1661033 , 0.14285882, 0.67154685}};

var A = Nd4j.create(array1);

 var B = Nd4j.create(array2);

System.out.println("A "+Arrays.toString(A.shape()));

System.out.println("B "+Arrays.toString(B.shape()));

//np.einsum('ij,jk->ik', A, B)   is just tensormmul

var DOT_CC = Nd4j.tensorMmul(A, B,new int [][]{ {1},{0}});

System.out.println(Arrays.toString(DOT_CC.shape()));

System.out.println("DOT_CC: "+ Arrays.toString(DOT_CC.shape()));

System.out.println( DOT_CC);

//np.einsum('ij,jk->ijk', A, B) . its complicates so lets use slicing dicing

INDArray CC = Nd4j.zeros( A.shape()[0],A.shape()[1],B.shape()[1]);

for(int i=0;i<A.shape()[0];i++){

   for(int j=0;j<B.shape()[1];j++){

         // assign elemet wise multiplication of A.getRow(i) * B.getColumn(j)
                      
       CC.get(NDArrayIndex.point(i),NDArrayIndex.all(),NDArrayIndex.point(j)).assign(

                                        A.getRow(i).mul(B.getColumn(j))

                                );

     }

}

System.out.println("CC: "+Arrays.toString(CC.shape()));

System.out.println( CC);

Ok. Thank you very much!