How to feed a network with data from GPU

I’d like to:

  1. decode video frame
  2. apply some simple processing (rescaling, channel mix etc…) probably with OpenCV
  3. feed a ComputationGraph

while staying GPU-side.

What’s the best way to do this while limiting memory copy/download/upload ? I can decode frames into OpenCV GpuMat, but then how to view the GpuMat as an INDArray (or the inverse : how to create a INDArray and build a GpuMat view ?)

Pinging @saudet here

Generally a NativeImageLoader can be used on Mats, but I’m not sure about GPUMat. Many of the ways we convert Mat to NDArray can be found here:

Thank you.
I’m aware of how to share memory between Mat and NDJ4, but GpuMat/UMat are different. You cannot get a normal pointer to their data.

AFAIK, that’s not possible at the moment because ND4J always requires GPU memory to be backed by a buffer in host memory, so you’ll need to transfer your data to CPU anyway.