As part of calculating the silhouette coefficient on a set of clusters, we run code to calculate the distance between INDArray vectors:
We currently run this for every point vs every other point in the same and other clusters.
With large amounts of points in each cluster, this can take quite a long time to compute (we are running with a CUDA backend).
We were wondering if the overhead was between CPU and GPU. Would it be possible to batch these distance calculations and run them all at once? I had a look at Transforms.allEuclideanDistances, but the JavaDoc doesn’t have an explanation on the expected input or output.
Any help is greatly appreciated.