Hi all, i was looking on dl4j a few days ago and i was trying to run code based on this post: Face Recognition in JAVA.
I found 2 problems:
-
Load the pretrained model is taken around 30 minutes. Is it ok?
-
Convert INDArray to double (featuresArray.data().asDouble()) is taken a lot of time.
Here is my pom.xml information:
<properties>
<dl4j.version>1.0.0-beta7</dl4j.version>
<ffmpeg.version>3.2.1-1.3</ffmpeg.version>
<javacv.version>1.4.1</javacv.version>
</properties>
<dependencies>
<dependency>
<groupId>org.nd4j</groupId>
<artifactId>nd4j-native-platform</artifactId>
<version>${dl4j.version}</version>
</dependency>
<dependency>
<groupId>org.deeplearning4j</groupId>
<artifactId>deeplearning4j-core</artifactId>
<version>${dl4j.version}</version>
</dependency>
<dependency>
<groupId>org.deeplearning4j</groupId>
<artifactId>deeplearning4j-zoo</artifactId>
<version>${dl4j.version}</version>
</dependency>
<dependency>
<groupId>org.deeplearning4j</groupId>
<artifactId>deeplearning4j-modelimport</artifactId>
<version>${dl4j.version}</version>
</dependency>
<dependency>
<groupId>org.datavec</groupId>
<artifactId>datavec-data-image</artifactId>
<version>${dl4j.version}</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.bytedeco/openblas-platform -->
<dependency>
<groupId>org.bytedeco</groupId>
<artifactId>openblas-platform</artifactId>
<version>0.3.9-1.5.3</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-jdk14</artifactId>
<version>1.7.25</version>
</dependency>
</dependencies>
Full class code:
package com.ajmakoni.fr.dl4j.math;
import java.io.File;
import java.io.IOException;
import org.datavec.image.loader.NativeImageLoader;
import org.deeplearning4j.nn.graph.ComputationGraph;
import org.deeplearning4j.nn.transferlearning.TransferLearningHelper;
import org.deeplearning4j.zoo.PretrainedType;
import org.deeplearning4j.zoo.ZooModel;
import org.deeplearning4j.zoo.model.VGG16;
import org.nd4j.linalg.api.ndarray.INDArray;
import org.nd4j.linalg.api.buffer.DataType;
import org.nd4j.linalg.dataset.DataSet;
import org.nd4j.linalg.dataset.api.preprocessor.DataNormalization;
import org.nd4j.linalg.dataset.api.preprocessor.VGG16ImagePreProcessor;
import org.nd4j.linalg.factory.Nd4j;
public class VGG16Model {
private TransferLearningHelper transferLearningHelper;
private NativeImageLoader nativeImageLoader;
private DataNormalization scaler;
private boolean isInitialized;
private ComputationGraph objComputationGraph = null;
public VGG16Model() {
try {
String path = "C:\\Java\\ajmackoni\\Face Recognition\\Processing\\VideoFace";
File f = new File(path, "model.zip");
System.out.println("Loading DL4J");
if(!f.exists()) {
ZooModel objZooModel = VGG16.builder().build();
objComputationGraph = (ComputationGraph)objZooModel.initPretrained(PretrainedType.VGGFACE);
objComputationGraph.save(f);
}
else {
objComputationGraph = ComputationGraph.load(f, false);
}
System.out.println(objComputationGraph.summary());
System.out.println("Loaded DL4J");
String name = "pool4";
transferLearningHelper = new TransferLearningHelper(objComputationGraph, name); //new TransferLearningHelper(objComputationGraph,"pool4");
nativeImageLoader = new NativeImageLoader(224, 224, 3);
scaler = new VGG16ImagePreProcessor();
isInitialized=true;
} catch (IOException e) {
e.printStackTrace();
}
}
public double[] getFeatures(File faceImageFile) {
if (!isInitialized) {
return null;
}
try {
INDArray imageMatrix = nativeImageLoader.asMatrix(faceImageFile);
scaler.transform(imageMatrix);
//objComputationGraph.feedForward(imageMatrix, false);
DataSet objDataSet = new DataSet(imageMatrix, Nd4j.create(new float[]{0,0}));
DataSet objFeaturized = transferLearningHelper.featurize(objDataSet);
INDArray featuresArray = objFeaturized.getFeatures();
int reshapeDimension=1;
for (long dimension : featuresArray.shape()) {
reshapeDimension *= dimension;
}
featuresArray = featuresArray.reshape(1,reshapeDimension);
DataType d = featuresArray.dataType();
double[] result = featuresArray.data().asDouble();
return result;
} catch (IOException e) {
e.printStackTrace();
return null;
}
}
public INDArray getFeatures2(File faceImageFile) {
if (!isInitialized) {
return null;
}
try {
INDArray imageMatrix = nativeImageLoader.asMatrix(faceImageFile);
scaler.transform(imageMatrix);
//objComputationGraph.feedForward(imageMatrix, false);
DataSet objDataSet = new DataSet(imageMatrix, Nd4j.create(new float[]{0,0}));
DataSet objFeaturized = transferLearningHelper.featurize(objDataSet);
INDArray featuresArray = objFeaturized.getFeatures();
int reshapeDimension=1;
for (long dimension : featuresArray.shape()) {
reshapeDimension *= dimension;
}
featuresArray = featuresArray.reshape(1,reshapeDimension);
DataType d = featuresArray.dataType();
return featuresArray;
} catch (IOException e) {
e.printStackTrace();
return null;
}
}
}
Computer hardware:
Sample Image:
Dl4j Output :
00:28:22.896 [main] INFO org.nd4j.linalg.factory.Nd4jBackend - Loaded [CpuBackend] backend
00:28:59.772 [main] INFO org.nd4j.nativeblas.NativeOpsHolder - Number of threads used for linear algebra: 4
00:28:59.785 [main] WARN org.nd4j.linalg.cpu.nativecpu.CpuNDArrayFactory - *********************************** CPU Feature Check Warning ***********************************
00:28:59.785 [main] WARN org.nd4j.linalg.cpu.nativecpu.CpuNDArrayFactory - Warning: Initializing ND4J with Generic x86 binary on a CPU with AVX/AVX2 support
00:28:59.785 [main] WARN org.nd4j.linalg.cpu.nativecpu.CpuNDArrayFactory - Using ND4J with AVX/AVX2 will improve performance. See deeplearning4j.org/cpu for more details
00:28:59.785 [main] WARN org.nd4j.linalg.cpu.nativecpu.CpuNDArrayFactory - Or set environment variable ND4J_IGNORE_AVX=true to suppress this warning
00:28:59.786 [main] WARN org.nd4j.linalg.cpu.nativecpu.CpuNDArrayFactory - *************************************************************************************************
00:29:01.756 [main] INFO org.nd4j.nativeblas.Nd4jBlas - Number of threads used for OpenMP BLAS: 4
00:29:02.463 [main] INFO org.nd4j.linalg.api.ops.executioner.DefaultOpExecutioner - Backend used: [CPU]; OS: [Windows 10]
00:29:02.463 [main] INFO org.nd4j.linalg.api.ops.executioner.DefaultOpExecutioner - Cores: [8]; Memory: [4.0GB];
00:29:02.463 [main] INFO org.nd4j.linalg.api.ops.executioner.DefaultOpExecutioner - Blas vendor: [OPENBLAS]
Could you give some ideas how to fix the problems?
Thanks,