NoClassDefFoundError - BasicNDArrayCompressor

1

Hello,

I have models trained on PC (windows) beta6 and beta7 lib versions.
When I try to run those models on Android i receive error:

Caused by: java.lang.RuntimeException: Error loading ND4J Compressors via service loader: No compressors were found. This usually occurs when running ND4J UI from an uber-jar, which was built incorrectly (without services resource files being included)
at org.nd4j.linalg.compression.BasicNDArrayCompressor.loadCompressors(BasicNDArrayCompressor.java:64)
at org.nd4j.linalg.compression.BasicNDArrayCompressor.(BasicNDArrayCompressor.java:44)
…
java.lang.NoClassDefFoundError: org.nd4j.linalg.compression.BasicNDArrayCompressor
at org.nd4j.linalg.compression.BasicNDArrayCompressor.getInstance(BasicNDArrayCompressor.java:96)

On Android, i tried to use beta5 and beta6 but same error is thrown.

If models are trained on beta4 or maybe beta5 - I can run them on Android. (I have some models trained long time ago, not sure on which version - beta4 or beta5)

How to run new models on Android?

Thanks on your time,
Aleksandar

2

@lavajaw could you show us your build.gradle?

3

Sure,

implementation (group: ‘org.deeplearning4j’, name: ‘deeplearning4j-core’, version: ‘1.0.0-beta6’) {
exclude group: ‘org.bytedeco’, module: ‘opencv-platform’
exclude group: ‘org.bytedeco’, module: ‘leptonica-platform’
exclude group: ‘org.bytedeco’, module: ‘hdf5-platform’
exclude group: ‘org.nd4j’, module: ‘nd4j-base64’
}
implementation group: ‘org.nd4j’, name: ‘nd4j-native’, version: ‘1.0.0-beta6’
implementation group: ‘org.nd4j’, name: ‘nd4j-native’, version: ‘1.0.0-beta6’, classifier: “android-arm64”
implementation group: ‘org.bytedeco’, name: ‘openblas’, version: ‘0.3.7-1.5.2’
implementation group: ‘org.bytedeco’, name: ‘openblas’, version: ‘0.3.7-1.5.2’, classifier: “android-arm64”
implementation group: ‘org.bytedeco’, name: ‘leptonica’, version: ‘1.78.0-1.5.2’
implementation group: ‘org.bytedeco’, name: ‘leptonica’, version: ‘1.78.0-1.5.2’, classifier: “android-arm64”

annotationProcessor group: 'org.projectlombok', name: 'lombok', version: '1.18.14'
4

@agibsonccc any news?

5

@lavajaw first off, could you upgrade to beta 7 with the later javacpp versions as well?

@saudet can help you a bit more with android here.

6

Well with beta7 we have this problem:

7

@agibsonccc @saudet Sorry guys, but I need to push you, because my next release depends on this. What I can try in this situation? Any workaround to run beta7 on Android?

8

If you bundle libc++_shared.so in your app, it should work with beta7.

9

I tried that and application can run now, but anyway if I use new models trained on beta7 or beta6 its throw error - java.lang.NoClassDefFoundError: org.nd4j.linalg.compression.BasicNDArrayCompressor.
@saudet

10

Yeah, DL4J hasn’t been designed too much with backward compatibility in mind. You might need to resave your models to get them working. Among other things, since you’re using “compressors”, you’ll need to remove those and replace them with NDArrays of the right data type. In any case, ideally, it should be able to load them anyway, so please file an issue about it! Thanks

11

@saudet Sorry, but can you explane how to do it?
I just checked, old model and new model use same arhitecture (DenseNet example). Diferences are in :
Activation - leakyRelu vs swish
Regularization - none vs weightDecay
Updater - Adam vs Nesterovs

12

Does it work on the desktop? If it works with Java SE, it’s probably your version of Android Studio that’s stripping files it shouldn’t be. Make sure all resources for DL4J are found in the final APK.

13

I had the same issue. Please verify if NDArrayCompressor resource file is added to META-INF/services of nd4j jar file, NDArrayCompressor resource should contain the implementation of NDArrayCompressor interface. And also set the context loader for all threads with current class’s class loader while performing this operation.

14

@koushik.ragavendran actually yes it’s right here: deeplearning4j/org.nd4j.linalg.compression.NDArrayCompressor at master · eclipse/deeplearning4j · GitHub

it’s been there for years. Not sure what the issue could be here. It’s likely a classpath issue or the way you’re including resources in an uber jar.

15

@agibsonccc Yes it is there. My answer was related to way it is included to uber jar and also setting right context class loader so that service loader can find the resource and use the implementation.

16

@koushik.ragavendran I see. The classloader part is a great point. Note that in dl4j, you can actually set the classloader we use now in 1.0.0-M1 and newer. You can call Nd4jClassLoading.setClassLoader(…) now which may help the situation here as well.

If you do use that, just make sure to set the classloader you want to use before calling any Nd4j.create(…) like methods. Otherwise that will initialize the nd4j backend.