Using ML models with Amazon EMR

@mar why is hdf5 in there? Was there an import issue with the deeplearning4j-modelimport? I’d consider converting the model locally and then just using it like that in your jar file.

Also, where did you get the idea to manually import nd4j-backends pom? I don’t think we’ve ever told people to do that.

This smells a bit of copy and paste without knowing what it is you’re putting there. pom dependencies are not actual jar files and are usually just parent projects of real jars. Unless specified otherwise (usually in the context of a spring BOM) you shouldn’t use a pom dependency like that.

Could you try setting the jvm property org.bytedeco.javacpp.logger.debug=true and running it on the workers? Maybe there’s an old libc or something causing a load failure.

Minus the cleanup I gave you the rest should work here.