Read the h5 model fail H5Fopen failed

1

I run my demo local successful.But I build maven put it on reomote server.Can’t read my h5 model.
File is exist and the premission is work well.What can I do next to solve it

java.lang.RuntimeException: H5Fopen failed
at org.bytedeco.hdf5.H5File.allocate(Native Method)
at org.bytedeco.hdf5.H5File.(H5File.java:40)
at org.deeplearning4j.nn.modelimport.keras.Hdf5Archive.(Hdf5Archive.java:72)
at org.deeplearning4j.nn.modelimport.keras.utils.KerasModelBuilder.modelHdf5Filename(KerasModelBuilder.java:227)
at org.deeplearning4j.nn.modelimport.keras.KerasModelImport.importKerasModelAndWeights(KerasModelImport.java:171)
at com.sohu.tv.so.search.service.singlevideo.VideoSortServiceImpl3.init(VideoSortServiceImpl3.java:57)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:349)
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:300)
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:133)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:408)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1566)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:539)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:476)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:303)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:299)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194)
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:351)

2

I try it on local Spring project.
the exception still exist,but more inforation.
I run in easy java demo(not include spring),it work well.

HDF5-DIAG: Error detected in HDF5 (1.12.0) thread 0:
#000: H5F.c line 793 in H5Fopen(): unable to open file
major: File accessibility
minor: Unable to open file
#001: H5VLcallback.c line 3500 in H5VL_file_open(): open failed
major: Virtual Object Layer
minor: Can’t open object
#002: H5VLcallback.c line 3465 in H5VL__file_open(): open failed
major: Virtual Object Layer
minor: Can’t open object
#003: H5VLnative_file.c line 100 in H5VL__native_file_open(): unable to open file
major: File accessibility
minor: Unable to open file
#004: H5Fint.c line 1707 in H5F_open(): unable to read superblock
major: File accessibility
minor: Read failed
#005: H5Fsuper.c line 412 in H5F__super_read(): file signature not found
major: File accessibility
minor: Not an HDF5 file

java.lang.RuntimeException: H5Fopen failed
at org.bytedeco.hdf5.H5File.allocate(Native Method)
at org.bytedeco.hdf5.H5File.(H5File.java:40)
at org.deeplearning4j.nn.modelimport.keras.Hdf5Archive.(Hdf5Archive.java:72)
at org.deeplearning4j.nn.modelimport.keras.utils.KerasModelBuilder.modelHdf5Filename(KerasModelBuilder.java:227)
at org.deeplearning4j.nn.modelimport.keras.KerasModelImport.importKerasModelAndWeights(KerasModelImport.java:171)
at com.sohu.tv.so.suggest.job.SuggestIndexJob.afterPropertiesSet(SuggestIndexJob.java:99)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1633)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1570)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:539)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:476)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:303)

3

What is the code you are using to load it? Where is the file supposed to be?

4

I use spring and the class impliment InitializingBean
this class use @Component
the file put into resources dir

@Override
public void afterPropertiesSet() {
try {
String resourcePath = new ClassPathResource(
“model_functional_20200710.h5”).getFile().getPath();
ComputationGraph model = KerasModelImport.
importKerasModelAndWeights(resourcePath);
System.out.println(model.summary());
} catch (Exception e) {
e.printStackTrace();
}
}

5

bug

6

As far as I know h5 models can not be loaded from the class path. You will have to copy it into the filesystem somewhere and read it from there.

Edit: Overall, I also suggest that you import the model locally and then save it in DL4J format. That way you will not be surprised by changes to the import functionality down the line.

7

I use the maven read from resources successfully, so I think maybe it’s has something wrong
between with spring and model read. I train form ptyhon keras and save the h5 model.
I want to use model depence on spring

8

that is just a special case for the native h5 library that is used to read that file. It just can not read from the class path if the resource isn’t unpacked onto the file system.

9

Can I read h5 change to DL4J format and use it to online web, please give me some demo about DL4J format

10

You just save and load it like any other dl4j model. After importing from Keras, your model is effectively not much different from any other native dl4j model.

See https://deeplearning4j.konduit.ai/models/model-persistence

11

Thankyou for your reply.I try my best to do it

12

I try DL4J format,with the spring load method failed too.

Exception in thread “main” java.lang.NoClassDefFoundError: org/apache/commons/io/input/CloseShieldInputStream
at org.deeplearning4j.nn.graph.ComputationGraph.load(ComputationGraph.java:4559)
at com.sohu.tv.so.suggest.job.SuggestIndexJob.main(SuggestIndexJob.java:120)
Caused by: java.lang.ClassNotFoundException: org.apache.commons.io.input.CloseShieldInputStream
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
… 2 more

13

b
b1005×292 171 KB

14

That just looks like a missing dependency on apache commons. Have you excluded any dependencies?