A fatal error has been detected by the Java Runtime Environment

DL4J
1

I am trying to load a .h5 model (tf.keras) in scala using deeplearning4j.

I’m using SBT with following dependencies :

libraryDependencies ++= Seq(
  "org.deeplearning4j" % "deeplearning4j-core" % "1.0.0-M2.1",
  "org.nd4j" % "nd4j-native" % "1.0.0-M2.1",
)

When I try to load models containing LSTM or RNN layers, the following error is raised:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x000072104b173475, pid=76220, tid=76222
#
# JRE version: OpenJDK Runtime Environment (11.0.24+8) (build 11.0.24+8-post-Ubuntu-1ubuntu322.04)
# Java VM: OpenJDK 64-Bit Server VM (11.0.24+8-post-Ubuntu-1ubuntu322.04, mixed mode, sharing, tiered, compressed oops, g1 gc, linux-amd64)
# Problematic frame:
# C  [libnd4jcpu.so+0x1573475]  sd::NDArray::getOffset(long long) const+0x25
#
# Core dump will be written. Default location: Core dumps may be processed with "/usr/share/apport/apport -p%p -s%s -c%c -d%d -P%P -u%u -g%g -- %E" (or dumping to /home/userr/IdeaProjects/Tor_scala/core.76220)
#
# An error report file with more information is saved as:
# /home/userr/IdeaProjects/Tor_scala/hs_err_pid76220.log

Process finished with exit code 130 (interrupted by signal 2:SIGINT)

However, models containing layers other than LSTM/RNN doesnt raise any errors. I tried loading models containing Dense layers and CNN. They worked perfectly. This issue was seen only when I try with LSTM/RNN layers.

I tried with other versions - M2, M1.1 and M1
The same issue was seen with M2 version too.

But when I used M1.1 and M1, I got the following error for all models regardless of the type of layers used in the model:

Debug: Deallocating org.bytedeco.hdf5.Group[address=0x71ce3568b530,position=0,limit=1,capacity=1,deallocator=org.bytedeco.javacpp.Pointer$NativeDeallocator[ownerAddress=0x71ce3568b530,deallocatorAddress=0x71ce026a7dc0]]
[error] java.lang.NullPointerException: Cannot invoke "org.deeplearning4j.nn.modelimport.keras.KerasLayer.getNumParams()" because the return value of "java.util.Map.get(Object)" is null
[error]         at org.deeplearning4j.nn.modelimport.keras.utils.KerasModelUtils.importWeights(KerasModelUtils.java:235)
[error]         at org.deeplearning4j.nn.modelimport.keras.KerasModel.<init>(KerasModel.java:190)
[error]         at org.deeplearning4j.nn.modelimport.keras.KerasModel.<init>(KerasModel.java:99)
[error]         at org.deeplearning4j.nn.modelimport.keras.utils.KerasModelBuilder.buildModel(KerasModelBuilder.java:311)
[error]         at org.deeplearning4j.nn.modelimport.keras.KerasModelImport.importKerasModelAndWeights(KerasModelImport.java:150)
[error]         at tesst$.main(tesst.scala:13)
[error]         at tesst.main(tesst.scala)
[error]         at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[error]         at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
[error]         at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error]         at java.base/java.lang.reflect.Method.invoke(Method.java:569)
[error] stack trace is suppressed; run last Compile / run for the full output
[error] (Compile / run) java.lang.NullPointerException: Cannot invoke "org.deeplearning4j.nn.modelimport.keras.KerasLayer.getNumParams()" because the return value of "java.util.Map.get(Object)" is null
[error] Total time: 3 s, completed 21-Oct-2024, 2:26:27 pm

My model was compiled and trained in Python and saved as .h5. Following is my scala code for loading the model:

import org.deeplearning4j.nn.modelimport.keras.KerasModelImport
import org.deeplearning4j.nn.graph.ComputationGraph


object tesst {
  def main(args: Array[String]): Unit = {
    // Path to your .h5 file
    System.setProperty("org.bytedeco.javacpp.logger.debug", "true")
    System.setProperty("org.bytedeco.javacpp.logger", "sl4j")
    val modelPath="/home/userr/Models/lstm_actual.h5"
    val model: ComputationGraph = KerasModelImport.importKerasModelAndWeights(modelPath,false)
    println(model.summary())
  }
}

I’m new to DL4J. Can someone guide me and let me know why it happens?

2

@MRV could you post the configuration and keras version?

3

@agibsonccc
Model Type - Functional
print(model.class) >> <class ‘keras.src.engine.functional.Functional’>
TensorFlow version: 2.15.0
Keras version: 2.15.0
Keras backend: tensorflow

4

@MRV that’s not what I needed. Can you dump the json of the file you’re importing or use model.summary()?

5

@agibsonccc

The details of the model is given below:

Model configuration:
{‘name’: ‘model_6’, ‘trainable’: True, ‘layers’: [{‘module’: ‘keras.layers’, ‘class_name’: ‘InputLayer’, ‘config’: {‘batch_input_shape’: (None, 43), ‘dtype’: ‘int32’, ‘sparse’: False, ‘ragged’: False, ‘name’: ‘main_input1’}, ‘registered_name’: None, ‘name’: ‘main_input1’, ‘inbound_nodes’: }, {‘module’: ‘keras.layers’, ‘class_name’: ‘Embedding’, ‘config’: {‘name’: ‘embedding_10’, ‘trainable’: False, ‘dtype’: ‘float32’, ‘batch_input_shape’: (None, 43), ‘input_dim’: 10000, ‘output_dim’: 50, ‘embeddings_initializer’: {‘module’: ‘keras.initializers’, ‘class_name’: ‘RandomUniform’, ‘config’: {‘minval’: -0.05, ‘maxval’: 0.05, ‘seed’: None}, ‘registered_name’: None}, ‘embeddings_regularizer’: None, ‘activity_regularizer’: None, ‘embeddings_constraint’: None, ‘mask_zero’: False, ‘input_length’: 43}, ‘registered_name’: None, ‘build_config’: {‘input_shape’: (None, 43)}, ‘name’: ‘embedding_10’, ‘inbound_nodes’: [[[‘main_input1’, 0, 0, {}]]]}, {‘module’: ‘keras.layers’, ‘class_name’: ‘InputLayer’, ‘config’: {‘batch_input_shape’: (None, 9), ‘dtype’: ‘float32’, ‘sparse’: False, ‘ragged’: False, ‘name’: ‘main_input2’}, ‘registered_name’: None, ‘name’: ‘main_input2’, ‘inbound_nodes’: }, {‘module’: ‘keras.layers’, ‘class_name’: ‘LSTM’, ‘config’: {‘name’: ‘lstm_2’, ‘trainable’: True, ‘dtype’: ‘float32’, ‘return_sequences’: False, ‘return_state’: False, ‘go_backwards’: False, ‘stateful’: False, ‘unroll’: False, ‘time_major’: False, ‘units’: 64, ‘activation’: ‘tanh’, ‘recurrent_activation’: ‘sigmoid’, ‘use_bias’: True, ‘kernel_initializer’: {‘module’: ‘keras.initializers’, ‘class_name’: ‘GlorotUniform’, ‘config’: {‘seed’: None}, ‘registered_name’: None}, ‘recurrent_initializer’: {‘module’: ‘keras.initializers’, ‘class_name’: ‘Orthogonal’, ‘config’: {‘gain’: 1.0, ‘seed’: None}, ‘registered_name’: None}, ‘bias_initializer’: {‘module’: ‘keras.initializers’, ‘class_name’: ‘Zeros’, ‘config’: {}, ‘registered_name’: None}, ‘unit_forget_bias’: True, ‘kernel_regularizer’: None, ‘recurrent_regularizer’: None, ‘bias_regularizer’: None, ‘activity_regularizer’: None, ‘kernel_constraint’: None, ‘recurrent_constraint’: None, ‘bias_constraint’: None, ‘dropout’: 0.0, ‘recurrent_dropout’: 0.0, ‘implementation’: 2}, ‘registered_name’: None, ‘build_config’: {‘input_shape’: (None, 43, 50)}, ‘name’: ‘lstm_2’, ‘inbound_nodes’: [[[‘embedding_10’, 0, 0, {}]]]}, {‘module’: ‘keras.layers’, ‘class_name’: ‘Dense’, ‘config’: {‘name’: ‘dense_14’, ‘trainable’: True, ‘dtype’: ‘float32’, ‘units’: 64, ‘activation’: ‘relu’, ‘use_bias’: True, ‘kernel_initializer’: {‘module’: ‘keras.initializers’, ‘class_name’: ‘GlorotUniform’, ‘config’: {‘seed’: None}, ‘registered_name’: None}, ‘bias_initializer’: {‘module’: ‘keras.initializers’, ‘class_name’: ‘Zeros’, ‘config’: {}, ‘registered_name’: None}, ‘kernel_regularizer’: None, ‘bias_regularizer’: None, ‘activity_regularizer’: None, ‘kernel_constraint’: None, ‘bias_constraint’: None}, ‘registered_name’: None, ‘build_config’: {‘input_shape’: (None, 9)}, ‘name’: ‘dense_14’, ‘inbound_nodes’: [[[‘main_input2’, 0, 0, {}]]]}, {‘module’: ‘keras.layers’, ‘class_name’: ‘Dropout’, ‘config’: {‘name’: ‘dropout_16’, ‘trainable’: True, ‘dtype’: ‘float32’, ‘rate’: 0.5, ‘noise_shape’: None, ‘seed’: None}, ‘registered_name’: None, ‘build_config’: {‘input_shape’: (None, 64)}, ‘name’: ‘dropout_16’, ‘inbound_nodes’: [[[‘lstm_2’, 0, 0, {}]]]}, {‘module’: ‘keras.layers’, ‘class_name’: ‘Dropout’, ‘config’: {‘name’: ‘dropout_17’, ‘trainable’: True, ‘dtype’: ‘float32’, ‘rate’: 0.1, ‘noise_shape’: None, ‘seed’: None}, ‘registered_name’: None, ‘build_config’: {‘input_shape’: (None, 64)}, ‘name’: ‘dropout_17’, ‘inbound_nodes’: [[[‘dense_14’, 0, 0, {}]]]}, {‘module’: ‘keras.layers’, ‘class_name’: ‘Concatenate’, ‘config’: {‘name’: ‘concatenate_8’, ‘trainable’: True, ‘dtype’: ‘float32’, ‘axis’: -1}, ‘registered_name’: None, ‘build_config’: {‘input_shape’: [(None, 64), (None, 64)]}, ‘name’: ‘concatenate_8’, ‘inbound_nodes’: [[[‘dropout_16’, 0, 0, {}], [‘dropout_17’, 0, 0, {}]]]}, {‘module’: ‘keras.layers’, ‘class_name’: ‘Dense’, ‘config’: {‘name’: ‘dense_15’, ‘trainable’: True, ‘dtype’: ‘float32’, ‘units’: 1, ‘activation’: ‘sigmoid’, ‘use_bias’: True, ‘kernel_initializer’: {‘module’: ‘keras.initializers’, ‘class_name’: ‘GlorotUniform’, ‘config’: {‘seed’: None}, ‘registered_name’: None}, ‘bias_initializer’: {‘module’: ‘keras.initializers’, ‘class_name’: ‘Zeros’, ‘config’: {}, ‘registered_name’: None}, ‘kernel_regularizer’: None, ‘bias_regularizer’: None, ‘activity_regularizer’: None, ‘kernel_constraint’: None, ‘bias_constraint’: None}, ‘registered_name’: None, ‘build_config’: {‘input_shape’: (None, 128)}, ‘name’: ‘dense_15’, ‘inbound_nodes’: [[[‘concatenate_8’, 0, 0, {}]]]}], ‘input_layers’: [[‘main_input1’, 0, 0], [‘main_input2’, 0, 0]], ‘output_layers’: [[‘dense_15’, 0, 0]]}

Model architecture:
Model: "model_6"
__________________________________________________________________________________________________
 Layer (type)                Output Shape                 Param #   Connected to                  
==================================================================================================
 main_input1 (InputLayer)    [(None, 43)]                 0         []                            
                                                                                                  
 embedding_10 (Embedding)    (None, 43, 50)               500000    ['main_input1[0][0]']         
                                                                                                  
 main_input2 (InputLayer)    [(None, 9)]                  0         []                            
                                                                                                  
 lstm_2 (LSTM)               (None, 64)                   29440     ['embedding_10[0][0]']        
                                                                                                  
 dense_14 (Dense)            (None, 64)                   640       ['main_input2[0][0]']         
                                                                                                  
 dropout_16 (Dropout)        (None, 64)                   0         ['lstm_2[0][0]']              
                                                                                                  
 dropout_17 (Dropout)        (None, 64)                   0         ['dense_14[0][0]']            
                                                                                                  
 concatenate_8 (Concatenate  (None, 128)                  0         ['dropout_16[0][0]',          
 )                                                                   'dropout_17[0][0]']          
                                                                                                  
 dense_15 (Dense)            (None, 1)                    129       ['concatenate_8[0][0]']       
                                                                                                  
==================================================================================================
Total params: 530209 (2.02 MB)
Trainable params: 30209 (118.00 KB)
Non-trainable params: 500000 (1.91 MB)
__________________________________________________________________________________________________

Model type: Functional

6

@MRV apologies please ping me if I don’t get back to you. looking at the model again it might be an issue with functional models specifically. Could you try converting this to a sequential and see if you still have that issue?
It’s still hard to tell why a nullpointer is being thrown there. It looks like a native crash but could also be a java one. There’s a mixed cause there…