Predicting with a keras model: data types and byte orders

DL4J
1

I verified that I need to reverse byte order of predictions in keras python to be read in java. Now that I’ve loaded a trained .keras model in DL4J and made some predictions, these naive questions arise:

  1. Is byte order of model weights an issue on load of .keras format? Is it detected and handled, or otherwise not an issue?
  2. As input for prediction, I’m making an INDArray from an array of double - should I worry about byte order? (Would an array of float work? Be slower?)
  3. The example(s) I’ve seen use model.output.getDouble(0), so native java double seems to be used throughout, as maybe native python double is used.

I tried reversing order on the array of double passed to Nd4j.create(), and predictions got less believable, e.g. all the same in cases with variety using java byte order in the input.

2

@Phobrain Don’t use onheap data structures (float etc) - byte order: I maintain that module and I am honestly not sure what you’re talking about.

Do you mean order of the array like c or f? If so, then we default to C and so does keras. It should be fine.

Native java is mainly used in the samples to show the result. Usually you convert the array inline to a usable result. Ideally you want that to be a bulk operation. That way you minimize the performance impact an array crossing to on heap has.

3

Do you mean order of the array like c or f?

I mean endianness of e.g. a binary float on disk, maybe it relates to array order too?

Here’s how I am saving float32 predictions to disk for reading in java:

marray.byteswap().tofile(outfile)

where

marray = numpy.lib.format.open_memmap(npoutfile, mode='w+', shape=(N_PICS*N_PICS,), \
            dtype='float32')

My understanding is that whether java/python are different will depend on the machine architecture. Does that clear up my question?

If we can’t use onheap data structures, why does INDArray allow a heap array as input?

INDArray aaa = Nd4j.create(data2d);

I’m just doing one prediction til I get understanding. I’m suspecting ND4J might be handling endianness ‘under the hood’.

More:

The x86 architecture, including x86-64, uses little-endian byte ordering, meaning the least significant byte of a multi-byte value is stored at the lowest memory address.

And Java is big-endian, but:

nativeOrder

public static [ByteOrder](https://docs.oracle.com/javase/8/docs/api/java/nio/ByteOrder.html) nativeOrder()

Retrieves the native byte order of the underlying platform.

This method is defined so that performance-sensitive Java code can allocate direct buffers with the same byte order as the hardware. Native code libraries are often more efficient when such buffers are used.

I think interoperability should be tested on machines with both native endiannesses, in case you have only been testing on a machine where native order matches java.

Here is my pre-Nd4j data structure, I suppose it’s faster to build the INDArray array a number at a time:

double[] data = new double[DATA_SZ];

double data2d[][] = new double[1][];
data2d[0] = data;
// pack in values aaand
INDArray aaa = Nd4j.create(data2d);

As a designer, I’d be tempted to add a toString-like method that inspected the native byte order and returned a description of what to do and how it’s handled. I can’t count how many times I (and the mailing list) answered this kind of question when supporting molecular simulation code, once upon a time. :slight_smile: