Hi
Just getting to grips with dl4j, I was wondering if dl4j supports neural networks using int8 for layers,weights etc, instead of fp16 or fp32? I have a pascal gpu which is useless for fp16 but can do int8 with a 4x speedup over fp32 but apart from an int8 compressor in the source I could not see if dl4j has options to use integer ops for the layers such as 1dconvolution, maxpooling etc.
Thanks in advance.
You typically don’t want to build models by default with int8. Int8 is typically used for inference. Dl4j does have the int8 data type but I wouldn’t recommend trying to use it for training.
yeah , I thought that might be the case, a shame really, looks like I will have to save up for a Turing GPU. Some of the benchmarks for int8 performance are insane, 120 TeraOPS per card, even if the memory bandwidth is the limiting factor in the real world.
I can’t in principle see why since byte values could not be used for everything, there is enough precision with 256 discrete values to capture most patterns, there are some research papers showing that even fp16 is overkill for most problems as the extra precision makes no major difference to the overall performance of the network. I’m guessing that the backprop,activation functions, convolution kernels etc. might need to be reworked to actually train using int8.