Error running dl4j library in android studio

OK,thanks. Thats what
For saving the models, am I supposed to save it in a json file? or a txt file?(I saved it as a txt file and it came out as 60 mb!)

*Thats what I did I was just not sure I did it correctly

‪On Thu, May 26, 2022 at 2:01 PM ‫חגי השינג‬‎ <ciheching@gmail.com> wrote:‬

@ciheching you don’t have to think about that. We have save methods on the models and a ModelSerializer class for that if you would like to have a bit more control. That format is a zip file.

I think I’m doing something wrong with the loading and saving of the models because it’s taking lots of time for each save/load (15min+)
Maybe because its getting encrypted?
the txt file looks like this(not the entire file just a small part):
PK o �T configuration.json�X[o�6 ~ϯ0��
��t����b�U �E
E@�� � H�� ��#� �n3lC����9G �s�>�_Nz=o���\�|�) ��{ϋ b � w�� �r\���hR�9[Ic��}�χ#� #� �pv5\� �y��x����++Q �"�u< } a ���GB�� �H} �S@� ���Ϙo6����?�l�ia�g 鴆A�" �f�\��
k\J ��߾%}�Ω?l s��k�5� }4 ��Ա2Hu� b���<�L��8ŵ�r=4 _ $���Y��;�w��|�/ �K �X %�b�TY���M� �4� "L���m �j/D� ���P`Ɔ4�M��| N�w�0^L�E’��m�(5�h•(`7�bk�l0�{� BB�� ��[�׉ �U+ zp� �>4�T�N��Iq* &�Td�H�� ��8sR�7n�V�E͐�е��`=��P����-�m= v6H�G ��۬KPhPvԛG�2�q �KB˳�L^������=�&� :m| � ��g(K �� � v~#�.;�>�� kx�tk�� �i� �z %z���w� � E©��S3� �ς�.���`�R�W��TqA� �G�q�����7� 63���W� �C��g�;�։ЃO0� E�0&&`Y(�-}�4� H�)31_��.[��J���x�6D7w�c�Ȉ � � a 7*�3� �+$ٴ ��y E �@�v����U7:f�+c�85�x��jt�Ym:��s> O’��n<�G���9 P zpV[ �&ͺ�� �r
�� ��x����y� '�t?����R"-���Xn7�5�t� Z`�Fx��dC$��� � �� juL��J�&�.��� �
�A!x:� �= �ǜ�9}aL�z:�i����� �9’� �10 ���{�)��k��� E���r��} ��PA��wv�\���y���\�n_�l/w� & W�� �_kD�W���3!�gG�gm )��)]c�WՐU}3Ӧ��/�x:#�@I�9v�-˖ Q�O������o%$�d�2r��EE^T��T�� ��b�Nx��@ K���{��� �I4��M�^ �9�Z��� H� .��y;���b� �k, �؂�`�f� v jybu���n����x�׻� �U�����(CL
۝ % <�T=�G=� ��Z�J]�� ��B�p�� #<��_ɰ�Fb�ޑב%��N � PK @وs � PK o �T coefficients.binl�
XLy�� <�0 ! B �Vs� °a B $! �A� �H� B2 �0z�sOwM�a�ifXlh���v7֮����~ ��w]���ﺦ33�9�s���y�_ ̛ 4� ����ik ��g.f�Ϗ �>`��������n����۝�_���V�� �_��� ��#t� �`Z E��� 9�� ��F�H� ��?� �ߣ����w�A� f�1���od�T���� ��=�o��s’ ��ɾ?�J� �J��
��Y�oV�b���CJ�j l�� ��#�Wb�׽���-y[֋��%���w�+�t�o��_}c 7P��}iO ���ƒ�n�hP����ǜ�64~)] ��7�JNw��C}K Z� �
����Y8s��t���%Ϋ||z �J e��f -�[y� � � �V��Z�-i� �){N��c&�� ��� ����� ��#� ��s��~���9����C_�0��~��]�gr��fO�R�Ii��=��� ��D�� xӡtĐ�� >�/��F �u9�{��a e�}O o�JW.�����}���Ïv��Wݦ�ɾ�.����Z +��� z ��i ��f� �
9UB�����Z �P5�-

‪On Thu, May 26, 2022 at 2:02 PM ‫חגי השינג‬‎ <ciheching@gmail.com> wrote:‬

my saving code is this

nn.save(file, true);

@ciheching it’s a zip file of course it’s going to be bigger and not readable in plain text. Please don’t paste binary blobs directly in the message. It’s just a zip file. Treat it as such. If you want to explore it, open it using a file viewer. Make sure if you are using an app to open it that you have a proper .zip extension on the file.

Is there a way for me to make it load the nn faster because its taking lots of time to load 10+ min?

right now its already going for 25mins will switching the code to ModelSerializer help with the problem?

‪On Thu, May 26, 2022 at 7:07 PM ‫חגי השינג‬‎ <ciheching@gmail.com> wrote:‬

@ciheching you mean the phone is taking a while? Could you elaborate on how you’re timing it? Like are you loading it an app? Models don’t take that long to load unless you’re running up against memory constraints. Java will tend to be slowed down by the garbage collection when your memory is right up against the limits imposed by the runtime.

I have an emulator of a phone and when it reaches the line of loading the NN for predicting/training it takes a very long time you think i need to give the emulator more memory?

so do you know what might be the problem?

‪On Fri, May 27, 2022 at 1:50 PM ‫חגי השינג‬‎ <ciheching@gmail.com> wrote:‬

@ciheching you’re not giving me much to go off of but of course a system with less resources while running ML workloads will not be the fast. I need more specifics to go off of.

im not sure what kind of specifics your looking for i set the android emulator on 4gb RAM it took it around 10 mins to run in the debug (the model.save)
does that make sense?

@ciheching debuggers are known to slow things down. I can’t comment on the overhead of the debugger. What does it do when it just runs without that?

I know for a fact that saving models can be disk intensive as well as memory intensive so if you are running up against the memory of the emulator that would explain the behavior.

without the debugger i get a jank error I think the solution to that is to that is to add it on another thread

@ciheching “jank error” doesn’t tell me anything and only running it in the debugger is not a workaround. If you intend to save models in your application you should understand the problem or just not do it.

well i put it on a separate thread and it seems to be working but im not sure will update
thanks for the help

@ciheching yeah just make sure to actually have it work for real. Workarounds like that don’t really help us fix your problem. Sorry I know some of this stuff can be annoying to work with.

Just an update so that you know if someone else ever asks:
In android studio many times it wont run the model if your on the main UI thread and for that reason you need to create another thread and run the training/predicting on another thread
In addition to that training and opening the model and saving it is around 20 times slower or more in debugging mode
Thanks for all the help. If I need anything else I’ll let you know :slight_smile:

I have many images in a directory (one directory for each letter/number) and all these directories are in a directory called TrainData and i did this
but i think its not working correctly the training

MultiLayerNetwork nn;
  File file = new File("/data/data/com.example.train/files/TrainData");
   File file1 = new File("/data/data/com.example.train/files/Text1Mode.bin");
   Random random = new Random();
   //we create a custom datasets so let's read the happy and sad images

   //we shuffle the images at random
   FileSplit train = new FileSplit(file, NativeImageLoader.ALLOWED_FORMATS, random);

   ParentPathLabelGenerator labelMaker = new ParentPathLabelGenerator();
   ImageRecordReader recordReader = new ImageRecordReader(28,28, 1, labelMaker);
   recordReader.initialize(train);
   //we want to end up with a DataSetIterator (as the container for the images)
   DataSetIterator dataIter = new RecordReaderDataSetIterator(recordReader, 100, 1,36);

   //normalize the dataset [minRange=0,maxRange=1]
   DataNormalization scaler = new ImagePreProcessingScaler(1, 0);
   scaler.fit(dataIter);
   dataIter.setPreProcessor(scaler);

   //check the images and labels accordingly
   DataSet ds = dataIter.next();
   System.out.println(ds);
   System.out.println(dataIter.getLabels());

....network.....