I sent you a sample .csv data file via email
The email message didn’t work. Do you have an email address I can send it to?
@agibsonccc Happy holidays! send me an email address I can send the sample data file to when you get a chance. Thanks
@agibsonccc ND4J code in my Android application which used to run without problems is now failing to complete execution posting the following error log
03-03 05:51:37.096: E/NeuralNetworkService(10709): java.lang.RuntimeException: No serializer strategy found for given header NormalizerSerializer.Header(normalizerType=STANDARDIZE, customStrategyClass=null)
03-03 05:51:37.096: E/NeuralNetworkService(10709): at org.nd4j.linalg.dataset.api.preprocessor.serializer.NormalizerSerializer.getStrategy(NormalizerSerializer.java:181)
03-03 05:51:37.096: E/NeuralNetworkService(10709): at org.nd4j.linalg.dataset.api.preprocessor.serializer.NormalizerSerializer.restore(NormalizerSerializer.java:117)
03-03 05:51:37.096: E/NeuralNetworkService(10709): at com.android.contextq.neuralNetwork.NeuralNetworkService$Dl4jModelCache.getOrLoad(NeuralNetworkService.java:394)
03-03 05:51:37.096: E/NeuralNetworkService(10709): at com.android.contextq.neuralNetwork.NeuralNetworkService.neuralNetworkloadAndRun(NeuralNetworkService.java:2003)
03-03 05:51:37.096: E/NeuralNetworkService(10709): at com.android.contextq.neuralNetwork.NeuralNetworkService$NeuralNetworkServiceRunnable.run(NeuralNetworkService.java:9830)
03-03 05:51:37.096: E/NeuralNetworkService(10709): at java.lang.Thread.run(Thread.java:1564)
03-03 05:51:37.096: E/NeuralNetworkService(10709): - Dl4jModelCache - model restore failed
03-03 05:51:37.096: E/NeuralNetworkService(10709): java.lang.RuntimeException: No serializer strategy found for given header NormalizerSerializer.Header(normalizerType=STANDARDIZE, customStrategyClass=null)
03-03 05:51:37.096: E/NeuralNetworkService(10709): at org.nd4j.linalg.dataset.api.preprocessor.serializer.NormalizerSerializer.getStrategy(NormalizerSerializer.java:181)
03-03 05:51:37.096: E/NeuralNetworkService(10709): at org.nd4j.linalg.dataset.api.preprocessor.serializer.NormalizerSerializer.restore(NormalizerSerializer.java:117)
03-03 05:51:37.096: E/NeuralNetworkService(10709): at com.android.contextq.neuralNetwork.NeuralNetworkService$Dl4jModelCache.getOrLoad(NeuralNetworkService.java:394)
03-03 05:51:37.096: E/NeuralNetworkService(10709): at com.android.contextq.neuralNetwork.NeuralNetworkService.neuralNetworkloadAndRun(NeuralNetworkService.java:2003)
03-03 05:51:37.096: E/NeuralNetworkService(10709): at com.android.contextq.neuralNetwork.NeuralNetworkService$NeuralNetworkServiceRunnable.run(NeuralNetworkService.java:9830)
03-03 05:51:37.096: E/NeuralNetworkService(10709): at java.lang.Thread.run(Thread.java:1564)
```
Is this the "same" error fixed by the most recent snapshot?
Is this the “same” error fixed by the most recent snapshot?
My dependencies in build.gradle are:
def dl4jVersion = '1.0.0-M2.1'
def openblasVersion = '0.3.21-1.5.8'
implementation (group: 'org.deeplearning4j', name: 'deeplearning4j-nn', version: dl4jVersion)
implementation (group: 'org.nd4j', name: 'nd4j-native', version: dl4jVersion)
implementation (group: 'org.nd4j', name: 'nd4j-native', version: dl4jVersion, classifier: "android-arm64")
implementation (group: 'org.bytedeco', name: 'openblas', version: openblasVersion)
implementation (group: 'org.bytedeco', name: 'openblas', version: openblasVersion, classifier: "android-arm64")
Will running ``mvn -U``` resolve this issue?
Thanks
@adonnini you need to start with org.eclipse.deeplearning4j if you’re using snapshots. I mentioned this before the snapshots use a new namespace. You’ll also need to update the openblas versions.
Thanks @agibsonccc
I am confused. The dependencies have been like the ones I listed in my message above for more than three years and have always worked.
Could you be a little more specific as to the changes I need to make?
Is there an alternative to using snapshot?
Update openlabs versions to which version?
Use org.eclipse.deeplearning4j as the group entires in the dependencies?
Thanks
@adonnini could you clarify what you want exactly? I thought you were trying to use snapshots? You have to change the versions if you want to use those. If you change the versions back, you can always use those again. When I publish version changes I will have to change the version as well.
mvn -U as we discussed in the email does 1 thing: forcibly redownloads versions of snapshots
there were bugs in the normalizer there is also the new model serialization format. You have to use snapshots in order to even load your model.
@agibsonccc I am sorry to be a nuisance. I need to resolve this issue urgently as it affects my production application.
I still do not understand what I need to change in my set-up, build.gradle and import statements, to resolve the problem.
Could you please help me out by telling me specifically what I need to change and how in order to fix the bug?
I am also concerned by this “You have to use snapshots in order to even load your model.”. The trained model I use is produced with code I have been using for years and I did not make any changes to that code for years either. It does use snapshots.
Thank you
@adonnini Let’s establish a few things:
-
Software with bug fixes means version changes. You used snapshots a few months back and noticed a few issues. That was with your normalization pipeline. You since sent me an email with all the data.
-
I am sorry but I am really not sure where this is coming from.To my understanding, it was physically impossible for you to load your newer model without using snapshots because it surpassed the 2g in size flatbuffers (the original file format) supported. The snapshots switches to a new but similar format that allows loading bigger models. I was under the assumption that you still had that code laying around as it was. Those “snapshots” involved the new namespace.
-
I need to clarify what we are talking about here. I really need to understand where the disconnect is. You mentioned using snapshots above in your statement. What you gave me just now was the previous stable version. That is not snapshots.
-
You are mentioning a production application and not needing to change code. The only thing I can try to guess here is your production application and training code are separate where training uses snapshots and your stable application does not. If you are already using snapshots then you should be able to copy that config I thought?
Please try to answer these questions first. I am sorry I can’t read your mind and only have scant recollections of what has been a very long conversation here. Help me understand the gap in understanding here and I will do my best to help you. I am going about it this way because if you already have a local setup that works with snapshots we can try to start from a frame of reference you already have.
@agibsonccc I apologize for the confusion. I mixed things up.
I have two projects where I use models built with dl4j/samediff:
A) The project where I am building my model. This thread is about problems with this project
B) A production Android application where I only run models for inference. The models I run for inference are older models which I created using the older dl4j framework not samediff. Until recently, I was able to run older models for inference in my Android application without problems. Now execution fails with a normalizer failure (see error log below).
I made two (main) mistakes:
-
I used this thread to report the normalizer failure in my Android application. I thought this would be OK as the failure was a normalizer failure which is something discussed in this thread
-
I mistakenly told you that in my Android application I use snapshots. As you correctly pointed out in my Android application I use stable versions
Below, to try and make your life easier I am reporting again the error log from my Android application and the build.gradle dependencies.
Resolving the normalizer problem in my Android application is a higher priority than resolving issues in the model I am building (project A))
Should I open a new thread for the normalizer error in my Android application?
Please let me know if you need anything in order to take the next step.
Sorry about all this confusion.
DEPENDENCIES
def dl4jVersion = ‘1.0.0-M2.1’
def openblasVersion = ‘0.3.21-1.5.8’
implementation (group: ‘org.deeplearning4j’, name: ‘deeplearning4j-nn’, version: dl4jVersion)
implementation (group: ‘org.nd4j’, name: ‘nd4j-native’, version: dl4jVersion)
implementation (group: ‘org.nd4j’, name: ‘nd4j-native’, version: dl4jVersion, classifier: “android-arm64”)
implementation (group: ‘org.bytedeco’, name: ‘openblas’, version: openblasVersion)
implementation (group: ‘org.bytedeco’, name: ‘openblas’, version: openblasVersion, classifier: “android-arm64”)
ERROR LOG
03-03 05:51:37.096: E/NeuralNetworkService(10709): java.lang.RuntimeException: No serializer strategy found for given header NormalizerSerializer.Header(normalizerType=STANDARDIZE, customStrategyClass=null)
03-03 05:51:37.096: E/NeuralNetworkService(10709): at org.nd4j.linalg.dataset.api.preprocessor.serializer.NormalizerSerializer.getStrategy(NormalizerSerializer.java:181)
03-03 05:51:37.096: E/NeuralNetworkService(10709): at org.nd4j.linalg.dataset.api.preprocessor.serializer.NormalizerSerializer.restore(NormalizerSerializer.java:117)
03-03 05:51:37.096: E/NeuralNetworkService(10709): at com.android.contextq.neuralNetwork.NeuralNetworkService$Dl4jModelCache.getOrLoad(NeuralNetworkService.java:394)
03-03 05:51:37.096: E/NeuralNetworkService(10709): at com.android.contextq.neuralNetwork.NeuralNetworkService.neuralNetworkloadAndRun(NeuralNetworkService.java:2003)
03-03 05:51:37.096: E/NeuralNetworkService(10709): at com.android.contextq.neuralNetwork.NeuralNetworkService$NeuralNetworkServiceRunnable.run(NeuralNetworkService.java:9830)
03-03 05:51:37.096: E/NeuralNetworkService(10709): at java.lang.Thread.run(Thread.java:1564)
03-03 05:51:37.096: E/NeuralNetworkService(10709): - Dl4jModelCache - model restore failed
03-03 05:51:37.096: E/NeuralNetworkService(10709): java.lang.RuntimeException: No serializer strategy found for given header NormalizerSerializer.Header(normalizerType=STANDARDIZE, customStrategyClass=null)
03-03 05:51:37.096: E/NeuralNetworkService(10709): at org.nd4j.linalg.dataset.api.preprocessor.serializer.NormalizerSerializer.getStrategy(NormalizerSerializer.java:181)
03-03 05:51:37.096: E/NeuralNetworkService(10709): at org.nd4j.linalg.dataset.api.preprocessor.serializer.NormalizerSerializer.restore(NormalizerSerializer.java:117)
03-03 05:51:37.096: E/NeuralNetworkService(10709): at com.android.contextq.neuralNetwork.NeuralNetworkService$Dl4jModelCache.getOrLoad(NeuralNetworkService.java:394)
03-03 05:51:37.096: E/NeuralNetworkService(10709): at com.android.contextq.neuralNetwork.NeuralNetworkService.neuralNetworkloadAndRun(NeuralNetworkService.java:2003)
03-03 05:51:37.096: E/NeuralNetworkService(10709): at com.android.contextq.neuralNetwork.NeuralNetworkService$NeuralNetworkServiceRunnable.run(NeuralNetworkService.java:9830)
03-03 05:51:37.096: E/NeuralNetworkService(10709): at java.lang.Thread.run(Thread.java:1564)
@agibsonccc Android application normalizer problem solved!!! I inadvertently made a one line code change.
Again, very sorry about all this fire drill
@adonnini great. Please end this thread. Congrats. I will publish a stable version soon. I’m just trying to get the final parts of the rewrite in a good place performance and testing wise.
This thread should not be closed.
What I should end is the “interference” of issues regarding my Android application. I will not mention it any longer.
The issue I opened this thread for is still very much unsolved.
Its solution was put on hold because of the flatbuffers issue which you solved.
Then, I ran into the normalizer bug which you fixed recently.
When you fixed it, you told me to simply run mvn -U in order to get the snapshot with the normalizer bug fix.
As you know, I am not familiar with maven. I did not realize you did not mean simply running mvn -U
I found out that simply running mvn -U does not work.
After doing some seach online, I switched to my project’s directory and ran
mvn clean install -U
That seemed to run successfully. Was this the right command?
If it is, I will run my model and see what happens.
Thanks
Sorry yes -U is a flag it’s not a command. Let’s get back to what you were trying to do. Yes you need to install and then everything will be there.