Next Snapshot or release

DL4J
1

Hi,

Can anybody give a short info if there are any near snapshot updates or milestone releases?

Best regards

Thomas

Tried also to build from source on windows with the msys shell but got an link error:

/sources/deeplearning4j/libnd4j/include/legacy/NativeOps.h:1077:27: error: external linkage required for symbol ‘long int _numpyHeaderLengthWordSize(sd::Pointer, long int)’ because of ‘dllexport’ attribute
1077 | SD_LIB_EXPORT static long _numpyHeaderLengthWordSize(sd::Pointer shapeBuffer,long wordSize) {
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
C:/msys64/home/mrrobot/sources/deeplearning4j/libnd4j/include/legacy/NativeOps.h:1095:27: error: external linkage required for symbol ‘long int numpyHeaderLengthWordSize(sd::Pointer, long int)’ because of ‘dllexport’ attribute
1095 | SD_LIB_EXPORT static long numpyHeaderLengthWordSize(sd::Pointer shapeBuffer,long wordSize) {

2

@thomas I just published a build a bit ago.

I’m currently troubleshooting a paid customer issue atm so I haven’t been able to get to the CI yet. I’ve done a significant amount of the release work (run tests, double check everything etc) but am currently blocked.

I want to get this cleaned up this weekend. Either way release should be going out soon.

3

Thanks for the quick answer and your hard work.

4

@thomas I published the windows build. Let me know if you have any issues.

5

Same error after git pull. But i can wait for official release if you are on target.

Best regards

Thomas

6

@thomas You might want to check with @partarstu - he had success with it.

7

@thomas - my scope is to simply get the newest SNAPSHOT JAR in order to proceed with my custom Op implementation. The newest snapshot version has the merged PR changes which allow me to do it, but this version doesn’t have the Windows backend (Android, Linux and Mac are available), so you can’t run it on Windows. The corresponding jobs in GitLab Actions fail so no updated Windows artifacts are published. We just need to patiently wait until @agibsonccc is done with the current blockers and will have some time to fix the pipelines in order to get new Windows artifacts.

8

@partarstu oh sorry I misunderstood you! I kicked off a windows build already. The build here actually worked: .github/workflows/build-deploy-windows.yml · deeplearning4j/deeplearning4j@21d0372 · GitHub

It just had an old tmate plugin that was stalling the build. I think in some new version (I hadn’t changed that in a while) some sort of logic change was forcing it to kick in which actually just stalled the build after it completed. I removed that.

You can see from the job that it manages to upload the jars.

I’m trying to figure out if after the snapshots are updated you guys still have that problem.

9

@agibsonccc , actually I’m sorry - seems like my browser hasn’t correctly refreshed the snapshots page so I didn’t see those artifacts, but now I double-checked it and they are there! Thanks a lot !!! I’m currently re-loading all the maven dependencies (in order to avoid wrong sources versions - had those issues a couple of times while using snapshots) and after it’s done I’ll let you know if I managed to start the engine

10

@agibsonccc , unfortunately it’s not starting - the problem is with this part of new code:

List<Class<?>> classModules = AnnotationDetector.scanClassPath(ND4JClassLoading.getNd4jClassloader())
                    .forAnnotations(UserDefinedOp.class)  // one or more annotations
                    .on(ElementType.TYPE) // optional, default ElementType.TYPE. One ore more element types
                    .collect(AnnotationDefaults.getType);

Caused by: java.lang.ExceptionInInitializerError: Exception kotlin.jvm.KotlinReflectionNotSupportedError: Kotlin reflection implementation is not found at runtime. Make sure you have kotlin-reflect.jar in the classpath [in thread "main"]
	at kotlin.jvm.internal.ClassReference.error(ClassReference.kt:45)
	at kotlin.jvm.internal.ClassReference.getQualifiedName(ClassReference.kt:28)
	at dorkbox.updates.Updates.<clinit>(Updates.kt:25)
	at dorkbox.annotation.AnnotationDetector.<clinit>(AnnotationDetector.java:155)
	at java.base/java.lang.Class.forName0(Native Method)
	at java.base/java.lang.Class.forName(Class.java:495)
	at java.base/java.lang.Class.forName(Class.java:474)

But one very important fact is that the previous issue with javacpp UnsatisfiedLinkError seems to be gone now! :partying_face:

11

@partarstu oh interesting! Let me add that in my current working branch here. That never came up in my tests. I"m guessing due to the unit test classpath or intelliij having something already. Thanks for catching that!

1 Like
12

hi, @partarstu i tried to check the sonatpye repository, which repository do you checked in your browser. i still cannot see the updates. or do you download from the links inside the build logs?

EDIT: Ah think i found it. Didn’t used the s01 only oss.sonatype.org.

13

Hi @thomas , the back-end should be here : https://s01.oss.sonatype.org/content/repositories/snapshots/org/nd4j/nd4j-native/1.0.0-SNAPSHOT/

14

@agibsonccc , adding kotlin-reflect dependency in my POM resolved the issue. I’m now able to start the engine! Thanks for building the working snapshot version!

Also javacpp UnsatisfiedLinkError is not present any more!

15

@partarstu that current PR has merged. I added kotlin-reflect to model import for you. I"m kicking the snapshots build off now.

16

@agibsonccc , thanks a lot! I’ll give it a try tomorrow. For now I’m able to work further and already using my UDF!

17

@agibsonccc sorry, got to it just now. Kotlin problem persists, the same error.