I need to implement my own BP algorithm for the Op CompareAndSet. I did this in the same way as I did for Gather Op, but it didn’t work. So I’ve been struggling with this problem for like 3 hours and I had to give up. Unlike Gather Op, CompareAndSet is a base Op and it’s inside the SameDiff'sops field. I tried all the possible ways of overriding it, including replacing this Op with my custom one directly in SameDiff etc. but I’m stuck in FlatBuffersMapper#fromFlatNode because it creates this Op instance based on a hardcoded numerical value of OpType thus bringing the original version to life. That one is incorrectly cloned (I have no idea why cloning is needed) and some props are missing so the execution fails. Overriding the props setup method doesn’t help because the original version is hardcoded in LegacyOpMapper. Changing this type to a CUSTOM_OP in my custom class also doesn’t help because the execution fails on another part then (which makes sense because this Op is the part of the SameDiff Ops). Could someone please help me here?
P.S. It would be really great to have some flexibility to doing this stuff for all Ops, not only the custom ones. Like the way it works for Gather Op.
@partarstu all you need to do is override the doDiff(…) normally.
Most of the time we implement a _bp op in c++ to handle this if possible though.
That involves adding a _bp kernel in c++ and the associated java class to handle delegation to the underlying kernel.
In your case though, just overriding doDiff and in CompareAndSet should be enough.
That or you could also just add your own op.
Regarding the flexibility, adding your own op is very doable. The main problem is integrating it with the rest of samediff where we also have the codegen and the like. File an issue and we can discuss it over there. As usual, some expectations would be nice.
My main issue is I’m not sure what you’d expect when it comes to all the custom steps.
My initial hunch here would be to have a registry where we lookup custom ops by the user with annotations. I already implemented something like that for our model import. I don’t see why I couldn’t do the same thing here.
Did that from the very beginning, it doesn’t work. I forgot to mention that my custom Op extends CompareAndSet, because I don’t need a new Op, I just want to modify the BP, not the whole Op. I thought that adding the instance of my custom Op to the ImportClassMapping should do the job (it does for the case of my custom implementation of the Gather Op BP), but unlike Gather Op, CompareAndSet is ignored if it’s added to the ImportClassMapping. The core issue is cloning/creating a new instance from scratch instead of first looking up any class mapping. I think it has something to with the OpType - Gather Op is CUSTOM, CompareAndSet - TRANSFORM
Will do that today. Having an easy way to integrate the custom Ops would definitely be a great feature to the whole DL4J.
The idea is to bring something that could be specific to a project but not required to be present for the whole framework. In my case it’s only the BP part. Actually the original one is also implemented directly in Java, so I don’t think it’s an issue. But if any BP algorithm or the Op execution needs to be a little tuned/adjusted for specific case purposes, it would be great to have the ability to do this without a huge effort. Right now it’s not taken into account architecturally (e.g. private fields in all Ops instead of protected etc.). Regarding c++ implementation - tried that once, spent the whole day on the env setup and still failed to reach the normal setup.
It would be a great solution. Because the current workaround of modifying ImportClassMappingOP_NAME_MAP and replacing the Op instance with your own custom override having a custom BP is definitely not working for all Ops (otherwise I wouldn’t raise this topic). Would be nice to have a normal quick way to add custom Ops (at least with BP overrides) to adjust the graph to specific project needs.
@agibsonccc , until this issue is fixed - is there any alternative to using CompareAndSet ? Otherwise I’m kind of blocked, because I haven’t found in SameDiff any activation function which gives me the behavior I need (binary activation based on the threshold)
@agibsonccc , I’ve taken a look at your PR and it seems fine at first sight. I’m however a little confused regarding all the overrides, specifically exec() method. Does it mean in my case using INDArray.replaceWhere() directly inside the exec() method and writing my own BP in doDiff() ?
On the other hand, since I’ve not been using SNAPSHOT for a long time now (I always had some issues with it), right now I get exception for each constant/variable introduction into the graph. Never seen it in M 2.1. Is this a known issue? Without having the SNAPSHOT running I can’t test your changes.
The stack trace (I’m running in Windows 11 with Java 19): Exception in thread "main" java.lang.UnsatisfiedLinkError: 'void org.nd4j.linalg.cpu.nativecpu.bindings.Nd4jCpu.setShapeBuffer(org.bytedeco.javacpp.LongPointer, int, org.bytedeco.javacpp.LongPointer, char, int, boolean)' at org.nd4j.linalg.cpu.nativecpu.bindings.Nd4jCpu.setShapeBuffer(Native Method) at org.nd4j.linalg.cpu.nativecpu.ops.NativeOpExecutioner.createShapeInfo(NativeOpExecutioner.java:1800) at org.nd4j.linalg.api.shape.Shape.createShapeInformation(Shape.java:3262) at org.nd4j.linalg.api.ndarray.BaseShapeInfoProvider.createShapeInformation(BaseShapeInfoProvider.java:68) at org.nd4j.linalg.api.ndarray.BaseNDArray.<init>(BaseNDArray.java:169) at org.nd4j.linalg.api.ndarray.BaseNDArray.<init>(BaseNDArray.java:310) at org.nd4j.linalg.cpu.nativecpu.NDArray.<init>(NDArray.java:170) at org.nd4j.linalg.cpu.nativecpu.CpuNDArrayFactory.createUninitialized(CpuNDArrayFactory.java:206) at org.nd4j.linalg.factory.Nd4j.createUninitialized(Nd4j.java:4388) at org.nd4j.linalg.factory.BaseNDArrayFactory.valueArrayOf(BaseNDArrayFactory.java:792) at org.nd4j.linalg.factory.Nd4j.valueArrayOf(Nd4j.java:4503)
@partarstu yes a UDF will override exec and exec(OpContext) (used when the op doesn’t have the input arrays associated with it)
The extra stuff is just for serialization. Ops often have properties that need to be propagated to the underlying arguments that actually get passed to c++. You can see these here:
That is stuff like the tArguments, iArguments, …
Those can either be on the op or op context. Often times ops that are implemented in c++ have an associated op class with it. For ease of use those have arguments as part of the constructor that then get passed down to the base classe’s lists.
@partarstu no I’m pretty sure this is recent. I did see someone else have issues with snapshots I’m just not sure what the source of this is. If you could, could you get set org.bytedeco.javacpp.logger.debug=true
and give me the logs for that?
Seems like there is something missing. I tried changing the version of the native backend from windows-x86_64-onednn-avx512 to the generic windows-x86_64, but it didn’t help. I have no idea if it’s the issue of javacpp-1.5.8-windows-x86_64, or of nd4j package