Custom SameDiff ops missing from graph after upgrade to M2

SameDiff
1

Hi. I’m using a custom SameDiff Gather Op which extends a standard (native) one by overriding the doDiff() method. The rest remains the same, including the Op’s name. It worked all the time with different DL4J versions but after upgrading to M2 this Op is being ignored and automatically replaced by the standard one. After debugging the code I noticed that it happens because of always cloning the Ops (not sure why it happens all the time, even when there’s no importing etc.) in org.nd4j.autodiff.samediff.serde.FlatBuffersMapper#cloneViaSerialize(). Manipulations with ImportClassMapping.getOpNameMapping() by replacing the standard op with the custom one do the trick, but only when I train the model from scratch. If I load an existing same model (as usually) in order to copy the weights from it (the graph itself is not changed during this copying), somehow my custom Op, still remaining in ImportClassMapping.getOpNameMapping() map, is again being ignored.

Would appreciate any tips on how to correctly configure my custom Op so that it doesn’t get kicked off and replaced by the standard one.

Thanks in advance!

2

@partarstu could you file an issue with more info? We’re going to do a follow up release in the next week or so to address some issues with M2. Anything you can tell me would be great. I’d appreciate a stack trace.
IMportClassMapping isn’t relevant anymore. It was relevant only for the old import.

The serialization I’d be curious to see a reproducer for though.

3

@agibsonccc I’ve filed the issue here: Custom SameDiff ops missing from graph after upgrade to M2 · Issue #9670 · eclipse/deeplearning4j · GitHub .

I couldn’t add much more interesting info though than it was before. There’s no stack trace or error. The core idea is that my custom Op which used to work for a couple of different releases is now being replaced by the native one and the new cloning mechanism is the root cause. There’s no import of any Op in my case. If current logic is correct then there’s some Op registration mechanism missing for any custom implementation which doesn’t require libnd4j changes and is thus not part of the native Ops.