Hello guys, do you have/know of an implementation in DL4J of the Uncertainty Aware Attention model in DL4J?
See: https://arxiv.org/pdf/1805.09653.pdf
Couldn’t find much googling it. There is an implementation in TensorFlow.
Thanks,
Juan
For future replies (sorry for the late reply @jfarjona ) there’s always new papers coming out every day.
It doesn’t make sense for us to try to keep up with all of that. The reality is, we need to bootstrap a lot of this innovation from other frameworks. We already have the tools necessary to run these models via model import and many ops from these frameworks tend to reuse the same ops that already exist in the core framework.
The dl4j and samediff suite of tools are just that: a tool. The focus is more on providing the tools to build these things (eg: providing fast math) , not to add every new paper that comes out.
Not all new papers get added to the tensorflow or pytorch core codebases: they are usually just examples or community repositories published by independent researchers. These things are not core contributions to the actual framework itself.
That being said, people use a tool because it solves a problem they have. Typically that comes with building tools on top of the framework itself. In order to achieve that, and actually keep up with the innovation, it should be easy for any user to go to a github repository and map whatever community repository they’re looking at.
These frameworks all have the same ops and do the same thing, maybe just with a slightly different API. Even then these frameworks also tend to have the same APIs.
What we’ll be doing is adding code generation to the framework that can look at these github repositories and pull out the necessary implementations of these things enabling a form of “auto porting” of various research repositories to the framework.