Attention Layer

DL4J
1

Team,

Is there any example available in DL4J for Attention Layer (SelfAttentionLayer class)?

Regards

2 Likes
2

I would also be very much interested in such an example.

2 Likes
3

At the moment there are no examples that use it.

4

Would als be interested in seeing Attention NNs working with DL4J, especially given their recent popularity.

1 Like
5

Yea, I’d like to see an example also, especially how to make the example of the paper ‘Attention is All You Need’. imo, it would be very to see how to implement this model: https://ricardokleinklein.github.io/images/transformer/transformer.png .

1 Like