"Attention Is All You Need" model implementation using dl4j

[adonnini]

@agibsonccc would the suggestion you made in

Reduce rank and set values according to 3rd dimension

[ [[1,0,0,0],[0,1,0,0],[0,0,1,0]], [[1,0,0,0],[0,1,0,0],[0,0,1,0]] ] How to get below result from above SDVariable? Reduce 3rd dimension, and the value = 0.1 when 1 exists in the first or 2rd position, otherwise value = 1. [ [0.1,0.1,1], [0.1,0.1,1] ]

@agibsonccc

is this for weighted loss? If not why not make that a placeholder?

apply in my case? May I ask what you meant by “make that a placeholder”? I am not sure I understand.

Thanks