Global vs Local
Scoring Functions
Dot
score(s, h) = sᵀh
Efficient directly identifies alignment.
General
score(s, h) = sᵀWₐh
Learns a mapping between spaces.
Concat
vₐᵀ tanh(Wₐ[s;h])
Multilayer Perceptron alignment.
Play
Next
Reset
Global
Local
Window (D):
2
Global attention
: all source rows in
H
receive non-zero weight.
Local mode restricts support to a window around predicted position p_t (radius D).