Build the original Transformer

Assemble one encoder layer and one decoder layer from “Attention Is All You Need” (2017). The complete model repeats each layer N times.
query key value probability score
A guided diagram of the original encoder-decoder Transformer Source tokens move up the encoder stack on the left. Shifted target tokens move up the decoder stack on the right. Residual paths bypass every sublayer, and decoder cross-attention reads keys and values from the encoder output. ENCODER STACK · N× DECODER STACK · N× source tokens The cat sat down embedding + position shifted target tokens 〈BOS〉 Le chat s'est embedding + position whole token rows enter together multi-head self-attention Q · K · V all from encoder input Add & Norm + residual position-wise feed-forward same MLP · applied to every position Add & Norm + encoder output residual masked multi-head self-attention each position sees only earlier target tokens Add & Norm + encoder–decoder attention Q from decoder · K, V from encoder K V Add & Norm + position-wise feed-forward same MLP at every target position Add & Norm + linear + softmax → next-token probabilities training / prefill positions parallel generation: one new token, then loop