Parafrontier

We're yet another neolab trying alternative paths to AGI, specifically architectures and learning algorithms.

stateofthe.tech A map of research papers, and a place to read them.

StateofTheTech. Attention Is All You Need2017 M
GraphPaperSessions

Agent

I already know LSTMs. What should I read before this?

  1. Thinking
  2. Read references of Attention Is All You Need · 41 refs
  3. Marked Long Short-Term Memory known · pruned 1
  4. Read Learning to Align and Translate · §3

Seven papers remain on your path, and four of them carry most of the weight. Start with Sequence to Sequence Learning for the encoder–decoder the Transformer keeps, then Learning to Align and Translate, where its attention comes from. Deep Residual Learning and Layer Normalization explain the two tricks wrapped around every sublayer.

Ask about this paper