Ilya's Top 30 Papers

"If you really learn all of these, you'll know 90% of what matters today" — Ilya Sutskever

Progress

0/26

Time Invested

0 min

of 62h 30m total

Next Up

The First Law of Complexo...

#1blogComplexity Theory

The First Law of Complexodynamics

30 min

Advanced

#2blogRNNs

The Unreasonable Effectiveness of Recurrent Neural Networks

45 min

Easy

#3blogRNNs

Understanding LSTM Networks

30 min

Easy

#4paperRNNs

Recurrent Neural Network Regularization

Moderate

Prerequisites needed

#5paperInformation Theory

Keeping Neural Networks Simple by Minimizing the Description Length of the Weights

1h 30m

Advanced

Prerequisites needed

#6paperSequence Models

Pointer Networks

1h 15m

Moderate

Prerequisites needed

#7paperCNNsAlexNet

ImageNet Classification with Deep Convolutional Neural Networks

Easy

#8paperSequence Models

Order Matters: Sequence to Sequence for Sets

Moderate

Prerequisites needed

#9paperScaling

GPipe: Easy Scaling with Micro-Batch Pipeline Parallelism

45 min

Moderate

Prerequisites needed

#10paperCNNsResNet

Deep Residual Learning for Image Recognition

Easy

Prerequisites needed

#11paperCNNs

Multi-Scale Context Aggregation by Dilated Convolutions

45 min

Moderate

Prerequisites needed

#12paperGraph Neural Networks

Neural Message Passing for Quantum Chemistry

1h 30m

Advanced

#13paperTransformersTransformer

Attention Is All You Need

Moderate

Prerequisites needed

#14paperAttentionBahdanau Attention

Neural Machine Translation by Jointly Learning to Align and Translate

1h 30m

Moderate

Prerequisites needed

#15paperCNNs

Identity Mappings in Deep Residual Networks

45 min

Moderate

Prerequisites needed

#16paperReasoning

A Simple Neural Network Module for Relational Reasoning

Moderate

#17paperGenerative Models

Variational Lossy Autoencoder

1h 30m

Advanced

#18paperRNNs

Relational Recurrent Neural Networks

Advanced

Prerequisites needed

#19paperComplexity Theory

Quantifying the Rise and Fall of Complexity in Closed Systems: The Coffee Automaton

Expert

Prerequisites needed

#20paperMemory Networks

Neural Turing Machines

1h 30m

Advanced

Prerequisites needed

#21paperSpeech

Deep Speech 2: End-to-End Speech Recognition in English and Mandarin

1h 15m

Moderate

Prerequisites needed

#22paperScaling

Scaling Laws for Neural Language Models

1h 30m

Moderate

Prerequisites needed

#23paperInformation Theory

A Tutorial Introduction to the Minimum Description Length Principle

Advanced

#24paperAGI

Machine Super Intelligence

Advanced

Prerequisites needed

#25bookInformation Theory

Kolmogorov Complexity and Algorithmic Randomness

10h

Expert

#26courseCNNs

Stanford's CS231n Convolutional Neural Networks for Visual Recognition

20h

Easy