# ilya-sutskever-recommended-reading

**Repository Path**: julio2023/ilya-sutskever-recommended-reading

## Basic Information

- **Project Name**: ilya-sutskever-recommended-reading
- **Description**: No description available
- **Primary Language**: Unknown
- **License**: Not specified
- **Default Branch**: main
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2024-06-03
- **Last Updated**: 2024-06-03

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# Deep learning reading list from Ilya Sutskever
> 深度学习精炼秘笈

> til, Ilya sutskever gave john carmack this reading list of approx 30 research papers and said, ‘If you really learn all of these, you’ll know 90% of what matters today.’
<br>

[[Twitter Post]](https://twitter.com/keshavchan/status/1787861946173186062) [[Arc.net Link]](https://arc.net/folder/D0472A20-9C20-4D3F-B145-D2865C0A9FEE)
<br>

## 

- **The Annotated Transformer.** Sasha Rush, et al. [[Blog]](https://nlp.seas.harvard.edu/annotated-transformer/) [[GitHub]](https://github.com/harvardnlp/annotated-transformer/)
- **The First Law of Complexodynamics.** Scott Aaronson. [[Blog]](https://scottaaronson.blog/?p=762)
- **The Unreasonable Effectiveness of Recurrent Neural Networks.** Andrej Karpathy. [[Blog]](https://karpathy.github.io/2015/05/21/rnn-effectiveness/)
- **Understanding LSTM Networks.** Christopher Olah. [[Blog]](https://colah.github.io/posts/2015-08-Understanding-LSTMs/)
- **Recurrent Neural Network Regularization.** Wojciech Zaremba, et al. [[ArXiv]](https://arxiv.org/abs/1409.2329) [[pdf]](https://arxiv.org/pdf/1409.2329)
- **Keeping Neural Networks Simple by Minimizing the Description Length of the Weights.** Geoffrey E. Hinton and Drew van Camp.
- **Pointer Networks.** Oriol Vinyals, et al.
- **ImageNet Classification with Deep Convolutional Neural Networks.** Alex Krizhevsky, et al.
- **Order Matters: Sequence to sequence for sets.** Oriol Vinyals, et al.
- **GPipe: Easy Scaling with Micro-Batch Pipeline Parallelism.** Yanping Huang, et al.
- **Deep Residual Learning for Image Recognition.** Kaiming He, et al.
- **Multi-Scale Context Aggregation by Dilated Convolutions.** Fisher Yu and Vladlen Koltun.
- **Neural Message Passing for Quantum Chemistry.** Justin Gilmer, et al.
- **Attention Is All You Need.** Ashish Vaswani, et al.
- **Neural Machine Translation by Jointly Learning to Align and Translate.** Dzmitry Bahdanau, et al.
- **Identity Mappings in Deep Residual Networks.** Kaiming He, et al.
- **A simple neural network module for relational reasoning.** Adam Santoro, et al.
- **Variational Lossy Autoencoder.** Xi Chen, et al.
- **Relational recurrent neural networks.** Adam Santoro, et al.
- **Quantifying the Rise and Fall of Complexity in Closed Systems: The Coffee Automaton.** Scott Aaronson, et al.
- **Neural Turing Machines.** Alex Graves, et al.
- **Deep Speech 2: End-to-End Speech Recognition in English and Mandarin.** Dario Amodei, et al.
- **Scaling Laws for Neural Language Models.** Jared Kaplan, et al.
- **A Tutorial Introduction to the Minimum Description Length Principle.** Peter Grunwald.
- **Machine Super Intelligence.** Shane Legg.
- **Kolmogorov Complexity and Algorithmic Randomness.** A.Shen, V. A. Uspensky, and N. Vereshchagin.
- **CS231n: Convolutional Neural Networks for Visual Recognition.**