Hyperlinks
Personal Websites
The best feature of the web is the kind of individual content that has nowadays been largely been segregated to social platforms. On the other hand, increasingly lower cost to hosting and serving your own content has made it easier for many people to publish to the web. Often times, this kind of stuff is hard to find and even connected to in the first place. Turns out much high-quality web content is highly interwoven!
Gwern Branwen frequently blogs a breadth of sometimes contrarian, unbashedly opinionated tech culture pieces
Patrick Collison curates pages on topics he explores, such as quick execution
Andy Matuschak publishes his personal notes online as an experiment of working in the open, writing about his experiences designing human empowering systems
Some more…
Deep Learning
Deep Learning by Goodfellow, Bengio, and Courville is a great, free online textbook
Efficient Backprop by Yann LeCun covers tips tuning neural networks through backpropagation
The Lottery Ticket Hypothesis by Frankle and Carbin explores the phenomenon of winning ticket network weights (generated by applying post-training, absolute weight-conditional binary masks on reset weights before re-training) that lead to better performance than their equivalent trained-to-convergence networks
Why Deep Learning Works Even Though It Shouldn’t by Ryan Moulton writes of his research intuitions and offers suggestions
A number of helpful sources for approaching linear algebra:
- Matrix Cookbook by Petersen and Pedersen is a collection of identities, approximations, inequalities, etc. about matrices
- Essense of Linear Algebra by Grant Sanderson (3Blue1Brown) is a video series that provides fantastic visual comprehension of linear algebra concepts (notably created in Manim)
- Viewing Matrices & Probability as Graphs interprets matrices as relations
- A Programmer’s Intuition for Matrix Multiplication describes matrices as collections of data and functions
Some sources for learning on time series data:
- A Recipe for Training Neural Networks by Andrej Karpathy
- Tips for Training Recurrent Neural Networks by Danijar Hafner
- LSTM: A Search Space Odyssey by Greff et al.
- The Sorcerer’s Apprentice Guide to Training LSTMs
Graph Neural Network resources, a Twitter thread by Petar Veličković, contains great resources covering the state of graph neural networks
The Neural Network Zoo by the Asimov Institute is a fun exploration of the breadth of deep learning network architectures
Distill is a highly curated research journal which publishes quality articles that summarize areas of research in machine learning, often accompanied by interactive visualizations for better explainability of intricate techniques
einops is a PyTorch tensor modification library which easily abstracts complex transformations with cool intro docs
Weight-Agnostic Neural Networks demonstrates the success of neural networks without training, focusing on optimal weight initialization for reinforcement learning and classical supervised learning tasks
OpenAI Microscope visualizes features of a collection of convolutional neural networks using lucid
VGG Image Annotator (VIA) is a simple, self-contained image, audio, and video annotation tool
Writing
Many of Paul Graham’s essays can satiate interests in technology, startups, and philosophy relating to the two
The Bitter Lesson by Rich Sutton reminds of the reality of working with persistent computational power growth
You and Your Research by Richard Hamming reflects on his years at Bell Labs and research success
How to do Research at the MIT AI Lab is full of useful advice for any researcher
Advice for Undergraduates Considering Graduate School is an essay to prospective undergraduate students interested in research, from a series of essays by Philip Agre
The Illustrated Guide to a Ph.D. explains what a Ph.D. is with a set of diagrams, starting from your earliest education
You Cannot Serve Two Masters by Ben Recht discusses industry misdirection of academic work away from “curiosity driven research”
A List of Programming Essays curated by Ben Kuhn covers a variety of technical topics
Your Life is Driven by Network Effects describes levels of networking over the course of your life
Notes from the Mystery Machine Bus by Steve Yegge is a legendary rant which frames software engineering as a political affiliation
Farmer & Farmer (Jonathan Harris & Sep Kamvar) write about the bridge between humans and technology
Utilities
Detexify converts character sketches to LaTeX
Vimium lets you quickly browse the web using your keyboard Vim-style with alternatives available for most modern browsers [1] [2] [3]
LibreOffice is a free office software suite including a word processor, spreadsheet editor, etc.
LyX is a word processor built on top of LaTeX made for great writing right out of the box
Other
Learn X in Y Minutes is a quick reference tool for refreshing familiarity with a programming language and follows a unique tutorial format
Paul’s Math Notes is full of great math notes from grade school algebra to calculus in college
Cambridge Notes by Dexter Chua is a collection of curated notes on many math and physics courses
Artvee is a source of high resolution, public domain art
The Eye is a digital library that includes datasets and National Gallery paintings
Vimgifs is a collection of gifs exemplifying usage of Vim