Blog
This is my blog. I try to use the previous article as the quality benchmark for the next one (so they get progressively better). I also try to write moderately frequently (which may conflict with the previous goal). Either way, this is just the tip of the iceberg of what I'm working on.
Thoughts on LLMs - February 2024
Some thoughts on the state of large language models.
Advancements in PPO - November 2023
A series of experiments that contributes to and advances research on the PPO RL algorithm.
Successor Heads - September 2023
Recurring, interpretable attention heads in the wild.
Residual Streams - July 2023
An analysis of residual streams in ResNet and transformer-based models.
Lessons Learned from AI Curling - June 2023
Lessons learned from 6 months of attempting to reach superhuman performance in a curling simulation.
BERT Embedding Math - March 2023
Modifying the embedding layer of BERT to change attributes of words and semantics of sentences.
Image Classifier Attacks - December 2022
Creating images that large-scale image classifiers misclassify by applying gradient descent to the image.