Blog Posts
-
Zero-Shot Localization with CLIP-Style Encoders
Think of your favorite pre-trained vision encoder. I’m going to assume you’ve chosen some variant of a CNN (Convolutional Neural Network) or a ViT (Visual Transformer). The encoder is a function that maps an image into a \(d\)-dimensional vector space. In the process, the image is transformed into a sequence of feature maps:
-
Geographic Position Encoders: A Deep Dive
Designing input features for a neural network involves a trade-off between expressiveness and inductive bias. On one hand, we want to allow the model the flexibility to learn patterns beyond what we humans can detect and encode. On the other hand, a model without any inductive biases will struggle to learn anything meaningful at all.
-
Sigmoid: the "Natural" Choice
One of the very first models studied in any introductory machine learning course is the logistic regression model for binary classification: