Overview
We remember images in words, like "the foggy street with a single red umbrella", long after we've lost the file. This project closes that gap: a retrieval system that searches a collection of images using natural-language descriptions instead of tags or filenames.
It leans on CLIP, a model trained to place pictures and the sentences that describe them into the same embedding space. Encode every image once, encode the query on the fly, and the nearest neighbours by cosine similarity are the pictures that best match what you asked for.
How it works
- Index: each image is passed through the CLIP image encoder and stored as a vector, building a searchable gallery of embeddings.
- Query: the user's sentence is run through the CLIP text encoder into the same space.
- Rank: results are ordered by similarity between the query vector and every image vector, with no manual labelling anywhere in the pipeline.
What I learned
How much reach you get from a good shared representation. Once pictures and words live in the same space, "search" becomes a distance measurement, and a model that never saw your gallery still understands what you're looking for.