Look-a-Ryks matches your face against portraits in the Rijksmuseum art collection. It uses AWS Rekognition

Amazon thinks I look like the creepy older brother of painter Frederik Weissenbruch.

A couple of years ago I ran into a picture of a crashed facial recognition system in a Oslo pizza place.

I thought it foreshadowed a pretty bleak world where you’ll be filmed, scanned, recognised, analysed and remembered in every store you’ll ever visit. This will be to provide a “superior customer service” of course, but we all know what happens next.

I wondered if we could come up with use cases for facial recognition that weren’t completely evil.


First, we’ll need the facial recognition technology. This can be provided entirely by Amazon Rekognition. This is the service under fire for selling to US law enforcement agencies.

Secondly, we’ll need a massive dataset of faces to play with. Luckily, the Rijksmuseum in Amsterdam has an amazing API that exposes almost 600.000 works of art.

I named it Look-a-Ryks.

The actual Rijksmuseum

How it works

To start us off we’ll need all relevant objects from the collection. I decided to go for the easy route by downloading all data for the search term “portret” (“portrait”), of which there are 37,000.

Using some one-off scripts on an EC2 instance, I downloaded all of the images, put them in an S3 bucket and then forwarded them to Rekognition. The code for the download and upload is here, but it’s not super reproducible.

After this, I created a small application where users can upload a photo and see their doppelganger from the Rijksmuseum collections.

The result

For a lot of people, the application manages to find a portrait that at least has a passing resemblence. It was enough for a delivery manager in GDS to print out the doppelgangers of team members for on the wall.