fast-neural-style: Real-Time Style Transfer

I followed up a reference to fast-neural-style from Twitter and spent a glorious hour experimenting with this code. Very cool stuff indeed. It’s documented in Perceptual Losses for Real-Time Style Transfer and Super-Resolution by Justin Johnson, Alexandre Alahi and Fei-Fei Li.

The basic idea is to use feed-forward convolutional neural networks to generate image transformations. The networks are trained using perceptual loss functions and effectively apply style transfer.

What is “style transfer”? You’ll see in a moment.

As a test image I’ve used my Twitter banner, which I’ve felt for a while was a little bland. It could definitely benefit from additional style.

twitter-banner

What about applying the style of van Gogh’s The Starry Night?

twitter-banner-1

That’s pretty cool. A little repetitive, perhaps, but that’s probably due to the lack of structure in some areas of the input image.

How about the style of Picasso’s La Muse?

twitter-banner-2

Again, rather nice, but a little too repetitive for my liking. I can certainly imagine some input images on which this would work well.

Here’s another take on La Muse but this time using instance normalisation.

twitter-banner-6

Repetition vanished.

What about using some abstract contemporary art for styling?

twitter-banner-4

That’s rather trippy, but I like it.

Using a mosaic for style creates an interesting effect. You can see how the segments of the mosaic are echoed in the sky.

twitter-banner-7

Finally using Munch’s The Scream. The result is dark and forboding and I just love it.

twitter-banner-8

Maybe it’s just my hardware, but these transformations were not quite a “real-time” process. Nevertheless, the results were worth the wait. I certainly now have multiple viable options for an updated Twitter header image.

Related Projects

If you’re interested in these sorts of projects (and, hey, honestly who wouldn’t be?) then you might also like these:

Lightning Activity Predictions For Single Buoy Moorings

[speakerdeck url=”https://speakerdeck.com/exegetic/lightning-activity-predictions-for-single-buoy-moorings”]

A short talk that I gave at the LIGHTS 2013 Conference (Johannesburg, 12 September 2013). The slides are a little short on text because I like the audience to hear the content rather than read it. The objective with this project was to develop a model which would predict the occurrence of lightning in the vicinity of a Single Buoy Mooring (SBM). Analysis and visualisations were all done in R. I used data from the World Wide Lightning Location Network (WWLLN) and considered four possible models: Neural Network, Conditional Inference Tree, Support Vector Machine and Random Forest. Of the four, Random Forests produced the best performance. The preliminary results from the Random Forests model are very promising: there is good agreement between predicted and observed lightning occurrence in the vicinity of the SBM.