Stable Diffusion Holiday Cards

This was a quick project, so it gets a quick (and surprisingly timely) writeup. I wanted to make holiday cards for my friends and given the recent popularity of text-to-image models, figured that I could make custom cards using one.

There's a wide assortment of text-to-image models available online, each with varying amounts of accessibility. DALL-E 2 requires credits, Midjourney requires interacting with a Discord bot, and Stable Diffusion is just available. This sort of made the choice of choosing Stable Diffusion for me, plus there happens to be a handy-dandy example Colab notebook. I am nowhere near familiar enough with these models to accurately explain how they work, for more information check out these neat links.

Working off the example notebook, it was pretty easy to use the model to generate 5x7 card-shaped images. The front of the card is the generated image, the backside has a flipped version of the image with a frosted glass effect. The prompt and a signature is also printed on the backside of the card.

The workflow is:

1. Fill out the missing words in the prompt: "A {adj} holiday card for {person} with {thing1} and {thing2}"
2. Run the model a couple of times until the image looks reasonable
3. Generate the backside of the card
4. Save both images (and upload to Walmart Photo to print)
5. Rinse and repeat

I noticed that when the prompt contains "holiday card" the model tends to produce much lower quality images (so basically all the time). This is probably due to the dataset the model was trained on - Stable Diffusion v1.5 was finetuned on the laion-aesthetics dataset and skimming through some images from the dataset there doesn't seem to be a lot of card-like pictures.

Colab link here.