How I Built Procedurally Generated Gothic Comic Compositions
Inspiration: A Devil’s Composition
Mike Mignola is a legend. On the surface of it, he makes comics. But that is just the entry point to the quality of his craft as an artist, writer, and world-builder. In my book, Hellboy stands shoulder to shoulder with Neil Gaiman’s Sandman, both of which tower with long shadows over anything Marvel and DC have recently put on the screen.
So it’s not surprise I am obsessed with Mignola’s sense of composition, light, design and dialogue. In my own work, an architectural sense of composition is a requisite. The artist will usually know if a line is “right” or “wrong” by its combined relationship with other objects, something that these photos of accidental Renaissance compositions make more accessible. Abstracted away from human layering, simple light and color can have an identical effect. Think the wiggles of Kandinsky or the balances of Mondrian.
What Mignola does with tight composition is echoed in a divine choice of light and color. The color is done by the fantastic Dave Stewart, who bounces off Mignola’s dark shapes with stark, gorgeous simplicity. The color palettes are stark and mystical, with deep, lush colors in a constrained selection. They look like the muted mathematics of analytical cubism, beige with red, or Picasso’s blue period when rendering the fish man Sapien. Color like this alone could instantiate the characters, bring them forward and reveal them.
The other thing that drives me bonkers is Mignola’s line. It’s not straight, like that of a practiced draftsman. And it’s not bendy, or wobbly, like that of a whimsical illustrator. It is purposefully bent out of shape, reminding me of Paul Klee or Matisse’s cut outs. A confident and unique signature.
I systematically began searching through Mignola compositions to get a sense for how he does it, how he creates these stand-alone objects that are perfectly balanced in color, weight and line, while creating narrative mystery. And given that the works span over twenty years, a pattern began to emerge. Yes, Mignola has a rendering style. But he also has a compositional style. This is what I wanted to capture in essence. And then I wanted to write software that does it to infinity.
Plan: Breaking down the Mignola Formula
I am used to building intricate compositions, but my formula is different from Mignola’s — usually a single fantastical piece in front of an equally packed background. These comic covers have a very different vibe. First of all, they start out all black. Second, they can be jammed with detail but are minimal due to color selection (i.e., detail is barely seen as it is dark). Third, they are not abstract, other than in the composition, but feature a richly rendered central figure. Animals, architecture and cultural references are paramount.
To break it down further, you can divide most images into horizontal thirds, and then splice them into three sequential layers. The middle layer is usually taken up by the protagonist — Hellboy, Abe Sapien, and any other number of leads that feature a prominent color scheme. As an object, they are located in the bottom two thirds of the page, to leave room for title decoration and atmosphere at the top. They are also the brightest object on the screen. The colors often also pop on details, like eyes or weapons, but let’s put that aside.
The background of the covers tend to feature two elements, positioned in the top two thirds, which is default colored to black. The first is a textured background of some sort. This can be accomplished with architecture, broken columns, suits of armor. Or it can be a natural object, like the branches of a tree or the winding suckers of an octopus.
The second part of the background that you see above is that crazy smoke stack that trends in a diagonal. It plays a key function in the composition, both to balance against the direction and motion of the main characters, as well as to provide a contrasting background color to the highlight color (e.g., Red) of the main figure. Notice also that this color is often a muted variation on a different highlight color in the foreground.
The final piece is the foreground. Depending on the edition and artifacts on the different covers, the foreground is either an object, series of decorations (to hide feet), or a panel featuring details from the story. It’s not unusual to see, for example, flowers on a small hill in a bright color, or a panel with an insect on it, or a bunch of books, or skulls and swords in a pile. The foreground pulls attention, and then passes that attention to the central figure. It also holds the figure in place between the two separate uses of the mood color.
So that’s it. That’s our machine. You can see in my sketches brainstorming about what kind of imagery could go into each one of these categories, as it becomes important to build real volume within each object type in order to generate a sense of surprise and randomness — everything from jelly fish to statues of the Queen.
Code and Art: Building the generative system
Surely, I could have just made a version of a Mignola cover and called it a day. But what made this project worth doing was learning new tools, and building a system that plugs into Twitter and generatively creates new artworks. The workhorse to create the system is Processing (the desktop Java version). I highly recommend Daniel Shiffman’s saintly tutorial videos to get up to speed.
The first step was to figure out the color code, which I think makes or breaks a lot of these visuals. I pulled out a (1) dark, (2) mid and (3) highlight value for about 5 different colors from the originals — red, orange, yellow, teal and green. Note there are generally no blues or purples in the color scheme. The next part is trivial to anyone with a programming background, but a pain to me! The colors were loaded into arrays, so that I could call them by index without having to shift the saturation or brightness of the colors within Processing, and the selection was randomized. You can see the initial color experiment result here.
On the left, the skulls are tinted with the Hellboy red, and a desaturated line cuts through the background. On the right, the mustard yellow goes into the skulls, and into the background as well. There is no mechanism yet to cycle the colors so that the back/high colors are different from the mid color.
In thinking about the background line, I wanted to approximate the free-hand feeling captured in the covers, while also providing a randomized energetic anchor to the rest of the composition. In the next set of experiments, I broke up the line into multiple, random length segments that have minor directional jitter (i.e., shaking of end-points) in their orientation. Color and width were also allowed to vary, decreasing in opacity and size as the line count increased.
You also see also the first load of font into the system below, and the randomly generated titles starting to take shape. For those curious, the font is Exocet, made famous by the Diablo video game. Something about Hellboy and Diablo seems to fit together. I wanted each of these covers to be a door to a potential story, balancing the text against the cover in the way that Hellboy’s subtitles (e.g., Conqueror Worm, The Storm and the Fury) support the enigmatic art with equally powerful narrative. The system here is simply [Adjective] & [Profession], lists of about 500 words each. While the combinations can get super arcane by picking ridiculous professions, usually the adjective rescues the title back into sense.
Once we had the color systems and the background elements in place, what remained missing was all the actual drawing and rendering. I am not in a place to bang out twenty years worth of character designs, but thankfully the Internet provides in a time of need. There is a ton of Creative Commons imagery on the web, but not everything is going to fit into our style. With a few criteria in mind — hand drawn or engraved, fantastical, beautiful, mysterious — I went searching for inspiration.
And boy is it there. It’s easy to make beautiful things when you use masterpieces as the source. In particular, three sources were of great use. The first is Old Book Illustrations, filled with gems like this:
The second incredible resource is the Wikimedia Commons, with a plethora of various biological illustrations. While these are unlikely to function as either backgrounds or characters, they have great potential for foreground or other detail, such as skeletons, anatomical hearts, or ancient fauna.
Once I got going with characters, I realized that Western renderings only scratch the surface of what’s possible, and Japanese woodblocks beckoned. This opened up a treasure trove of samurai and spirits fighting various sea monsters, from giant crabs to octopuses, as well as beautiful landscapes rendered in immaculate gradient. The below are Hellboy from another era.
The rest of the work went into carving out the images from their rich homes, and into folders separated into (1) backgrounds, (2) objects and (3) decoration. These were then sized, oriented and rotated into the composition algorithmically. Horizontal orientation flips occasionally to create more surprises, and the direction of the background lines changes as well. Also, in transplanting the character, I found an effect that allows the create of a light and shadow on the character. Yes, the effect is flat. But, it’s scalable and does not require me to, let’s say, pre-highlight eyes or other details on dozens of cut-outs. Here are the first combined covers, with minimal randomness.
Two more points of polish are worth mentioning. The first is that building gradient transparency into the backgrounds has allowed the jagged billowing smoke to really connect more seamlessly into the picture. When Mignola does a cover, his composition is a flat cut of a three dimensional arrangement, while mine is at best a parallax arrangement. But this hack makes it feel more seamless.
And second, my experiments on using foreground elements as a way to blend the scene together didn’t work. So instead, I opted out to bring in a bit of decoration, paneling, and text to drive the composition home. You can see this technique on just a few of the BPRD originals, but they are not a consistent element and have a different design.
One could play with the idea of having multiple randomly generated panels with elements like crows or frogs, creating a quick story interplay between them. But the imported elements in this project are already much more heavily rendered than Mignola’s style. So sometimes, enough is enough. The other part of the decoration that got added in text is a description of the story. Another simple string of randomized text: [Adjective] [Synonym for Story] of [Emotion Noun] and [Functional Noun]. You know, A Sad Tale of Love and Betrayal.
Outcome
Here are the results.
They are wired up to Twitter to generate a new cover every 12 hours, and I am constantly adding new objects and backgrounds to create diversity. Getting lost in constructing objects is the pleasure of remixing the Masters.
As in any procedural system, some outcomes are stronger than others. But there is a story-telling that is happening here that triggers the imagination. Each of these covers is a door to a new book — one that will never be written in our universe, but perhaps in another one.
Which one would you read? Maybe The Different Exterminator, An Afraid Drama of Training and Shame but not The Fast Herpetologist, A Useless Nonfiction of Difference and Resentment.
Which one would you write and illustrate?
The other takeaway here is about working with the machine. A project like this is only possible in this interface:
The combination of machine tool with human craft allows for a very different artistic outcome than a single point solution. It also leads to surprises between the interplay of machine elements, like randomness generation, and human constraints, like layout design and image rendering. Putting everything into balance shifts from just the particular image to the template of the image, to the parameters of variation for the code. Too much randomness and the output is nonsense. Too much control and the output is boring.
I am tempted to keep fiddling, to keep making things more stark, louder, deeper. But it’s time to let this bot fly free.
Congrats, you’ve made it to the footer! I am Lex Sokolin, an artist, futurist and entrepreneur working at the intersections of finance and technology.
Give this article a few ❤ claps, and follow my art robot and tech robot.
