We can't deny that artificial intelligence takes over how we deal with and operate the world. Arts have been subjected to AI too. There are tools that we can use to create digital illustrations and artworks or transfer art styles. One tool is VQGAN+CLIP, which I had been experimenting with for the past few weeks.
The first round of experiments was a disappointment. I get a digital illustration that is patches of the image used to train the model but, going through iterations, find out that we need to be specific with the concept and the art style. Throwing a word or sentence without a vision of art style can be disastrous to the output image. I think AI models generate illustration by relating our word prompt, and without giving a prompt for the art style throws a curveball to what we look forward to in the output illustration.
After a disappointing run, I tried to be very specific with my prompt, accounting for what art style I wanted to see. So I tried to experiment with concept art and dungeons. I tried to have an initial prompt, the entrance to the dungeons. I am still disappointed with the initial results cause the digital illustration has patches of entrance signage. After changing the articles in the same prompt, I was able to get what I envisioned as the entrance to the dungeon.
After getting accustomed to how I should write my prompt, I continued experimenting with VQGAN+CLIP. The next phase was to create a series of illustrations that somehow tell a story inside the dungeon. Reflecting on that, I adjusted my prompt to what I look forward to in the dungeons' interior. I did add details like a pathway, monster, remnants of war, and others. It was a fun experiment, and I observed tendencies of AI from misspelled prompts to the art style. I was in awe of how two illustrations from misspelled prompts have similarities. I guess AI can still understand despite language errors in our prompt.
However, there are still illustrations that result in patches or tape of images from time to time. The output illustration is good, but I find it inconsistent with minor flaws from here to there. What I presented here is a more defined illustration. I added details and some elements in Photoshop and Illustrator to fix inconsistencies in the image. It ties up the digital artwork to what I envisioned.
I did another experiment to fully explore how I tie up my prompts to create an illustration. I did misspell words in the prompt like water well to water weel. I was astonished at how AI responded. At first, a misspelled prompt without context results in disruptive and bizarre patches of illustration. When I did put context like water well and water weel, the result was quite satisfying. It generates a correct representation, sometimes almost exactly similar output. It may be a coincidence since I may not try it for a handful of experiments, but, knowing it works like that, it was an awe-moment.
Also, I did run a similar prompt to see if AI replicates. I initialize the AI model with the same parameters and iterations. I run the experiment with different prompts. I wasn't able to see AI replicate the prior illustration. I couldn't fully understand yet how AI can work without plagiarizing, but it works. I did a reverse image search, and all digital artwork (non-edited) yields no match. Is that amazing? For me, it is.
Art is something that we make so that we understand ourselves better. It is an expression of our creativity and even craziness. Our definition of art changed from our ancestors' burned sticks to AI-generated or enhanced art. Some will argue if AI arts are art. But I think artificial intelligence is an ideal partner for us to create or enhances art. My digital illustration is base on AI, but these are sometimes out of context, and I still need to fix some lines, shades, or colors to tie it up. I guess this is the same for those who use AI for the arts. AI seems too foreign at first, but we can agree that it is still art.
| Author's Note: |
|---|
All content including digital illustrations is from the author. The illustrations are generated by VQGAN+CLIP ( Notebook and Nightcafe), rescale resolution with Real-ESGAN, and added details and edited using Illustrator and Photoshop. |
