Table of contents
Open Table of contents
Design is iterative
Large language models (LLMs) have not replaced Graphic Designers because graphic design like any creative human endeavour is created iteratively by either taking feedback from people or/and artists own internal monologue. Multi Modal LLMs can create designs in one shot but they cannot make precise changes without also introducing unintended changes to it. Multiply it by the number of iterations and you will end up with something barely recognizable. This makes it very difficult for LLMs to create designs which require multiple iterations. Keep in mind that most non trivial designs require multiple iterations because neither thoughts nor natural language can perfectly describe an image without ambiguity so we need some kind of image to start with as a medium of communication either with ourselves or with stakeholders.
Some Experiments with LLMs
I am sharing a very small experiment you can repeat as well, it will barely take 10 minutes. The idea is to take a random design from Figma and ask AI to make a small change to it without introducing additional changes, that’s it. The picture is just a screenshot from Figma from its first page, you can use any non-trivial design yourself, just make sure its graphics are heavy, not text heavy.
I picked ChatGPT and Gemini 2.0 Flash with image generation because they have the best vision capabilities at the moment. I asked both to change the text Wi-Fi to Li-fi in the picture while keeping everything else the same. These were the results:
Original Screenshot.
Changes made by ChatGPT. ChatGPT made the requested change but it also changed a lot of other things like icons, text color etc
Changes made by Gemini 2.0 Flash Gemini could not make the requested change, in fact it replaced something else with Li-Fi instead. It messed up faces and icons too like ChatGPT as well.
It’s incredible what AI did here in my opinion and we should not discount it. Even 5 years ago, This would have felt like magic to us and somehow computers are able to do it now. That said, there are small differences and the closer we look, the more differences we will see. Here is a fun one. There are lots of tools you can use online such as im2go, diffchecker which can compare images pixel by pixel and they show differences at the pixel level. Ideally, AI should only change the pixels in desired regions of image while keeping everything else the same but current AI is simply not capable of doing it.
(If someone is inclined, there are free and open source image diff tools as well in case you to run some sort of large scale experiments, just search image diff Github on Google or ask ChatGPT or your favourite LLM)
Here is another fun one:
Ask AI to count the number of enclosed regions in the picture below:
So far all the AIs I tested failed at this task
AI does not understand images as well as text
While other creative endeavours like programming suffers from some similar flaws like AI making unintended changes to working code, modern LLMs make far fewer mistakes with text in general because they are primarily (pre)trained on a huge corpus of text. Images are an add on, so they don’t naturally understand images as well as text.
The cost of AI artifacts
Some people might think that we can get by with a 95% for 99% accuracy and that AI is good enough but we need to account for the number of iterations here and as a user we lack control over what AI can change and when. For all we know, it can introduce random subtle changes anywhere in any iteration and as we prompt it to change it back, it will introduce yet more changes we didn’t ask for.In fact, this is a universal frustration with any creative work right now with AI. Furthermore, even if AI does create something close to what we want in one iteration with JUST a small minor artifact, it’s still very problematic. Ask yourself this: If you see an ad on facebook which looks perfect but someone had 6 fingers for a second, will it change your perception or not? If so, by how much?