The prototype
Facetype is the name of Adam’s interactive project, in which the emotions detected from a person’s facial gestures control a variable font. To each detected emotion corresponds a specific typeface, which keeps transforming as your facial expression changes in front of the camera. This project does, in fact, link a variable font with face-api.js, an open-source AI that can detect and measure facial expressions in real-time.
The font contains five different, but continuous, styles that attempt to represent visually five different facial gestures: happy, surprised, neutral, sad, and angry. Before associating each emotion to each specific style in this project, Adam spent time researching how people react to and perceive different typefaces, analyzing what emotions are evoked when presented with a specific typeface.
The gestural interface created by Adam allows you to explore each typeface and style of the font only using your facial expressions. The cursors on the red and blue quadrants move in synch; the first indicates the registered facial gesture and the second shows the corresponding type.
Today we communicate so much with text. And when we do so, we remove all non-verbal communicators and rely on a static copy of the spoken language. In real life, facial expressions are visual signals used to modify or reinforce verbal communication and to establish empathy with the interlocutor.
Emoji represent a way to communicate emotions graphically and in a more emphatic way. Many emoji are in fact used to emulate the facial expression we would have while saying the things we are typing. But the font of text can’t be edited based on our mood or the emotions we feel. Adam’s prototype allows us to communicate with a personal typeface, where written words reflect the sender’s expression. Will variable fonts be integrated in our keyboards in the future, exactly like emoji, GIFs and stickers are integrated in the keyboards we use today?
What’s new
Variable fonts aren’t new. Dalton Maag has been experimenting with them for a couple of years already. In an interview we did with Bruno Maag last year, he claimed that variable fonts are one of the latest innovations in the type industry and that Dalton Maag are investing in them. He stated, “variable fonts have great potential, but their future is dependent on how people use them”.
What Adam did with variable fonts is something new: he used them in an interactive way, with an open-source AI.
Connecting inputs and outputs in new, unexpected, ways is something that Danish creative coder, Andreas Refsgaard, is famous for; he uses algorithms and machine learning to create unconventional connections between inputs and outputs. For example, he has worked on projects that allow people to play music using only eye movement and facial gestures, or to control games by making silly sounds (discover more). He also created an online bookstore selling sci-fi novels where everything —the stories, book covers, reviews, and prices — is entirely generated by AI.
Adam, in this project, ideated his own unconventional connection between input and output: he combined creative coding with type design.
We are looking forward to seeing the possibilities that this experiment will open up in the future, since this might change the way we communicate. The way in which this prototype might come into use is yet to be explored!
About Adam
Originally from Australia, Adam Lenzinger recently graduated in Graphic Communication Design from The Royal Danish Academy of Fine Arts, where he is now research assistant. With his project Facetype, hewas one of the finalists of the Danish Design Award 2020 in the category “Young Talent”.
