Meta AI published a report last week on its artificial intelligence (AI) text-to-image generation research. The technology, dubbed Make-A-Scene, allows users to draw a freeform digital sketch to accompany a text prompt. The AI then uses the two together to produce an AI-generated image.
Meta AI, an AI laboratory owned by Facebook and Instagram’s parent company Meta Platforms responsible, was started as a research laboratory in 2013. Make-A-Scene aims to “realize AI’s potential to push creative expression forward,” according to a recent Meta AI blog post.
“Our model generates an image given a text input and an optional scene layout,” reads the report. “As demonstrated in our experiments, by conditioning over the scene layout, our method provides a new form of implicit controllability, improves structural consistency and quality, and adheres to human preference.”
Related Articles
Generative art, or art that has been made using autonomous systems, has its roots in the 1960s when artists like Vera Molnar and Grace Hertlein began using computers to aid their practice.
However, the idea of making art through some kind of chance emerged much earlier. As technology has developed, the field of study has undergone dramatic shifts “in who can make generative art, how they make it, what it looks like, and even the themes and topics that it is capable of addressing. Because the tools and the work are tightly coupled, the history of generative art can be seen as a history of developments in software and hardware,” Jason Bailey wrote in an article for Art in America in January 2020.
Considering Facebook’s Artist-in-Residence program coupled with Meta AI, the unveiling of the Make-A-Scene program hardly emerges as a shock.
“To realize AI’s potential to push creative expression forward, people should be able to shape and control the content a system generates,” reads the blog post. “It should be intuitive and easy to use so people can leverage whatever modes of expression work best for them.”
Researchers used human evaluators to assess images created by the AI program. “Each was shown two images generated by Make-A-Scene: one generated from only a text prompt, and one from both a sketch and a text prompt,” explains the post.
The addition of the sketch feature resulted in a final image that was more aligned by 66.3 percent with the text description, the post said. For those who do not wish to draw, however, the sketch option is not necessary to generate an image; users can choose to use solely the text-only descriptions.
While the software is not yet available to the public, it has been tested by employees and select AI artists Refik Anadol, Sofia Crespo, Scott Eaton, and Alexander Reben.
“I was prompting ideas, mixing and matching different worlds,” noted Anadol. “You are literally dipping the brush in the mind of a machine and painting with machine consciousness.”
Meta program manager Andy Boyatzis, on the other hand, “used Make-A-Scene to generate art with his young children of ages two and four. They used playful drawings to bring their ideas and imagination to life.”
Meta has increased the potential resolution of Make-A-Scene’s output four-fold up to 2048-by-2048 pixels since the report’s release. Additionally, the company plans to provide open access demos but has not yet announced a release date. Until then, those curious about the developing technology will have to wait until October, when the project will be discussed at the European Conference on Computer Vision in Tel Aviv, Israel.