UC teams with Bay Area tech company for new lab that hopes to make AI work for artists

The battle lines between creative artists and the companies on the frontier of artificial intelligence have been drawn. The Hollywood writer’s strike in 2023 protested the use of AI in screenwriting. Record labels like Universal Music Group have asked streaming services to block AI companies from accessing their artists’ work. And Facebook’s Artists Against Generative AI community has more than 160,000 members.

Meanwhile, in the federal courthouse in San Francisco, artists are suing companies Stability AI, DeviantArt, Midjourney and Runway AI. Their legal complaint describes the firms’ AI-powered image generators as “copyright-laundering devices, promising customers the benefits of art without the costs of artists.”

Artists are already losing their livelihoods as a result, according to Bay Area illustrator Karla Ortiz, one of the artists behind the lawsuit. “This job displacement is real, it’s fast, and it’s affecting our industry in unprecedented ways,” she said.

This is the warzone into which UC Santa Cruz has launched a peacemaking initiative, in the form of a new lab that teaches tomorrow’s artists to harness the power of AI to enhance their creativity, rather than face off against it as a cut-price competitor.

Filmmaker Celine Parreñas Shimizu, UC Santa Cruz’s arts dean, hatched the idea for the Arts, AI, Augmentation and Acceleration Lab—A4 for short—while playing with OpenAI’s ChatGPT during a storyboarding session.

“I felt like I was walking into volatile terrain: the dawn of new technologies,” she said. “How does it impinge upon our ability to imagine what has not yet been authored?”

Shimizu is as concerned as any other artist about AI putting creative people out of a job. But the technology isn’t going away, and she reasoned that the way forward is for artists to learn how to use AI tools to enhance the value of their own work.

So she approached Silicon Valley computing company Advanced Micro Devices—where her husband was once chief architect of video game technologies—with the idea of collaborating. AMD donated six high-performance computers worth $50,000 to the lab.

The lab launched in October with an A4 summit, inviting students, university IT staff, faculty members, and notable alumni to weigh the pros and cons of AI and wager whether the initiative might ultimately enhance or undermine students’ creativity.

AI systems that generate text, images, music, or video do so through the power of “deep learning” models that have been trained on millions of original creative works. Artists like those behind the case against Stability AI and its co-defendants argue that AI threatens their livelihoods through the theft of copyrighted works. But artists’ concerns about AI run deeper still, into the quality of the works it generates.

One alumni attendee was Kevin Nolting, an editor at Pixar who has worked on animated films like Finding Nemo and Up. During the summit, Nolting was concerned that AI might sanitize art, removing its human imperfections and leading to bland, unoriginal works.

He also worries that dependence on technology to generate cinematic worlds may prevent students from studying more crude, traditional methods of designing sets and special effects.

“I think of Godzilla or Night of the Living Dead, with all these cheesy fake effects, and they’re still the most impactful movies I’ve seen,” he said. “I would encourage students to play with these effects first before they’re even allowed in the lab.”

Meanwhile, alumnus Rick Carter, a production designer who’s worked with Steven Spielberg and Robert Zemeckis, thinks AI could also help reduce barriers to entry into the film industry.

“My honest opinion is that there’s going to be a whole movement of people wanting to create stories and get them out there in the social media—partly because the old system of making a movie is so expensive,” he said.

Shimizu hopes the A4 lab can help to diversify the AI world—a priority for UC Santa Cruz’s Arts Division, with about 60 percent students of color and nearly a third being the first from their family to go to college. She also aims to address concerns surrounding “stolen” art by standardizing consent for artists to willingly add their work to a pool of training data.

After this school year, however, it will do so without her; Shimizu has announced that she is leaving UCSC in June to become dean of the School of Theater, Film, and Television (TFT) at UCLA.

But the lab will continue at UCSC, and for recently appointed A4 director and art department chair Elliot Anderson, the best way to tackle the looming threat of AI in creative spheres is through artists’ active involvement in the industry.

“The answer to those concerns is going to be having artists involved in AI with a diversity of students and faculty,” he said. “It’s really crucial to give artists a voice in how these technologies develop.”

Matthew Schumaker, a music professor at UC Santa Cruz, thinks ethically sourced AI can also open doors to new avenues of creativity. “We’re not trying to imitate a certain person’s style or steal anything from anyone else,” he said. “We’re looking for new pathways, new possibilities that otherwise we wouldn’t be able to execute.”

As part of his research at the A4 lab, Schumaker will use AI to enhance music composition. He plans to use recorded sounds as a model for entire orchestrations. Given the sound of an airplane flying overhead, an AI program could dissect the recording’s various frequencies and compare them to a database of orchestral sounds. This way, the technology could turn anything from the sound of a car engine to a human voice into an orchestral arrangement.

Other research includes assistant professor Chari Glogovac-Smith’s VR work. By using AI to create large-scale worlds accessible through VR headsets, Glogovac-Smith plans to experiment with new forms of immersive historical storytelling. “AI is really a tool,” they said. “Dare I say, I would equate it to a paintbrush.”

Critics of AI-generated art remain unconvinced that it has a place in an institution training the artists of tomorrow. Students paying hefty tuition fees are being robbed of their creative skills, Ortiz argued.

“The only ones who like AI are the executives who will get to save money by displacing these students’ future jobs,” she said

Glendon Mellow, a science illustrator based in Toronto, Canada, agrees. As long as AI is trained on copyrighted material, it remains a deceitful collaborator, he said: “We can argue about the output all day, but at its root, this system is poisoned.”