Pixar’s innovative productions have redefined the boundaries of technology integration in visual storytelling. Its collaboration with VAST has set the foundation for AI-powered cinematography while allowing quick adaptation to evolving artistic needs. Eric Bermender, Head of Data Center and IT Infrastructure, Pixar, discusses the data centre’s capacity for artistic growth.
VAST Data, a data platform company for the AI era, has announced that Pixar Animation Studios, a subsidiary of the Walt Disney Company, selected VAST as the key data platform collaborator for the company’s data-intensive feature animation production, including its latest, Elemental, which hit US theatres on June 16 2023.
Pixar began working with VAST in 2018 as it first pioneered new animation techniques using volumetric rendering for their acclaimed 2020 film titled, Soul. Its newest feature, Elemental, required a new way to scale these detailed volumetric characters and environments. Unlike the geometric surfaces and materials used in earlier Pixar projects, the broadly applied volumetric animation methods used in Elemental created six times the data footprint and computational demands for data than that of Soul.
To meet the technical requirements of the Pixar creative team, Pixar turned to the VAST Data Platform to power the demanding render pipeline for Elemental as well as Pixar’s other ongoing and future productions. By moving 7.3 Petabytes of data to a single datastore cluster that is managed across one high-performance namespace organised from low-cost flash – VAST provides real-time access to keep Pixar’s render farm constantly busy while enabling improved observability and analytics, simplified collaboration and development across multiple productions running at the same time and laying the foundation to leverage AI and Deep Learning for future films.
“Pixar is the world’s leading innovator in animation and in order to continue to deliver new stories, breath-taking visual environments and memorable characters, we require the industry’s most innovative technologies to help us bring the animators’ vision to life,” said Eric Bermender, Head of Data Center and IT Infrastructure at Pixar Animation Studios. “Elemental is the most technically complex film that Pixar has ever made, but with VAST we were able to build beyond our current level of animation and consider new techniques that no one had ever considered before because we didn’t have the technology in place to support it. VAST’s technology has allowed us to change the way we store and access data while also opening the door to new potential visual pipelines.”
In collaboration with VAST, Pixar has:
- Optimised data access: The compute requirements for Elemental demanded fast and concurrent data access from hundreds of thousands of processors used in the render pipeline. VAST delivered fast, uninterrupted performance even during the film’s peak rendering usage, requiring nearly two Petabytes of capacity at one time (compared to previous films over the last five years using only about 300 to 500 Terabytes of capacity).
- Improved operational resilience: With different productions in different stages of rendering at any given time, Pixar depends on VAST’s Data Platform to handle the volumetric demands of several films being developed at the same time. With VAST, these pipelines enjoy the uptime needed to meet Pixar’s intense film production schedules and release dates.
- Engaged audiences with visually stunning storytelling: The parallel data access, high performance and scale of the VAST Data Platform allowed Pixar to render nearly 150,000 volumetric frames in Elemental alone – each of which helped give these brilliant characters and spectacular environments their new and unique look.
- Built the foundation for future generations of AI-powered cinematography: Using VAST’s Data Platform, Pixar projects are now available for the types of high-speed data processing required to employ new animation techniques using machine learning and deep learning training models for automated and improved production processes.
“At VAST, we’re so proud to work with Pixar as they create timeless cinematic magic by simultaneously pushing the boundaries of storytelling and technology,” said Jeff Denworth, Co-founder of VAST Data. “For Elemental and future films, we’re delivering a data platform that powers the animation and rendering workflows for their most data-intensive and computationally heavy projects, while enabling its AI and ML pipeline for the future in order to further the ambitions of Pixar artists and the stories they’re able to tell.”
Eric Bermender, Head of Data Center and IT Infrastructure, Pixar, offers insight into his role and the influences of data management for creative endeavours.
Can you provide an overview of your role and what this looks like day-to-day?
I manage two systems teams, the data storage team and the data centre operations team. My typical day is fairly atypical in that my focus changes depending on many production, technical and studio factors. In general, about a third of my meetings during a week is with various production artists and teams, another third is with various systems technology engineers and teams, while the remainder is with our studio staff and teams. I’m very privileged to hold a position that allows me to engage with all these various teams and individuals in a creatively dynamic environment.
How does the data centre play a part in supporting/enabling the continuous innovation of the film industry and projects such as this?
One of the most creatively challenging and enjoyable parts of my job is working with our engineers to extend the envelope of technology in service to production. We have a saying at Pixar – art pushes technology. The data centres we have on campus are the core pieces of infrastructure that enable us to do this. Periodically, we need to reconfigure and upgrade that infrastructure to accommodate the latest computer technologies. Our data centre team is tasked with maintaining the ‘state-of-the-art’ to enable the ‘art’.
How has the industry transformed in recent years in terms of data output and the ways in which it relies on data centres today?
Although the technology within the data centres has changed significantly over the years, and some of the types of data have changed, the core requirement of needing low latency access to our data has not – low latency is key to our success. Our artistic process relies on, and is accelerated by, quicker iteration and computational feedback from our computer systems. We continue to invest in systems which reduce our I/O latency with larger capacity and IOP data sets.
How quickly must data centres scale to support computational demands including the enormous amounts of data being processed and how is this managed efficiently?
The data centre’s capacity for artistic growth is complicated by the fact that the concept of artistic growth, like art itself, is subjective. However, to manage this growth I subscribe to the concept of two important observations, or ‘laws’, that bring some predictability to the process. The first is the omnipresent Moore’s Law which I’ve modified in terms of Data Capacity Growth and Computational Processing growth. By trending both over time we can be fairly predictive about how often we double these two factors. Over a long enough sampling period, CPU and data both tend to normalise. I can then use these doubling rates to plan data centre cooling and power needs in the future, as well as our computation and data storage needs.
The second less-known Law is Blinn’s Law which states ‘rendering time tends to remain constant, even as computers get faster’. This is mainly due to the observation that as artists have more computational resources they tend to invent new technologies, or as they would say ‘looks’, that will utilise those additional resources. This has a lot to do with the creative process of Dailies in which artists typically gather in the mornings to review the renders that were generated the night before. To fit the overnight window of time needed to have Dailies in the morning, artists tend to naturally create renders that fit within the two-to-four hour per job timeframe.
How crucial is a Disaster Recovery strategy for avoiding downtime at an organisation such as yours?
This is a question I grapple with constantly because there can be a significant financial cost associated with various levels and aspects of recovery. I tend to break recovery up into three categories: immediate individual recovery, medium-term production workflow recoveries and long-term fileserver recoveries. Each has its own methods and costs associated with the recovery process depending on the time required and the technology used.
What new techniques were you able to consider since working with VAST?
From a production data storage perspective, Pixar traditionally would tier our data from high-performance/low-capacity storage to low-performance/high-capacity storage. However, in about 2016, this model started to break down as our artistic processes evolve. In general, the higher performance tiers that vendors were offering were not growing capacity fast enough to keep up with our creative pipelines. The VAST platform became the best solution for us as it allows us to continue to expand and grow our high-performance and high-capacity pipelines without artistic compromise. Specifically, the all-flash tier allows us to expand without having to consider slowness at the disk aggregate level. The ability to expand compute and capacity independently allows us to make changes to the platform based on need as opposed to vendor architecture. Finally, the persistent memory layer enables various compression and deduplication techniques within the VAST software that gives us the best overall compression we’ve ever experienced.
How do you maintain the smooth and seamless delivery of the technology?
Using any methods we can! I think it’s safe to say that the production process is not necessarily smooth and predictable, and it’s not really meant to be. Each project we work on can have its own look and therefore its own I/O profile. We need to be able to pivot quickly, which for us means being willing and able to rearchitect per project if necessary. Therefore, the storage team tends to focus on technologies that can integrate scale-out performance and capacity within a short duration of time.
How has your work with VAST enabled you to continue producing timeless cinematic magic and what does the future hold?
VAST has been a great collaborator over the past few years and was the foundational technology that we leveraged to produce volumetric characters for our features Soul and Elemental. Their engineering and sales teams engage with us to help us incorporate solutions that allow us to continuously evolve our artistic vision. Additionally, we have moved all data which could be considered for AI training to VAST. In collaboration with the VAST team, we continue to develop new production techniques and technologies to grow at the speed of art.
Click below to share this article