In an innovative leap forward for the visual effects industry, DNEG, an acclaimed leader in digital animation and VFX for major films, has adopted Red Hat OpenShift. This strategic move significantly enhances artist productivity and optimizes data centre operations.
Red Hat, a leading provider of open source solutions, has announced that DNEG, a world-leading visual entertainment services company, is using Red Hat OpenShift as a consistent global platform to speed up software innovation and improve artist productivity for competitive differentiation.
DNEG is a visual effects (VFX) and animation studio for feature film and television with seven Academy Award wins for ‘Best VFX’ in recent years, including Inception, Interstellar and Dune: Part One. What makes DNEG unique is its software developed by hundreds of engineers around the world and its artists accessing those creations to deliver DNEG’s renowned work. A fundamental part of any visual effects workflow is the ability to track and manage a large volume of digital assets and their metadata to assemble everything from cityscapes to jostling crowds to flowing water.
From its roots 25 years ago as a small company based in London, DNEG has been steadily growing and now has more than 9,000 employees in 10 locations around the world. DNEG needs to move fast to be the best in its field, yet aspects of its asset tracking system, a large monolithic application, had become a bottleneck. The steady increase in artists and workload brought a corresponding increase in the amount of metadata, all competing for access to the same database services.
It was clear that the asset tracking system needed a significant architectural overhaul in order to scale to support this expanding, globally-distributed team. DNEG also continually faces increasing compute demands as the industry’s technology evolves, such as with the introduction of the latest industry-standard collaboration-based solution for constructing disparate elements into animated 3D scenes – Universal Scene Description (USD).
DNEG decided to break up key parts of the asset tracking system into microservices and manage these in containers using Red Hat OpenShift, the industry’s leading hybrid cloud application platform powered by Kubernetes. The first and most acutely critical, component of this was a high-volume metadata interrogation API, Pipe Query. DNEG had a clear vision and leaned on Red Hat consultants and training to enhance its expertise in DevOps, Infrastructure-as-Code, API management and other relevant areas, developing first a proof of concept, then a global infrastructure.
Now in production in six data centres around the globe, Red Hat OpenShift provides an abstraction and orchestration layer between artists and the database, enabling developers to deliver Pipe Query in a standardised, cloud-native way to artists, wherever they are in the world.
Red Hat OpenShift automates the application lifecycle to reduce manual processes for DNEG’s developers and enable workloads to be managed dynamically. DNEG has taken a progressive DevOps approach to delivering Pipe Query from development and build, through quality assurance (QA) and all the way to production. Red Hat OpenShift Pipelines, based on the upstream open source Tekton project and Red Hat OpenShift GitOps, which uses Argo CD, are helping DNEG to create advanced workflows for continuous integration and continuous deployment (CI/CD) using industry best practices.
This helps developers deliver Pipe Query updates in a faster, more stable and flexible way – deploying features when they are available rather than waiting for rigid release cycles and with minimal disruption: releasing a change now takes minutes where before it took hours. Red Hat OpenShift gives DNEG visibility into application usage so that it can streamline the creative process of asset design for artists and help them be productive instead of waiting for the system to fetch data.
With its deployment of Red Hat OpenShift Platform Plus, DNEG can also take advantage of:
- Red Hat OpenShift’s in-built security features, enabling DNEG to control its trusted sources of content, testing and deployments, to defend applications from attacks and vulnerabilities in all layers of the platform and to extend secure services through standard interfaces and APIs.
- Red Hat Advanced Cluster Management for Kubernetes to control clusters and applications from a single console, with built-in governance and compliance policies.
- Red Hat Advanced Cluster Security for Kubernetes to help protect containerised Kubernetes workloads in all major clouds and hybrid platforms.
- Red Hat OpenShift Data Foundation for persistent software-defined storage and essential data services.
- Red Hat Quay, a scalable, central registry of available software to help distribute it efficiently to multiple clusters.
DNEG is planning to onboard more of its software to Red Hat OpenShift for ease of distribution and orchestration, prioritising services that deliver the most value. This includes its asset provisioning system and its file transfer system, which carries 40 petabytes of data. As industry technology evolves, DNEG anticipates that this could also include workloads that use Machine Learning (ML), as these have high compute demands and can save hours of laborious manual work by helping artists to apply animation principles from one digital entity to another. DNEG is also in the process of implementing Red Hat 3scale API Management to help manage workload, such as balancing requests from artists with batch requests from a server to reduce congestion. Red Hat 3Scale API Management can help DNEG to prioritise artist’s computing consumption based on the importance or delivery date of the work. This helps artists learn the most efficient way to use resources to deliver their work and enables DNEG to better understand how much resource each production is using, so it can more accurately quote for future jobs.
We asked Paul Salvini, DNEG CTO, further questions to find out more about the project:
Can you elaborate on the strategic vision behind adopting Red Hat OpenShift and how it aligns with DNEG’s long-term goals in the visual effects and animation industry?
DNEG’s long-term technology goal is to provide the best possible creative platform for artists, production managers and clients. Our company has grown to more than 9,000 employees in 10 locations around the world. At any given time, we need to juggle dozens of complex, multi-petabyte visual effects and animation projects. Adoption of Red Hat OpenShift makes it possible for us to adjust our complex proprietary software such that we can leverage best-practice architectural and development techniques. This promotes faster and more stable development of the applications which serve our artists across our six globally distributed data centres.
What were the most significant challenges faced during the implementation of Red Hat OpenShift and how did your team overcome these obstacles?
To best leverage Red Hat OpenShift, we decided to focus on some key parts of our existing desktop-based asset tracking system, reimagining them in a cloud-native architecture. While we had a clear vision of what we wanted to achieve, we had little practical experience of working with cloud-native architectures and platforms. The biggest challenge for us was to architect a system that would be efficient and reliable at scale: managing metadata for over 100 million digital assets, while in constant use by all of DNEG’s artists around the world.
How has Red Hat OpenShift enhanced artist productivity and creativity at DNEG and what measurable improvements have you observed since its deployment?
We have observed improvements in both flexibility and efficiency, as well as reductions in risk. Red Hat OpenShift has allowed us to fundamentally change the nature of our software, with some parts now able to be tested and deployed to production in minutes, rather than days or weeks. This makes it much quicker and safer to release updates to artists, who eagerly anticipate the creative and productivity improvements each software release brings.
Could you describe the nature of the collaboration between DNEG and Red Hat throughout this project? How did this partnership influence the outcome of the implementation?
We have had a fantastic collaboration with Red Hat throughout this project. We were able to leverage the knowledge and skill of Red Hat consultants and training to expand our knowledge and expertise in platform architecture and administration, DevOps techniques, Infrastructure-as-Code and API management.
What are DNEG’s plans for further integrating Red Hat technologies, particularly in areas like Machine Learning and API management? How do you envision these integrations impacting your operations and services?
In addition to expanding our migration from desktop-based to cloud-native architecture, DNEG is in the process of implementing Red Hat 3scale API Management to help reduce congestion through managing and balancing interactive and batch requests. Red Hat 3Scale API Management can help DNEG to prioritise artists’ computing consumption based on the importance or delivery date of the work.
Based on your experience with this project, what key lessons have you learned and what advice would you offer to other organisations considering a similar Digital Transformation initiative?
My advice would be to leverage the passion and experience of the team at Red Hat. They were with us every step of the way to make sure we gained the knowledge we needed to have a very successful initial outcome.
Click below to share this article