Australian Sites Chop Render Hours on Marvel Project with Single Submission Tools and Detailed Log Tracking
Last Updated: December 18, 2015 7:35 pm GMT
(Los Angeles, California--December 18, 2015) When many studios are limited to one facility and a small bank of computers, Australia’s Cutting Edge was sitting on a gold mine – two big VFX studios and all the hardware that came with them. Two times the resources doesn’t mean two times the efficiency, though, especially when your farms don’t talk to each other. To create that bridge, the team invested in PipelineFX’sQube!, a render farm management tool tailored to the needs of big VFX projects.
Since 1992, Cutting Edge has built a string of studios that employs over 150 artists and reaches from Brisbane to Sydney. Like most professional shops they can do it all: films (Predestination, The Age of Adaline); commercials and TV series, including
, PlayStation Network’s first scripted show that’s based on Marvel’s beloved graphic novel series. What they couldn’t do was run all render requests off a single dispatcher, and the minutes kept adding up.
Plans to link the farms in Brisbane and Sydney started with a test. The idea was to put render farm managers against each other in a multi-site battle, utilizing both live and offsite workers. This became the proving ground for key needs like: global log files tracking, scripting, platform compatibility and project mirroring. After tests totaling 10,000 jobs, Qube! was declared the winner, in large part, because of how well it could handle the logs.
“With other render management tools, reading log files for machines that aren't local was problematic, and reading log files is an incredibly important part of seeing what's going on and troubleshooting,” said Rangi Sutton, VFX Supervisor at Cutting Edge. “Our TDs and support crew can now dial up logs for a given job through GUIs or the command line tools supplied on any workstation. It’s the only way to see how everything's going across the sites.”
By running off a single dispatcher, artists can submit jobs onsite or remotely through VPN and get access to the power of two farms. With this single point of control, Cutting Edge can now count the software licenses for applications like NUKE for all users over the group. Previously, license allocations needed to be predefined between Sydney and Brisbane. During the day the farms consist of dedicated slave units, but after 7pm, all idle workstations join the pool, offering a 20-30% boost in rendering power every night.
Another key feature that drew the team was Qube!’s native compatibility with OSX and Linux, and the Unix-like design. “We already have in-house tools to manage our environment for our major applications; Maya, C4D, NUKE, Houdini, so we can control the specifics of the software used per project and even per shot. While Qube provides a lot of built-in job types that would help smaller studios kick things off, they don’t get in our way when we choose to ignore them. Essentially, the way we initialize a session at the workstation on the terminal, is exactly the same as what happens on the render farm. We don’t run into new sets of problems for farm-rendered jobs such as user permissions or application versions.”
Running simulations in Houdini proved to be another source of time-savings thanks to per-frame dependencies. In the past, the team found their system was dependent on stages completing before the render could move on. With Qube! that all changed.
“Now we can send stages to the Supervisor and tell Qube! how to perform the operations: pre-processing, simulations, post-processing, followed by rendering. We can configure the granularity of these dependencies; simulations will require all the pre-processed geometry to be available before commencing, whereas a Mantra render need only wait for all the geometry for its given frame.
Using the Qube! Python API, we can describe all these stages from inside of Houdini's Python interpreter, and submit the whole dependency tree in one go,” said Sutton. “Retakes of big effects scenes can be re-simulated and re-rendered through on a single-button click, without any unnecessary idle render nodes or wrangler intervention.”
In the near future, Cutting Edge will also be able to do something that was impossible before: track farm data for every project. “We had access to resource tracking – how long a crew has been working on individual shots, different storage stats – but never a stat like comparing CPU cycles against project size. With 50,000 render jobs over the last six months, it’s the aggregate data that Qube! is recording in its data warehouse that will help us make a lot of important strategic decisions. Now we can apply cost-accounting to the render usage, as well as make software license decisions based on the actual usage over time.”
About Qube!™ and Smart Farming™
Qube! is an intelligent, mature and highly scalable render management solution that can be quickly integrated into any production workflow, and is backed by world-class technical support. Smart Farming delivers intelligence to production pipelines by providing business-critical insight into render pipelines, maximizing investment in rendering infrastructure and automating manual processes. Qube! works out of the box with all leading content creation applications and is truly cross-platform with all software components available on Windows®, Linux®, and Mac OS®X operating systems.
As the leading provider of intelligent render farm management solutions for digital content creation, PipelineFX provides software, support, consulting and training services worldwide. Qube! runs on more than 30,000 render nodes at over 700 customers in 51 countries and is used to render film and visual effects, post production, broadcast, design, games and education media. Clients include Apple, Arc Productions, Base FX, BBC, Deluxe, Dyson, Electronic Arts, Google, HBO, Hasbro, L’Atelier Animation, L.M.U., Lockheed Martin, Mackevision, Method Studios, MPC, NBC Sports, Next Media Animation, Nvidia, NHK, NYU, Original Force, Pratt Institute, Procter & Gamble, ReelFX, South Park Studios, SVA, Target, Technicolor and many others. PipelineFX is headquartered in Honolulu, HI, and has offices in Los Angeles, London, Portland, Las Vegas, and Vancouver.
About Cutting Edge
Cutting Edge is an Australian VFX, Picture and Sound Post Production Studio located in Brisbane, Sydney and The Gold Coast. It is a privately owned independent company employing 160 people across 3 sites. Credits featured for 2015 alone include multiple local and international productions; Warner Brothers’ “Our Brand is Crisis”; Sony Pictures’ “Powers” and “Underground” television series; Independent Features’ "The Moon and The Sun”, "Last Cab to Darwin”, “The Shallows” and “Infini”; Plus Television Features and Series’ “Deadline Gallipoli”, "House of Hancock", “Wonderland”, "A Place to Call Home", "No Activity”, and "Jack Irish" to name a few. With an unprecedented number of productions in progress and slated for 2016, Cutting Edge’s infrastructure requirements have been growing at a rapid rate.
Copyright 2015 PipelineFX, Inc. PipelineFX is a copyright of PipelineFX Inc. Qube! and Smart Farming are Trademarks of PipelineFX Inc. All other brand names, product names, service marks or trademarks belong to their respective holders.