TL;DR: Testing the private beta Python API from Qarnot
About a month ago, while keeping an eye on the Libre Software Meeting (RMLL) in Beauvais and watching the news about WordWars spread on some blogs and magazine, I stumbled on a video presentation about open source software tools for animators and Blender 3D rendering using Qarnot (in french). And I was struck.
Qarnot Computing is a high performance computing service provider, which means they can do a lot of complex calculations, super fast, using a large amount of computers. This service, of course, is not new. For example, 3D content creators often use these external services, also called “render farms”. Some 3D renderings require a lot of computing power in case of complex scenes, long movies or highly detailed realistic simulations. And by externalizing that work to a large amount of processing power, this can be done at a fraction of the time necessary compared to what it would require on a single machine.
Where Qarnot stands out from its competitors is that instead of spending a lot of money on cooling and infrastructure, they decided use the excessive heat produced by those computers to maintain temperatures in private houses and buildings. So, instead of having one giant warehouse with thousands of computers that needs to be constantly refrigerated, they have installed a network of “radiators”, made of the same computers, in hundreds of private interiors. So no more energy is required to cool down the computers, the heat is used to make a home comfortable and no more horrible window-less farms either, the infrastructure is distributed across a city. Using this approach, they claim to have reduced the carbon footprint of such services by 78%. And I could not be more happy to test this on my own projects.
The next cool thing about Qarnot is that they provide rendering services for Blender. The Blender Foundation uses them for their latests open movie projects. They offer some free credits to test their services. And they are cheap too. All these were so many good reasons to give them a try.
My WordWars project uses Blender to render the daily war news from The New York Times into a movie that looks like the intro scene from StarWars. So every day, a new Blender file is generated and rendered. On my home machine, it takes about 1h30 to render but on my dedicated server, it takes up to 8 hours. And I’m running this project from the server because I want to keep my home computer for other things and its also not constantly up or connected. The server, well, is up 24/7. So I decided to contact Qarnot.
I needed to get in touch with them, because so far, the only way to launch a rendering using their system is via a web interface. It’s really practical for a one time project, but having a file that automatically updates every day needs a different approach. I do want to keep things automated. So I asked if they had any API, any way to interact with the rendering farm via scripts. And as a response, I was happy to get invited to test their Python API, still in private beta.
They provided me with the necessary Python files, documentation, and a small, but feature complete, example that I could start with. It took me a while to figure out how to operate it correctly though. Maybe because I never interacted with a render farm before or maybe also because I got a bit confused with some of the parameters and the limitations of the private beta.
One thing that I found confusing was the ‘framecount’ parameter. The ‘advanced_range’ indicates which part of the movie you want to render but I did not understand, at first, how this related (or not) with ‘framecount’. ‘framecount’ is also named ‘frame_nbr’ in another function (‘create_task’) and that also puzzled me for a while as what this variable really represented. After testing different approaches, I understood ‘framecount’ as being the maximum number of frames that you want to process at the same time (the whole point of speeding the process of rendering). I say maximum, because it “feels” different depending on the amount of processing power available for the task. And in the end, the whole range of frames you want to render will be rendered, it might just take more time.
As this is still private beta, I could only get access to some of their processing power and was limited in storage space for the resulting renderings (limitations that you don’t have normally). So in the end, I could only run portions of my project − 300 frames per task − and that would take around 10 minutes to process (rendering the whole clip would take a little more than an hour using this method). So I’ll keep all these as tests for now and wait for the public release, which they should announce by the end of this year. Testing the same render (full power) using their web interface, the whole clip was rendered in less than 10 minutes. And the cost for the service would be around 0.75€ for that particular case.
So using Qarnot, I could be delivering a WordWars video clip, 10 minutes after the news have come out, at an affordable cost of less than a euro and without feeling guilty by double heating the planet.
I’m also guessing that, since the Python API they provided runs under Python 3.4, you could also expect a Qarnot-plugin that will integrate directly in Blender interface. And if they are not the ones to create this, someone will do it for sure.
I want to thank them for allowing me to test this and the patience they took to respond to the many questions I had. I also wish them success as this is truly an inspiring way to create a computing company.