RED ONE Workflow in detail
- August 6th, 2010
- Posted in DI Workflow . Digital Cinema
- Write comment
Dennis Alaniz is a Chapman alum and former student of mine who has been a very busy editor and on-set DIT. As I have said many times, the new camera technology has created tremendous opportunities for young filmmakers to get hired and get on-set and go to town managing data, looks and how the final edit is going to be made. It takes organization, discipline and passion because there is a mountain of raw data out there ready to be corralled and whipped into shape. I think you will find his breakdown to be very useful.
RED WORKFLOW by Dennis Alaniz August 5, 2010
DISCLAIMER: As an emergent technology, a RED workflow that works today will likely need to be refined in as short a time as a month. The workflow I describe below suited my particular situation at my particular time. It is my hope that what I learned will serve as a guideline for any future workflow using RED technology.
And remember, the newer the technology, the bigger the risk. It hurts living on the bleeding edge. Trust me! The list below is what we specifically worked with. If you can get better, faster machines, do it!
Hardware
2 RED Cameras with Mysterium X Sensor (30.4 firmware and 30.5 firmware)
Please do what you can to have the same firmware on both cameras. It can save you a major headache later in the process.
6 16GB Flash Cards for RED Camera
You need more! Ideally, you would never reuse a card. We were stressed to keep cards clear so the film could keep shooting. An external drive is also an option, but moving parts make things much more precarious.
2 MacBook Pro Laptops for DIT
A laptop of decent speed that can connect to firewire or e-sata is necessary to keep cards copied and cleared. It doesn’t need to be laptops, but you should have one or two separate machines to copy the flash cards over to external drives. Laptops are more helpful if you need to stay mobile.
2 Flash card readers
As many as you need and can plug in to your machines. Just keep flash cards copied and empty IF the data has been verified by an MD5 after the copy.
Mac Pro (8 core, 16BG RAM) for Asst. Editor
You’ll need a fast machine to facilitate copying files, rendering video, and keeping several apps running. Most importantly, you need to stay as much in pace with the production as possible.
REDRocket video card
An extremely expensive accessory, but if you can afford it, it will keep you moving at an impressive pace and allow you to manipulate and play back the image with very little performance hit. Something your Director and Cinematographer will thank you for. Every single day!
E-SATA Card for Asst. Editor
The $20 from Best Buy will suit you just fine. As long as you’ve got a lot of very fast connections (that are hot swappable) to your Mac Pro because you’ll be shuffling a lot of drives.
6TB Redundant storage
8TB G-TECH RAID 5 with G-TECH E-SATA Card (not the same card as listed above)
6 2TB G-TECH External RAID 0 (Set Drives)
These six drives were broken into groups of three: A, B and C. When one card was copied off, it was copied to both drives in a group at a time. (ex. Card 001 was copied to Drive A1 and A2. What card was copied to what group depended entirely on availability. This redundancy will be explained in the Appendix 1 below. Exactly how much you’ll need depends on a great many factors. 6TB barely suited us. I would have been happy to have an additional 2.
6TB Work Storage
2 4TB G-TECH External Non-RAID
2 1TB External FireWire
The two 4TB drives were my personal raw RED storage, Import/Export, SFX Drives. Again, I’d have been happier with about another 4TB. This redundancy is also explained in Appendix 1. The 2 1TB were used as an emergency back-up to the editor’s drive and as a delivery drive respectively. Ideally, you’ll have a back-up drive onsite and offsite for every machine being used.
Optional Hardware
Laptop and USB External Drive (at least 500 GB)
Delivering low res dailies for use on set (more on this later).
Software: RocketCine-X (depreciated)
Viewing dailies and transcoding footage
REDCine-X
Transcoding for upres (the official replacement for RocketCine-X)
FileMaker Pro
Maintaining a shot database
ClipFinder (2.2 and 2.5)
Exporting shot information and still images for use in FileMaker Pro. Different versions will be explained later.
R3D Data Manager
Used to verify the MD5 of your clips after they’re copied off the card. This program is a must!
Final Cut Pro (Or the NLE of your choice)
Shot organization and synching dailies
Workflow
To reiterate the critical pieces of data from above, we shot on a newer model RED with a Mysterium-X sensor. That alone drastically altered my intended workflow. I won’t go into all the technology as to why, but simply say, know the color science of your camera! The Mysterium-X uses a new color science called FLUT and was the specific reason for this workflow.
This we know, the higher the testosterone, the more ready, willing and anxious a guy is to cost of viagra satisfy his sex itch. Key ingredients in Mast Mood oil are tadalafil cheap prices among those herbal products that have already gained highest acclamation. Vitamin D can treat impotence completely order levitra online is the opinion of qualified medical practitioners before taking them. It is a pure herbal online viagra australia supplement.
Knowing the color and gamma space your finishing in isn’t necessary, but it saved me a step early on in the process. Whatever you choose, be consistent in it when transcoding from RED. We were working in a color space of Camera RGB and gamma space of REC709. (ADDENDUM: Changing post houses last minute may facilitate rerendering all the red footage with a different gamma space. This is still an unknown)
As the director was also editor for this film, I was required to work from a trailer on set. This made my job easier and allowed me to keep pace with the production. Equipped with the above hardware, our day-to-day workflow is detailed below.
On-Set Workflow:
Footage shot on the RED to Flash Cards.
Footage offloaded using R3D Data Manager to verify MD5 and to duplicate on two of the above 2TB set drives.
Data from one of the two set drives is copied to one of two 4TB G-TECH Non-RAID drives. I was always delivered only A2, B2 or C2. This was to ensure that, if I corrupted a drive, I never corrupted the other in the group.
Using RocketCine-X (Now depreciated), newly copied footage is transcoded to 720p Proxies and saved to USB external 500GB drive. Footage is then transcoded again to 2K and saved on the 6TB G-TECH RAID.
720p footage is loaded into Final Cut. Organized into bins and properly labeled.
Raw RED footage is again loaded into clipfinder to export an XML of all data along with highest quality screen captures, one every second.
XML is imported into FileMaker Pro. Cross referencing organized footage in Final Cut, missing data (i.e. Shot name) are filled in FileMaker Pro.
Up to 3 screen captures per shot are added to the FileMaker Pro database for visual reference.
At the beginning of the next day of shooting, audio from the prior day is delivered and copied to the 6TB RAID
All footage captured the prior day should be in its own daily Final Cut Pro project. The prior day’s media is then reconnected to the 2K media on the 6TB RAID. This “online” project is synced to the newly received audio.
The day begins again from the top.
In post, the workflow isn’t so dogmatic. Some shots have been pushed in beyond what the 2K image can handle and so is being reconformed to a 4K online. VFX shots are also being reconformed to 4K.
4K Upres:
Place all footage to be reconformed into a new timeline.
Export XML from Final Cut.
Using ClipFinder 2.5 (THIS IS IMPORTANT! ClipFinder 2.2 CANNOT handle the following steps!) import the XML using the Import FCP XML with relevant settings defined (ex: VFX requires 8 frame handles).
Replace the MOV with the R3D media.
Send everything to a RedCine-X Bin. (This, specifically, is what you need ClipFinder 2.5 for)
Double check the color settings are correct for each clip and export them as 4K files (VFX clips go to their own drive, online 4K shots go back to the 6TB RAID).
For footage to be reconformed in FCP, go back to ClipFinder and choose Run Conform on FCP XML file. Make sure you point the application to the folder with the NEW 4K FOOTAGE!
Import this new XML into FCP. The newly imported clips can now be dropped into the show timeline and will be pointing to the higher res 4K footage. Double check that your push-in, repositions, filters, etc., match the clip it’s replacing. Resting it above the 2K allows you to keep the 2K as a reference in case something goes wrong.
The advantages to this particular workflow (on and off set) are:
It allowed me to run onto set with the low res proxy to answer any continuity questions with great ease.
I was able to show the director, cinematographer, costumer, etc., high res footage approximately 1-2 hours after it was shot. Using RocketCine-X, I was also able to grade some shots on the fly to verify we were on the right track.
Finding shots in the FileMaker Pro database was easy as I could cruise the database looking for a particular image, or search by day it was shot, scene name, etc.
Editing online could begin immediately. While on set, I put together a teaser to show the crew incorporating footage shot as recent as the day prior.
The delivery file to the post house will require no reconform later down the line. They intend to take our semi-graded 2K ProRes file, with an EDL to notch out the shots, and do any touch-ups to that file.
Reconforming to a 4K image is extremely easy as I have the raw RED material handy and, since the file names were honored, sending it to RedCine-X is painless and the REDRocket cuts down transcoding time drastically.
Disadvantages to this workflow are:
As the 6TB drive is connected to the editor’s machine and not mine, streaming 2K over an ethernet cable is not reliable. Pulling that much data and hitting the drive from two machines can cause a massive performance hit.
Sharing Final Cut projects isn’t easily done. To solve this, any changes I made on my end had to be copied to the editor’s project by hand on his machine to ensure he wasn’t editing the same project at the same time.
Editing online takes a LOT of hard drive space and render times can escalate quickly. To create a DVD of the full timeline can take anywhere between 5 and 8 hours.
Here’s a link to the FileMaker Pro template I used on set. It’s hard coded to extract the reel name based on where I had the raw RED files stored, so that will need to be modified to suit your needs. The included XSLT file is to handle the translation from ClipFinder’s particular XML format into something FileMaker Pro can handle. When you import an XML file into FileMaker Pro, you’ll see where to use this XSLT file. (NOTE: This has not been tested with an XML from ClipFinder 2.5)
That, in a nut, is the majority of my workflow . There were a number of subtleties in variation I had to employ to accomplish any particular tasks (ie. uploading for web, creating specific screener DVD’s) and can describe any of those in greater detail if you need assistance with your own workflow.
Appendix
The intent of our redundancy was to ensure integrity and safety of all critical data. After the film finished shooting, there were now 8 drives with 3 different copies of all the raw RED footage: one copy was split amongst drives A1, B1, and C1, another on A2, B2, and C2 and the third was split among my two 4TB drives Cold Storage – 1 and Cold Storage – 2. My two drives served as the drives I would use when, for whatever reason, I had to return to the RED footage to re-render anything. The two remaining sets were separated from each other and both put in a safe place in different parts of town. This ensured that the destruction of no one building would completely destroy the footage.
Do you know anyone on the east coast that has worked with 3D?
Thank you for taking the time to write this up.
It’s very clear, and takes something terrifying and makes it seem mundane.
If the show was being cut on AVID would the workflow have been cleaner?
Sharon:
My colleague Tashi Trieu has worked on 3D. For the East Coast, you might contact the owner of 21st Century Digital. He builds and rents 3D rigs and has been an advertiser on this blog. Their link is listed on the bottom right of this blog under LINKS.
Hi Bruce,
Avid has the toolset (AMA) to open and play .R3D files but when RED ONE first started building their camera systems they had an exclusive agreement with Apple and FCP. Final Cut can open and play the proxies at quarter or half resolution and their log and transfer tool is very easy to use. Most folks have Macbook Pros and they are usually a must have on set.
Because the RAW data coming off the camera is so large, a transcode is almost always the solution. In the case of this low budget feature, they are cutting on FCP and have a tower attached to a RAID using Apple Pro Res 4:4:4:4
Had it been assigned to Avid the codec likely would be something like DnxHD 115 or the like. Metafuze (an Avid product) would be used to do the transcode.
Then later on the metadata would be used to generate a final EDL and conform of the original files. They could then be transcoded to DPX 10 bit LOG files and graded on a Quantel or Autodesk Lustre.
Thanks for this article. It’s a very interesting read, especially since every DIT seems to have their own workflow. All together we seem to be working towards a standardised way of cataloging/organising/transcoding data.
Was this workflow developed over an amount of time over small projects or was it developed in downtime?
I’m very curious to see how the Filemaker database works but the file seems to be down. Any chance this could be fixed?
Thank you again.
Dan
Hi Dan,
I updated the link and here it is again.
http://dl.dropbox.com/u/364909/vault/RED_FMP_template.zip
Dennis developed this out of necessity as he knew he would be receiving the files and need to edit them using PRO-RES 4:4:4:4. I hope to have a follow up discussion with him in the fall as the editing progresses.
Hi
I’m the original author of the workflow. To answer a couple of quick questions, I was brought onto the film after a 2K ProRes 4×4 pipeline was discussed. This was conceived by the asst. editor before me and confirmed as a good idea by our post house. When I came on to replace the former asst., I was asked to flesh out the workflow. Believe me, I would have loved to do an offline/online!
My workflow consisted of avoiding the REDRocket due to budgetary reasons, but that changed when I learned about the new color science in the Mysterium X (damn bleeding edge) and I’m so very thankful it did. The REDRocket is absolutely essential to a smooth RED workflow.
As for the day-to-day, that was very much cooked up over the first two or three days as I learned the pace of the crew and the intentions of the director. I was originally under the impression that I’d be cutting along with the film to make sure enough coverage was shot. My duties very quickly became one of organizing and cataloging (with occasional editing for producers/director etc).
As to the question of Avid, the workflow would have very likely required an offline/online as I’m not aware of a 4×4 codec available for the avid, though I could well be mistaken. And because Final Cut did have exclusivity with RED for so long, their ProRes codecs proved to be very flexible and helpful.
Hope that gets everything for now.
thanks!
dennis
p.s. Dan, if you find a way to suss out the REEL ID from the file without having to look into the directory structure, please do share. That’s the one thing about my FileMaker Pro template that drives me nuts!