It Came From The Searches Volume 2
Matt Stratton turned me on to Clicky Stats awhile ago, and through Clicky, I’m able to see what web searches come up with my blog as a result. I also see when these searches lead them to my blog – but then they leave because they didn’t find the answer. I thought since the search engines think I already have it on my site, perhaps I should. Below is a sampling of the search queries (that I can decipher from cryptic keyword searches) that my website supposedly already has the answers for. So, I present to you:
It came from the Searches, Volume 2.
March 10th, 2010 – March 18th, 2010.
optimum use of cores on macpro using redrushes
As many as humanly possible. Unfortunately, Red Rushes is not as optimized as it could be, so there will be plenty of horsepower left over, even when red rushes is chugging away. We have to wait for Red to enable distributed transcoding, or, use another encoding solution.
FinalCutServer Slows down
What is slow? The transcoding? Check in / check out? Network traffic will slow check in / check out. Background processes and concurrent users will slow down Final Cut Server – especially if transcoding is happening in the background.
“Prores Avid Import” , “importing FCP into Avid MC” , “convert DNXhd to pro res 422” , “how to import mac prores into avid”
Right here: http://michaelkammes.com/editorial/getting-final-cut-pro-projects-andor-media-into-avid-media-composer/
P2 or DNxHD?
What flavor of P2? You can edit more tracks within Media Composer in real time with DNxHD. You also have more latitude when color grading and compositing, but files sizes are traditionally larger (assuming DNxHD 145 or greater).
unity nab rumours
Yeah, sorry. NDA.
avid dnxhd convert to prores
I assume you are on a PC. Windows cannot encode into ProRes – Apples limitations. If you’re on a Mac, you can export to ProRes from the Export menu, and change your codec / compression to ProRes.
Avid-qualified Windows-based with quad core processors
http://www.avid.com/products/Media-Composer-Software/system-requirements.asp
I will caution that if this a single quad (and not a dual quad) then only the newest Nehalem based Processors (HP z400, for example) can handle DNxHD encoding in RT.
Avid Mass Storage
Avid has several flavors of their own storage –VideoRaid SR (local) and Unity (shared) and ISIS (enterprise shared). However, Avid can use just about any storage that your OS can understand. I recommend firewire400 at the bare minimum (no USB) and esata, infiniband, or fibre if you can afford it.
“michael kammes”, “mkdc”, “micahel kammes”, “Michael Kammes Burbank”
Why yes, that’s me. Whats up? Say hi to your mother for me.
“DNxHD PE codecs IN FCP 6” , “quicktime output avid media composer for final cut hq“
Ya gotta download them.
http://avid.custkb.com/avid/app/selfservice/search.jsp?DocId=263545
FCP can export to any codec QuickTime understands…so, once you download the codec you’re golden. Keep in mind , this will export a DNxHD .mov file, NOT a DNxHD .mov in an MXF wrapper, which is what Avid needs to understand a file without transcoding or a quick import.
getting broadcast quality for fcp mov
That’s tough. ProRes is not a SMPTE standard, therefore, delivering a ProRes file will never be required for a broadcast deliverable. So, you can either A) lay it off to tape or B) transcode to a SMPTE compliant file. DNxHD, for example, is a SMPTE standard. DNxHD 145 closest in quality to ProRes 422.
“”final cut” network storage best practice” , “final cut server edit in place”
Permanent storage, like fibre. FCSvr does not like storage to go offline. Ever. Thus, get permanent storage, and set it as an “Edit in Place”, so FCSvr does not need to copy media to and from it to do conversions. EiP tells FCSvr that the storage is permanent. I recommend a RAID and several spindles, A) for speed B) for redundancy and C) faster performance for multiple users.
inexpensive final cut pro server
Buy a new, base level Xserve. Buy a 10 seat FCSvr license, buy a small RAID5 array, and buy Matt Geller’s book “Getting Started with Final Cut Server.” For anything more advanced, you’ll probably have to contract out.
the best fibre storage for final cut server
How much do you need? I’ve always been a fan of Sonnet or G-Tech.
better than final cut server
CATDV, in most cases. http://www.squarebox.co.uk/
edit in place final cut server rights share
If I understand this correctly…. Yes, you need to have Read AND Write permission to any volumes FCSvr will use. Yes – even if you don’t intend on writing to the volume, you still need that ability. It’s a limitation of FCSvr.
mxf wrap dnxhd
High(er) end encoding solutions can handle it, like most of Telestreams product line.
do stereoscopic 3d mxf files take up more storage space
Nope. Avid dumps half of the picture information to get to signals (left eye and right eye) into 1 HD Frame size. The remaining 2 halves, when combines, equal one regular frame. Thus, same file size.
“avid media” samba share
Nope, can’t do it – if you’re trying to share an OMFI or MXF folder within Avid.
Mac Pro 8 Core 2.93 Nehalem install red rocket
Red Rocket is not entirely stable on Mac, and the workflow is kluge. Wait a little bit.
difference between pro res 422 & pro res 422HQ
Higher bitrate. Both are 10bit, but HQ has the ability for higher quality because of less compression.
quality loss transcode to 220x?
What was the original source? If uncompressed, you should see no difference on the scopes.
“Ruby on rails and final cut server” , “final cut server Web-Based Review and Approval issues”
One of my favorite “wow!”s during a demo. Ruby on Rails released a “module” for FCSvr which gives basic functionality and framework for a web based review and approval process. It’s buggy and kinda unreliable, but gives people a GUI concept of what FCSvr can do.
dnx36 file format
Great Avid Offline Codec. Not broadcast quality, but looks great. DNxHD36 is for 23.976fps material, DNxHD45 is for 29.97fps material – but both belong to the DNX36 family.
optimum compressor config for final cut server
Use those virtual clusters!
http://www.kenstone.net/fcp_homepage/compressor_multi_cores_stitzer.html
http://www.fcsoutlet.com/home/Studio_Outlet/Entries/2008/9/1_Virtual_Clusters_-_Compressor.html
Hum in Avid sdi Mojo analog audio signals
Sounds like (ha!) you have unbalanced audio going into your Mojo SDI. This is common – with no ground, hums (60Hz, usually) often occur because of dirty power and/or a power cable being to close to your audio gear. Put a ground lift on your AC plugs or power-strips
encode red r3d episode engine
As long as you have the codecs on your machine, yes. However, it’s the Wavelet extraction only, at 2K.
final cut pro and avid unity bugs
Decreased performance for starters. But that’s been known. Expect a 20-30% speed drop-off when using FCP on Unity.
p2 versus xdcam HD in Avid MC 4.0.5 2010
What flavor of P2? I like how XDCAM HD 35mbit looks compared to DV100. However, I like the way AVC based P2 Media looks, but it needs transcoding (for now)
“Macbook Pro”+”dual boot”+Avid+FCP
Yes! I do it on my laptop. You need to blow your HDD and OS away, and partition the drive off into at least 2 partitions, and install an OS on each one. Then, install FCP on one partition and Avid on the other. I do this so I can independently update my OSes without hosing up the other. Hold down the Apple key on boot to select which partition to boot into.
HD AT dnx36 broadcast quality
No. DNxHD145 is the baseline for broadcast quality. Yes, you can transcode DNX36 material to DNX145, but while that may meet the deliverable spec, it won’t look good.
final cut server metadata dnxhd
If it’s DNxHD in an MXF wrapper – no. Final Cut Server cannot parse all of the metadata in an MXF wrapper –currently.
“mac mini” NLE
Yikes. Holy horsepower Batman. You’re gonna need it, and the mini doesn’t have it. I’m not even sure FCP will install.
edit dnxhd in fcp
Not natively. Downlaod the DNxHD codecs, and put the timeline in unlimited. Or, pre-render into Pro Res for an easier time editing.
Encoding During Editorial
Last week on my POST Magazine blog, I briefly discussed Encoding in Post: The Four Hot Spots. I figured, “Why not elaborate on one of those areas?” Thus far, I’ve discussed the concept of Pre-encoding, and various facets of the final encode. Let’s talk about the most vital and often overlooked area: During Editorial.
Of course I’m (initially) referring to the almighty DVD, which all of you assistant editors need to burn daily – all, naturally, with different watermarks and circle takes or various cuts. Plus, that Blu-Ray for that Director with the fancy car – you know who you are.
However, more and more above your paygrade individuals who foot the bill for your paycheck are using non-traditional devices to view media. The web has undoubtedly become the new standard, with petaflops of media files being FTP’d daily. Flash, Windows Media, and heavily Compressed Quicktimes are all popular options – and many times, are all needed. New fangled video enabled mobile devices – like the iPhone, Blackberry, and Droid – have increasing become the way to view video while on the road between business meetings and Starbucks pit stops. Those facilities that have been wise to adopt Digital Asset Management (DAM) internally will also need a version for tracking, as well.
So, yes, dear assistant editor, you now have several DVDs, web versions, and mobile versions to create. Plus, there is always – always – that oddball version you need to create. It’s inevitable. All in time for the next edit session tomorrow.
This is when you need encoding. And a hellova lot of it.
I can only hope that all of you, dear readers, are familiar with the concept of a Quicktime (QT) Reference. No? Then Unca Michael will enlighten.
A QT Reference is simply a pointer file.
Quicktime file vs Quicktime Reference file (Click to Enlarge)
It’s a small media file that appears as a full movie file to most media players, but during playback, it points back to the initial media used in the generation of said QT reference file. Think of it like that bookmark in your web browser. That bookmark isn’t the entire website, but it pulls up the website when you click on it. It’s a link.
QT references are your best friend. Why? Most NLE’s can generate a QT Reference much faster than exporting a whole “complete” file. Thus, less time watching a progress bar. (Although, ironically, having the wait time does give you time to get coffee to stay awake and watch the progress bar longer. Interesting.) This speed boost is accomplished by having the NLE only create new (render) files for media that has some sort of effect on it in the timeline. All other media is untouched, and thus, the QT Reference can point to it. Brilliant!
So, I’ve already saved you oddles of time. Waiting.
Now, I need to earn that bonus, and you need to take this QT reference goodness and utilize the strength of it’s pointers to create the compressed files needed for your deliverables.
At this point, it may be a good time to check out this post, as I explore choosing the right encoding solution. In a nutshell, you need an encoding solution that can accommodate all of the formats you need to deliver. It’s mighty beneficial to have a solution that can encode into multiple formats at once. Again, less thumb twiddling time.
Here is where I drop the “But…” bomb. I must tell you something vital when dealing with QT references: Your encoding solution MUST sit on the same network as your edit system and media storage. Huh? The QT reference is aware of where the original media is in relation to itself when it is created. If a user were to say, I don’t know, email the file, or even move the QT reference to another folder (even on the same system), the link to the original media is lost. Yeah, big gotcha. That’s why I recommend having your encoding weapon of choice on the same SAN/NAS as your edit bay…or have a firewire drive on standby to copy the QT reference and media (retaining the same hierarchy) to move from the edit bay to the encoding solution.
Moving a Quicktime Reference out of it's natural habitat (Click to Enlarge)
Solutions like Root6’s Content Agent (see my review here) are actually built explicitly for this purpose: not only can Content Agent sit on any Windows compliant SAN, but the system can handle concurrent encodes, auto FTP files, email status updates, and automate DVD burning with watermarks. And while I’m sure burning and labeling those DVDs fulfills your life’s purpose, there is even a more efficient way to streamline that. Look into Rimage: Automated network DVD authoring and label printing, scalable from small runs to large runs.
While I am sure this will not keep you from many late nights in the edit bay, it very well may give you a few more hours to crash on that old couch in the back of the room.
Conceptual Workflow: Conditional Encoding Part 2- Web and Distributed Encoding
Last week, we covered the concept of Conditional Encoding (KCM: Kammes Compression Methodology, still looking for investors) and you may have noticed that I put an odd node on the graphic workflow. The LOGIN. Why? Because the next phase of this workflow is going to piggyback on not only the Internet, but on your LAN as well.
So, we’ve already built the logic which determines the parameters for the encode(s) via the web browser interface, and delivered that to the encoder. But now we need to get the files to be transcoded to the encoder. If you’re on a LAN (internal network) then it’s time to call the I.T. nerd, and have them map the path from your encoding solution to your computer. This creates a link to the encoding solution and your computer which houses the media. Both systems now “see” one another, and given the proper permissions, they can read and write each others files. A user can now move the files to be transcoded (sometimes as easy as drag and drop) into the appropriate folders on the encoder. At this point, the encoder utilizes the settings from the web interface, applies them to the files, which the user has just moved and WHAMMO – encoding begins.
Further automation is very possible, including setting up watch folders on the encoding solution to make copying of files easier for the end user. Instead of having to navigate to a specific and oddly named folder, a user cpuld drop the file on an easy to read and locate network folder. Since the encoding solution is constantly watching this folder, it’s as if the files were deposited directly on the encoder. Many solutions can also send emails on the status of the encodes, and can even do their own housekeeping – moving and deleting files to maintain free space and organization.
To streamline the process further, solutions like StorageDNA have the ability to copy files over your network faster than a traditional drag and drop. Add that feature to delta synchronization, encryption, and a host of other features, and now you have a pretty efficient solution.
This solution is perfect for several users needing to access the encoder on a private, local network. And guess what? Short of LAN acceleration, some solutions actually already do all of this. Which is why this workflow doesn’t stop here.
Let’s say you’re remote. Or – better yet – you want to start a business for encoding for the teeming millions out there?
That presents a bit of a problem. How do we get the media to the Encoder? How to we marry the logic based encode parameters to the media? And most importantly, how do we bill for it?
With DAM. Digital Asset Management.
DAM solutions are best suited for their own tech column post (forthcoming) but essentially, it’s a piece of software that organizes all types of data – media or otherwise. It catalogs every conceivable piece of data about the file (metadata), and enables a user to search across all of the fields – including custom ones. By depositing the web based encode parameters into the DAM, we now have a record of the user, their encode parameters, and a place for the uploaded media to reside where it can be referenced by the DAM. All of this data can then be dished off to the encoder, where the machine chugs away. The encoded media is then returned to the DAM, where information like “encoding time” is cataloged and organized. A report can then be generated for the client. Client’s can then be billed for the encode time. Some DAMs, like CatDV and Final Cut Server are extensive enough to allow for backend coding to allow for a review and approval process. (See my last entry on FCServer.) This would allow a client to, say, watch several versions of their encoded file (with a watermark, natch), pick which file best suits them, and then download. Oh, and pay.
Storage DNA also falls into this nicely. It offers the same benefits on a LAN as it does a WAN. Faster transfer times, resuming broken connections, and even a web interface to track the files.
I know that’s quite a bunch of tasks and features for these puzzle pieces to “automatically” do. While the basic framework (“hooks”) for this workflow is built into many DAM solutions, don’t kid yourself – this will require some forethought and engineering time. But it is possible.
The last and final step is to optimize your encoding solution. For high volume batches, consider using software that scales. That is, software that can distribute the encoding work to other machines to get the trancoding done quicker. Telestream’s Episode Engine family is excellent at this. Apple’s Compressor can also accomplish this – albeit with limited format support.
Here is our finished workflow. Pat yourself on the back.
Completed workflow. (Click to enlarge)
Conceptual Workflow: Conditional Encoding
Venn Diagram of Encoding Ability
The post industry lives and dies around the concept of deliverables. What specifications have to met to appease the viewer, server, or engineer on the other end. Many times, just getting the deliverable out is a chore in itself. The last encoding format sheet I read from a leading encoding manufacturer had 5 pages of supported input / output formats. Being able to decipher these often cryptic encoding acronyms and numeric values appears to need a degree in engineering.
This is Science.
Translating from one medium to another has it’s own inherent problems. Does the image blur or breakup when the camera pans? Is the audio intelligible after it’s compressed? Do the lettering or graphics remain legible when the image is resized? Often it takes a slow hand (and a smooth touch) to know when to finesse parameters to convey the artistic vision when transcoding.
This is Art.
Depending on the deliverable, one may have options as to what path is the easiest to walk down. Is a certain codec really the right choice? Will it play nice in all browsers or devices? Will it cause the media to load slowly? These kinds of decisions are based on thorough knowledge of the medium, and the gotchas associated with the platform.
This is Experience.
There was a time when all three of these traits of these were the core component of a now fading job: The Compressionist. Like a Colorist, their job was highly specialized, and had a job to ensure the best quality and visually appealing image once the media left the edit bay. Our attention span has now waned, and our deadlines have been cut short, and one click presets are rapidly becoming commonplace when outputting.
There has got to be a better way.
Replacing the job of a Compressionist – the human eye, the human attitude, the human heart – cannot be replaced. However, we can begin to bridge the gap with what I am calling “Conditional Encoding”, or for you coding geeks, “Boolean Encoding” (patent pending). Of course, both my employer and myself would prefer KCM: Kammes Compression Methodology.
There are baselines that can be assembled for specific criteria. This, coupled with easy to understand non-techie based concepts yield a easier path for users to navigate: An example:
Does the source media involve fast motion?
User: YES
Encoding System: increase data rate. Change motion estimation to a higher level. De-interlace if the file is interlaced. Perform a 2-pass encode.
User: NO
Encoding System: maintain average data rate, reduce motion estimate factor, perform 1 pass
Does the media include large amounts of dialogue?
User: YES
Encoding System: Use less compression on audio, apply equalization favorable to vocal frequences, reduce volume on non vocal frequencies.
User: NO
Encoding System: Use more compression, apply broadband equalization.
Logic Based Workflow
So, we can essentially build rules based on the human perception of the source media. By utilizing these within a Boolean conditional set (IF, THEN, ELSE logic), we can very quickly create a set of encoding parameters “closer” to what a Compressionist would choose. While they will not replace the role, they will certainly get the dart closer to the bull’s-eye.
Think of Choose Your Own Adventure books. Each subsequent decision is based on a prior decision. By incorporating this methodology, this logic could create the desire encode. Once applied to each deliverable, the process becomes streamlined.
Another key to this is presenting users with options and descriptions which translate between technical and laymen. Bitrates could instead be replaced by “need more space”, or “higher quality”, for example. Logic would have to be configured and scripted in to the parameters to balance the users options when answers conflict.
To further efficiency, this could be incorporated into a familiar interface which users could also readily utilize and understand, like a web page. This means an easy to use GUI for the user, the possibility for remote access, and the ability to be ported over to many platforms.
The webpage could then conceivably generate a data file that the encoding solution could parse and understand. Formats like XML have long been a web standard, and several encoding solutions already utilize this format: Root6’s Content Agent, Digital Rapids Stream software, and several of Telestream’s product offerings already have this functionality (albeit limited) built in.
Several consumer software solutions have “wizard” functionality, which incorporates a slimmed down version of this concept. Apple’s Compressor, for example, has once-click device encodes (DVD, iPod, etc.), or time-based encodes (how much material do you need to fit on a given medium). Each of these selections loads up a preset of encoding parameters. This Boolean Encoding brings this concept to the next level.
Compressor Format 'Wizard'
Compressor's DVD Encoding Presets
I can see it now: The Microsoft Paper Clip A.I. turned encoding vehicle. I think I just died a little inside.
Next week, I’ll explore a much larger scale workflow which not only incorporates this concept, but utilizing it as a possible side business – or even a current business enabler.
The Money Room
Off and on for several years, I was involved with a post facility that had what they referred to as “The Money Room” Quite apropos, not only for the greenish hue to the walls, but what they *did* in that room. Unbeknownst to them (but now beknownst to me) the so-called castoff activities and backroom chores which took place in that space are now the new(er) ways to make money at your post facility…and even be a marketable service.
In this room, aside from the usual barrage of CD and DVD authoring, download and uploading of files, temp graphics and label creation, they did basic encoding, usually by a lesser paid assistant. Certainly not glamorous, but essential. Definitely not the first notions of what a post facility does: Offline and Online suites. Finishing. Audio rooms or dub stages. The flagship rooms.
Why?
Well, typically your talent – the editors who have named clients – command more in terms of pay than the backroom assistants, and the talents’ workspaces also have a lion’s share of the gear with which to make them shine.
Yes, I am of course referring to the almighty R.O.I.
With the current race to zero, rates for the client focused suites are continually dwindling, definitely at odds with the cost of gear and talent operating within them. Possessing a ‘Money Room’ already begins with less overhead – both financially and technologically.
So, what can you do in this money room to earn some of dat cabbage?
First and foremost: Encoding. Every website nowadays is content rich, from youtube to Facebook. Everyone has multimedia on their phones. All of these media riddled avenues necessitate a *special* and unique format. Those have to be created somewhere. Why not from you?
Utilizing your offline / online bays to chomp through 100 different formats for deliverables – when you could be billing for editorial or finishing in the room – is simply poor planning. A facility could conceivably upgrade to a new computer for the Online suite, and use the older machine as an encode station in the Money Room. This not only boosts the productivity (and marketability) of your Online suite, but also gives your Money Room a CPU to encode with.
So, I have a box with some processors. Now what?
Quicktime References. Have your offline / online bays and your encoding station see the same storage over a SAN. Whether it be via SMB re-share / Ethernet, Fibre, or simply cloned / portable firewire drives, these are a sure shot to not only increase productivity, but create a more efficient workflow. Have your editorial room export a QT Reference which links to the original media, then have the encode machine pick up that reference file, and let the number crunching commence. You’ve now significantly cut down the export time out of your edit bay (QT References are much faster to generate than a complete export) and also freed up the client bay for other activities. Hopefully more glamorous. Hopefully billable.
I recommend you create encodes for the following:
• H.264 or the like for web review and approval, or FTP uploads. Perhaps even iPod or iPhone versions.
• Flash versions for embedding in websites.
• MPEG formats for DVD or Blu-Ray dailies and/or screeners.
• Predefined proprietary formats for youtube, Facebook, myspace, Hulu, etc. Each site has its own requirements for submissions. Perhaps you can charge per location’s format?
Advanced encoding software packages allow for multiple simultaneous encodes on one machine, and some allow for distributed encoding over many machines. Others still utilize watch folders that are always looking for a QT Reference to begin encoding from, and even sets of parameters for multiple groups of encodes. What a value add it would be to tell a client that you could not only do the editorial, but give them deliverables for any destination they could desire – same day.
So, you’re not billing out your editorial room enough to justify something like this. I get it. As an example, this is where the promises of highly compressed formats (such as RED) being quicker can actually backfire and allow other revenue streams.
These abnormally large sized and compressed files are still a very, very intensive process for editorial machines. It takes a great deal to chomp through a 4K file – especially when 99%, if not more, of this material will never be viewed at 4k, or even on a playback system that would do it justice. More often than not, you will see it at less than 50% of its original quality – HD – or even less on broadcast TV or on the web. Given this truth, one can make an argument for simply downrezing the raw 4K footage to a manageable frame size and codec; like DNxHD for Avid or ProRes422 for Apple. This previously difficult to manage format in now in a much more edit machine friendly format for use in the editorial process. These formats can exceed broadcast quality standards – very appealing.
So you’re a purist – I get it. You want to have a 4Koutput. That doesn’t mean you can’t do a pre-encode (in this case, transcode) to an offline format for ease in editorial. Despite being suitable for broadcast, DNxHD and ProRes 422 – as well as DVCProHD – make create offline codecs, too. Provided the computerized tool (or the assistant!) does things right, your facility can matchback from the offline material to the 4K when onlining. Sound familiar? This is what telecine houses have been doing for years to DVCam –and charging you a ton for it.
I’m amazed at just how under-utilized this concept is: not only as a pure way to tighten ones belt, but to simply be more efficient. As an example, I happened to be at one of the studios here in Hollywood that gets the editorial output – rough cut and fine cuts – in one, and only one, format. A format that is antiquated and popular almost 10 years ago. Each department downwind of that facility spent hours encoding into a format they could use with their systems, meeting their visual and technological specifications. Imagine the amount of money spent in manhours working around this issue.
The film? Catch it this summer in theaters. Budget is $200 million.
UPDATE: More Extensive RED Benchmarks
**v 1.1 – added RedRushes for DNx36 & 220 & Pro Res 422 HQ batch encoding
I’ve decided to expand my testing after inquiries regarding other encoding solutions…and it developd into benchmarking single & batch encode times into various codecs with various encoding solutions.
Specs, standards and universal notes:
2.93 GHz MacPro, 6GB RAM.
10.5.6 / QT 7.6
Avid Codecs 2.0 (shipped with Media Composer family 3.5+)
All media local on OS Drive.
R3D Proxy _H quality was used for all tests.
Builds tested: 16, 17, 18.
10 clips ranged from 00:26 seconds to 04:31. Median was 02:19. Since every editor’s batch will be different, this was a ballpark for an average shoot.
All clips were resized to full frame HD frame sizes during encoding. As a side note, the frame resizing from the native 2048 x 1024 to HD frame sizes was not a significant factor in the delta for encode times.
No LUTs or image adjustments (aside from resizing) were used.
Pro Res 422 HQ -The highest quality compressed HD codec that Apple offers. Exceeds Broadcast standards.
DNx36 is 1080i/29.97 8bit. The lowest resolution of HD Avid offers. Used for offline editorial.
DNx220x is 720p/59.94 10bit – The highest quality compressed HD codec that Avid offers. Exceeds Broadcast standards.
REDCODE RAW Quicktime Codec: 3.5.0
FCP: v6.05 (FCStudio 2)
RED Final Cut Studio installer 1.0
RedRushes: v3.60
Compressor: v3.05
Compressor Local Virtual Cluster: 16 instances, all local.
Episode Pro (Desktop): v5.1
Episode Engine (16 Processor License): 5.1.2. Split and Stitch disabled, as there seems to be a bug in the stitch process.
Final Cut Pro L&T: Batch not applicable; Log & Transfer only processes 1 file at a time. DNxHD codecs are not traditionally used within FCP.
Red Rushes: Batch not applicable, only 1 file processed at a time. Quarter Res Debayer Quality.
Compressor: Batch not applicable, only 1 file processed at a time.
Episode Pro: Batch not applicable, only 1 file processed at a time.
Findings:
Amazingly, those of you who use Final Cut Pro as your editor will find you have the seemingly fastest encoder out of the bunch – and free. It does require some basic setup to get the cluster working – and is known to be flakey, but seemed to be a rockstar during my testing.
It should be noted that the free RED codec for Mac OS – REDCODE – is *still* a Quicktime component. That means no matter what encoder you use, the QT component will be the bottleneck. In addition, whatever bottleneck Redcode with QT causes, it’s only part of the equation: The codec (in this case, ProRes and DNxHD) you are encoding to must be written to be able to take advantage of multi threading.
RedRushes utilizes REDline as their encoder, and seems to be the best at utilizing available CPU horsepower. It averaged 15-20% more processor usage at any given moment then any other non batch encoder (FCP L&T, non VC Compressor, and Episode Pro). That being said, this was usually only around 50-55% at best. Batch encoders seemed to be able to take advantage of the remaining processor cycles, although Compressor with a VC seemed to be average 95-100%, whereas Episode Engine lagged behind between 80-85%. Unfortunately, the Stitch function of Telestream’s Split & Stitch technology seemed create a playable but greenscreen media file after stitching, so that feature had to be omitted. This feature may yield better results.
Pro Res 422 HQ, across the board, yielded slower encode times. Avid DNx220 would be the Avid equivalent to ProRes 422 HQ (although, technically, it should be vice versa) and was always done quicker. This is by no means a visual quality test, this was raw speed.
Although I cannot prove it (aside from my results here) it seems some encoders just “play well” with some codecs and data rates (i.e. high compression/low data rate DNx36 vs. lower compression / high data rate DNx220 & ProRes 422 HQ). This contrasts with the Episode Family, whose encode times were pretty similar across codecs.
All testing was done local (internal OS drive), as the differences in mass high speed storage varies from user to user and therefore difficult to baseline. I define mass high speed storage as RAID sets with Firewire eSATA, Fibre, or SCSI connection. While I expect times to be similar when Firewire/USB drives are used as the source drive (as most batch encoders write locally to a cache for processing, then write back out to the destination drive), I certainly expect encode times to decrease when mass high speed storage is used, as larger files require more time to write after the encode is done and the cache has to copy out to the destination drive. I do not expect this to be drastic, but it may save a few minutes each hour.
I attribute the increased times with batch encoding with Compressor with VC to this. (I know there is a Cluster option setting for this, however altering it seems to break the Virtual Cluster) I could have decreased the encode time by up to 20% if the application did not have to write out locally, as the merging of the distributed quicktime segments took almost as long as the length of the clips (RT) themselves.
When batch encoding with Compressor, it’s important to remember that the application is splitting the transcode up to the available processors. This is great if a batch of 2 clips are the same length, but if one clip is, lets say, 1 minute longer than the other, then the longer clip will no longer benefit from the distributed encoding when the first is finished. Normally this is only an issue at the tail end of a batch encode (as once one encode finished, another will start). For long encodes, this bares mentioning.
Across the board, encode times are cut in 1/2 to 3/4 from the last gen of Mac Pro (Harpertown, 3.2Ghz 8 core)..making the new Mac Pro, in the RED realm, a great investment for high volume encoding.
It should also be noted that even though some of the more expensive encoders (Episode Family) are not the fastest, the increased encoding options and variables, codec support, templates, watch folders, and bells and whistles they contain may be worth the investment.
http://www.telestream.net/pdfs/datasheets/EpisodeSeries_Format_Support.pdf
As of this writing,Telestream’s Desktop Products: Episode & Episode Pro will run you $495 / $995, based on options and their Enterprise line Episode Engine & Episode Engine Pro runs $3950 / $8450.
Final Cut Studio 2 (with compressor) is $1299.
RedRushes is a free download from red.com.
Questions? Comments?
Review: Root6 Technology’s ContentAgent
Everyone has an encoder nowadays. Final Cut Studio has shipped compressor for years. Avid ships with Sorenson Squeeze, and I typically suggest some flavor of Telestream’s Episode family line. All of these have varying degrees of quality and format support, and some go even above the call of duty with watch folders.
One fatal flaw is that they all rely on someone else’s engine with which to encode through. Quicktime. Quicktime, while being the pipe which leads to all things NLE, becomes vary narrow when it comes to efficient processor usage. In fact, it’s pretty bad.
Ever viewed your system processing usage while encoding? So much to be desired.
Root6 Technology, a player in the encoding and media market for over 6 years now, (BeamTV) has taken an innovative approach to this problem thus created an intelligent workflow device.
ContentAgent, at its core, is a software application, which can dig its claws into several hardware components. Utilizing Root6’s solution to the Quicktime Bottleneck: their Platinum Encoder, it can regulate CPU usage during encodes. Intelligently, ContentAgent farms out processor cycles for single or multiple encodes, understanding the limitations of each codec so as to efficiently disperse the workload across available resources. ContentAgent also has the added ability to utilize GPU horsepower on a per encode process.
By way of comparison, an Avid on 8 core 3.2 GHz MacPro is processing RED files around 5:1 RT. (1 minute of footage = 5 minute encode). ContentAgent is encoding that file in approximately 2:1 (DNX36). It should be noted that Avid is simply utilizing the QuickTime wavelet extraction files – not the R3D file. ContentAgent has full R3D support. This yields better quality at a faster pace.
ContentAgent RED Workflow
While that certainly enhances the turnaround time for encodes, it doesn’t solve the issue of baseband encodes. This is where ContentAgents hardware hooks become that much more powerful. Utilizing a Digital Rapids I/O card, either in an SD or HD Flavor – the ContentAgent now has the functionality of batch or crash capture and layback from any SD or HD deck source with RS422 control.
So, now we have an encoding solution that is faster than most anything else out there AND can pull tape. What else can it do?
Glad you asked.
Avid, for all its ability, has always had a proprietary media management system. Depending on whom you ask, this is a curse or a blessing. As it pertains to raw media files, it requires virtually all media (exception: AMA Volumes in v3.5) to be used in an Avid timeline to be encoded into OMF or the newer MXF format. This ties up a considerable amount of time on a billable suite. ContentAgent has the ability to encode media into OMF and MXF formats (even in the DNxHD codec) so as to bypass the “wrapping” OMF/MXF encode Avid does with non Avid Media. This means instantaneous media access in the suite. Those of us on Unity will appreciate the ability of ContentAgent to write the data to the Unity database for even faster usage.
Store
Metadata, especially when going tapeless, is almost as important as the pixels themselves. Getting the metadata inherent in the filename AND header (*cough* RED *cough*) becomes a massive chore when the file conversions are not followed precisely. ContentAgent has remedied that as well, utilizing an SQL database as the system’s backend to store and manage all data imported or encoded for referencing. This means the link between the 2 files (pre and post encode) plus all pertinent data located within those file can be referenced (or in some cases, parsed) by 3rd party applications.
Oh yeah, ContentAgent has that feature, too. XML schemas; which enable custom XML files to be written, containing the data from the pre and post encoded file, ready to dish out to applications supporting it. While not the easiest to understand, the functionality is there.
Those of you who have edited on higher end finishing applications (Autodesk, for one), or even some DVD authoring applications (DVD Studio pro, for one), may be familiar with the concept of node based editing. That is, each node (or “room”) contains parameters for a specific task. The results of that node can then be fed to single or multiple other nodes, where the process continues. This means intelligent, decision based workflows, easy troubleshooting and – drum roll – automation. Watch folders, FTP, file copying, status emails, copy / delete responses – all handled by developing these node based actions – called workflows. Once a workflow is devised, it can be saved and utilized by a few button pushes later. This means your Tape Operator in your Core can initialize a workflow without reinventing it. User permissions and a user interface with larger buttons create an environment designed for touch panel usage in a machine room.
Content Agent Workflow
Support for other hardware interop include Rimage support, for automating CD and SD and Blu-Ray DVDs with disc graphic design capability – either local or over a network. Another facet of this is the ability to create basic DVD menu layouts, with slates, watermarks, and chapter breaks. While rudimentary, for dailies and simple, trackable dupes, this fits the bill.
DVD Authoring Pane (Click to Enlarge)
DVD Label Editor (Click to Enlarge)
Saving you the headache of building some kluge machine is the fact that ContentAgent is built upon the same machine your PC Avid or Autodesk products are using. HP Workstation CPUs. As of this writing, the XW 8600 running Windows XP SP2 is the current config. The addition of 4 15K SAS drives allows for the system to capture uncompressed HD locally prior to encode. With the software understanding any volume the OS sees, shared storage is always an option. Facilis and Unity showed no issues. ContentAgent utilizes the ATI Fire GL card for its GPU acceleration. My tests have shown a 10-15% speed increase when this is enabled.
So, what are the downsides?
- Support. 8 hour time difference from SoCal to SoHo means some delays in troubleshooting. U.S. Support is very knowledgeable, but spread thin.
- Interface. Although it is designed purposely for pudgy fingers on a touch panel, it’s decidedly not Mac / not PC interface takes away some typically standard conventions on where to click next.
- Modular. Many of the codecs and abilities ContentAgent has are built upon a basic licensing structure. Pay to play. This means a system, after all is added up, (qualified computer and varying degrees of software licenses) can run the gamut: Between $30K-$50K. However, in all fairness, no other application – or collection thereof – can do all of this so as eloquently as ContentAgent.
Update: With NAB around the corner, updates promised in 2.5 and 3.0 software include: distributed encoding (Content Central) to further utilize horsepower on multiple machines, most notably, Blade servers. Blu-Ray authoring support is also in Beta.
*Obligatory Disclaimer: My opinion on the technology contained therein is independent of my affiliation with the reseller of this product.
-
Follow Me
