Entries tagged with compression - Channel 10

Netflix’s Neil Hunt shares encoding workflow info

benwaggoner — Thu, 20 Nov 2008 01:50:00 GMT

Now, this is a blog post! Neil Hunt, Chief Product Officer for Netflix, has just put up a great blog post talking about their encoding workflow for their video streaming services.

It’s full of awesomeness, but I wanted to excerpt the section describing their 1st gen, 2nd gen, and HD encoding settings and workflow.

First Generation Encoding

Our first set of encodes are based on WMV3 and WMA in ASF with WMDRM10 (Janus). We chose these standards because the Janus components have been widely adopted by our CE partners such as Roku, LG Electronics, Samsung, TiVo, and of course Microsoft Xbox.
We encode most content at 500, 1000, 1600, and 2200kbps VBR, but some titles whose source quality merits it have also been encoded at 3400kbps. The highest bitrate encodes are fit into 720x480 non-square pixels (the usual 1.2 PAR for widescreen content, 0.9 PAR for 4:3), but optimum encoding at lower bitrates is achieved with fewer pixels. Encoded films are normally at 24fps to match the source, while shot-to-video and mixed material is de-interlaced to 30fps (or 25fps for PAL content).

Netflix has been using anamorphic video all along, which I think is an underused feature of Windows Media and other formats. When you’re limited to 720x480 pixels, you want to encode all the pixels you’ve got, without having to synthesize any extra.

Second Generation Encoding

The new Silverlight player (that some users are helping us test as I write) uses VC1 Advanced Profile encoding with PlayReady DRM. A key property is that each GOP header includes frame size and resolution, which allows us to assemble a stream on the fly from different bitrate encodes as your broadband bandwidth fluctuates. (Another key feature is more coverage, including Intel Macs and Firefox users.) We expect to switch completely to the new player later this year.
The VC1 encoders are more efficient than the WMV3 encoders, so we are currently encoding VC1AP at slightly lower birates: 375, 500, 1000, and 1500kbps, all square pixel. At some point we are likely to add a couple more resolutions of non-square pixel encodes capturing the original pixel-aspect-ratio of the source.
We are also re-wrapping the VC1AP encodes in WMDRM10 for CE devices, which will gradually switch to the more efficient encodes in future firmware upgrades.

This is a great example of the improved efficiency of the VC-1 Encoder SDK and tools based on it. Not all Windows Media/VC-1 encoding is equal; the latest tools can offer a very meaningful reduction in bitrate required for a give quality level, improving user experiences and the cost of delivering the content.

The new encodes are backwards compatible to older hardware and software encoders, so even older devices can take advantage of the improvements.

High Definition Encodes

Today we have rights to deliver about 400 streams in HD (720p). More titles will be added over time. We experimented with first-generation WMV3 encodes at 4000kbps and 5500kbps, but settled on second-generation HD encodes with VC1AP at 2600kbps and 3800kbps, which extends their accessibility down to lower home broadband connections. As with SD, encodes of film material are at 24fps, and encodes of shot-to-video material are at 30fps (or 25fps for PAL), rather than the 60fps that would come from a Blu-ray disc - we judged the 60fps content as too expensive of bandwidth for now. In general, these encodes are definitively better than SD, but won't challenge well-executed Blu-ray encodes - that would require a bitrate out of reach for most domestic broadband today. We believe Moore's law will drive home broadband higher and higher enabling full 1080p60 encodes in a few years.

There’s codecs for you – a good 720p experience at 3800 Kbps, which is probably a little below the average 480p bitrate used for MPEG-2 on the DVDs that fly around the nation in those Netflix envelopes.

I’m not that worried about 1080p60 encoding myself. The vast majority of fictional content is shot 24p, including 99% at least of Blu-ray discs. While Blu-ray players may output 60p, the encodes, and hence bitrate requirements, are still 1080p. I find that typical film content in VC-1 wants around 6-8 Mbps for a 1080p24 experience enough better than 720p to be worth the trouble.

I just got my Xbox set up to handle the Netflix streams – now I’ve got to go check out some of those 400 HD titles! Hmm. Pan’s Labyrinth and Heroes.

Anyway, it’s great stuff. I love it when partners can share this kind of detail about what they’re doing.

Silverlight 2 powering Netflix on Mac

benwaggoner — Tue, 28 Oct 2008 00:38:00 GMT

It's been discussed for a while, but the time is now - Netflix has just announced Silverlight 2 will power their streaming video service, which had been Windows-only until now. Silverlight 2 adds support for PlayReady DRM, so we now have the same DRM support on Mac and Windows. The service will be available on Xbox 360 as well.

You have to be a Netflix subscriber of course, but the content is commercial-free and much higher quality than the various free video services. They encode up to a full 720x480, in proper 4:3 or 16:9, at a range of bitrates. The higher rates definitely outperform standard def cable and satellite in video quality.

Lots of good stuff in the press release, but I had a couple of things I wanted to call out (bolding mine).

Netflix, Inc. (NASDAQ: NFLX), the world's largest online movie rental service, today announced it has begun the deployment of Microsoft Silverlight to enhance the instant watching component of the Netflix service and to allow subscribers for the first time to watch movies and TV episodes instantly on their Intel-based Apple Macintosh computers. The deployment, which will initially touch a small percentage of new Netflix subscribers, is the first step in an anticipated roll-out of the new platform to all Netflix subscribers by the end of the year.

Note that this won’t be immediately available to all customers.

Silverlight is designed for delivery of cross-platform, cross-browser media experiences inside a Web browser. It is expected that Netflix members who watch movies and TV episodes instantly on their computers will enjoy a faster, easier connection and a more robust viewing experience with Silverlight, due to the quality built directly into the player. Among the viewing enhancements with the new player is a breakthrough in timeline navigation that vastly improves the use of fast-forwarding and rewinding. The new Netflix player takes advantage of PlayReady DRM, which is built into Silverlight, for the playback of protected content on both Windows-based PCs and on Macs. That had not been possible with previous generation technologies.

This is one of the first big rollouts of PlayReady with Silverlight in the USA. When you see the new navigation, it’s a real testament to the difference between a media player plugin like the WMP OCX, and a rich application runtime that plays media well like Silverlight.

The rich user experience support and DRM, what else does Silverlight bring to the equation? One big thing is that PlayReady DRM supports http delivery of pre-encrypted content, unlike other platforms that require a proprietary streaming server and protocol be used to deliver protected content, applying DRM on the fly. Pre-encrypted content is both more scalable (since the encryption processing is only done once, not per user), and more secure (since an in-the-clear media file doesn’t need to be transmitted and stored on the server).

There’s been two big technical challenges in web video for the last dozen years or so.

How do we deliver the best experience to each user, in a world of variable bandwidth and hardware power
How do we deliver that experience as cost effectively as possible, enabling new business models in video distribution.

We’re at the point now where a user with a fast connection and a decent computer should have a darn good video and audio experience. Netflix is a testament to what’s possible as the cost to deliver a MB of video gets low enough, from a business perspective. While finally delivering a user experience better than VideoCD.

Silverlight 2 RC0 is out

benwaggoner — Wed, 01 Oct 2008 15:01:00 GMT

The first public release canditate (RCo) for Silverlight 2 is now available for download by developers. While anyone can install it, the main goal is for developers to make sure they aren't hit by any breaking changes for Silverlight 2 RTM so they can fix any issues. No one will be auto-updated to this release. Users will be auto-updated to Silverlight 2's final release when it's posted.

All Silverlight 1.0 projects should work fine, but there may be some Silverlight 2 Beta projects that require updates.

For the media folks, there's a couple of new things I want to mention.

There's a new scaling algorithm that's a lot faster and much higher quality - a nice compromise when you can get it!
Additional optimizations have been done for the VC-1 video decoder, so playback will be faster. The gains are biggest for content using B-frames.

Since the default Expression Encoder templates have "always on" scaling and its presets use B-frames by default, the above will provide a nice performance boost for existing content. The above are "always on" features - you don't need to update anything to take advantage of them. So expect smoother frame rates on lower-end machines.

Further information:

EDIT: Added ScottGu link

Expression Encoder Service Pack 1 preview

benwaggoner — Wed, 24 Sep 2008 03:19:00 GMT

James Clarke has a blog post up describing some of the new features in the forthcoming Expression Encoder 2 Service Pack 1.

It's got some great new stuff, and we're demoing it here at Streaming Media West this week.

Read the whole post - there's lots of good stuff in there. A couple of my favories are

A new Silverlight 2 base player using .NET. Among other things, this will enable players that display the video at 100% scale by default, improving quality and performance.
New A/B compare modes. Awesome stuff for high-touch encoding and codec tweaking
Some even more VC-1 advanced options for High Codec Nerditry.
The first public release of Microsoft H.264 compression technology.

Anyway, a lot of good stuff for a SP1. I'm looking forward to it.

H.264 and AAC support coming in Silverlight

benwaggoner — Tue, 09 Sep 2008 19:09:00 GMT

So, our big IBC press announcement went out this morning. Lots of blog-worthy stuff in there, but as a compression nerd, it's the codec stuff I'm going to talk about first.

The big news is that, yes, we're going to add support for MPEG-4 to Silverlight, in the version coming after the fast-approaching fall release of Silverlight 2. Specifically, this will be H.264 for video and AAC for audio.

Why?

So, why MPEG-4 support in Silverlight? It's pretty straightforward, really. We have customers with libraries of H.264 content they wanted to publish to Silverlight, but didn't want to reencode to VC-1. Silverlight's strengths go far beyond media playback, and customers wanted the choice to deploy a wide variety of existing content within Silverlight. Silverlight aspires to provide as much choice as feasible as to how Silverlight can be authored and delivered. H.264 support is something we'd considered for past versions, but there were higher priority features we needed to deliver first. Silverlight 2 provides us a very rich base for delivering web apps, so we can start spreading our wings a bit.

What?

We've got more technical details we'll be sharing at IBC and later, but today, I'll just quote the details from Scott Guthrie's interview:

PressPass: Will you be showing or announcing anything new at the IBC conference this week?

Guthrie: Yes. At IBC 2008 we will be demonstrating a technology preview of H.264 video and Advanced Audio Coding (AAC) audio playback support in Silverlight, and H.264 authoring using Microsoft Expression Encoder and Windows Server 2008 for delivery. Until now, Silverlight has supported the SMPTE VC-1 and Windows Media formats, as well as MP3 for audio, enabling customers to take advantage of broad support across the Windows Media ecosystem, including third-party tools, service providers and content delivery networks.

We’ve always wanted Silverlight to support a variety of formats, so today we’re announcing that H.264 and AAC support will be available in a future version of Silverlight, which will offer content owners greater flexibility and choice to deliver video and audio.

PressPass: Historically, people have associated Microsoft with VC-1. Does this signal a change in direction?

Guthrie: No. Although we have been working with VC-1 for some time, it’s not widely recognized that Microsoft has also been an active participant in the standardization of H.264/MPEG AVC for many years, and we’ve included H.264 support in several Microsoft products. Microsoft’s Gary Sullivan was the chairman of the Joint Video Team (JVT), which developed the H.264 standard, and he recently accepted an Emmy Award on behalf of the JVT.

PressPass: Does this mean that Silverlight is moving away from Windows Media?

Guthrie: Not at all. This is about offering our customers more choice. Media producers and distributors around the world have enjoyed the high quality, flexibility and affordability of Windows Media formats for over a decade. As a testament to its pervasiveness, Windows Media can be found in almost every conceivable media scenario from desktop home video to feature films and TV broadcasts.

Again, don't read this as as a big change around our strategy for media formats. We've long-supported WMV and MPEG-4 side-by-side in products like Xbox and Zune. As I posted last week, Microsoft has been contributing to H.264 since its inception.

This doesn't represent any change in our support for Windows Media. Windows Media continues to work well for today's Silverlight customers. I expect (not a goal, just a prediction) that the majority of Silverlight content will remain in WMV well after we release MPEG-4 support.

In particular, VC-1 will likely remain the codec of choice for HD for some time. Comparing VC-1 Advanced Profile to H.264 High Profile with all the bells and whistles turned on, VC-1 only needs about half as many MIPS per pixel for playback. This won't matter as much for lower resolution content, or podcasting stuff that's in the simpler Baseline profile, but makes for a big reduction in system requirements for 720p and higher.

Where/When can I learn more?

If you're at IBC, swing by! If you want a head start on using MPEG-4 in Silverlight, feel free contact me directly. And we'll have plenty of more technical info to share down the road a bit.

Technical Emmy for H.264/MPEG-4 AVC

benwaggoner — Thu, 28 Aug 2008 19:05:00 GMT

I have many ways of reminding my wife I'm an hopeless video nerd. Among them is that I only pay attention to the Emmy Awards she's watching when the recap of the technical awards comes on.

So, I was really excited that our own Dr. Gary Sullivan, chairman of the Joint Video Team, was on hand last week to receive a 2008 Primetime Emmy^® Engineering Award for developing H.264/MPEG-4 AVC High Profile. In the words of the Academy:

Joint Video Team Standards Committee (JVT) for the development of the High Profile for H.264 / MPEG-4 AVC. The High Profile enables high definition images in the H.264 video coding system used today to deliver HD video over satellite and cable TV as well as Blu-ray Disc. The JVT is a standardization team comprised of members from the International Standardization Organization (ISO), the International Electrotechnical Commission (IEC), and the International Telecommunication Union (ITU).

The award recognized Broadcom, NTT DoCoMo, Dolby Laboratories, Fraunhofer Heinrich Hertz Institute, Microsoft, Motorola, Panasonic, Sony, and Thomson as contributing companies, as well as standards organizations ISO, IEC, and ITU.

Gary has long represented Microsoft as as chair and co-chair of the Joint Video Team that coordinated the H.264/MPEG-4 AVC effort between ITU-T VCEG and ISO/IEC MPEG, amongst his many other efforts like DXVA; his page on Wikipedia has more details.

I suppose some people might be surprised at Microsoft's inclusion here, but they shouldn't be. While Microsoft has a long heritage of making our own ground-up codecs like VC-1, we've also long been involved in standards-based codecs, including the original MPEG-4 Part 2 as well as MPEG-4 Part 10/H.264. And we've got a number of products that incorporate H.264 today, including Mediaroom, the Zune 4/8 and 80, and Xbox 360. H.264 are and VC-1 are both great codecs, with somewhat different sweet spots, and we use either or both as appropriate in our products, platforms, and services.

The picture above has, left to right: Malcolm Johnson (director of the ITU standardization bureau), Ajay Luthra (JVT vice-chair from Motorola), Gary Sullivan (JVT chair from Microsoft), Thomas Wiegand (JVT vice-chair from Fraunhofer HHI), and Scott Jameson (chair of ISO/IEC JTC 1)

ANSI

EE Times

Dolby

Minoru "Mick" Etoh's blog (who represented DoCoMo at the ceremony)

Alex Zambelli's blog is reborn with Olympics info roundup

benwaggoner — Fri, 15 Aug 2008 19:55:00 GMT

Yikes! No posts since early July. I've been a bad, bad blogger. Fortunately, compression wizard Alex Zambelli's blog has relaunched to take up the slack!

He's started off with a series of Olympics posts.

NBC Olympics 24/7 - a great roundup of information and details

NBC Olympics video without Silverlight? - showing how to play back the Olympics without Silverlight on Windows (and why you'd want to use the Silverlight version if given a choice).

Why no full screen mode in the NBC Olympics player?, citing this silverlight.net forum discussion including information from our own Tom Taylor.

Hopefully he'll inspire me to get back at it...

Silverlight Skinning in Flip4Mac

benwaggoner — Thu, 03 Jul 2008 22:54:00 GMT

Much has been made of our making the Mac a first-class citizen of Silverlight. We build the Mac and Windows versions together, and release them simultaneously. We've intentionally structured Silverlight to minimize dependencies on the underlying hardware and OS, and carry around all our codecs inside the runtime.

What I haven't talked a whole lot about in too long is our support for authoring Silverlight video experiences on the Mac. While our own Expression Encoder 2 is Windows-only, we work with partners to provide WMV and VC-1 encoding on other platform. Telestream has been a great partner here, with their Flip4Mac QuickTime component and their Episode stand-alone compression tool (originally known as Compression Master, and which came along with their acquisition of Popwire).

Flip4Mac is a testament to both the portability of the Windows Media technologies and the flexibility of Apple's QuickTime architecture. It works as a QuickTime component, enabling QuickTime to play Windows Media content like any other supported format, with full access to all of QuickTime's features like hardware accelerated full-screen playback (all supported in the free version). For example, in the QuickTime Pro Properties window:

Looks just like any other QuickTime file. Even the ASF metadata shows up.

Flip4Mac also supports content authoring in the paid version, as well as import into content creation apps like Final Cut Pro. It's got pretty deep control, even exposing features like B-Frames that'd take a registry key with Windows Media Encoder.

So, how about a quick walkthrough through the Flip4Mac encoding settings?

Flip4Mac is accessed like any other exportable format from QuickTime (like MPEG-4). It also comes with a bunch of presets for typical scenarios.

It has the normal set of basic video export features you'd expect, nicely Macified. I appreciate the "Size: Current" so I don't have to always type in the frame size of the source when I'm not scaling.

The Advanced button reveals some deeper options, including a complexity control for speed/quality tradeoffs and B-Frame distance. You can also set Flip4Mac to deinterlace interlaced sources, or pass interlacing on through to the final encode. Since it can both encode and decode interlaced VC-1, Flip4Mac can be a great way to transfer editable 480i video sources over slow connections. It can match DV quality in 20% the data rate.

For Silverlight audio, Flip4Mac supports both WMA 9 Standard and 9 Professional. It doesn't encode the 32-96 Kbps WMA 10 Pro low bitrate modes that Silverlight 2 will be able to decode, but does have the full variety of WMA encoding modes.

And the coolest new feature of Flip4Mac: built-in Silverlight templates! This lets you build a complete Silverlight player (using Expression Encoder templates we provided) directly from any app that support QuickTime export.

Low Latency webcasting with Windows Media and Siverlight

benwaggoner — Sun, 22 Jun 2008 05:25:00 GMT

So, Streaming Media is doing a special "Europe edition" of Streaming Media, and I'm doing an article about webcasting for it. I've getting a bunch of questions about how to deliver low-latency live streaming to Silverlight, and so with their permission, I'm excerpting this section on that topic.

EDIT: A correction was made; turning off Fast Cache in Windows Media Services does not have any effect on webcasting latency.

While by default, Windows Media can offer 15-20 second end-to-end delay, it's possible to drive it down to 2-3 seconds with best practices on a good network, and we're looking at what we can do to push it to below even that. The critical thing is to tune the encoder, server, and player latency together.

There's an older but more detailed "Reducing Broadcast Delay" document on this topic over at the Windows Media portal that may be worth perusing.

Low Latency Webcasting

Latency is the measure of how much time goes between when video enters the encoder and leave the user display. Latency is something that doesn’t matter at all in some markets, and matters a lot in others.

The reason we have latency is buffering, and the reason we have buffering is for quality and reliability. By having the server wait several seconds after a video stream is received before sending it out, it’s able to support more peaks and valleys in the data rate, and makes it possible for a dropped packet to be detected and resent before its needed. In the same way, buffering in the player lets it average out data rates and recover dropped packets as well.

Large buffers were extremely important in the modem era, and are still useful in many kinds of networks today. The defaults are good for delivering high quality content over a variety of networks. But when minimizing the latency is important, and reliable networks are available, the end to end delay can be reduced substantially.

Since total end-to-end latency is the sum of the encoder latency, server latency, and player latency, plus how long it takes the packets to travel between each of those, improving latency requires tweaks to the encoder, server, and player in parallel.

Encoder

On the encoder side, reducing buffer size reduces startup latency. WME and Expression Encoder both enable you to go down to one second.. Using Lookahead or Lookahead Rate Control will increase latency beyond the buffer value (typically about another half a second for LRC).

For audio, Windows Media includes the “WMA Low Delay” audio codecs from 192-64 Kbps which provide lower latency than the normal WMA modes. If you’re targeting Silverlight 1.0 or WMP 10 or earlier, you’ll want to use that for low latency. If you can require WMP 11 or Silverlight 2, you can use the lower delay yet WMA 10 Pro codec at 32-96 Kbps.

Server

WMS features like Advanced Fast Start can dramatically reduce latency for on-demand content, but don’t apply to live webcasting as the server has to wait for content to arrive from the encoder it in real time, and so can only play out at real time.

Server buffering can be turned off entirely in WMS for Windows Server 2003 and 2008, yielding a significant drop in latency.

EDIT:

The previous version of this post suggested that turning off Fast Cache would help webcasting latency. On further research, this turns out not to be the case. Disabling Fast Cache will slow down on-demand startup time, but will have no effect one way or the other for live content.

Player

Normally the player isn’t under control of the streamer. By default WMP dynamically picks an optimum buffer size based on its measurements of network and stream performance. However, it’s possible to lower the buffer size in the player’s options. This can help reduce latency when watching streams with a good connection, but could produce pauses in the video when watching video from the general Internet.

Silverlight makes the player buffer size a controllable parameter, so an optimal setting can be applied for the content, and even adjusted on the fly. This is controlled by the BufferingTime parameter in a Silverlight MediaElement.

Good article about the Olympics in Silverlight

benwaggoner — Mon, 16 Jun 2008 23:31:00 GMT

Max Bloom has written a good article about NBC Universal''s upcoming Olympics broadcasts in Silverlight.

Read the article yourself, but here's a few choice quotes from it:

"NBCU plans to offer 3,600 hours of live programming from Beijing. That’s 212 live hours for each of the 17 days of the Olympics... In addition to the sheer volume of live content to be delivered—three times what was offered in 2004— what’s notable is that most of NBCU’s live programming—2,200 hours—will be delivered online at NBCOlympics.com."
"The 2008 Summer Games in Beijing will mark the arrival of streaming as a viable alternative to the Olympics’ television broadcast. This summer, NBCOlympics.com will offer 4,400 hours of on-demand streaming in addition to its 2,200 hours of live programming, making the Beijing Olympics the most ambitious streaming media project in history."
"To help meet that challenge, the NBCOlympics.com player offers a “metadata overlay” feature, which allows the player to display transparent data and navigation tools over the video window. This enables users to access statistics and other data without covering up, pausing, or leaving the primary video display. For example, play-by-play announcers’ dialogue can be keyed into an XML data stream, then rendered as a timecoded, scrolling text caption that transparently overlays the bottom of the video display. The player also enables the TiVo-like experience of pausing, rewinding, and replaying content, and these two features together allow viewers to use either the timecode or the play-by-play captioning to rewind to a specific point in the on-screen action and replay it."
"A slew of DRC-Stream software and encoder boards from Canada-based Digital Rapids are being deployed in Beijing to populate NBCOlympics.com’s encoding farm, but other than committing to VC-1, NBCOlympics.com has yet to confirm encoding bitrates, frame rates, or frame sizes. (Without offering more specifics, Miller says NBCOlympics.com will be streaming through a managed bitrate solution to optimize the user’s connection, with a target maximum bitrate of 650KB/sec.) Digital Rapids is also supplying software to enable transcoding from other digital media formats into VC-1.Miller promises hundreds of hours of online HD video..."

Note that there's already a bunch of content up at NBCOlympics.com if you want to get an early taste of what's in store on 8/8/08.

Expression Encoder SDK is out!

benwaggoner — Thu, 05 Jun 2008 22:01:00 GMT

One of the neat features of Expression Encoder 2 we've been talking about is its .NET object model interface for automation. I'm happy to report that it's now available for download!

Here it is

Sample projects

It includes a number of sample projects (quoting from the help file). These are a good jumping off point for various scenarios.

Simple encode

The simple encode sample is a console application that demonstrates how to encode a file from a console application when showing progress. To use it, just run the built application (Simple.exe) at a command prompt.

Asynchronous encoding

The WPFEncode application demonstrates how to encode a file from a UI application when displaying the progress of the encoding process. To use the application, click the Browse button to point to a file that you want to encode and then click the Encode button to encode it. When the file is encoding, a bar displays the encoding progress.

When you press the Encode button, the application creates a thread to perform the encode to make sure that the UI thread is not blocked. The progress events are called from a non-UI thread. Consequently, we use the Windows Presentation Foundation Dispatcher to marshal the updates to the UI thread.

If you try to close the application when the encoding is still occurring, the application prompts you to confirm that you want to stop the process. If you confirm, the application stops the encoding thread but delays closing the application until the thread actually stops.

MediaInfo

The MediaInfo sample is a console application that demonstrates how you can use the MediaItem class to extract media and metadata information from a media file. To use it, just run the built application (MediaInfo.exe) at a command prompt with the full path of a media file as the parameter, as shown in the following example:

Publishing plug-in

Publishing plug-ins enables you to take video and Microsoft Silverlight template assets that you have created as part of an encoding job and then do something with encoded assets. For example, you can upload the media asset by using FTP, WEBDAV, or METAWEBLOG API, publishing the asset to an asset management system or web service, or creating a data CD.

The following code sample demonstrates how to use a publishing plug-in to add the output of an encode job to a Zip file by using the Open Packaging Convention components found in Microsoft .NET 3.0.

A publishing plug-in is required to derive from the PublishPlugin class, which in turn derives from the EncoderPlugin class:

Template plug-in

The template plug-in lets you customize template settings and interact with the job through the IPluginHost interface. The plug-in also lets you add or remove files from the template output when the template is published.

The following code sample demonstrates how to add the default template parameter control to Microsoft Expression Encoder with only several lines of XAML. It also implements a custom script command creation feature that shows how the template plug-in can interact with the current job by adding a script command at the current play head position of the selected MediaItem.

Within Expression Encoder, every template receives a template plug-in. If the template doesn’t specify one, a default one will be assigned. A template plug-in derives from the TemplatePlugin class, which in turn derives from the EncoderPlugin class.

Videos from the "Secret Compressionist's Ball"

benwaggoner — Thu, 29 May 2008 03:39:00 GMT

I've always called my annual compressionists party something like the "Ben Waggoner Compressionist's Party" but David Sayed from the Expression Encoder team suggests the much better name of the "Secret Compressionist's Ball"

Better yet, he's got a couple of video interviews from the party! You'll see me in the background in my mildly famous purple shirt in a few shots.

First, Tony Houghton from Promoscape, talking about Silverlight.

Second, Bruce Lidl from MainConcept talking about their VC-1 codec implementation they're now licensing out, including support on Mac and Linux.

The clips are embedded at smaller than they're encoded; double-click to see them in their full glory.

Ben at Streaming Media East

benwaggoner — Mon, 19 May 2008 15:37:00 GMT

I'm at Streaming Media East this week in New York.

On Monday I'm doing a three-hour workshop on "Deploying On-Demand and Live Media Experiences with Microsoft Silverlight."

My handouts are available here.

I'll also be at the Microsoft booth on Tuesday and Wednesday.

Tuesday:
10 am - 12 pm
2 pm - 6 pm

Wednesday:
10 am - 12 pm
2 pm - 4 pm

Expression Suite 2 and Expression Encoder 2 now shipping!

benwaggoner — Thu, 01 May 2008 22:43:00 GMT

May 1st is a happy day for me! I went over to www.microsoft.com/expression, and was delighted to find that the Expression Studio 2 is now shipping, including the final version of Expression Encoder 2. There's a 30-day trial available for download. Boxed copies of Studio are available today at CompUSA and Office Max, with other retailers coming. There will also be the Microsoft Expression Professional Subscription for $999/year (academic and volume pricing coming soon), which includes Expression Studio along with Visual Studio, Office, Visio, XP, Vista, Virtual PC, and Parallels Desktop (to run all of the above on Mac).

As I've said before, I feel that Expression Encoder 2 is the best compression tool at fulfilling the needs of its market since Terran Interactive's Media Cleaner Pro 3.1 (so long ago I can't even find anything on the web about it!). Here are some of my favorite things about it from my obsessive compression nerd's perspective.

First desktop priced VC-1 Encoder SDK based product ($199).
And the VC-1 implementation gives you great quality by default, and gives you an advanced mode that accesses the useful features, while handling the more esoteric ones behind the scenes. For example, the rule for WMV PowerToy was use Static Motion Vector Cost if using P-frame DQuant, otherwise use Dynamic. So Expression Encoder 2 just has a DQuant setting, and uses the right Motion Vector Cost based on that.
High quality scaling and deinterlacing modes
WMA Pro support, targeting Silverlight 2's support for that. So danceable music at 48 Kbps.
"Smart by default" - you can use the same preset and feed it a PAL 16:9 interlaced file and a NTSC 4:3 progressive file, and it'll do the right thing without having to do a single click: you'd get a 640x360 deinterlaced 25p file and a 640x480 29.97 file.
Which means you don't have a huge combination of settings to deal with all the video formats. Instead, the presets are scenario based; you just need one "1 Mbps streaming" option that'll handle all the video file variants.
Great multi-clip templates, so you can make a video gallery, with each clip having its own optional thumbnail-based chapter navigation (hmm, that deserves a nice demo).
Easy creation of thumbnail-based chapter navigation!
Captioning and metadata support, including ISAN.
Available soon, an integrated publishing module to go straight to Silverlight Streaming.
Support for a wide variety of source formats, including QuickTime, AVI, Windows Media, VOB (DVD), many other flavors of MPEG-2, MPEG-4, and AVISynth.
Automation available via both command-line and a .NET object model.
Simple editing features, like in/out points, preroll/postroll videos, and image and animated overlays.
GPU acceleration for previews
A great A/B compare feature, enabling short sections of the file to be encoded with a variety of compression settings, and then compared (playing in real time! zoomed in!, with an A/B split screen slider!) to both the source and each other.
An unbelievably cool live encoding module, which supports multiple cameras with live switching, streaming from files, including looping, live metadata insertion, and big quality improvements from lookahead rate control and dynamic complexity. I owe you a blog post describing that as well.

So, in summary, I guess I'd say I like it.

I've had a some blog posts highlighting different projects done with Expression Encoder 2 which you can follow along with.

Hands on with high-touch encoding: Streaming Media All-Stars Redo

Encoding screen recordings for Silverlight in VC-1 with Expression Encoder 2

"What Happens in Vegas" - 720p Movie trailer at 2 Mbps via Silverlight Streaming

I've got other information about Expression Encoder and how it fits into Silverlight at

My NAB Presentations

Silverlight Media technologies overview in Expression newsletter

And, of course I'll be covering Expression Encoder 2 as one of the products we cover in my

My June 23-27th class at Stanford

Hands on with high-touch encoding: Streaming Media All-Stars Redo

benwaggoner — Wed, 30 Apr 2008 06:47:00 GMT

Introduction

As Expression Encoder 2 approaches its immenent release, I've been using it for more and more real-world projects. This recent one was particularly chewy fun, and I thought it would make a good tutorial for a high-touch workflow.

As you may remember from a few weeks ago, I was one of the inaugural class of Streaming Media's Streaming Media All-Stars. There was a fun video montage of all of us on baseball cards being announce by ballpark-style narration. Good stuff, but the FLV compression wasn't quite up to my standards for this rare intersection of compression obsession and personal vanity. So I contacted Streaming Media and asked if I could take my own whack at it.

I'll have an expanded version of this post as an article in an upcoming issue of Streaming Media Magazine. If you don't get it, you can sign up for a free subscription.

The Source

One thing I noticed in the original is that the background graphics and a few of the animations were interlaced, as you can see in the last "before" image at the very bottom of the page.

While deinterlacing it may have been possible, the heavyweight motion-adaptive deinterlacers available for technologies like AVISynth can be finicky to configure, and extremely slow. And in the end, nothing beats getting the source fixed in the first place. Compression is the art of getting output that's as close to the original as possible with the bits you have available; often getting access to higher quality sources can provide a much bigger improvement to final quality than all the codec tweaking in the world.

So, I contacted the post house, and they fixed the background interlacing (it was just a matter of properly flagging the source as interlaced in After Effects) and re-rendered it for me as a lossless RGB PNG codec QuickTime .mov file. However, there were two shots that snuck through where one layer was still interlaced. I didn't want to wait for another disc, so I dived into After Effects (in the end, all difficult preprocessing jobs seem to wind up in After Effects). I used the "Reduce Interlace Filter" with a softness of 1 to blend the two fields together. Traditional deinterlace methods messed up the text on the cards too much. However, the softness increase from that filter wound up causing a slight visual discontinuity when it kicked in. So, I broke out the two shots with interlacing into layers, and then used a five-frame cross-dissolve transition from the original progressive frames to the start of the interlaced shot which hid the slight loss of focus (masked in part by the motion). Both interlaced shots ended on a hard cut, so I was able to switch back to the original video without a transition.

I then rendered the new version out from After Effects in 32-bit float (to reduce the risk of introducing banding via an 8-bit to 8-bit conversion) into the Lagarith codec in YV12 mode, which uses the native 8-bit 4:2:0 colorspace of VC-1 and other codecs. This means that Expression Encoder doesn't need to do any color space conversion, making compression slightly faster.

Markers

The other notable issue with the original clip was "keyframe popping"; when an obvious "jump" in the video happens at the keyframe rate of the video. Watch the original FLV, and you'll see it during any of the longer static shots. Since the whole section with the cards is one single long shot over 3 minutes long without any hard cuts, there wasn't a place for natural keyframes (automatically inserted at a hard cut) to go. Thus keyframe transitions would happen while the cards were otherwise static, making even a slight change visible.

I also wanted to show off the Expression Encoder templates a bit by doing thumbnail navigation. In EEv2, I'm able to graphically set markers on particular frames, and set them to be keyframes and/or thumbnails. A thumbnail becomes an image file which, with the supported templates, automatically gets included in the menus for navigation (think a chapter on a DVD). Normally you also want to make the chapter points keyframes, since keyframes support immediate random access, as no other frames need to be decoded before displaying a keyframe.

This was an opportunity to kill two birds with one stone; if I set the markers on the first static frame of every card, it'd be nice high quality image that all the later frames that reference that I-frame can be based on, propagating its quality forward. If I set my keyframe spacing long enough, there wouldn't be any other keyframes in that interval to cause keyframe popping, and so the static card would be very consistent.

So, I set a marker for each person, flagged to be both a thumbnail and a keyframe. The audio doesn't always sync up exactly so that the person's name begins after their card is down, so sometimes the first name is cut off. This would have been easy to fix by just delaying the audio a second.

You can also use non-thumbnail keyframe markers; these become keyframes without showing up in navigation. I stuck a few of those in as well in the intro/outro sections, on the first full frames after the logo gets built. Since the sponsor pays the bills (Ripcode in this case), I always want to make sure that logos remain nice and crisp.

Setting keyframes has been around in compression projects for ages now; I did a lot of this in Premiere 4.0 for Cinepak encodes in the pre-Media Cleaner days, since Cinepak was prone to keyframe popping issues. Modern codecs like VC-1 do a much better job of finding good natural keyframes, and also to reduce popping issues. The Silverlight version would have looked a lot better than the Flash even if I hadn't set them, but they did get a further boost in quality. But don't think this is something you should be doing in every case; this clip is unusual in having minutes without cuts with a mix of static and moving elements, at an extremely low bitrate.

Encoding Settings

Now, what encoding settings do we want to use?

Video

Frame Rate: Source. We want to capture all the motion in the source perfectly (29.97 frames per second in this case).
Keyframe Interval: 15 seconds. The longest gap between markers in the source is a hair left than 15 seconds, so this will prevent keyframe popping between cards.
Profile: VC-1 Advanced Profile. So we can use DQuant, as discussed below.
Mode: VBR Peak Constrained. This is a progressive download project, so VBR Peak Constrained gives us optimum quality.
Bitrate (average): 488 Kbps. Matching the original FLV's actual bitrate (400 Kbps was requested, but VP6 overshot by over 20%).
Peak bitrate: 896 Kbps. So video + audio + overhead (9 Kbps in this case) max bitrate is a consumer broadband friendly 1000 Kbps total.
Peak Buffer Size: 15 seconds. So the buffer duration can contain an entire Group of Pictures (a keyframe and frames that reference it).
Width and Height: 640x480. Same as source. The original project had both 320x240 and 640x480, but they used the same data rate, so I'm doing just the 640x480 and Silverlight embed can be set to the desired size.

Audio

Codec: WMA. We're targeting Silverlight 1.0 compatibility, so WMA Pro isn't an option
Mode: VBR. Always better quality for progressive download.
Bitrate: 48 Kbps. Matching the data rate of the FLV source. Also this is minimum bitrate for WMA VBR. I always try to use at least 48 Kbps for WMA progressive for that reason; it's a massive quality jump from 32 Kbps CBR for typical content.
Sample Rate: 44.1 KHz. Same as source. Also, 44.1 is the native audio rendering mode for Silverlight, and so offers the same quality and better performance versus 48 KHz.
Bits per Sample :16. The only option for WMA
Channels: Stereo. VBR audio requires stereo. I'd use mono if I needed to do 32 Kbps, since there isn't stereo separation important to the experience here.
Audio peak bitrate: 96 Kbps. Again, so total peak comes out as 1000 Kbps. The audio isn't that difficult or variable, so higher likely wouldn't sound any different.
Audio peak buffer size: 1.5 Kbps. The default is almost always fine.

Advanced Codec Settings

Video Complexity: Best (5). It's a short clip at a reasonable frame size. Complexity 3 probably would have been just as good, but the encode only takes about 12 minutes at 5, so I didn't bother doing anything less than the max (I love my new 8-core workstation!).

Perceptual Optimizations

Adaptive Dead Zone: Conservative. The normal default. It softens out edges that might ring or get too blocky, but not by too much. I tried both Off and Aggressive, and Conservative definitely looked the best, as usual.
DQuant: I-Frames Only. There aren't many I-frames (mainly the few dozen we set manually, and perhaps a few more natural ones), but they contain the important visual data of the faces on the cards, so we want them to be as high quality as possible. DQuant spends too many bits on smooth parts of the image to use on every frame, but upping the bitrates on a few dozen I-frames won't hurt quality much, and improves the quality of the static parts of the card we wind up staring at for those many seconds.

Filters

In-Loop: On. Always on unless using Simple Profile; it helps reduce artifacts and improve quality, particularly at these aggressive bitrates
Overlap: On. Further hides artifacts, which are a challenge with motion graphics at such a low bitrate
Denoise: Off. The source doesn't have a hint of noise. If there were a lot of textures, Denoise can help to soften them some for easier encoding, but there's not much texture either.
Noise Edge Removal: Off. This is really only useful for noisy edges of analog captures, and even then we're better off cropping. It obviously doesn't apply here.

Group of Pictures

B-Frame Number: 2. Normally we use 1 for film and video sources, but for this kind of motion graphics, 2 is more efficient. This gives us an IBBP pattern, so each B-frame is adjacent to a P-frame. 3 B-frame is less efficient in this case, probably since with the IBBBP pattern the middle B-frame is two frames away from a reference frame (only I and P frames can be reference frames), and the P-frames are too far apart and so require more bits to store the change over four frames instead of three since the previous I or P frame. Using 2 also gives us better random access than 1, since worse-case random access time is based on the maximum number of P-frames between I-frames. With 15 seconds between keyframes at 30 fps, that gives us 450 frames per GOP maximum. With 2 B-frames, that'll gives us 149 P-frames per GOP, the same (and thus the same random access) as if we had a 5 second GOP without P-frames (the old Windows Media Encoder default).
Scene Change Detection: On. This will give us natural keyframes where need them. The codec seems to do a good job of putting them in the right place. I've never changed this in EE.
Adaptive GOP: On. Always have this on.
Closed GOP: Off. This is required to be on for CBR encodes in EE, but slightly reduces quality with VBR encoding. In particular, it can increase keyframe popping, since an Open GOP pattern starts with B-frames before the first keyframe/I-frame, you get BBIBBPBBP..., with the B-frames able to reference the last P-frame of the previous GOP. This helps smooth over changes between GOPs, since you have the leading B-frame(s) to spread the change over.

Motion Estimation

Chroma Search: Full True Chroma. Motion Graphics is a canonical time we want chroma search. The encode is so fast, there's no reason to not go for the full meal deal and do Full True Chroma.
Match Method: SAD. For this kind of content with very simple, flat areas, the Sum of Absolute Differences Motion Match is actually both higher quality and faster than either Hadamard or my normal video/film default of Adaptive.
Search Range: Adaptive. The smallest range works for most of the frames, but there's some very fast motion when the cards zoom in which need the bigger range. Adaptive it is.

Output

The Output pane has some of my favorite usability features of Expression Encoder, letting us apply rich templates and automatic publishing.

First, the Template. I picked the "Clean" template, which has a nice subtle overlay control, and a popup navigation via the thumbnails we made above when you mouse over the top of window. It also supports going full screen with a double-click. One thing I like about Clean is that the video fills the frame exactly, without having to account for the control bar or other elements. So I can embed at exactly 640x480 for a 640x480 clip.

The publish mode (I've got the optional Silverlight Streaming publishing plugin installed) lets me automatically or manually upload the final project to our Silverlight Streaming service. This is a great way to test or deliver Silverlight projects. You can sign up for a free account with 10 GB of storage and 5 TB/month of bandwidth.

Before/After

So, how much did all this help? Here's a couple of the more pronounced before/after shots. All the below are inserted as 100% scale PNG, so there's no scaling or further compression to complicate the comparison. Note that the FLV came out darker for some reason. I'm not sure what the cause of that was; the VC-1 brightness matches the source. Perhaps something to do with the Mac/Windows gamma difference on the platform the FLV was encoded on? This actually makes VC-1's job relatively harder, since the motion graphics are easier to see.

And you can see the actual clips in action here:

Before: FLV VP6 in Flash

After: WMV VC-1 in Silverlight

Detail improvements

I grabbed a frame right after the transition that really shows the detail difference between VP6 and VC-1 here; it's especially striking in the texture of the shirt. The VP6 gets sharper after a keyframe pop, but this is how it starts. VC-1 quality in the card is maintained perfectly throughout.

FLV VP6:

WMV VC-1:

Deinterlacing improvements

In this frame (man, do I look like a stiff!), you can see the effect of my blend deinterlace to hide the fields. Notice the ringing artifacts in the original frame. Encoding fields as progressive is extremely challenging for codecs, since you have high motion 1-pixel high horizontal lines, combing high frequency and high detail. I normally don't like doing a blend, since those double-images are also hard to encode, but it was only for a very short duration in this clip, and the deinterlacing filters I had handy had a lot of trouble preserving the text perfectly.

FLV VP6:

WMV VC-1:

Silverlight Media technologies overview in Expression newsletter

benwaggoner — Mon, 28 Apr 2008 16:47:00 GMT

In this month's Expression Newsletter, I've got an article giving an overvew of Media Technologies in Silverlight.

There's not much in there that hasn't been discussed elsewhere, but it's a nice one-stop overview of what's available in 1.0 and coming in 2.

Sample Encoder Test Clips

benwaggoner — Wed, 23 Apr 2008 02:24:00 GMT

I've been working for what seems like forever on a good 1080p24 film-sourced content with clear rights that can be used for codec testing by the broader community.

Since I've finally realized that a big purpose of a blog is an auxilliary memory for myself, and to have a place to point people towards things I get tired of typing over and over, here's two test clips showing what's available.

Both of these are from a shoot on the Lady Washington shoot a bunch of interepid Microsoft employees and and a hand-picked crew did before my time at the company. It was shot on a mix of 35mm and 1080i. There's also a nice if not incredibly challenging 5.1 orchestral soundtrack with it. Being of a tall ship on the water, it's got lot of good imagery to stress gradient preservation (sky) fine random details (water) and sharp diagonal lines (rigging).

Both of these comps are 960x540 QHD ("Quarter HD") @ 3 Mbps CBR, for relatively easy playback off Silverlight Streaming. I'll get around to doing full-rez encodes and the source clips down the road (we'll see if Microsoft.com lets me host 20 GB of downloads)...

Clip 1: 1080i
This is from 1080i30, obviously. And just as obviously, this one was edited and color-corrected by an actual professional, not me. A good test for HD interlaced compression (note the QHD sample is actually deinterlaced to 540p30). I wouldn't do anything else to it to make it a test clip. The usage rights for the clips are on the final frame.

Clip 2: 1080p24
This is from 35mm film sources. And it was thrown together in a few hours, sans any color correction, by myself (not an editor!) for a rush job to get some DLNA test bitstreams together. There's a thread over at Doom9 discussing what we should do to this to make it a better codec test clip for feature film content.

My NAB Presentations

benwaggoner — Tue, 22 Apr 2008 23:46:00 GMT

As promised, here are the final decks for my presentations at NAB.

Encoding for the Next Generation: MPEG-2/H.264/VC-1

This session was part of the all-day Next Generation DVD track sponsored by the DVD Association.

I talked about the various options for and issues in encoding for high definition, focusing on Blu-ray but also talking about digital downloads. It's apparently become somewhat infamous, with mentions of it over at Doom9 and AVSForum. Although now that I look, they're both from the same guy...

Here's the deck: Encoding for the Next Generation.pdf

In-Depth Microsoft Silverlight

This was a three-hour media-focused overview of Silverlight and the Expression tools. It took a look at encoding and hosting video and audio assets for integration, and incorporating those into a Silverlight Rich Interactive Application. I used a shorter version of the same presentation for the "Successfully Set up your own Streaming Media Solutions in a Worship Environment" track, so I won't bother to include both.

Here's the deck: In-Depth Microsoft Silverlight.pdf

I've been blogging versions of the tutorials. So far I've got:

Tutorial 1: Default Settings (no blog post there, as it's just a demo of the pretty-good quality that EEv2 can do without changing ANY settings, automatically adapting to the 480i 16:9 source).

Tutorial 2: Movie Trailer at 2 Mbps

Tutorial 3: Encoding screen recordings (less the navigation demo in the online version)

Now I need to figure out a good .pptx to Silverlight conversion workflow.

And if you like this stuff, don't forget about my class at Stanford in June.

An exhaused me talks to James Clarke about Expression Encoder 2

benwaggoner — Wed, 09 Apr 2008 21:23:00 GMT

Oops, forgot to post this video clip of a VERY exhaused me at Mix talking to Expression Encoder's James Clarke about 2.0. The clip itself is actually quarter HD (960x540), so double-click on it to take it full screen.

Could be a good case study for the captioning features in Silverlight...

Ben at NAB

benwaggoner — Tue, 01 Apr 2008 00:20:00 GMT

Hard to believe that NAB is starting at the end of next week!

Here's my schedule for booth duty and conferences if you'd like to swing by and say hello.

Encoding for the Next Generation: MPEG-2/H.264/VC-1

Saturday, April 12th 3:30 pm - 4:45 pm

Las Vegas Convention Center N252

This session is part of the all-day Next Generation DVD track sponsored by the DVD Association.

I'll be talking about the various options for and issues in encoding for high definition, focusing on Blu-ray but also talking about digital downloads.

Successfully Set Up Your Own Streaming Media Solutions in a Worship Environment

Sunday, April 13th 9:00 am - 5:00 pm

Las Vegas Convention Center N115

This is going to be a fun event. We're going to be demonstrating the end-to-end experience of live broadcasting for both live events and on-demand HD, emphasizing the hands-on and best practices aspects.

Microsoft Booth

Monday, April 14th 1:30 pm - 6:00 pm

Wednesday, April 16th, 9:00 am - 1:30 pm

I've got two shifts at the booth, and I always happy for more company. I'll be manning the Silverlight pod. Come on down to get some questions answered or see some demos.

In-Depth Microsoft Silverlight

Tuesday, April 15th, 10:00 am - 1:00 pm

Las Vegas Convention Center N256

This is a three-hour media-focused overview of Silverlight and the Expression tools. We'll take a look at encoding and hosting video and audio assets for integration, and how to incorporate those into a Silverlight Rich Interactive Application.

Annual Compressonist's Party

Tuesday, April 15th, evening

The Wynn

Yep, it's time again for the my annual compresionist's party at NAB. And now that I work for Microsoft, the snacks are better than ever! We're not exactly sure what time and which room yet; I'll share details when we get closer. But we try to start early enough and run late enough that we're a good stop on the way to or from dinner. Drop on in and let's talk about the news of the compression world from the show.

RSVP is not required, but if you think you're coming, drop me an email so I can track a rough headcount. My email is first name period last name at Microsoft.com