Tuesday, April 20, 2010

No such thing as a multimedia data format 100% unencumbered by patents

Speculations abound that Google may release the VP8 codec developed by On2 Technologies, a company it acquired recently, under a FOSS license.

One of the FOSS pundits advocating this move, Dana Blankenhorn, writes on his blog: "Open source — free, unencumbered open source — is going to become the default for basic Web video."

Wishful thinking vs. patent reality

I understand what Dana means and wants. It would be great if it were possible. But no one will ever be able to guarantee "free, unencumbered open source" in terms of a FOSS-based multimedia codec (encoder/decoder) module against which no one could bring up patent infringement assertions.

This BetaNews article offers a lot of insight and a headline that serves as a stern warning: "Google may face legal challenges if it open-sources VP8 codec"

That article explains that the patents Google acquired along with On2 cover "three principal video compression technologies" but all codecs are "developed around several bedrock technologies. Companies that either have claims to those technologies, or at least believe they do, could very well file suit if they believe Google was never licensed to give away methodologies they contend they have created, and thus own."

Ogg Theora: supposedly patent-free but Apple and others doubt it

Earlier today I commented on the "Patent Absurdity" movie. That one was made entirely with Free Software and is distributed in the Theory format (which is commonly referred to as either the "Ogg format" or the "Ogg Theora format"). The download page of the PatentAbsurdity web site claims that Ogg Theory is "a free and open video compression format" that "can be used to distribute film and video online and on disc without the licensing and royalty fees or vendor lock-in associated with other formats."

Sounds like the thing Dana Blankenhorn asked for. But is it really? Reliably?

According to TheRegister, at least Apple doubts it, referring to an "uncertain patent landscape" concerning Ogg Theora.

This is a general problem. It doesn't matter whether it's Google or any other vendor or a FOSS project: there's no such thing as a multimedia data format that anyone can absolutely guarantee to be unencumbered by patents.

Multimedia data formats and codecs: one of the worst patent minefields

Multimedia (audio/video) data formats and codecs are one of the worst patent minefields of all. One has to tread carefully, and some of those mines go off all the time.

Think of MP3.

Every year at CeBIT, the big IT trade show taking place in the northern German city of Hannover, there are raids where the police goes to trade show booths and confiscates product samples that violate different patents, many of them MP3 patents. This CNET article talks about how that effort was stepped up significantly in 2008 versus the previous years. Enforcement is becoming ever more rigorous. Unfortunately available only in German, this press release issued by the Local Court of Hannover talks about how the authorities set themselves up this year. They brought mobile printers along to CeBIT so they could immediately issue any warrants and court orders they needed. For a trade show that only lasts a week, if they needed several days to produce and deliver such documents they might not achieve the desired effect, at least not in full.

Let's be very clear: MP3 patents are simply software patents. While the most ingenuous aspect of the MP3 compression technique is to perform a lossy compression (intentionally "losing" data) because the human mind won't need certain sounds to still have a very similar acoustic impression, MP3 isn't just a single patent but instead a whole family of patents. I haven't checked lately but I once heard that there are several dozen MP3-related patents. Obviously they're not all related to psychoacoustic research. Many of them are purely mathematical. If MP3 players infringe them, they use algorithms covered by the patents, and those patents will read on hardware as well as software implementing the methods taught.

Some of those alleged "pirates" pursued by the German police at CeBIT may even use FOSS components that play a role in the asserted infringements. But using FOSS obviously won't make them immune from patent law.

The impossibility of reliable patent clearance

Even if Google now made its very best effort to identify any patents that could read on VP8 (and Google has a level of sophistication and quantity of resources matched by only a few in the world economy), there would never be a 100% reliable guarantee that no one would ever be able to (or at least be able to attempt to) assert some patents against that codec.

What professionals call "patent clearance" -- ensuring that a given product doesn't infringe any of a group of patents -- is impossible in this case because to do clearance, you need to have a finite quantity of patents to check on. If I owned 145 patents and gave a list to you, you could check if any of your products is in any way related to them. Many of them might turn out at first sight to be irrelevant. Others may appear potentially relevant and require a closer look. Anyway, sooner or later you'd be done.

However, when you don't know where the risk is coming from -- from which large patent holders, from which small one, or from which troll --, then you don't even know where to begin with your clearance effort.

Would full-text search help? Not really. In some areas such as chemical and pharmaceutical substances, there's a certain nomenclature and you could run pretty specific searches for patents covering methods to heal a particular disease. But for multimedia data codecs there isn't a particular terminology. There are words that will help find some of those patents. But other patents of this kind may not use any of those terms and still be relevant.

What makes it even harder is that you'll never be able to look at patents that are in stealth mode, meaning the applications have been filed but not yet published. Those you will only see after a certain period of time, such as 18 months. But theoretically you can infringe them any day after they've been filed and be held responsible later.

Repatenting (not reinventing) the wheel

Also, patent offices have resource constraints but are inundated with patent applications. As a result, it happens all the time that they grant patents on "inventions" for which there actually is prior art. In that case you may very well be able to get the patent invalidated in court, but the risk is yours and since the outcome is never 100% certain, you might be forced to pay royalties nonetheless.

To give you a funny example (strange kind of humor, some might argue) of how easily a patent office may grant a patent on a very old "invention", here's a story from Australia on a patent granted nine years ago ago on the concept of a wheel. Yes, a wheel.

In all fairness, I have to point out (as the article does) that it wasn't a regular patent. It was what they call an "innnovation patent" in Australia, a kind of fast-track low-cost patent that undergoes very limited scrutiny. But even with the smallest amount of scrutiny imaginable, granting a patent on a wheel in 2001 is incredible. Still, it happened. And if this can happen with something as simple as a wheel, think of what this means for multimedia data formats, compression techniques, and codecs.