Wednesday, August 3, 2011

Apple's favorite make-Android-awkward patent (with videos)

In its lawsuits against the three leading Android device makers, Apple asserts a variety of patents ranging from mid-1990s Macintosh patents (such as the two HTC was found, on a preliminary basis, to infringe) to more recent technologies. Apple's number one priority clearly is to ensure that consumers will perceive the iPhone and iPad user experience as unique. That, in turn, means that Apple shoots for a substantial degradation of the Android user experience.

Patent enforcement is a "you win some, you lose some" kind of business. Apple litigates over a variety of multi-touch patents, and some of them are really very broad. Just two examples: U.S. Patent No. 7,663,607 covers a multipoint touchscreen (read its claim 1 to get an idea of the scope) and U.S. Patent No. 7,479,949 (first inventor on the list: Steve Jobs) lays exclusive claim to the use of heuristics (meaning experienced-based problem-solving strategies) for letting the operating system make educated guesses to interpret a user's intent based on where the user's fingers make contact with the screen. Those are, potentially, killer patents, but it's possible that a court could be convinced that Apple didn't really invent multi-touch as a whole (nor the use of heuristics for implementing it), and as a result, those patents could be invalidated or narrowed down to whatever technological progress Apple truly achieved. I guess it's worth trying to enforce those patents, but it would be difficult for Apple to rely just on patents of that kind. Apple is more likely to win with multiple narrower patents than a single nuclear bomb patent.

One patent stands out because Apple appears to be extremely convinced of its validity and enforceability. U.S. Patent No. 7,469,381 on "list scrolling and document translation, scaling, and rotation on a touch-screen display" is

Apple also asserted this one against Nokia, a dispute that was settled in June.

Let's look at what this patent covers and why it appears to be exceptionally solid in legal terms.

The scope of the '381 patent

I found two YouTube videos (out of presumably many) of demos of mobile devices that happen to show what list scrolling on a touch screen is like with and without the invention claimed by the '381 patent.

Here's a Samsung Galaxy S II demo by TechnixMedia:

About 16 seconds into the video, you can see that the user scrolls the list of settings. You can see the "Wireless and network" item at the top of the list. In reaction to the user's scroll gesture, the menu moves down for an instant, but it then bounces back so that the first list item ("Wireless and network") aligns with the top of the screen.

By contrast, here's a SlashGear video in which you can see what a similar operation looks like on a Google Nexus S:

The scroll operation I mean happens about 40 seconds into the video. It's the same Android menu (Settings), just that the operation is performed in the other direction. As you can see, the list does not move in the scrolling direction and then bounce back to the edge of the screen. When the end of the list is reached, the operation stops abruptly. An orange glow at the bottom of the screen signals that the scrolling operation hit one end of the list.

Based on those two videos, you may think that the difference between the two ways to handle list scrolling on a touch screen isn't significant. At the end of the day, they both serve the basic purpose of scrolling. But those are demo videos, and in actual "hands-down" use, there's a very important difference: the first kind of scrolling operation is smooth and intuitive because the list keeps moving along with your finger as long as you move it. Eventually, your finger will stop moving -- at the latest, it will stop when it hits the edge of the touch screen. But until then, the screen content sticks to your moving finger before it realigns with the edge of the screen. However, if you use a phone that doesn't have that feature, the scrolling operation stops abruptly while your finger keeps moving.

You get the same result either way, but Apple's patent covers the only way that looks and feels intuitive. Having a scrolling operation stop despite continued finger movement is counterintuitive. I experienced it first-hand on a Nokia phone that didn't have that feature (their more recent devices have that feature, and I heard that Windows Phone has that feature, so it won't be an issue for future Nokia devices). If scrolling stops though the finger keeps moving, many people intuitively press their finger more firmly onto the screen because it's almost as if the device wasn't reacting.

The "glow" is a very suboptimal solution. It tells you that you've reached the end of the list, but you still have the counterintuitive experience I described. That's because it will take your brain longer to notice and interpret the glow than to realize a disconnect between the movement of your finger and the scrolling of the screen content. Basically, the glow just limits the amount of time you'll spend continuing to move your finger. As soon as your brain has parsed the message the glow sends, you at least know that your touch screen isn't broken and you stop trying to scroll and pressing harder.

In the description part of the '381 patent document, Apple actually describes the scrolling behavior of devices without this patented feature as "awkward":

"[...] the limitations of conventional user interfaces can cause these actions to be awkward to perform.

Furthermore, scrolling displayed lists and translating electronic documents can be awkward on both portable and non-portable electronic devices with touch-screen displays. A user may become frustrated if the scrolling or translation does not reflect the user's intent. Similarly, a user may become frustrated if rotation and scaling of electronic documents does not reflect the user's intent.

The scope of the patent (which is defined by its claims, not its title or abstract) is obviously not as broad as covering "anything multi-touch". However, it's still broad in the sense that there's no workaround. The patent does not relate to only one particular way of implementing the behavior I described, such as one particular algorithm. It covers all ways to do this.

One such element of the user experience can't render Android totally unsalable. But if Apple can force Android device makers to remove this feature (which Google's own Android distribution doesn't even have because Google is apparently afraid of the '381 patent), consumers using those devices will be less satisfied. If they get to try out an iPhone or iPad, this feature will contribute to an overall impression of iOS sporting the more intuitive and enjoyable user interface.

And with several such patents, it's possible for Apple to create a very serious user experience gap between its own products and those of its competitors.

Death by a thousand cuts. Well, it won't literally take a thousand patents. But the key thing is: one such patent has an effect, and several of them can collectively have major impact.

Apple's confidence in the legal strength of this patent

In the first part of this post I already mentioned the disputes in which Apple has put this patent to use. It seems that they now tend to use it in their most time-sensitive and crucial actions.

This is how Apple emphasized the strength of this patent in its motion for a preliminary injunction against Samsung in the U.S.:

"The '381 patent is a strong, easily understood patent. It is directed to a clever method for displaying images on touch screens: when one uses a finger to drag a displayed page past its bottom edge, for example, and releases the finger, the page bounces back to fill the full screen. The claims of the '381 patent are so clear that in recent litigation between Apple and Nokia Corporation, neither side asked the court to construe any terms. A simple demonstration proves that Samsung's products infringe. Moreover, the Patent Office confirmed the validity of all twenty claims of this patent in a reexamination initiated by Nokia, which included the best prior art references Nokia could find."

The fact that all 20 claims of this patent survived reexamination by the patent office means that this patent enjoys an even stronger presumption of validity than a patent that hasn't yet proven its ability to withstand such scrutiny.

The clarity argument that Apple makes is also very important. Judges are rarely programmers, and jury members tend to struggle even more with those technical issues. When a patent is crystal clear, it's also a more predictable winner than technically more complex patents, such as the virtual machine patents that Oracle asserts against Google.

Claim construction -- meaning that the court has to decide on how to interpret the key terms that appear in the language of the asserted patent claims -- is always a big issue. But for this patent it doesn't even appear to be necessary because it speaks for itself:

"In view of the easily understood and straightforward language of the claims of the '381 patent, neither party [meaning: neither Apple nor Nokia] asked the Delaware [federal district] court to construe any of its terms during the May 15, 2011 [claim construction] hearing.

It also seems that prior art is a tough call. Quite possibly Apple was the first company to come up with the idea for how to solve this awkwardness issue affecting scrolling operations on touch screens. From a policy point of view, I don't believe it's good for innovation and competition that such an idea (even though it was undoubtedly a brilliant one) can be patented at a general level -- the patent monopolizes the right to implement this kind of behavior as opposed to just one particular technical solution for doing this. But we're now talking about litigation, not politics. And in patent litigation, prior art is the most common basis for having patents declared invalid (or at least having their scope narrowed). This is what Apple writes about Nokia's attempts to take this patent down:

"On April 28, 2010, Nokia requested ex parte reexamination of all twenty claims of the '381 patent based on four prior art references, three of which were not cited in the original prosecution. [...] Nokia relied on the same prior art for its invalidity contentions in supplemental interrogatory responses that it provided six months later, in effect conceding that it had not been able to find any better prior art, despite a year of litigation. [...] The Patent Office subsequently found Nokia’s arguments unpersuasive and confirmed the patentability of all twenty claims."

Note that this doesn't mean the patent can never be invalidated. But unless someone finds prior art that even Nokia -- which has vast resources and had a strong incentive to fight this patent -- wasn't able to find, it's unlikely to happen.

Again, I'm just desribing the way things are. It's the law of the land. And this patent exists not only in the U.S. and Australia (the two countries in which Apple is already suing over this patent) but Apple also filed applications elsewhere, including in Europe. In order to have some preliminary protection prior to getting a European patent granted, Apple also obtained this German utility model on the invention, which is a kind of fast-track patent with limited examination and will lose relevance once Apple gets a "real" European patent granted. Even though this is clearly a software patent (since the only innovative element here is a matter of operating system functionality as opposed to new types of hardware components), patents of this category are issued by the European Patent Office all the time, and many of them are upheld by the courts. So this is a patent that Apple can assert against Android in several major markets.

With invalidation being a long shot (to say the least) and with infringement being easily proven, the only realistic way for an Android device maker to get a license from Apple is to have enough intellectual property that Apple will feel forced to enter into a cross-license. But even if Apple agreed to a partial cross-license, it would try very hard to keep touchscreen patents like this one excluded from any deal.

If you'd like to be updated on the smartphone patent disputes and other intellectual property matters I cover, please subscribe to my RSS feed (in the right-hand column) and/or follow me on Twitter @FOSSpatents.

Share with other professionals via LinkedIn: