SmartGlass: A Big Step toward Convergence of TV and the Web Across Screens

Microsoft Xbox SmartGlass is CoTV 1.0 (maybe) -- on to CoTV 2.0!

With the release of Xbox SmartGlass, I am gratified to see many of the concepts I described as "Coactive TV" in 2002 finally being realized. I had been seeing increasing progress in recent years, as noted in my January post, but those have been very fragmented, partial steps, and I was being optimistic to refer to it then as CoTV 1.0. SmartGlass might be considered a major complementary step toward what (when integrated with those other pieces) will become representative of what I had in mind as CoTV 1.0.

The basic concept of CoTV is that we have multiple screens and input devices, and multiple content sources that have a Web of interconnections.  What we really want (even if most do not realize it yet) is to use the right combination of screens and input devices, at the right time, in the right way -- to work with whatever content we want at a given time. What connects them is the cloud, and our devices should use the cloud to support our media use seamlessly, not constrain it.

As noted in that January post, and more fully in my January "CoTV Now" summary, we are getting there, but there is still much more to come -- what might be looked to as CoTV 2.0 and beyond.  Now we seem to be at a significant milestone.  That makes this a good time to review where we are now, and to look to what will follow.  Based on the announcement materials, it seems as follows.

Now/emerging (CoTV 1.0):

• Numerous  iPad, iPhone, Android (and soon Surface) companion apps
• Social TV
• Producer and third party enhancements on the second screen
• AirPlay (and Miracast) screen-shifting 
• and now a much richer any-screen experience with SmartGlass that includes rich remote control and enhancements, and steps toward full multi-screen hypermedia browsing.

Still to come (CoTV 2.0):

• Selectable, Alternative "Enhancement Channels" 
• Screen targeting 
• Flexible session-shifting
• Link-and-pause (and sync bookmarks)
• Full multi-screen hypermedia browsing  
• TV Context parameter/API
• Full Coactive Internet commerce and advertising
• Third-party linking rights/fees

Some links expanding on this are listed below.

-------------

I want my CoTV!  ...SmartGlass promises to be a reasonable start!

(Apple, your move. AirPlay was nice, but SmartGlass goes much farther.  Google?  Others?)

-------------

On SmartGlass:


A very nice video overview:
Xbox SmartGlass and Internet Explorer for Xbox - E3 2012 HD

Some descriptions:
Introducing the New Entertainment Experience from Xbox
Xbox SmartGlass goes beyond the second screen
Introducing Xbox SmartGlass

More video:
E3 2012: Xbox Media Briefing Smartglass Highlights
E3 2012: Xbox SmartGlass
Xbox SmartGlass Walkthrough

On CoTV:

CoTV Now

The Necessity of Steve Jobs: ...Inventor? ...or Necessitor?

The recent comparisons of Steve Jobs to Edison and Ford brought me back to an important point: Invention is the mother of necessity. We don't realize we need something until an "inventor" shows us what it can be, and what it can do for us.

Which came first? Is necessity the mother of invention? (as the saying goes) ...or is invention the mother of necessity? Is inventing unrecognized necessities the real heart of inventing? As Jobs famously said: "It’s not the consumers’ job to know what they want.”

Jobs was more important as a necessitor, than as an inventor.  It struck me that the point some have raised -- that Jobs did not invent the technologies he popularized -- has some validity, but fails to balance the picture with this important point.  It is true that the mouse, the "drag-and-drop" graphical user interface, hypertext, music downloads, MP3 players,smartphones, tablets, touchscreens, computer animation, and many more key "inventions" applied by Jobs were not invented by him.  It seems widely recognized that Jobs' key contribution was that he saw how such things could be put to use in new configurations, and to serve needs that others did not see or saw less clearly (and also that he had the drive and resources to realize his visions...)

This resonated with me, because I have often felt that my own history as an inventor has a similar focus (even if hardly on the scale of Jobs').  The contribution is not so much in solving a recognized technical problem, but in seeing what technical problems should be solved, and why, and what else that would mean.  (That is why the theme of this blog is "user-centered media" -- that is pretty much the theme of much of my work.)

In a sense, this relates to innovation at the level of "systems thinking."  The necessitor does not just solve a problem, but creates a whole new system, within the larger system of people, technology, economics, and culture.  Jobs saw that what was missing in the music business was a new model for aggregated, simplified sales of music, and integration of an e-commerce system (the iTunes store) with a user agent (iTunes) and a device (iPod).  Once people saw that, they needed it.  No one created the wholistic vision that enabled that necessity to be recognized and acted on until Jobs did.

Similarly, some argue that Edison's real impact was not the light bulb, but the electric distribution system and related infrastructure that he recognized as needed to make the light bulb broadly useful.  It is perhaps more apparent that Ford was not so much an inventor of cars and mass production, but a necessitor, who realized that we needed simple black cars, and lots of them.  Often such cases are not simple inventions, but whole systems of invention.  One necessity/invention leads to other necessities/inventions, to whole ecologies of inventions.

So which came first? the necessity or the invention?  I suggest, as in most things, the answer is a non-dualistic "yes, both."  It is hard to separate the two.  Our patent system seems to think of inventions as the thing that matters.  The constitution defines patents to be for "any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvements thereof."  This has always seemed to me a limited view of what inventors do.

I suggest an equal form of "invention" is what Robert Kennedy spoke of:  "I dream of things that never were, and ask why not?"  Once we take that step, we may need to invent some technology, but often what we need to do is take the vision, understand all that it entails, and assemble a whole system from technologies that may have previously existed, but not been combined and adapted in the right way.  This kind of systems thinking, is on a much different level than the more commonly recognized engineering tasks of solving the technical problems to meet a previously recognized need.

...This also has led me to questions about the place for such contributions in the patent system.  It seems to me that such contributions may be equally deserving of some kind of patent protection, to reward the creative thinking that advances our "useful arts" and our civilization in general.  Just as with more narrow senses of technical invention, this takes not just inspiration, but perspiration (to paraphrase Edison).  But just how this kind of invention of necessity fits (or could be fit) with our current patent system seems a bit unclear.

------

[Should anyone know of any good thinking by others on this theme, I would welcome references.]

Social TV -- The "Killer App" for Coactive TV -- Ready for Ubiquity

Social TV promises to be the killer app for coactive TV (CoTV).  (A "killer application" is an application that is so desirable to users that it drives the adoption of a larger technology.  The concept emerged when spreadsheets and word processors drove the adoption of PCs, which have obviously broadened to far wider importance.)

There are a number of signs that Social TV is emerging as such a killer app (some mentioned in previous posts).

• IntoNow launched in January 2011 and was quickly acquired by Yahoo on 4/25/11, and Spot411 re-launched 7/18/11 as TVplus.  Both have gotten prominent press and both do fully automatic syncing to any program, without need for any involvement by the TV distributor. 
• The Wikipedia article on Social Television was created in 5/07 with 3,244 bytes, grew to 5,528 by the end of 2009, then grew to 10,469 by the end of 2010, and to 16,851 by 8/23/11.  It now includes a list of 32 such systems (not all of which involve two-screens).
• One of the most popular FIOS TV apps was the Twitter app.

Being a killer app does not mean it will ultimately dominate the use of the platform, but only that it drives early adoption.  I suggest there are other killer apps for coactive TV as well, and that the long term value will span a wide range of apps.

• From a user viewpoint, EPGs (electronic program guides) are another important killer app, not least because it is one the MSOs (multi-system operators, TV distributors) are embracing along with users.  EPGs showcase the value of the companion device to allow interaction with a nice UI, and without interfering with current viewing.   The irresistible power of the iPad UI and relatively open ecosystem has finally convinced the MSOs that they must go outside the box (at least as to the set-top box and the TV screen).  Comcast and Time Warner Cable have moved quickly to offer tablet-based EPGs and DVR programming.  The coactive EPG will evolve into the full "Media Concierge" service that I have been blogging about since 2005). 
• The real money to drive all of this is in advertising.  Obviously this will drive the service providers and advertisers, but I submit that users too will recognize and increasingly demand the value of well targeted ads that exploit the flexibility of coactive UIs to be unobtrusive.  Well targeted ads can be a valuable service, as long as they are no more intrusive than the viewer wants them to be (which may vary from time to time, and from ad to ad).  Coactive ads--driving from a short spot to a companion microsite (whether linked to live, or deferred using a bookmarking feature)--can be far less intrusive and far more useful than a longer TV ad with no coactive companion element. A good UI can give the user control over when and how such ads appear.

All of these promising killer apps have synergy with one another.  Coactive TV is at heart hypermedia, and thus "everything is deeply intertwingled." (Quoting Ted Nelson, who also coined the terms hypertext and hypermedia.)

• Social TV apps can work both as program enhancements and to provide program guide/media concierge services.  
• Social TV can also be about ads, such as during the Superbowl, or when any ad of interest to my social circle appears.
• All of these will drive usage of enhancement content (such as IMDB pages), which will create further synergies.

But there is one more thing that is essential, and that is ubiquity. While full, ubiquitous coactivity is not central to all Social TV, I suggest it is essential to enabling it to reach scale.

• Synchronizing Web browsing to TV can be done manually, and has for decades.  Viewers have created their own Social TV ever since the first two people sat with a laptop in front of a TV, and ever since the first online chat about a TV program.  It can also be automated with program specific apps.  ABC did it a decade ago with Enhanced TV for the Oscars and other shows, and now on the iPad for Grey's Anatomy, but program and network apps cannot create massive synergy.
• What is essentially to enabling Social TV (and most other CoTV apps) to cross the chasm is ubiquity.  Siloing companion apps to a separate app for each network or program or advertiser is hugely self-defeating.  How many users will load more than a few apps, and how many will bother to open those apps more than once?  Just as the Web eliminated the need for separate apps for every content service, a ubiquitous CoTV service will require only a single context-linking app to reach services for every program, to every Web service. There will be all kinds of mashups driven by that context, but an effective context-linking service must be essentially universal.

A truly ubiquitous coactive TV service will be always on, and always aware of a viewer's TV context (except when disabled).  Such a ubiquitous service can activate any Web service and any application, in a rich ecology much like that on the Web.  That way a user can just set up the coactive companion context service just once, and get synchronized for any program or ad, to any social networking service, content service, or whatever -- whether directly, or via mashups.  (Just how such services can be structured to enable flexibility and user control was described in my published patent disclosures, and will be a subject of  future posts.)

It now appears that Social TV is the next big thing in TV, and will drive full coactivity -- but a whole lot of other functions will ride its coattails.

TVs, iPads, Time Warner Cable, and Viacom -- Copyright vs. Copyrape

The latest overreach of copyright owners over the Time Warner Cable iPad TV app is an interesting encapsulation of all that is wrong with the current excesses of copyright.

I see this as a key policy issue relating my theme of "user-centered media" -- one that gets to the heart of the social contract behind copyright and all intellectual property. The key question is the balance of what is good for users, and what is fair incentive to content creators. The Constitution wisely embraced that balance, but many have lost sight of it.

The case reported by the NY Times Media Decoder is a classic of overreach (and one in which I find myself in the surprising position of supporting the cable companies).  As the Times reports, Viacom pleads that Time Warner Cable’s actions “will interfere with Viacom’s opportunities to license content to third-party broadband providers and to successfully distribute programming on its own broadband delivery sites.”  Let's think about that, both from a technical and a policy perspective.  This is not really a question of TWC vs. Viacom, but of the public vs. the rights-holders.

First, an quick look shows how silly this is from a technical perspective:

• Doesn't Time Warner's distribution of Viacom channels to TVs in the home limit "Viacom’s opportunities to license content to third-party broadband providers?"  If I could not get The Daily Show from TWC, I would certainly be much more inclined to use Hulu for it.  Why does Viacom allow that?
• How is an iPad different from a TV?  (Answer keeping in mind that this is the 21st Century.) 
• I have a Blu-Ray player and a Mac Mini both HDMI-connected to my TV, so I can watch any "third-party broadband provider" programming on my TV, and don't need TWC if Viacom is available through other providers. I can view such broadband provider content on any screen I like, and they are completely free to compete with TWC.  (Such any-screen connectivity will soon be the norm.)
• Why should TWC be locked off the iPad when Hulu is sold rights to distribute Viacom programming to any Internet-connected screen, including both TVs and iPads.
• Remember also that both TWC cable TV and broadband in TWC-served homes both arrive as RF signals running in different channels on the same coaxial cable!
• Once more, what century is this?

But this is really a deeper question of policy, and both content owners and content users seem to forget the basic social contract that drives copyright.  Let's step back to those basics:

• Copyright is designed to maximize social welfare by encouraging content creation
• Copyright owners are given limited rights as their incentive to create content
• The public pays to compensate creators for content.
• The public also may pay distributors and device manufacturers for facilitating access to content, but that has nothing to do with copyright.  (I have to pay for a book of Shakespeare plays or the Bible, or to download it to my cell phone in Timbuktu, but the content is free.)

Thus various parties have rights to compensation:

• The creators of Viacom content have rights to compensation for their content.
• Viacom is entitled to collect such copyright-related compensation, as well as compensation for their contribution to distribution (as well as some profit).
• Distributors and device providers are entitled to compensation for distribution and devices (which, again, has nothing to do with copyright).

But the viewer who pays for content has purchased the right to enjoy that content.  That can take a number of forms, but none are tied to what screen the viewer is using.  For a single viewer (or household, or whatever unit is bought):

• I can pay for time-limited access (such as a streamed subscription) or for permanent access (such as a download or DVD).
• Those costs may be bundled with distribution and device costs, but the underlying copyright fee is a simple and distinct component of that bundle.
• Any copyright-based limitation to content by device or location or technology that goes beyond the simple distinction of time-limited or unlimited access is without foundation.  Such limitations might be forced by technical limitations, but once those technical limitations disappear, they have no basis.

So if I pay for a Viacom program (content) one time, for a month, or forever, I should have unlimited rights to enjoy viewing that content, one time, for that month, or forever.  There might also be software fees for apps, and bandwidth fees for distribution, but there is no basis for any further content viewing fee.*

The copyright owners seem to have forgotten that the maze of licensing that they have so many lawyers working on is mostly an accident of technological history.  Hopefully the courts will not lose sight of the basics and let the tail wag the dog, now that technology is liberating us.  Hopefully they will not let copyright turn into copyrape.

Content does not want to be free, if its creator wants compensation (subject to his limited copyright).  But once I have paid for a license, I should be free to view it as I like, for whatever amount of viewing I have paid the creator for.  Watching multiple times might be a multiple use, but watching on one screen rather than another is not a different use.

Ask not for whom the copyright tolls, it tolls for thee -- for the public welfare.

---------------------------------------
*A related issue is the current idea that cable-sourced access might be limited to in-home viewing.  That too is an artificial limitation,with no inherent justification.  It may be hard to prevent abuse of licenses, if I can let all my friends view my TWC content in their homes, but as long as TWC has the technical means to limit viewing to valid subscribers (and those viewing with them) why should there be any geographic restriction limiting out of home viewing? 

The Daily, iPad, and Apps ...or Web browsing with HTML5 -- Which paradigm?

The appearance of  The Daily from News Corp. is seen as a big step in the online journalism business, as described in a WSJ article.

I played with it briefly and it brings me back to some key questions about the future of media.  It will be very interesting to see how it does.  There are a range of important issues, and here are some impressions.

The interesting business issue is how app models are seen as a last chance to give publishers another bite at the monetization apple (pun intended) vs. free Web content.  This depends to some extent on whether Apple and other app stores let publishers keep enough money and enough control of the customer relationship (which Apple clearly hates to do, but Google is more open to).  But with HTML5 Web apps as alternative, that may become a harder sell than Murdoch now hopes.

Underlying this is the big technology question of whether the app fad loses out to HTML5 Web browsing.  In many respects, the app/widget model is a giant step backward.  Pre Web, there were "apps" for every online service, and they were all unique and non-interoperable with a clutter of invocations and divergent UIs.  The Web/browser brought a "World Wide Web" of consistency and interoperability that still enabled flexibility and varying look/feel.  A key issue is how to benefit from apps/widgets without going back to another age of islands and silos?  I built some of the first pre-Web publisher "apps" for TV Guide (hello again News Corp), Golf Magazine, Sierra, and others in the early '90s, and saw first hand how much the Web simplified things for both publishers and users.

The question is why bother downloading apps, when it seems HTML5 will soon give pretty much the same UI with no download?  Most of the current UI benefits of apps will soon go away.  The lasting benefit of the app store is central merchandising/sales (and a home page UI), and as Google shows with their Web app store, this can be done as little more than a Web site.  A few useful links are an Engadget article, the Chrome Webstore, and its FAQ.

Check out NY Times and SI Snapshot Chrome apps for an app-like experience in a browser, with little or nothing to download.  The NYTimes chrome site actually runs in Safari on the iPad and looks/acts much like the iPad app (but seems to give a different content mix).  The only essential thing the app store really adds is the home page array of icons (and maybe a different way to get people to pay).

I will bet on the browser.  It offers the best overall and most open user-centered experience.  And I think there are other ways to solve the monetization problem.  (One in particular is my FairPay pricing process, with an example of usage for a newspaper on my FairPay Zone blog.)

The awakening of TV to the 21st Century ...Real Soon Now?

CoTV was ahead of its time in 2002...  Now the stars may really be aligning for TV "companion" apps.


When I talked about CoTV to people at major TV and Web companies in 2002-5, they thought it was a good idea and assured me "Yes, I get it."  Some did, and some just thought they did.  Like all forms of "interactive TV" it has been "just around the corner" for many years "waiting for the stars to align."  But now the stars really do seem to be aligning.


At the recent TV of the Future "TVOT NYC Intensive"  from iTVT and Canoe, it was evident that important things are happening:

• iPad has awakened he giants:  Comcast, Time Warner, TV networks, TiVo, and many others are jumping into coactive "companion" apps for tablets (and phones).  iPad and other tablets are nearly ideal companion devices, and already in millions of laps.
• Platforms for interaction (CableLabs/Canoe, ETV, EBIF, ...) are enabling real innovation and increasing openness from within the distribution establishment.  EBIF is in over 20 million homes, and growing rapidly, not only in cable systems.  ETV is getting real.  The PayPal Buy Button is a nice example.
• Over-the-top alternatives are real -- the incumbent system operators know they need to get into the 21st century or watch their content distribution business get bypassed.

At the same time, others are moving in the same direction, and users are doing it themselves, manually and awkwardly, but in growing numbers:

• External plays based on TiVo, Blu Ray (Pocket BLU), and sound recognition (Spot411 Entertainment Tonight) show how this can be done outside the cable plant, even for shows distributed on cable.
• Social TV apps (about what you are watching now) are making the viewer value proposition even more powerful.

What is missing is for a smart player to provide an "always-on" TV sync connector -- a single app and context portal that drives any companion content for any show (and any ad) to a large base of households.  The problem has been that nearly all attempts to provide TV companion apps have been siloed, and limited to a single program or network.  

• In the early 2000's ABC ETV and Goldpocket did second-screen companion apps for major network shows (Millionaire, Sunday/Monday Night Football, Academy Awards, etc.) but only if you navigated to an ABC or program-specific Web site.  Up-take was rarely even 1% of  viewership, hardly a basis for a business.
• Now iPad and iPhone apps are creating similar experiences, but for the most part it is still a different app for each show or network.

How can anyone really expect significant uptake when users must know there is a particular app, bother to get it, then bother to use it, and then do the same every time they change channels or programs?  Even now at TVOT, I spoke to someone from Canoe who seemed to think I must be some kind of idiot to view this as a problem.  Saying (my paraphrase): "The user can just get the right app, or just go to the right Web site.  That BMW ad you want to sync to is a network ad, not a cable ad, so the network has to provide the app -- or the viewer can just go to bmw.com. That is simple -- why can't you see that???"  

One more time:  The viewer should not have to switch from a Comcast app to an ABC app to an MTV app to a BMW app (or enter a different URL) every time a program or ad changes. Only when there is one app (or Web portal) that seamlessly syncs enhancements for any show and any ad will this be easy for the viewer.  I should just be able to turn enhancements on, and have them appear on my tablet with no further effort (until I turn them off).  And when it is that easy, companion enhancements might quickly grow to 10-20-30% of viewership or more.  Just the linkage revenue from linking those ads would be worth many billions.

So does anyone get it yet?  Yes.  My contacts with well-placed industry players indicate that more and more of them now do get it, and some see it beginning to happen in the next year or two.  The cable operators have finally recognized that set-top boxes are good MPEG engines, but hopelessly inadequate platforms for user interfaces, and that they must open up to partners using Web-based technologies.  Canoe is seeking outside partnerships and ideas.  Maybe the system operaors will actually do what they need to do.  One interesting hint of this new direction is the eBay companion TV app, which can sync an iPad with any program on an EBIF-enabled set-top.  A demo by RCDb at TVOT Intensive showed a similar app for syncing iPad enhancements to deliver IMDB pages and other content.  Cable operators are starting with companion program guides, but a program guide that does not know what you are watching right now is pretty lame (as they are aware).  Once they provide that added smarts to the companion, linking to program-specific enhancements will be (relatively) easy.

And if the distributors do not get their act together, outsiders will do it.  The Spot411 effort shows one approach, and there are many others.  TiVo is well positioned to do it (and could still win big if it did).  And if it comes too slowly to the legacy providers, the IPTV players will soon have enough viewership on big screens to lead the way.

So who will it be that realizes this is a critical race, does it right, and wins it? TV is ready to be reborn for the 21st Century.  Once someone makes it easy to use across the board (and does not cripple it), it will happen very fast.

Building on Google TV: TV meets Webpad. Webpad meets TV.

As the Google blog says, "these features are just a fraction of what Google TV can do."

One hint, reported in the Wall Street Journal, is that "Linkages between Android phones and Google TV bring some unusual benefits. A Google engineer, for example, demonstrated how a person could use voice recognition in his cellphone to search for a TV program by speaking its name.

I see this as an opening for the kind of rich "Coactive TV" (CoTV) applications that coordinate TV viewing with enhancements and controls on a second screen device, such as an Android phone or tablet or notepad. While putting the Web onto the TV is desirable for some use cases, the real opportunity for convergence is to coordinate a Web device with a TV device. This can provide content and Social Web services on a Web device that knows what you are watching (such as to tweet about a program), and let the Web device control what you are watching) such as to swing a video from your tablet or phone to your TV.

Hopefully Google is well aware of this potential. I spoke to some senior people about these ideas some time ago (and as noted in a prior post, there is some interesting Google research, including a paper by some guy named "Brin, S"). But in any case, independent developers should be able to provide this (as long as the app police don't prevent them).

As noted in my previous post, iPad is beginning to show what a Webpad can do as a second screen. Google TV seems another big step in the right direction.

Comcast, Time Warner on iPad -- an ideal device for Coactive TV

The 5/12 announcement of Comcast's Xfinity iPad app, and comments by Time Warner that all cable operators are going to have similar offerings, looks like a big step toward a platform for Coactive TV on a mainstream basis. Two videos show the current steps.

First, this looks like a major step to position iPad (and presumably future tablets from other sources) as a well-integrated second screen for use with TV viewing. It will offer a rich remote control usable on the sofa for enhanced control of the big-screen TV, with full program guide and DVR control functions having an excellent UI, including its handy soft keyboard.

Second, it looks like a first step toward a simple coactive Social Web app. The Comcast demo shows how it lets you send an invitation to a friend, so he can tune in to the program you are watching with just a single click (if he has equivalent service).

iPad looks like a poster child for the kind of Webpad that would be an ideal second screen for Web services related to what you are watching on TV. With this device, and the level of TV context awareness in Comcast's demo, it would be very easy for the cable operators to add a full suite of coactive services. This could enable the iPad to show arbitrary Web content related to what you are watching on your TV.

----
Some related news that is also encouraging in using such mainstream devices as TV adjuncts is from Crestron. This leading high-end whole house entertainment control company is embracing iPad as an alternative to its very expensive custom tablet remote controls.