SmartGlass: A Big Step toward Convergence of TV and the Web Across Screens

Microsoft Xbox SmartGlass is CoTV 1.0 (maybe) -- on to CoTV 2.0!

With the release of Xbox SmartGlass, I am gratified to see many of the concepts I described as "Coactive TV" in 2002 finally being realized. I had been seeing increasing progress in recent years, as noted in my January post, but those have been very fragmented, partial steps, and I was being optimistic to refer to it then as CoTV 1.0. SmartGlass might be considered a major complementary step toward what (when integrated with those other pieces) will become representative of what I had in mind as CoTV 1.0.

The basic concept of CoTV is that we have multiple screens and input devices, and multiple content sources that have a Web of interconnections.  What we really want (even if most do not realize it yet) is to use the right combination of screens and input devices, at the right time, in the right way -- to work with whatever content we want at a given time. What connects them is the cloud, and our devices should use the cloud to support our media use seamlessly, not constrain it.

As noted in that January post, and more fully in my January "CoTV Now" summary, we are getting there, but there is still much more to come -- what might be looked to as CoTV 2.0 and beyond.  Now we seem to be at a significant milestone.  That makes this a good time to review where we are now, and to look to what will follow.  Based on the announcement materials, it seems as follows.

Now/emerging (CoTV 1.0):

• Numerous  iPad, iPhone, Android (and soon Surface) companion apps
• Social TV
• Producer and third party enhancements on the second screen
• AirPlay (and Miracast) screen-shifting 
• and now a much richer any-screen experience with SmartGlass that includes rich remote control and enhancements, and steps toward full multi-screen hypermedia browsing.

Still to come (CoTV 2.0):

• Selectable, Alternative "Enhancement Channels" 
• Screen targeting 
• Flexible session-shifting
• Link-and-pause (and sync bookmarks)
• Full multi-screen hypermedia browsing  
• TV Context parameter/API
• Full Coactive Internet commerce and advertising
• Third-party linking rights/fees

Some links expanding on this are listed below.

-------------

I want my CoTV!  ...SmartGlass promises to be a reasonable start!

(Apple, your move. AirPlay was nice, but SmartGlass goes much farther.  Google?  Others?)

-------------

On SmartGlass:


A very nice video overview:
Xbox SmartGlass and Internet Explorer for Xbox - E3 2012 HD

Some descriptions:
Introducing the New Entertainment Experience from Xbox
Xbox SmartGlass goes beyond the second screen
Introducing Xbox SmartGlass

More video:
E3 2012: Xbox Media Briefing Smartglass Highlights
E3 2012: Xbox SmartGlass
Xbox SmartGlass Walkthrough

On CoTV:

CoTV Now

Filtering for Serendipity -- Extremism, "Filter Bubbles" and "Surprising Validators"

[The Augmented Wisdom of Crowds:  Rate the Raters and Weight the Ratings, (2018) puts this in a much broader framework and outlines an architecture for augmenting social media and other collaborative systems.]

[A post-2016-election update on this theme:
2016: Fake News, Echo Chambers, Filter Bubbles and the "De-Augmentation" of Our Intellect]


Balanced information may actually inflame extreme views -- that is the counter-intuitive suggestion in a NY Times op-ed by Cass Sunstein, "Breaking Up the Echo" (9/17/12).   Sunstein is drawing on some very interesting research,* and this points toward an important new direction for our media systems.

I suggest this is especially important to digital media, in that we can counter this problem with more intelligent filters for managing our supply of information.  This could be one of the most important ways for technology to enhance modern society. Technology has made us more foolish in some respects, but the right technology can make us much smarter.

Sunstein's suggestion is that what we need are what he calls "surprising validators," people one gives credence to who suggest one's view might be wrong.  While all media and public discourse can try to leverage this insight, an even greater opportunity is for electronic media services to exploit this insight that "what matters most may be not what is said, but who, exactly, is saying it."

Much attention has been given to the growing lack of balance in our discourse, and there have been efforts to seek to address that.

• It has been widely lamented that the mass media are creating an "echo chamber" -- such as Fox News on the right vs. MSNBC on the left.  
• It has also been noted that Internet media bring a further vicious cycle of polarization, as nicely described in the 2011 TED talk (and related book) by Eli Pariser, "Beware online "filter bubbles," services that filter out things not to one's taste.
• Similarly, extremist views that were once muted in communities that provided balance are now finding kindred spirits in global niches, and feeding upon their own lunacy.

This is increasingly damaging to society, as we see the nasty polarization of our political discourse, the gridlock in Washington, and growing extremism around the world. The "global village" that promises to bring us together is often doing the opposite.

It would seem that the remedy is to try to bring greater balance into our media. There have been laudable efforts to build systems that recognize disagreement and suggest balance, such as services like SettleIt, FactCheck, and Snopes, and, a particularly interesting effort, the Intel Dispute Finder (no longer active).

• The notable problem with this is Sunstein's warning that even if we can expose people to greater balance, that may not be enough to reduce such polarization, and that balancing corrections can even be counter-productive, because "biased assimilation" causes people to dismiss the opposing view and become even more strident. 
• Thus it is not enough to simply make our filter bubbles more permeable, to let in more balanced information.  What we need is an even smarter kind of filter and presentation system.  We have begun to exploit the "wisdom of crowds," but we have done little to refine that wisdom by applying tools to shape it intelligently.

From that perspective, consider Sunstein's suggestions:

People tend to dismiss information that would falsify their convictions. But they may reconsider if the information comes from a source they cannot dismiss. People are most likely to find a source credible if they closely identify with it or begin in essential agreement with it. In such cases, their reaction is not, “how predictable and uninformative that someone like that would think something so evil and foolish,” but instead, “if someone like that disagrees with me, maybe I had better rethink.”

Our initial convictions are more apt to be shaken if it’s not easy to dismiss the source as biased, confused, self-interested or simply mistaken. This is one reason that seemingly irrelevant characteristics, like appearance, or taste in food and drink, can have a big impact on credibility. Such characteristics can suggest that the validators are in fact surprising — that they are “like” the people to whom they are speaking.

It follows that turncoats, real or apparent, can be immensely persuasive. If civil rights leaders oppose affirmative action, or if well-known climate change skeptics say that they were wrong, people are more likely to change their views.

Here, then, is a lesson for all those who provide information. What matters most may be not what is said, but who, exactly, is saying it. 

This struck a chord with me, as something to build on.  Applying the idea of "surprising validators"  (people who can make us think again):

• The media and social network systems that are personalized to serve each of us can understand who says what, who I identify and agree with in a given domain, and when a person I respect holds views that are different from views that I have expressed that I might be wrong about.  Such people may be "friends" in my social network, or distant figures that I am known to consider wise.  (Of course it is the friends I consider wise, not those I like but view as misguided, that need to be identified and leveraged.)
• By alerting me that people I identify and agree with think differently on a given point, such systems can make me think again -- if not to change my mind, at least to consider the idea that reasonable people can differ on this point. 
• Such an approach could build on the related efforts for systems that recognize disagreement and suggest balance noted above.  ...But as Sunstein suggests, the trick is to focus on the surprising validators.
• Surprising validators can be identified in terms of a variety of dimensions of values, beliefs, tastes, and stature that can be sensed and algorithmically categorized (both overall and by subject domain).  In this way the voices for balance who are most likely to be given credence by each individual can be selectively raised to their attention.  
• Such surprising validations (or reasons to re-think) might be flagged as such, to further aid people in being alert to the blinders of biased assimilation and to counter foolish polarization.

This provides a specific, practical method for directly countering the worst aspects of the echo chambers and filter bubbles.

More broadly, what we need to counter the filter bubble are ways to engineer serendipity into our information filters -- we need methods for exposing us to the things we don't realize we should know, and don't know how to set filters for.  Identifying surprising validators is just one aspect of this, but this might be one of the easiest to engineer (since it builds directly on the relationship of what we know and who we know, a relationship that is increasingly accessible to technology), and one of the most urgently needed.

Of course the reason that engineering serendipity is hard is because it is something of an oxymoron--how can we define a filter for the accident of desirable surprise?  But with surprising validators we have a model that may be extended more broadly--focused not on disputes, but on crossing other kinds of boundaries--based on who else has made a similar crossing--still in terms of what we know and who we know, and other predictors of what is likely to resonate as desirable surprise. Perhaps we might think of these as "surprising combinators."

[Update 10/21/19:] Serendipity and flow. Some specific hints on how to engineer serendipity can be drawn from a recent article, "Why Aren't We Curious about the Things We Want to Be Curious About?"  This reinforces my suggestion (in the paragraph just above) that it be "in terms of what we know and who we know" adding the insight that "We’re maximally curious when we sense that the environment offers new information in the right proportion to complement what we already know" and suggesting that it has to do with finding "the just-right match to your current knowledge that will maintain your curiosity." This seems to be another case of seeking a "flow state," the energized and enjoyable happy medium between not so challenging or alien as to be too frustrating, yet not so easy and familiar as to be boring. I suggest that smart filtering technology will help us find flow, and do it in ways that adapt in real time to our moods and our ongoing development.

This offers a way to more intelligently shape the "wisdom of crowds," a process that could become a powerful force for moderation, balance, and mutual understanding. We need not just to make our "filter bubbles" more permeable, but much like a living cell, we need to engineer a semi-permeable membrane that is very smart about what it does or does not filter.

Applying this kind of strategy to conventional discourse would be complex and difficult to do without pervasive computer support, but within our electronic filters (topical news filters and recommenders, social network services, etc.) this is just another level of algorithm. Just as Google took old academic ideas about hubs and authority, and applied these seemingly subtle and insignificant signals to make search engines significantly more relevant, new kinds of filter services can use the subtle signals of surprising validators (and surprising combinators) to make our filters more wisely permeable.

That may be society's most urgent need in information and media services.  Only when we can bring a new level of collaboration, a more intelligently shaped wisdom of crowds, will we benefit from the full potential of the Internet.  We need our technology to be more a part of the solution, and less a part of the problem.  If we can't learn to understand one another better, and reverse the current slide into extremism, nothing else will matter very much.

[Update:]  Note that the kind of filtering suggested here would ideally be personalized to each individual user, fully reflecting the "everything is deeply interwingled" and non-binary nuance of their overlapping Venn diagram of values, beliefs, tastes, communities of interest, and domains of expertise. However, in use-cases where that level of individual data analysis is impractical or impermissible, it could be done at a less granular level, based on simple categories, personas, or the like. For example, a news service that lacks detailed user data might categorize readers based on just a current session to identify who might be a surprising validator, or what might be serendipitous.

[Update 12/7/20:] Biden wins in 2020 with Surprising Validators!

A compelling report by Kevin Roose in the NY Times shows how Surprising Validators enabled Biden's "Rebel Alliance" to cut a hole in Trump's "Death Star" -- "…the sources that were most surprising were the one who had the most impact." "Perhaps the campaign's most unlikely validator was Fox News." This was by the campaign, external to the platforms' algorithms, but think how much more powerful this could be when fully integrated.

---
See the Selected Items tab for more on this theme.

[See also my earlier post on this theme:
Full Frontal Reality: how to combat the growing lunatic fringe.]

-------------
*The work Sunstein apparently refers to can be found by searching for "Biased Assimilation and Attitude Polarization," the title of a much-cited 1979 paper. I found some very interesting research and plan to review this further, seeking methods suited to algorithmic use. One interesting current center of study is the Yale Law School Cultural Cognition Project.

(On a personal note, this is an effort I have seen as having huge benefit to society since my first exposure to early work on computer-aided conferencing and decision support systems in the early 1970's.  I continue to see this as a vital challenge to pursue, and I welcome dialog and collaboration with others who share that mission.)  

i[Carter]Phone? -- Apple and Anti-Competitive Tying

Apple is pushing the laws prohibiting anti-competitive behavior, as noted in an interesting article by James Stewart in today's NYTimes, with reference to Maps and the iTunes Store.  It considers how Apple's efforts at total control of their ecosystem may be both harmful and illegal--at some point, if not yet.

For some time I have had similar concerns, and have been wondering how long until we see an "iCarterPhone Decision."  What do I mean by that?  Followers of communications history will remember the Carterfone Decision (1968) as a landmark step toward the breakup of the Bell System monopoly. Until then it was illegal to attach a phone not approved by AT&T to the US telephone network.  This was based on the AT&T argument that attaching any device not fully tested and approved by them to the network might introduce voltages or other electrical effects that would run through the wires and harm their central office equipment, potentially causing widespread harm.  The only permissible way to add a specialized device like the one sold by Carterfone was to use a Rube Goldberg-like acoustic coupler, with rubber cups that relayed sound in or out of a standard Bell telephone handset ear and mouthpiece  with no direct electrical connection (and with issues of signal quality).  Some of you remember early modems that connected to computers that way. The Carterfone Decision changed all that, and opened the way for the vibrant market in phones, answering machines, faxes, modems, etc. that we now take for granted.

The iPhone/iTunes ecology smacks of much the same kind of anticompetitive control, with restrictions that limit consumer rights, raise consumer costs, and limit competitive innovation.  The Times addresses the current flap over Apple's inferior maps app, as well as Department of Justice price fixing charges against Apple relating to e-books sold through the iTunes Store.  Similar issues apply to control of apps in general that Apple does not like for one reason or another --such as has been the case with Skype, Google, Flash, and many others.  Contrast this with Microsoft PCs that allow you to run any software from any source, with no involvement of Microsoft whatsoever.  Of course we are free of migrate to the Android ecosystem to get greater openness, and many have chosen to do just that.

As the Times article notes, Apple is not dominant the way Microsoft was (or AT&T), and thus its tying sales in the App Store may not reach a level actionable under antitrust laws. (Its alleged price fixing is another story.) But at an ecosystem level, given its disproportionate number of apps, it does already have a level of dominance that might warrant correction.

Other areas in which Apple is riding roughshod on the market (and consumers) relate to other kinds of proprietary behavior.  Apple champions open standards like HTML5 over proprietary standards like Flash when the proprietary standards belong to the competitor, and it suits their interests to smash them , but insists on proprietary standards of its own, such as for its iPhone connectors and its AirPlay protocol, for which it charges exorbitant prices (adapter retail $29?) or licensing fees (AirPlay speakers retail price bump $100?).

It will be very interesting to see how this develops -- whether the market rebels or the government finds cause to draw a line, or they just fail to maintain their edge.  From the market perspective, Apple is walking a very fine line, balancing the positive perception of product quality against the negative perception of arrogance and rapaciousness. Jobs was able to ride that balance for a very profitable run, but the maps fiasco, and the increasing success of Android (and maybe Microsoft, or someone yet to appear) suggests that this is a precarious and anti-consumer position, and that Apple's days of dictating to consumers and its ecosystem partners may be numbered.

A New Age in Patent Liquidity -- NYC 2/15 -- MIT Enterprise Forum Panel Session

This is a panel that should be very relevant to all entrepreneurs who have an interest in getting and monetizing patents, as well as those who work with them. "A New Age in Patent Liquidity: New Opportunities for Entrepreneurs," is presented by MIT Enterprise Forum of NYC.

I will be on the panel to present the perspective of an entrepreneur/inventor who has successfully navigated the Kafkaesque world of patents, which can be rewarding, but also hugely frustrating, costly, and risky.  I described some of the twists and turns of my adventures in a 2008 blog post "'The Six Phases of a Technology Flop' ...Patents, and Plan B." The theme was how I started seeking to build a software/services business, but also sought patents as a hedge to protect my investment -- a "Plan B." When the business failed to keep up with better-connected competitors with deeper pockets, I turned to the patents to try to capture value for my innovations.  Working with partners who brought the expertise and funding needed to do that, and eventually to undertake a patent suit, I went part way through infringement cases against Microsoft and Apple.  Some additional background on that is in last year's post that tells how Intellectual Ventures changed the game with a very creative, win-win deal.

I also expect to touch on my 2008 sale of another portfolio of patents to another very innovative company, RPX, as well as my ongoing work developing other patents.  I am pleased that Kevin Barhydt, VP, Head of Acquisitions for RPX (and formerly at IV) will also be on the panel.

From my perspective, IV, RPX, and others are making a real difference is offering inventors and other patent owners a way to monetize their IP for reasonable compensation -- in a market that is rational, and has a middle ground between "take a hike" and the nuclear option of litigation, with its huge costs in money, time, and disruption.

It is a pleasure to be a panelist and organizer for this event, especially given that I was the moderator and an organizer of MITEF's well-received 2000 panel session  "Patents for Dot-coms," which had an equally distinguished panel.

Coactive TV -- The World of TV is getting there, and more is yet to come...

The kind of advanced "coactive" TV that I been promoting since 2002 is finally reaching the mainstream, but there is still much more to come.

As noted in a new page on the CoTV Web site, "Coactive TV: User-centered Convergence Today and Tomorrow:" 

The increasing prevalence of "media multitasking" (simultaneous use of TV and the Web) on laptops and smartphones began to change perceptions, and 2-screen ITV began to be seen as desirable in itself. Users were creating their own manual ITV experiences by finding relevant Web services on their own.  That set the stage for the emergence of CoTV 1.0, which was then kick-started by the iPad.  One indication of CoTV crossing the chasm into mainstream attention was the survey by Katherine Boehret of the influential Mossberg/Wall Street Journal/All Things D team on 12/20/11.

Another indication this is getting real was the number of announcements at CES. As reported by Bill Niemeyer in the 1/13 OTT Monitor from The Diffusion Group:

One key takeaway from CES that has floated above the noise pertains to Automated Content Recognition (ACR) for TV and video platforms. CES saw announcements from a number of ACR vendors including Audible Magic, Civolution, Gracenote, and Zeitera.

What is ACR? It's a variety of technologies that allow a device or service to recognize automatically a specific piece of content and synchronize to it within seconds. ACR can be based on audio/video watermarking or fingerprinting (i.e., cloud-based pattern matching used by mobile music app services like Shazam). Let your cell phone hear a brief bit of a song and Shazam will tell you what it is and even provide synchronized lyrics.

How can ACR be used in OTT [Over The Top]? It can synchronize interactive experiences for programs - whether viewed live or time-shifted - as well as advertising or e-commerce apps. Distinct from watermarking, which requires insertion in the content, fingerprinting can be done completely outside the realm of content providers, networks, and PayTV operators. That said, developing third-party synced apps without infringing on copyrights could be tricky.

With ACR, literally "the possibilities are endless" (to use a trite phrase). It's a powerful tool that needs to be put in the hands of creatives to realize fully its artistic potential, as well as clever business-side types to see how much "extended revenue" it can create.

But this is just the start. To look further into the future of advanced TV and video-based hypermedia, check out the section on "CoTV Tomorrow -- CoTV 2.0" on that new CoTV page.  A partial list of advanced features:

• Selectable, Alternative "Enhancement Channels" 
• Screen targeting
• Flexible session-shifting
• Link-and-pause (and sync bookmarks)
• Full hypermedia browsing  
• TV Context parameter/API
• Full Coactive Internet commerce and advertising
• Third-party linking rights/fees