Tuesday, May 17, 2022

Boiling Elon Musk – Jumping Out Of The Pot Of Platform Law?

My take on the deeper issues for democracy of Musk's on-again/off-again bid for Twitter, Boiling Elon Musk – Jumping Out Of The Pot Of Platform Law?, has been published on Techdirt.  
The boiling frog syndrome suggests that if a frog jumps into a pot of boiling water, it immediately jumps out — but if a frog jumps into a slowly heating pot, it senses no danger and gets cooked. Mark Zuckerberg’s Facebook has been gradually coming to a boil of dysfunction for a decade – some are horrified, but many fail to see any serious problem. Now Elon Musk has jumped into a Twitter that he may quickly bring to a boil. Many expect either him – or hordes of non-extremist Twitter users – to jump out.
The frog syndrome may not be true of frogs, and Musk may not bring Twitter to an immediate boil, but the deeper problem that could boil us all is “platform law:” Social media, notably Twitter, have become powerful platforms that are bringing our new virtual “public square” to a raging boil. Harmful and polarizing disinformation and hate speech are threatening democracy here, and around the world.

The apparent problem is censorship versus free speech (whatever those may mean) -- but the deeper problem is who sets the rules for what can be said, to what audience? Now we are facing a regime of platform law, where these private platforms have nearly unlimited power to set and enforce rules for censoring who can say what...

The article goes on to suggest ways to take the pot off the burner. 

Thursday, May 05, 2022

Musk, Twitter, and Bluesky -- How to Rethink Free Speech and Moderation in Social Media

Beyond the hope, fear, and loathing wrapped in the enigma of Elon Musk's Twitter, there are some hints of possible blue skies and sunlight, whatever your politics. A new architecture document from the Bluesky project that Jack Dorsey funded points to an important strategy for how that might be achieved -- whether by Twitter, or by others. Here are some quick notes on the key idea and why it matters.

That document is written for the technically inclined, so here are some important highlights (emphasis added):

It’s not possible to have a usable social network without moderation. Decentralizing components of existing social networks is about creating a balance that gives users the right to speech, and services the right to provide or deny reach.

Our model is that speech and reach should be two separate layers, built to work with each other. The “speech” layer should remain neutral, distributing authority and designed to ensure everyone has a voice. The “reach” layer lives on top, built for flexibility and designed to scale.

Source: Bluesky 

The base layer...creates a common space for speech where everyone is free to participate, analogous to the Web where anyone can put up a website. ...Indexer services then enable reach by aggregating content from the network. Moderation occurs in multiple layers through the system, including in aggregation algorithms, thresholds based on reputation, and end-user choice. There's no one company that can decide what gets published; instead there is a marketplace of companies deciding what to carry to their audiences.

Separating speech and reach gives indexing services more freedom to moderate. Moderation action by an indexing service doesn't remove a user's identity or destroy their social graph – it only affects the services' own indexes. Users choose their indexers, and so can choose a different service or to supplement with additional services if they're unhappy with the policies of any particular service.

There is growing recognition that something along these lines is the only feasible way to manage the increasing reach of social media that is now running wild in democracies that value free speech. I have been writing extensively about this on this blog, and in Tech Policy Press (see the list of selected items).

The Bluesky document also suggests a nice two level structure that separates the task of labeling from the actioning task that actually controls what gets into your feed:

The act of moderation is split into two distinct concepts. The first is labeling, and the second is actioning. In a centralized system the process of content review can lead directly to a moderation decision to remove content across the site. In a distributed system the content reviewers can provide information but cannot force every moderator in the system to take action.


In a centralized system there would be a Terms of Service for the centralized service. They would hire a Trust and Safety team to label content which violates those terms. In a decentralized system there is no central point of control to be leveraged for trust and safety. Instead we need to rely on data labelers. For example, one data labeling service might add safety labels for attachments that are identified as malware, while another may provide labels for spam, and a third may have a portfolio of labels for different kinds of offensive content. Any indexer or home server could choose to subscribe to one or more of these labeling services.

The second source of safety labels will be individuals. If a user receives a post that they consider to be spam or offensive they can apply their own safety labels to the content. These signals from users can act as the raw data for the larger labeling services to discover offensive content and train their labelers.

By giving users the ability to choose their preferred safety labelers, we allow the bar to move in both directions at once. Those that wish to have stricter labels can choose a stricter labeler, and those that want more liberal labels can choose a more liberal labeler. This will reduce the intense pressure that comes from centralized social networks trying to arrive at a universally acceptable set of values for moderating content.


Safety labels don’t inherently protect users from offensive content. Labels are used in order to determine which actions to take on the content. This could be any number of actions, from mild actions like displaying context, to extreme actions like permanently dropping all future content from that source. Actions such as contextualizing, flagging, hiding behind an interstitial click through, down ranking, moving to a spam timeline, hiding, or banning would be enacted by a set of rules on the safety labels.

This divide empowers users with increased control of their timeline. In a centralized system, all users must accept the Trust and Safety decisions of the platform, and the platform must provide a set of decisions that are roughly acceptable to all users. By decomposing labels and the resulting actions, we enable users to choose labelers and resulting actions which fit their preferences.

Each user’s home server can pull the safety labels on the candidate content for the home timeline from many sources. It can then use those labels in curating and ranking the user timeline. Once the events are sent to the client device the same safety labels can be used to feed the UX in the app.

This just hints at the wide array of factors that can be used in ranking and recommending that I have explored in a major piece in Tech Policy Press, and in more detail in my blog (notably this post). One point of special interest is the suggestion that a "source of safety labels will be individuals" -- I have suggested that crowdsourcing can be a powerful tool for creating a "cognitive immune system" that can be more powerful, scalable, and responsive in real time than conventional moderation.

The broader view of what this means for social media and society are the subject of the series I am doing with Chris Riley in Tech Policy Press. But this Bluesky document provide a nice explanation of some basic ideas, and demonstrates progress toward making such systems a reality. 

The hope is that Twitter applies such ideas -- and that others do.


Friday, April 29, 2022

"Delegation, or, The Twenty Nine Words that the Internet Forgot" -- A Series in Tech Policy Press

It is the policy of the United States…to encourage the development of technologies which maximize user control over what information is received by individuals…who use the Internet…” (from Section 230 of the Communications Decency Act)

***Background and running updates below*** 

This series is being published in Tech Policy Press -- co-authored with tech policy executive Chris Riley...

Part 1. (2/27/22)
Delegation, or, The Twenty Nine Words that the Internet Forgot

The series begins with an exploration of why this emphasis on user control is far more important than generally recognized, and how an architecture designed to make high levels of user control manageable can enhance the nuance, context, balance, and value in human discourse that current social media are tragically degrading.

While that portion of the much-discussed "Section 230" has been neglected, those ideas have re-emerged -- most prominently in the 2019 ACCESS Act introduced in the U.S. Senate, which included among its provisions a requirement to provide “delegatability” – enabled through APIs that allow a user to authorize a third party to manage the user’s content and settings directly on the user’s behalf.

This opening essay concludes: 

User choice is essential to a social and media ecosystem that preserves and augments democracy, self-actualization, and the common welfare – instead of undermining it. And delegation is the linchpin that can make that a reality.

Part 2. (4/27/22)
Understanding Social Media: An Increasingly Reflexive Extension of Humanity 

We shape our tools and thereafter our tools shape us. (Marshall McLuhan)

Social media do not behave like other media. Speech is not primarily broadcast, as through megaphones and amplification but rather, propagates more like word-of-mouth, from person to person. Feedback loops of reinforcing interactions by other users can snowball, or just fizzle out. Understanding how to modulate the harmful aspects of wild messaging cascades requires stepping back and, instead of viewing the messages as individual items of content, seeing them as stages in reflexive flows in which we and these new media tools shape each other. The reflexivity is the message. A media ecology perspective can help us understand where current social media have gone wrong and orchestrate the effort to manage increasing reflexivity in a holistic, coherent, inclusive, and effective way.


This page is to be updated as the series unfolds -- with my own personal perspectives and links to relevant materials. All views expressed here are my own (but owe much to wise insights from Chris). 

My other works related to this are listed in the Selected Items tab, above. Some that are most relevant to expand on the themes introduced in this first article:

This diagram from my The Internet Beyond... article may also be helpful:

Chris and I are very pleased with how this collaboration is synergizing our ideas, and how we draw on very complementary backgrounds: his in internet policy, governance, and law; mine in the technology and business of media as a tool for augmenting human discourse and intellect.

Running updates

[5/6/22:] Dorsey-funded Bluesky project published an architecture paper that helps clarify key ideas in the vision of decentralized, user-delegated control of social media filtering. Suggestive of possible directions by Twitter under Musk, and more broadly. I posted some excerpts from this (somewhat technical) document, with some light context and links.

[5/6/22:] Today I was reminded how much the media ecology of reflexivity augmented by human-machine loops has surprisingly early roots. I first dug into that around 1970, including Licklider's 1960 Man-Computer Symbiosis, which I now see again was very pointed about this symbiosis as going beyond the levels of "mechanically extended man" (a very McLuhanesque phrase that Licklider cited to 1954) and "artificial intelligence." Liclider inspired (and sponsored) Engelbart's "Augmenting Human Intellect," which inspired my views on making social media augment human society -- and also anticipates the related resurgence of thinking about more "human-centered AI," and AI Delegability. And of course Bush's 1945 As We May Think inspired all of this.

This reflexive intertwingling of ideas is also apropos of the question of our original attribution of our opening quote ("Man shapes his tools and thereafter our tools shape us") to McLuhan -- we removed any specific attribution because it may have been taken from others -- what matters to us is that McLuhan adopted it and gave it added attention.

[4/29/22:] Opening sections revised to add the second in the series.

[2/28/22:] Very pleased to see this:


My thanks to the many outstanding thinkers in this space who have been helpful in developing these ideas -- and especially to Justin Hendrix, co-founder and editor of Tech Policy Press for his support and skilled editing. ...And of course to Chris Riley for this very stimulating and enjoyable collaboration.

[This post was first published 2/27/22 when the series began, and has since been updated and expanded as additional essays are published.]

Friday, February 04, 2022

The Wrong Way to Preserve Journalism

Experts Spar At Hearing on Journalism, Tech and Market Power, as Justin Hendrix nicely reports today in Tech Policy Press.

Here is a brief commentary [still in progress] -- on why the proposed "Journalism Competition and Preservation Act of 2021" is harmful law in terms of its effects on journalism, competition, business models, and the essential nature of the Internet. 

This bill provides for a badly designed subsidy, in a way that is the very opposite of enhancing competition or access to information via the Internet, and removes motivation for news publishers to move beyond their failed business models.

There is a case for subsidizing the preservation of news (especially local news), and for limiting the monopoly rents that the platforms extract from advertisers. Until the market can do that on its own, the way to do that is with a tax on platform ad revenues that is used to fund a subsidy for journalism and to support efforts to find better business models so that journalism can sustain itself in the new digital world of abundance.

My work on the FairPay framework suggests how the latter might be accomplished, in ways that few yet understand, as outlined below. But in the meantime a tax + subsidy strategy seems the only viable option.

The problems with this approach

Hendrix links to a statement by multiple public interest organizations on why this is the wrong remedy"

Free Press led a letter signed also by Public Knowledge, Wikimedia and Common Cause, among others, that said the JCPA “may actually hurt local publishers by entrenching existing power relationships between the largest platforms and largest publishers. News giants with the greatest leverage would dominate the negotiations and small outlets with diverse or dissenting voices would be unheard if not hurt.”

He also cites the hearing testimony of  Dan Gainor and Daniel Francis, which make compelling arguments as to why the good intentions of the advocates are misguided.

Joshua Benton at NiemanLab provides an excellent analysis of why “Australia’s latest export is bad media policy, and it’s spreading fast” (see the "third" idea, part way down): 

The base problem here is that these governments are telling the tech giants that their use of their country’s publishers news content has a monetary value that is somehow different from all other content in existence. And that’s the important word here: use.

...You can have a million complaints about these companies — I do! — but at a fundamental level, the ways in which they “use” content are simply inherent to their natures as a search engine and a social platform.

...The core issue is misdirection. Publishers complain about Google and Facebook’s use of their stories — but that’s not what they’re actually angry about. What they’re angry about is that Google and Facebook dominate the digital advertising business — just as they used to dominate the print advertising business. And those are two really different things!

...It’s also why I get cross with media reporters who let sloppy language seep into their stories — like that this is all about setting “a price for news content published on the companies’ platforms.” None of this content is being published on Google and Facebook unless the publishers have specifically asked it to be. It’s being linked to, in the same way everything else in the world is being linked to. And unless you think the very concept of a search engine or a social platform is immoral, linking to things is just a fundamental part of how these things work.

...So tax them. Say you’re going to put a 1.5% tax on the targeted digital advertising revenue of all companies with a market cap over $1 trillion, or annual revenues over $20 billion, or whatever cutoff you want. That would generate billions of dollars a year in a way that doesn’t warp competition or let Google and Facebook use their cash as a tool for targeted PR payoffs.

Better approaches

As for the question of sustainable business models for journalism, my work on FairPay explains why current models based on artificial scarcity and flat-rate pricing fail in the world of digital abundance, where prices should map to highly diverse and variable customer value propositions -- and how more adaptively win-win models based on customer value in an ongoing relationship can change that.

A large body of work on that is cited on my FairPayZone blog, including work with academic co-authors in Harvard Business Review and two scholarly marketing journals. Some notable items are (start with the first):

Of course, getting to this kind of model will take time, experimentation, and learning, which the news publishers have been too distracted, stretched thin, or simply too unimaginative to do. So...

In the meantime, Congress should provide a stop-gap that sustains journalism and helps it move toward being self-sustaining in this new digital world:

  • Tax the ad revenues of the dominant platforms to limit their obscene monopoly profits on advertising and help drive them toward better business models of their own (see Reverse the Biz Model! -- Undo the Faustian Bargain for Ads and Data).
  • Temporarily, use much of that tax revenue to subsidize news -- directly to publishers of quality news, especially local, and to create new public interest publishers much like public radio and television. 
  • For long-term remedy, use a significant portion of that tax revenue to fund experiments in better business models for publishers, and for operational platforms that help them generate direct reader/patron revenue in consumer-value-efficient ways.
Importantly, publishers should not be rewarded for their lack of business model innovation. Subsidies should be narrowly channeled to preserving actual journalism work itself in the short term, as emergency relief, while also supporting business model innovation projects  -- including development of shared Revenue-as-a-Service platforms. (These qualifications on timing in the use of the tax revenue were spurred by a comment from Chris Riley, referring to his post from a year ago, The Great War, Part 3: The Internet vs Journalism.)

This alternative approach would address the symptoms of failed business models for journalism, with none of the damage that would be caused by embracing cartels of businesses that failed to adapt, eliminating "fair use," and destroying the fundamental structure of linking on the Internet that has created so much value for all of us -- even the publishers who downplay the significant promotional value it has created for them.

Thursday, December 23, 2021

Tech Policy Press Had A Great First Year -- Illuminating the Critical Issues

Democracy owes thanks to Tech Policy Press and its CEO/Editor Justin Hendrix for a great first year of important reporting, analysis, and opinion on the increasingly urgent issues of tech policy, especially social media. It is becoming the place to keep up with news and ideas. 

They just published their list of Top 50 Contributor Posts of 2021 from 330 posts from 120 guest contributors and their list of Top 10 Tech Policy Press Podcasts of 2021 from 54 episodes.

I am honored to be among the stellar contributors - and to have written two of the “Top 50” posts (plus four others) - and to have helped organize and moderate their special half-day event, Reconciling Social Media and Democracy.

Just a partial sampling of the many other contributors I have learned much from - Daphne Keller, Elinor Carmi, Nathalie Maréchal, Yael Eisenstat, Ellen Goodman, Karen Kornbluh, Renee DiResta, Chris Riley, Francis Fukuyama, Corey Doctorow, and Mike Masnick.

Great work by CEO/Editor Justin Hendrix.

Sign up for their newsletter!

Monday, December 20, 2021

Are You Covidscuous? [or Coviscuous?]

Are You Covidscuous? Have you been swapping air with those who are?

Covidscuous, adj. (co-vid-skyoo-us), Covidscuity, n. -- definition: demonstrating or implying an undiscriminating or unselective approach; indiscriminate or casual -- in regard to Covid contagion risks to oneself and those around one.

[Update 1/12/22:] Alternate form: Coviscuous, Coviscuity. Some may find this form easier to pronounce and more understandable.

We seem to lack a word for this badly needed concept. Many smart people who know Covid is real and have been vaccinated and boosted and wear masks often still seem to be oblivious to the cumulative and multiplicative nature of repeated exposures to risk. Many are aware that Omicron has added a new curveball, but give little thought to how often they expose themselves (and thus those they spend time with) by not limiting how much time they spend in large congregate indoor settings -- especially when rates and risks are increasing.

In July 2020, I wrote The Fog of Coronavirus: No Bright Lines, emphasizing that Covid spreads like a fog, depending on distance, airflow, and duration of exposure. That while a single interaction may have low risk, large numbers of low-risk interactions can amount to high risk. “You can play Russian roulette once or twice and likely survive. Ten or twenty times and you will almost certainly die.  We must weigh level of risk, duration, and frequency.” A gathering of six friends or relatives exposes six people to each other. A party with dozens of people chatting and mingling in ever-changing close circles of a few people has far higher risk – even if all are boosted.

We need to constantly apply the OODA loop to our exposures – Observe, Orient, Decide, Act, and repeat. When rates and exposure levels are low, we can be more relaxed. As rates or other risk factors increase, we need to be far more judicious about our exposures.

We should think in terms of a Covidscuity Rating. An index that factors in how many people you interact with (each having their own Covidscuity Rating), for what duration. More people, some with higher Covidscuity, and for more duration, closer, with less masking all multiply risk. Maybe epidemiologists can decide just how that math generally works and create a calculator app we can use to understand the relevant factors better (much like apps for home energy efficiency). Maybe display a Monte Carlo graph to show how this is never exact, but a fuzzy bell curve of probabilities. This could help us understand the risks we take -- and those we take on from those we choose to interact with.

But in any case, the OODA loops must be continuous. Not from months ago, but weekly, and whenever there is new information. Observe, Orient, Decide, Act, repeat.

And of course we have a social responsibility. This risk is not just to you, but those you might next infect. And to all of us, as you help provide a breeding ground for new and more dangerous variants.

This is not to say some Covidscuity is always wrong, only that we should maintain updated awareness of what risk we take, for what reward, and consider not just single events but budget your activities for the compounding effect of repeated exposure. Consider your own Covidscuity, and that of those you expose yourself to.

Sunday, December 19, 2021

Tech Policy Press: The Ghost of Surveillance Capitalism Future

My short article in Tech Policy Press focuses on The Ghost of Surveillance Capitalism Future, AKA, The Ghost of Social Media Future. 

Concerned about what Facebook and other platforms know about you and use to manipulate you now? The "mind-reading" power of "biometric psychography" will make that look like the good old days. 

Now is the time for policy planners to look to the future – not just to next year, but the next decade. Whatever direction we choose, the underlying question is “whom does the technology serve?” These global networks are far too universal, and their future potential far too powerful, to leave this to laissez-faire markets with business models that primarily exploit users.

Plus two additional references that add to the vision of abuses:

    Monday, November 29, 2021

    Directions Toward Re-Architecting Social Media to Serve Society

    My current article in Tech Policy Press, ProgressToward Re-Architecting Social Media to Serve Society, reports briefly on the latest in a series of dialogs on a family of radical proposals that is gaining interest. These discussions have been driven by the Stanford Working Group on Platform Scale and their proposal to unbundle the filtering of items into our social media news feeds, from the platforms, into independent filtering “middleware” services that are selected by users in an open market.

    As that article suggests, the latest dialogue at the StanfordHAI Conference on "Radical Proposals" questions whether variations on these proposals go too far, or not far enough. That suggests that policy planners would benefit from more clarity on increases in scope that might be phased over time and on just what the long-term vision for the proposal is. The most recent session offered some hints of directions toward more ambitious variations – which might be challenging to achieve but might generate broader support by more fully addressing key issues. But these were just hints.

    Reflecting on these discussions, this post pulls together some bolder visions along the same lines that I have been sketching out, to clarify what we might work toward and how this might address open concerns. Most notably, it expands on the suggestion in the recent session that data cooperatives are another kind of “middleware” between platforms and users that might complement the proposed news feed filtering middleware.

    The current state of discussion

    This is best understood after reading my current Tech Policy Press article, but here is the gist:

       The unbundling of control of social media filtering to users -- via an open market of filtering services -- is gaining recognition as a new and potentially important tool in our arsenal for managing social media without crippling the freedom of speech that democracy depends on. Instead of platform control, it brings a level of social mediation by users and services that work as their agents.

       Speaking as members of the Stanford Group, Francis Fukuyama and Ashish Goel explained more of their vision of such an unbundling, gave a brief demo, and how they have backed off to become a bit less radical -- to limit privacy concerns as well as platform and political resistance. However, others on the panel suggested that might not be ambitious enough.

       To the five open concerns about these proposals that I had previously summarized -- relating to speech, business models, privacy, competition and interoperability, and technological feasibility – this latest session highlighted a sixth issue -- relating to the social flow graph. That is the need for filtering to consider not just the content of social media but the dynamics of how that content flows among -- and draws reaction from -- chains of users, with sometimes-destructive amplification. How can we manage that harmful form of social mediation -- and can we achieve positive forms of social mediation?

       That, in turn, brings privacy back to the fore. Panelist Katrina Ligett suggested that another topic at the Stanford conference, Data Cooperatives, was also relevant to this need to consider the collective behavior of social media users. That is something I had written about after reflecting on the earlier discussion hosted by Tech Policy Press. The following section relates those ideas to this latest discussion.

    Infomediaries -- another level of middleware -- to address privacy and business model issues

    While adding another layer of intermediation and spinning more function out of the platforms may seem to complicate things, the deeper level of insight from the dynamics of the flow of discourse will enable more effective filtering -- and more effective management of speech across the board. It will not come easily or quickly -- but any stop-gap remediation should be done with care to not foreclose development toward mining this wellspring of collective human judgment.

    The connection of filtering service “middleware” to the other “middleware” of data collectives that Ligett and I have raised has relevance not only to privacy but also to the business and revenue model concerns that Fukuyama and Goel gave as reasons for scaling back their proposals. Data collectives are a variation on what were first proposed as “infomediaries” (information intermediaries) and later as “information fiduciaries.” I wrote in 2018 about how infomediary services could help resolve the businessmodel problems of social media, and recently about how they could help resolve the privacyconcerns. The core idea is that infomediaries act as user agents and fiduciaries to negotiate between users and platforms – and advertisers -- for user attention and data.

    My recent sketch of a proposal to use infomediaries to support filtering middleware, Resolving Speech, Biz Model, and Privacy Issues – An Infomediary Infrastructure for Social Media?, suggested not that the filtering services themselves be infomediaries, but be part of an architecture with two new levels:

    1. A small number of independent and competing infomediaries that could safeguard the personal data of users, coordinate limits on clearly harmful content, and help manage flow controls. They could use all of that data to run filtering on behalf of...
    2. A large diversity of filtering services – without exposing that personal data to the filtering services (which might have much more limited resources to process and safeguard the data)

    Such a two-level structure might enable powerful and diverse filtering services while providing a strong quasi-central, federated support service – insulated from both the platforms and the filtering services. That infomediary service could coordinate efforts to limit dangerous virality in ways that serve users and society, not advertisers. Those infomediaries could also negotiate as agents for the users for a share of any advertising revenue -- and take a portion of that to fund themselves, and the filtering services.

    With infomediaries, the business model concerns about sustaining filtering services, and insulating them from the perverse incentives of the advertising model to drive engagement, might become much less difficult than currently feared.

       Equitable revenue shares in any direction can be negotiated by the infomediaries, regardless of just how much data the filtering services or infomediaries control, who sells the ads, or how much of the user interface they handle. That is not a technical problem but one of negotiating power. The content and ad-tech industries already manage complex multi-party sales and revenue sharing for ads -- in Web, video, cable TV, and broadcast TV contexts -- which accommodate varying options for which party sells and places ads, and how the revenue is divided among the parties. (Complex revenue sharing arrangements through intermediaries have long been the practice in the music industry.)

       Filtering services and infomediaries could also shift incentives away from the perversity of the engagement model. Engagement is not the end objective of advertisers, but only a convenient surrogate for sales and brand-building. Revenue shares to filtering services and infomediaries could be driven by user-value-based metrics rather than engagement -- even as simple as MAUs (monthly average users). That would better align those services with the business objective of attracting and keeping users, rather than addicting them. Some users may choose to wear blinders, but few will agree to be manipulatively driven toward anger and hate if they have good alternatives. But now the platform's filters are the only game in the platform's town.

    Related strategies that build on this ecosystem to filter for quality

    There might be more agreement on the path toward social media that serve society if we shared a more fleshed-out vision of what constructively motivated social media might do, and how that would counter the abuses we currently face. Some aspects of the power that better filtering services might bring to human discourse are suggested in the following:

    Skeptics are right that user-selected filtering services might sometimes foster filter bubbles. But they fail to consider the power that multiple services that seek to filter for user value might achieve, working in “coopetition.” Motivated to use methods like these, a diversity of filtering services can collaborate to mine the wisdom of the crowd that is hidden in the dynamics of the social flow graph of how users interact with one another – and can share and build on these insights into reputation and authority. User-selected filtering services may not always drive toward quality for all users, but collectively, a powerful vector of emergent consensus can bend toward quality. The genius of democracy is its reliance on free speech to converge on truth – when mediated toward consensus by an open ecosystem of supportive institutions. Well-managed and well-regulated technology can augment that mediation, instead of disrupting it.

    Phases – building toward a social media architecture that serves society

       The Stanford Group’s concerns about “political realism” and platform pushback has led them to a basic level of independent, user-selectable labeling services. That is a limited remedy, but may be valuable in itself, and as a first step toward bolder action.

       Their intent is to extend from labeling to ranking and scoring, initially with little or no personal data. (It is unclear how useful that can be without user interaction flow data, but also a step worth testing.)

       Others have proposed similar basic steps toward more user control of filtering. In addition to proposals I cited this spring, the proposed Filter Bubble Transparency Act would require that users be offered an unfiltered reverse-chronological feed. That might also enable independent services to filter that raw feed. Jack Balkin and Chris Riley have separately suggested that Section 230 be a lever for reform by restricting safe harbors to services that act as fiduciaries and/or that provide an unfiltered feed that independent services can filter. (But again, it is unclear how useful that filtering can be without access to user interaction flow data.)

       Riley has also suggested differential treatment of commercial and non-commercial speech. That could enable filtering that is better-tailored to each type.

       The greatest benefit would come with more advanced stages of filtering services that would apply more personal data about the context and flow of content through the network, as users interact with it, to gain far more power to apply human wisdom to filtering (as I have been suggesting). That could feed back to modulate forward flows, creating a powerful tool for selectively damping (or amplifying) the viral cascades that are now so often harmful.

        Infomediaries (data cooperatives) could be introduced to better support that more advanced kind of filtering, as well as to help manage other aspects of the value exchange with users relating to privacy and attention that are now abused by “surveillance capitalism.”

    Without this kind of long-term vision, we risk two harmful errors. One is overreliance on oppressive forms of mediation that stifle the free inquiry that our society depends on, and that the First Amendment was designed to protect. The other is overly restrictive privacy legislation that privatizes community data that should be used to serve the common good. Of course there is a risk that we may stumble at times on this challenging path, but that is how new ecosystems develop.


    Running updates on these important issues can be found here, and my updating list of Selected Items is on the tab above.

    Wednesday, November 03, 2021

    Resolving Speech, Biz Model, and Privacy Issues – An Infomediary Infrastructure for Social Media?

    A Quick Sketch for Discussion: Formative thoughts on addressing open concerns, posted in anticipation of a 11/9 conference session at Stanford on the “middleware” unbundling proposals. (This also suggests linkage to an 11/10 session on “data cooperatives” at the same event). [Update: As noted at the end, there was some discussion at the 11/9 conference session that was generally supportive of the directions suggested here.]


    Recent proposals to unbundle filtering services from social media platforms to better serve user interests have generated support, tempered by concern -- notably about business models and privacy protection. Instead of the one-level functional unbundling that has been proposed, these concerns may be better handled by a two-level unbundling. 

    Between the platforms and the large numbers of unbundled filtering services that need resources and access to sensitive personal data to filter effectively on their users’ behalf, add a layer with a small number of better-resourced “infomediaries” that are fiduciaries for users. The infomediaries can manage coordination of services, data protection, and revenue sharing in service to user interests, and enable the many independent filtering services to share resources and run their filters in privacy-protected ways.

    The time may be ripe for the long-gestating idea of “infomediaries” to emerge as a linchpin for resolving some of the management and control dilemmas we now face with social media. The session with Francis Fukuyama and others that I moderated at the Tech Policy Press event on 10/7, Reconciling Social Media & Democracy: Fukuyama, Keller, Maréchal & Reisman (along with other speakers that followed) generated a wide-ranging discussion of issues with those proposals that provide context for the upcoming session on these proposals he will participate in at Stanford. 

    Knotty problems with the “middleware” proposal

    The unbundling proposals that Fukuyama, I, and others advocate have been viewed to have considerable appeal in principle, but the 10/7 discussion sharpened many previously raised questions about whether they can work -- relating to speech, business models, privacy, compatibility and interoperability, and technological feasibility.

    Reflecting on the privacy issues led me to refocus on “infomediaries” as an important part of a solution, and how they might clarify the business model issues, as well. Infomediaries were first proposed in the dot-com era, as agents of consumers that could negotiate with businesses over data and attention, to give consumers control and compensation for their information. The imbalance of power over consumers has grown in the world of e-commerce, but social media have given this even more importance and urgency.

    The unbundling proposal is to spin out the filtering of what users see in their newsfeeds from the platforms -- to create independent filtering “middleware” services that users select in an open market to serve as their agents. There is wide agreement that their ad-engagement-driven business model drives social media to promote harmful speech in powerful and dangerous ways. Fukuyama raised an even deeper concern that the concentration of power to control what we each see is a mortal threat to democracy, “a loaded gun sitting on the table” that we cannot rely on good actors to not pick up.

    Unbundling of the filtering services would take that loaded gun from the platforms (and those who might coerce them) and reduce its power -- by giving individual users more independent control of what they see in their social media newsfeeds and recommendations. But -- how can those unbundled services be funded, since users seem disinclined to pay for them? -- and how can the filtering services use the personal data needed to do filtering effectively without breaches of privacy?

    This problem is compounded because we would want a wide diversity of filtering services innovating and competing for users. Many would be small, and under-resourced -- and there is no simple, automated solution to understanding the content they filter and its authority.

    • How would they have the resources -- to not only do the basic filtering task of ranking, but also to moderate the overwhelming firehose of harmful content that already taxes the ability of giants like Facebook?
    • How would a multitude of small filtering services be able to protect non-public multi-party content, as well as the multi-party personal metadata, needed to understand the provenance and authority of what they filter?
    These are challenging tasks, and there is reluctance to proceed without a clear idea of how we might operationalize a solution.

    The role of infomediaries

    I suggest the answer to this dilemma could be a more sophisticated distribution of functions. Not just two levels:  of platform and filtering services (as user agents); but three levels: of platform, of infomediaries (as a few, privileged user agents), and of filtering services (as many, more limited user agents).

    "Infomediaries" (Information intermediaries) were suggested in 1997 in Harvard Business Review --as a trusted user agent that manages a consumer’s data and attention -- and negotiates with businesses on how it is used and for what compensation. Similar ideas re-surfaced in a law review article in 2016 as "Information Fiduciaries" and then in HBR in 2018 as "Mediators of Individual Data" ("MIDs").

    (As I was writing this, I learned that another session at the Stanford event is on more a recent variant, “Data Cooperatives.” Despite that coincidence, I am unaware a connection has been seen, except for the observation in this recent work that social media data is not individual but “collective.” If the participants at those two sessions are not in communication, I suggest that might be productive.)

    Why have infomediaries not materialized in any significant way? It seems network effects and the "original sin of the Internet," advertising, have proven so hugely powerful that infomediaries never got critical mass in commerce beyond narrow uses. (I was CTO from ‘98-‘00 for a basic kind of infomediary service that had some success before the crash.)

    But now, with the harms of social media bringing the broader abuses of “attention capitalism” to a head, regulators may see that the only way to systematically limit these harms – and the harms of attention capitalism more broadly -- is to mandate the creation of infomediaries to serve as negotiating and custodial agents for consumers. They offer a way to enable business models that balance business power with consumer power, especially regarding compensation for attention and data -- in ways that empower users to decide what to allow, for what benefit. They also offer a new solution to protecting sensitive multi-party social media messages and related metadata -- while enabling society to refine and benefit from the wisdom of the crowd that it contains -- to help us manage our attention.

    Here is a sketch of how filtering services might be supported by infomediaries. Working out the details will be a complex task that should be guided by a dedicated Digital Regulatory Agency with significant business and independent expert participation.

    • Put all personal data of social media users under the control of carefully regulated infomediaries (IMs) who interface with the platforms and the filtering services (FSs), as fiduciary agents for their users. Create a small number of infomediaries (five to seven?) to support defined subsets of users. After that, users would be free to migrate among infomediaries in a free market -- and very limited numbers of new infomediary entrants might be enabled.
    • Spin out the filtering services from the platforms – and create processes to encourage new entrants. The infomediaries would cooperate to enable the filtering services to benefit from the data of all qualified infomediaries, while protecting personally identifiable data.
    • Empower the infomediaries to negotiate a share of advertising revenue from the platforms on behalf of their users, in compensation for their data and attention – to be shared with the filtering services (and perhaps the users). Provide alternatively for a mix of user support or public subsidy much like existing public media. Ideally that could grow to include user support for the platforms as an alternative to some or all advertising.
    • Use regulatory power to work with industry to manage interface standards and the ongoing conduct of these roles and negotiations, much as other essential, complex, and dynamic industries like finance, telecom, transport, power, and other utilities are regulated. Creation of new infomediaries might be strictly limited by regulators, much like banks or securities exchanges.

    The virtue of this two-level unbundling architecture is that it concentrates elements of the infomediary role that have network-wide impact and sensitive data in a small number of large competitive entities -- they could apply the necessary resources and technology to maintain privacy and provide complex services, with some competitive diversity. It enables much larger numbers of filtering services that serve diverse user needs to be lean and unburdened.

    Because the new infomediaries would be accredited custodians of sensitive messaging data, as fiduciaries for the users, they could share that data among themselves, providing a collective resource to safely power the filtering services.

    This could be done in two ways: 1) by providing purpose and time-limited, privacy protected data to the filtering services, or perhaps simpler and more secure, 2) by acting as a platform that runs filtering algorithms defined by the filtering services and returning rankings without divulging the data itself. (More on how that can be done, and why, is below). Either way, the platforms would no longer control or be gatekeepers for the filtering.

    This multilevel breakup may sound very complex and raise questions of regulatory power, but it would be very analogous to the breakup of the Bell System, which unbundled the integrated AT&T into a long-distance service (AT&T), seven regional local-service operating companies (RBOCs), and a manufacturing service (Lucent), all of which were opened to new competitive entrants, unleashing a torrent of valuable innovation.

    As our social media ecosystem becomes the underlying fabric of most human discourse, a similarly ambitious undertaking is not only economically desirable and justifiable, but essential to the survival of democracy and free speech. Functional specialization multiplies the number of entities, but it simplifies the tasks of those entities – and enables competition, innovation, and resilience. To the fear of technical solutions to social problems that Nathalie Marechal spoke of, I submit that the problem of algorithms that select for virality (thus exacerbating a social problem) is a newly-created technical problem, driven by an incentives problem – one that this architecture (or some improvement on it) can help solve.

    A casual reader might stop here. The following sections dig deeper on how this addresses first, the business model challenges that infomediaries were conceived to solve, and then, the difficult privacy issues of middleware unbundling and other problems that they seem might help finesse.


    Looking deeper...

    Resolving the business model issues

    Even as one who had read it then, it is now enlightening to turn the clock back to 1997 to read the original HBR article on infomediaries by John Hagel III and Jeffrey F. Rayport, The Coming Battle for Customer Information,” for perspective on the current problems of surveillance and attention capitalism. The authors predicted:

    In order to help [consumers] strike the best bargain with vendors, new intermediaries will emerge. They will aggregate consumers and negotiate on their behalf within the economic definition of privacy determined by their clients. … When ownership of information shifts to the consumer, a new form of supply is created. By connecting information supply with information demand and by helping both parties involved determine the value of that information, infomediaries would be building a new kind of information supply chain.”

    A 1999 book co-authored by Hagel greatly expands on this idea (and is also worth a look). It specifically refers to “filtering services,” to include or exclude marketing messages to match the need or preferences of its clients.

    Growing due to network effects and scale economies, vendors like Amazon and ad-tech services like Google and Facebook have effectively usurped the vendor side of the infomediary function. These powers are now so entrenched and engorged with obscene profits that there is little hope that infomediaries that do represent user interests can emerge without regulatory action.

    The proposal that unbundled filtering services be funded by as revenue share from the platforms has struck critics as implausible and complex. But if that role is not dispersed, among large numbers of often-small filtering services, but managed by a small number of larger infomediaries who have a mandate from regulators, the task may be far more tractable.

    Yes, this would be a complex ecosystem, with multiple levels of cooperating businesses for which economically sound revenue shares would need to be negotiated. Ad revenues from platforms to infomediaries, to filtering services, and possibly to consumers. Or alternatively, from consumers or sponsors or public funding -- in whichever direction makes corresponds to the value transfer. But many industries – such as financial services. ad-tech, telecom, logistics -- flourish with equally complex revenue shares (whether called shares, fees, commissions, settlements or whatever), often overseen by regulators that ensure fairness.

    Once such a multiplayer market begins to operate, innovation can enable better revenue models. My 2018 article “Reverse the Biz Model” explored some possible variations, and explained how they could work via infomediaries, or directly between business and consumer. It also suggested how consumer funding to eliminate ads on an individual basis could be commensurate with ability to pay. The inherent economics are more egalitarian than one might first think because those with low income have low value to advertisers. They would have to contribute less to compensate for lost ad revenue. Mediated well, users could even benefit from whatever level of non-intrusive and relevant advertising they desire, and platforms would still bring in sufficient funding to disperse through the ecosystem -- perhaps more than now, given that there would be less waste. (Note that filtering services might specialize in advertising/marketing messages or in user-generated content to better address the different issues for each.)

    Some fear that having filtering services receive funding from advertising, even indirectly, would continue the perverse incentives for engagement that are so harmful. But revenue shares to the infomediaries and filtering services need not be tied to engagement – they could be tied to monthly average users or other user-value-based metrics. With a multitude of filtering services, the value of engagement to the platform would be decoupled, so that no individual filtering service would materially affect engagement. These services might be structured as nonprofits, benefit corporations, or cooperatives, to further shift incentives toward user and social value.

    Resolving the privacy issues

    The other key opportunity for infomediaries is to manage data privacy. This takes on special significance because key aspects of filtering and recommendations depend on either message content or the metadata about how users interact with those messages -- both of which are often privacy-sensitive. Importantly, as noted by the recent proposals for data cooperatives, that data is not individual, but collective.

    Infomediaries may offer a way to finesse the concerns pinpointed in the 10/7 discussion. I suggested that the most promising strategy for filtering to understand quality -- given the limitations of AI and of human review of billions of content items in hundreds of languages and contexts -- is to use the metadata that signals how other users responded to that content. Daphne Keller nicely delineated the privacy concern:

    … I think a lot of content moderation does depend on metadata. For example, spam detection and demotion is very much driven by metadata. And Twitter has said that a lot of how they detect terrorist content, isn’t really by the content, it’s by the patterns of connections between accounts following each other or coming from the same IP address or appearing the same– those aren’t the examples they gave, but what I assume they’re using. And I think it’s a big part of what Camille Francois has called the ABC framework, the Actors-Behavior-Content, as these three frameworks for approaching responding to problematic online content.

    And I think it just makes everything much harder because if we pretend that metadata isn’t useful to content moderation, that kind of simplifies things. If we acknowledge that metadata is useful, that is often personally identifiable data about users, including users who haven’t signed up for this new middleware provider, and it’s a different kind of personally identifiable data than just the fact that they posted particular content at a particular time. And all of the concerns that I raised, but in particular, the privacy concern and just like how do we even do this? What is the technology that takes metadata structured around the backend engineering of Twitter or whomever and share it with a competitor? That gets really hard. So I’m scared to hear you bring up metadata because that adds another layer of questions I’m not sure how to solve.

    This is what drove me to refocus on infomediaries as the way to cut through the dilemma. The platforms could have filtered using as much of this data as the wished, since they now control that data. Similar data is central to Google search (the PageRank algorithm that was the key to their success) -- but search is less driven by engagement than social media.

    Privacy has been a sore point for the unbundling of filtering. The kind of issues that Keller raised led Fukuyama and his colleagues to back off from the broadest unbundling to advocate more limited ambitions, such as labelling, that are content-based and rather than metadata-based. He points to services like NewsGuard that rate news sources for their credibility. As I have argued elsewhere, that is a useful service, but severely limited because it only applies to limited numbers of established news services (which do represent large amounts of content), not the billions of user-generated content sources (obviously significant in aggregate, but intractable for expert ratings). Instead, I suggest using metadata to draw out the wisdom of crowds, much as Google does. Recent studies support the idea that crowdsourced assessment of quality can be as good as expert ratings, and there is no question that automated crowdsourced methods that draw on passively obtained metadata are far more capable of operating at Internet scale and speed – the only solution that can really scale as needed.

    Thus, it would be a huge loss to society to not be able to filter social media based on interaction metadata --- an infomediary strategy for making that feasible is well worth some added complexity. A manageable number of infomediaries could manage this data to include most (but not necessarily all) users in this crowdsourcing. Each infomediary would only have a subset of the users’ data, but that data could be pooled among properly regulated infomediaries and restricted to use only in filtering.

    More technical/operational detail on filtering and data protection

    As noted above, and drawing on work on trust and data sharing by Sandy Pentland (one of the speakers in the Stanford Data Cooperatives session), and similar suggestions by Stephen Wolfram (in his 2019 testimony to a US Senate subcommittee), there seem to be two basic alternatives: 1) providing limited, privacy protected data to the filtering services, or perhaps simpler and more securely, 2) acting as a platform for running filtering algorithms defined by the filtering services and returning rankings, without divulging the data itself.

    Perhaps emerging technologies for secure data sharing (such as those described by Pentland) might allow the fiduciaries to grant the filtering services controlled and limited access to this data. But that is not necessary to this architecture -- as noted above, the simpler solution appears to be that of having the infomediaries act as a platform for running filtering algorithms defined by the filtering services without divulging the data itself. Send the algorithm to the data.

    Adapting the approach and terminology suggested by Wolfram, the infomediary retains operational control of the filtering operation, and all the data used for that -- working essentially as a “final ranking provider” – as a fully trusted level of user agent. But the setting of specific criteria for that ranking is delegated to one or more user-chosen filtering services to operate essentially as a “constraint provider” that instructs that rankings to be done in accord with the preferences they set on behalf of their users. (In contrast, the platforms now serve as both constraint providers and final ranking providers -- and users have very little say in how that is done.)

    Note that, ideally, these rankings should be done in a composable format, such that rankings from multiple filters can be arithmetically combined into a composite ranking. This might be done with relative weightings that users can select for each filtering service, such as with sliders, to compose an overall ranking drawn from on all the services they choose. Users might be enabled to change their filter selections and weightings at any time to suit varying objectives and moods. Thus, users control the filters by choosing the filtering services (and setting any variations they enable), but the actual process of filtering and the data needed for that remains within the cooperating array of secure infomediaries.

    Back to Keller’s concerns: The boundaries between the platforms and the infomediaries are clear and with well-defined interfaces, much as in any complex, evolving ecosystem. There is nothing shared with competitors, only with partners. It is co-opetition, among trusted peers, on how shared data is used and protected, at what price. The personal data never goes beyond a team of infomediaries, all trusted with purpose-specific portions of one-another’s clients’ data. There is no more implementation complexity than in Google’s ad business. It won’t happen in a day, but it is eminently do-able – if we really want.

    Improved functionality for filtering, blocking, and flow control

    Consider how this two-level architecture can enable rich functionality with diverse characteristics needed to address the multi-faceted challenges of filtering, blocking, and flow control as we face new technical/social issues like "ampliganda." Growing evidence favors not just filtering (ranking and recommenders) or blocking (takedowns and bans), but flow controls. These include circuit-breakers and other forms of friction that can slow the effects of virality (such as nudging users to take time and read an item before sharing it). The infomediaries could pool their real-time network flow data to serve as the empowered coordinating locus for such measures -- with diversity, and with independence from the platforms.

    The infomediaries might also be the independent coordinating locus for takedowns of truly illegal content in ways that protect user rights of privacy, explanation, and appeal, much as common carriers handle such roles in traditional telecom services. Criteria here might be relatively top-down (because takedowns are draconian binaries), in contrast to the bottom-up rankings of the filtering services (which are fuzzy, not preventing anyone from seeing content, merely making it less likely to be fed into one’s attention). The infomediaries could better shielded these functions from corporate or political interference than leaving it with the platforms. They would serve as an institutional later insulated from platform control. The infomediaries could outsource takedown decision inputs to specialized services (much like email spam blocking services) that could compete based on expertise in various domains. Here again, the co-opetition among trusted peers (and their agents) keeps private data secure.

    Note that this can evolve to a more general infrastructure that works across multiple social media platforms and user subsets. It can also support higher levels of user communities and special interest groups on this same infrastructure, so that the notion of independent platforms can blur into independent groups, communities, using a full suite of interaction modalities, all on a common backbone network infrastructure.

    Whatever the operational details, the primary responsibility for control of personal data would remain with the infomediaries, as data custodians for the data relating to the users they serve. To the extent that the platforms and/or filtering services (and other cooperating infomediaries) have access to that data at all, it could be limited to their specifically authorized transient needs and removed from their reach as soon as that need is satisfied -- subject to legal audits and enforcement. That enables powerful filtering based on rich data across platforms and user populations.

    This is not unlike how trust has long been enforced in financial services ecosystems. Is our information ecosystem less critical to our welfare than our financial ecosystem? Is our ability to exchange our ideas less critical than our financial exchanges?

    [Update 11/8/21:] Feedback from Sandy Pentland (a panelist for the upcoming Data Cooperatives session) led me to the introduction to his new book, which provides an excellent perspective on how this kind of infomediary can evolve, and be distributed in a largely bottom-up way. My description above highlights the institutional role of infomediaries and how they can balance top-down order to serve users -- but Sandy's book suggests how, as these new data technologies mature, they might provide a much more fully distributed blend of bottom-up control and cooperation that can still balance privacy and autonomy with constructive social mediation processes.

    [Update 11/10/21:] There was discussion of data cooperatives as relevant to filtering middleware in the 11/9 HAI middleware session. Panelist Katrina Ligett emphasized the need consider not only content items, but the data about the social flow graph of how content moves through the network and draws telling reactions from users. She referred to data cooperatives as another kind of middleware, and Ashish Goel also saw promise in this other kind of middleware. I will be writing more on that.
    Directions Toward Re-Architecting Social Media to Serve Society


    For additional background, see the Selected Items tab.