Google Still Charging Advertisers for Conversion-Inflation Traffic from WhenU Spyware updated January 7, 2010

When an advertiser buys a pay-per-click ad and subsequently makes a sale, it’s natural to assume that sale resulted primarily from the PPC vendor’s efforts on the advertiser’s behalf. But tricky PPC platforms take advantage of this assumption by referring purchases that would have happened anyway. Then, when advertisers evaluate the PPC traffic they bought, they overvalue this “conversion inflation” traffic — leading advertisers to overbid and overpay.

In this piece, I show Google and its partners still covering popular sites with PPC advertisements promoting those same sites. I present the role of InfoSpace, the Google partner at the core of these misplacements, and I argue that Google should long ago have severed its ties to InfoSpace. I cite specific Google promises that these placements violate, and I critique Google’s contractual disclaimers that claim advertisers must pay for these bogus placements. Finally, I propose specific actions Google should take to satisfy to its obligations to advertisers.

Google and Its Partners Still Covering Advertisers’ Sites with Spyware-Delivered Popups

WhenU covers Continental with its own Google ads -- charging ad fees for traffic Continental would otherwise receive for free
WhenU covers Continental with its own Google ads — chargingad fees for traffic Continental would otherwise receive for free

As shown in the thumbnail at right and detailed in screenshots, video, and packet log, WhenU continues to cover web sites with PPC popups. Crucially, those popups show Google ads — often promoting the very same sites users are already browsing.

In the example shown at right, I browsed the Continental Airlines site. WhenU opened the popup shown at right — covering the Continental site with a list of Google ads, putting a prominent Continental ad front-and-center. Thus, Google charges Continental a fee to access a user already at Continental’s site. That’s a rotten deal for Continental: For one, an advertiser should not have to pay to reach a user already at its site. Furthermore, advertisers paying high Google prices deserve high-quality ad placements, not spyware popups.

The details of the Continental ad, as shown in the WhenU-Google popup, further entice users to click. The ad promises a “low fare guarantee” — suggesting that users who book some other way (without clicking the ad) may not enjoy that guarantee. And the ad promises to take users to the “official site” — suggesting that users who don’t click the ad will book through a site that is less than official. In fact both suggestions are inaccurate, but a reasonable user would naturally reach these conclusions based on the wording of the advertisement and the context of its appearance.

The WhenU-Google popup lacks the labeling specifically required by FTC policy. In particular, all sponsored search ads are to be labeled as such, pursuant to the FTC ‘s 2002 instructions. But look closely at the popup screenshot. On my ordinary 800×600 screen, no such label appears. I gather the required label would ordinarily appear on a sufficiently large screen, but the FTC’s policies make no exceptions for users with small to midsized screens. Indeed, as netbooks gain popularity, small screens are increasingly common.

The diagram below (left) confirms the specific intermediaries passing traffic from WhenU to Google in this instance.

The money trail: how funds flow from advertisers to Google to WhenU
(three examples persisting over ten months)
December 2009

PPC advertisers
(e.g. Continental)
money viewers
   Google   
money viewers
InfoSpace
money viewers
LocalPages
money viewers
(unknown company*)
money viewers
WhenU

PPC advertisers
(e.g. RCN)
money viewers
   Google   
money viewers
InfoSpace
money viewers

*  LocalPages
money viewers
Nbcsearch
money viewers
LocalPages

money viewers
WhenU

PPC advertisers
(e.g. Verizon)
money viewers
   Google   
money viewers
InfoSpace
money viewers
LocalPages
money viewers
WhenU

This observation marks the third sequence by which I have observed Google paying WhenU to cover advertisers’ sites with the advertisers’ own Google ads. The center and right diagrams (above) show the intermediaries in my May 2009 and February 2009 observations of similar placements.

The Impropriety of Google’s Relationship with InfoSpace

In all three instances I reported (as summarized in the diagram above), Google’s closest link is to InfoSpace. That is, Google pays InfoSpace, and InfoSpace pays the various entities that follow. In my view, Google’s relationship with InfoSpace is ill-advised for at least three reasons:

First, InfoSpace has a track record of improper placements of Google ads. InfoSpace is implicated in all three of the placements detailed above — misplacements that have continued over a lengthy period despite ample notice and opportunity for correction. Furthermore, I have personally observed other improper placements by InfoSpace. (Perhaps I’ll post more in a futher piece.) Google need not continue to do business with a distributor with such a poor track record.

Second, Google does not need a distributor whose business model entails farming out ad placements to subdistributors. If InfoSpace’s subdistributors seek to distribute Google ads, and to be paid for doing so, let them apply directly to Google and undergo Google’s ordinary quality control and oversight. Inserting InfoSpace as an additional intermediary serves only to lessen accountability.

Third, InfoSpace’s corporate history undermines any request for lenience or forgiveness. The Seattle Times chronicles InfoSpace’s accounting fraud in a three-part investigative report, “Dot-Con Job“, presenting 12,000+ words of analysis as well as primary source documents and even voicemail recordings. The Seattle Times byline summarizes their findings: “Investors were cashing out millions, and faithful investors were left with pennies.” Hardly a mark of trustworthiness!

These Ads Violate Google’s Promises to Users

These ad placements fall short of Google’s promises to users. By paying spyware vendors to show advertisements, Google both enlarges and prolongs the spyware problem. In particular, Google’s funding supports software that users struggle to remove from their computers. Google’s payments make it more profitable for vendors to sneak such software onto users’ computers in the first place.

Furthermore, Google’s Software Principles specifically disallow WhenU’s practices. Google’s “installation” and “upfront disclosure” principles disallow deceptive and nonconsensual WhenU installations. (I have video proof on file showing nonconsensual WhenU installations.) Google’s prohibition on “snooping” prohibits certain WhenU privacy practices, including WhenU’s historic violation of its own privacy policy (transmitting full page URLs despite a privacy policy promising “As the user surfs the Internet, URLS visited by the user … are NOT transmitted to WhenU.com or any third party server”).

Crucially, Google’s partnership with WhenU directly contradicts Google’s call for software makers and advertising intermediaries to “keep[] good company” by supervising partners. Despite that commitment, present on Google’s site for 4+ years, Google inexplicably continues its relationship with WhenU.

These Ads Violate Google’s Promises to Advertisers

These ad placements also fall short of Google’s obligations to advertisers. For example, when Google describes its Search Network, Google promises:

Ads are targeted based on a user’s search terms.   (emphasis added)

But here, the user performed no search — so there was no proper cause to display a Search Network ad or charge an advertiser a high Search Network price.

Google confirms:

On the Search Network, ads are shown … on … the search results pages of … Google’s search partners … within the Search Network. On our search partners, your ads may appear alongside or above search results, as part of a results page as a user navigates through a site’s directory, or on other relevant search pages.   (emphasis added)

A placement through a spyware popup does not meet these criteria: A spyware popup is not a “page.” Furthermore, a user browsing an ordinary web site (like the Continental site shown above) is neither “search[ing]” nor navigating a “directory,” contrary to Google’s promise that search ads are shown to users at search engines and directories.

Despite these clear promises, Google’s AdWords Terms and Conditions purport to allow these placements and any others Google might choose to foist on unsuspecting advertisers. Google requires advertisers to accept the following form contract provisions:

Customer understands and agrees that ads may be placed on … (z) any other content or property provided by a third party (‘Partner’) upon which Google places ads (‘Partner Property’).   (emphasis added)

That’s circular, uninformative, and a rotten deal. Advertisers should demand better. Nor should Google’s fine print claim the right to impose such bogus charges. Google should amend its contract to disavow charges from spyware, adware, conversion-inflation, and other schemes contrary to Google’s affirmative promises.

What Google Should Do

Google’s first step is easy: Fire InfoSpace. Google doesn’t need InfoSpace, and there’s zero reason for this relationship to continue in light of InfoSpace’s repeated failings.

Google also needs to pay restitution to affected advertisers. Every time Google charges an advertiser for a click that comes from InfoSpace, Google relies on InfoSpace’s promise that the click was legitimate, genuine, and lawfully obtained. But there is ample reason to doubt these promises. Google should refund advertisers for corresponding charges — for all InfoSpace traffic if Google cannot reliably determine which InfoSpace traffic is legitimate. These refunds should apply immediately and across-the-board — not just to advertisers who know how to complain or who manage to assemble exceptional documentation of the infraction. (Indeed, in response to my May 2009 report, I know Google provided a credit to RCN — the specific advertiser whose targeting I happened to feature in my example. But I gather Google failed to provide automatic credits to all affected advertisers, even though Google’s billing records provide ample documentation of which advertisers faced charges from which Google partners. And I understand that Google denied requests for refunds or credits from other affected advertisers.)

More generally, Google must live up to the responsibility of spending other people’s money. Through its Search Network offering, Google takes control of advertisers’ budgets and decides, unilaterally, where to place advertisers’ ads. (Indeed, for Search Network purchases, Google to this day fails to tell advertisers what sites show their ads. Nor does Google allow opt-outs on a site-by-site basis — policies that also ought to change.) Spending others’ money, wisely and responsibly, is a weighty undertaking. Google should approach this task with significantly greater diligence and care than current partnerships indicate. Amending its AdWords T&C’s is a necessary step in this process: Not only should Google do better, but contracts should confirm Google’s obligation to offer refunds when Google falls short.

I’m disappointed by how little has changed since my year-ago reports of these same practices. In a conference presentation in February 2009, I demonstrated substantially similar WhenU placements, with Google’s Rose Hagan (Senior Trademark Counsel) present in the audience. In May 2009 I wrote up these WhenU placements on my web site in great detail. Yet ten months later, the problem continues unabated. Indeed, the other misplacements I identified in May 2009 also continue: Google continues partnering with IAC SmileyCentral (deceptive browser plug-ins that induce searches when users attempt navigations), placing ads on typosquatting sites (including sites that show a company’s own ads when users mistype that company’s domain name), and, through Google Chrome, inviting users to search (and click prominent top-of-page ads) when direct navigation would better satisfy users’ requests and avoid unnecessary advertising costs for advertisers. I’m disappointed by the lack of progress when, in each instance, the improper charges are clear and well-documented. Google’s intransigence confirms the need for the Bill of Rights for Online Advertisers I proposed this fall.

Measuring the Perpetrators and Funders of Typosquatting

Moore, Tyler, and Benjamin Edelman. “Measuring the Perpetrators and Funders of Typosquatting.” Lecture Notes in Computer Science. Springer-Verlag. Financial Cryptography and Data Security: Proceedings of the International Conference 6052 (2010). (Introduction, Web appendix.)

We describe a method for identifying “typosquatting”, the intentional registration of misspellings of popular website addresses. We estimate that at least 938,000 typosquatting domains target the top 3,264 .com sites, and we crawl more than 285,000 of these domains to analyze their revenue sources. We find that 80% are supported by pay-per-click ads, often advertising the correctly spelled domain and its competitors. Another 20% include static redirection to other sites. We present an automated technique that uncovered 75 otherwise legitimate websites which benefited from direct links from thousands of misspellings of competing websites. Using regression analysis, we find that websites in categories with higher pay-per-click ad prices face more typosquatting registrations, indicating that ad platforms such as Google AdWords exacerbate typosquatting. However, our investigations also confirm the feasibility of significantly reducing typosquatting. We find that typosquatting is highly concentrated: of typo domains showing Google ads, 63% use one of five advertising IDs, and some large name servers host typosquatting domains as much as four times as often as the web as a whole.

How Google and Its Partners Inflate Measured Conversion Rates and Inflate Advertisers’ Costs

When advertisers measure the effectiveness of their pay-per-click ad campaigns, advertisers systematically assume additionality, i.e. that the sales that follow a paid click are sales that would not have happened without the ad platform’s assistance. This assumption offers intuitive appeal: If a user clicked an ad and then bought the advertised product, by all indications the ad platform should be thanked for finding and sending an interested customer. Or should it?

As it turns out, Google and its partners systematically inflate advertisers’ conversion rates by interceding in transactions advertisers would otherwise have received for free. This conversion-inflation syndication fraud overstates the true effectiveness of the ads Google delivers — leading advertisers to pay more than they should.

In this piece, I offer four examples of Google and its partners inflating conversions to claim credit for traffic advertisers would otherwise have received for free. In each example, an advertiser intensely measuring its conversion rate would mismeasure the true effectiveness of its ads, and would end up overpaying for traffic that is far less valuable than reporting systems suggest.

Traffic source How users are found What would have happened had Google and its partners not interceded
WhenU – adware User requests advertiser’s site. Adware covers advertiser’s site with pay-per-click listings. Advertiser’s site displays as usual, with no covering popup. User stays at advertiser’s site, and advertiser pays no PPC fee.
SmileyCentral – toolbar Reconfigured browser tricks user into running a “search” for a site’s domain name. User’s browser retains its ordinary configuration. User runs a direct navigation, and advertiser pays no PPC fee.
Typosquatting User misspells advertiser’s domain name. User’s browser shows a list of alternatives, and user selects one — reaching advertiser’s site at no charge. Or, user sees an error page, notices the misspelling, and corrects the spelling to reach the advertiser’s site without a PPC fee.
Chrome – browser suggestions User typing a web address is encouraged to run a search instead. User finishes typing the site’s web address and reaches the advertiser’s site without a PPC fee.

WhenU Covers Advertisers’ Sites with Advertisers’ Own Google Ads

WhenU covers RCN with its own Google ads -- charging ad fees for traffic RCN would otherwise have received for free.
WhenU covers RCN with its own Google ads — chargingad fees for traffic RCN would otherwise have received for free.


PPC advertisers (e.g. RCN)
money viewers
   Google   
money viewers
InfoSpace
money viewers
Nbcsearch
money viewers
LocalPages
money viewers
WhenU

The money trail – how funds flow from advertisers
to Google to WhenU.

Through popups shown to users already at advertisers’ sites, WhenU (and its partners) charge advertisers for traffic they would have otherwise received for free. Google passes along these clicks through its advertising platform, and Google charges advertisers as if these were genuine leads, even though in fact these users were already on advertisers’ sites.

In testing of May 9, 2009, I browsed the web site RCN.com. Advertising software (generously, “adware”) from WhenU popped open the window shown at right — covering more than 80% of the RCN site with a list of pay-per-click ads, among them a prominent ad for RCN. I clicked the RCN ad, and I was taken back to RCN. See screenshots detailing the full sequence, or a screen-capture video.

As a provider of high-speed Internet access (with attendant concerns for user security, PC reliability, and tech support expense), RCN is unlikely to advertise with a notorious adware program like WhenU. Nor is RCN likely to want to show its ads to users already at rcn.com — users who RCN has already managed to reach. So how did RCN’s ads end up in this unfortunate placement? Using a network monitor, I confirmed the full sequence of intermediaries: WhenU and its MediaTraffic ad system sent traffic to LocalPages, which redirected to Nbcsearch, which forwarded the traffic to Infospace, which finally passed the traffic to Google. Google in turn paid Infospace, which paid Nbcsearch, which paid LocalPages, which paid WhenU. See the diagram at right and the full packet log.

For RCN, this placement is a rotten deal. RCN has already paid to get a user to its site — perhaps via a postcard, TV advertisement, display ad, or other paid search activity. But then WhenU intercedes and puts a roadblock in front of that user, in the form of the popup at right. A typical user presented with that popup will click the RCN entry to get back to RCN and continue the signup process. But then RCN pays twice to reach a single user.

Meanwhile, if RCN is tracking conversion rates, it will notice that this WhenU/Google placement seems to have a high conversion rate — reflecting that this ad was shown to users already on the verge of signing up with RCN. So in all likelihood, RCN will increase its Google bid to attempt to obtain more traffic of similar (apparent) quality. But the supposed high conversion rate is misleading at best: Since these are users who were already at the RCN site, without any intervention by Google or WhenU, it is nonsense to credit Google or WhenU for resulting sales.

This ad placement also lacks the labeling required by FTC policy and precedent. For one, all sponsored search ads are to be labeled as such, pursuant to the FTC ‘s 2002 instructions. But look closely at the popup screenshot. On my ordinary 800×600 screen, no such label appears. Furthermore, the FTC’s Direct Revenue and Zango complaints, decisions, and orders reveal an additional duty that adware vendors specifically and clearly label each popup with, among other information, a statement that the popup is an advertisement. (See Direct Revenue Decision and Order, provision VI.(1), and Zango Decision and Order, provision VI.(1).) Here, the popup bears the WhenU icon and even a phone number, but it lacks any disclosure that the window’s listings are advertisements. I gather the required label would ordinarily appear on a sufficiently large screen, but the FTC’s policies make no exceptions for users with small screens. Indeed, as netbooks gain popularity, small screens are increasingly common.

These ad placements are particularly troubling because they have continued for months on end. I first observed substantially similar placements on February 7, 2009, though I have reason to think the placements began well before then. Furthermore, Google has been on actual notice of these practices for 3+ months: I first reported these placements to the public in my lunch keynote at the INTA Trademark Law and the Internet conference on February 10, 2009. I posted my slides that very day, and slides 23-24 show substantially this same set of relationships. Importantly, Google’s Rose Hagan, Senior Trademark Counsel, was present in the audience, and she responded to this portion of the talk by promising to investigate. But despite her presence, my clear posting of the parties responsible, and a three month opportunity to act, Google has nonetheless failed to sever these relationships.

Plenty more could be said about WhenU. I have personally observed a wide variety of deceptive WhenU installations, including even WhenU installations via security exploits. (I never had occasion to post those videos to my public site, but I have them on file.) The Google-funded StopBadware declared multiple WhenU-bundlers to be “badware” (1, 2, 3) based on WhenU’s automatic startup, disruptive pop-ups, and other “bad or undisclosed” behaviors. But I doubt Google will defend its WhenU placements, so I’ll save full critique of WhenU for another day.

IAC’s SmileyCentral Grabs Advertisers’ Organic Traffic to Show Google Ads

Standard web browsers place a search bar at top-left. Click that box, type an address, and press enter to reach a site.
Standard web browsers place a search bar at top-left. Click that box, type an address, and press enter to reach a site.

With SmileyCentral installed, a 'direct navigation' request for www.verizon.com yields search results, not the Verizon site. With SmileyCentral installed, a “direct navigation” request for www.verizon.com yields search results, not the Verizon site.


PPC advertisers (e.g. Verizon)
money viewers
   Google   
money viewers
IAC/SmileyCentral

The money trail – how funds flow from advertisers
to Google to IAC/SmileyCentral.

By reconfiguring users’ web browsers, IAC’s SmileyCentral toolbars charge advertisers for traffic they would otherwise have received for free. Google passes along these clicks through its advertising platform, and Google charges advertisers as if these were genuine leads, even though in fact these are users who were specifically and unambiguously trying to reach advertisers’ sites directly.

Since the earliest web browsers, a user seeking a particular web site clicks in the browser’s upper-left text box, types the desired address, and presses Enter. See the first inset image at right — standard Internet Explorer 6, offering exactly this layout to request a site.

But suppose a user installs the “SmileyCentral” toolbar from IAC (the advertising powerhouse that also owns Ask.com, Match.com, and more). SmileyCentral modifies longstanding browser layout by pushing a user’s Address Bar to the right, and inserting at left a search box where users naturally expect the Address Bar to appear. Then, when a user goes to the top-left box and enters a domain name, seeking a direct navigation to the specified site, SmileyCentral runs a search instead. See the second inset image at right — the result of typing www.verizon.com, then pressing Enter, into the top-left box. Notice that the user receives a list of search ads for Verizon, even though the user asked for Verizon by its correct web address.

IAC/SmileyCentral’s search results increase advertisers’ costs. Consider user behavior upon reaching the listings shown at right. In all likelihood, a user facing these listings will click one of the top links — perhaps the first listed (to verizonwireless.com) or the second (which specifically indicates it is the “Verizon Official Site”). But these are sponsored links. Every time a user clicks one of these links, Verizon pays a fee to Google, which in turn pays IAC/SmileyCentral. (To confirm Google’s role, see the packet log.)

As in the WhenU example above, this placement is a bad deal for advertisers. Had it not been for IAC/SmileyCentral’s tricky toolbar, these users would have directly reached the sites they requested. Instead, IAC/SmileyCentral grabs the users and sends them to lists of ads — saddling advertisers with unnecessary advertising expense.

Here too, advertisers are likely to be tricked if they attempt to assess the value of this traffic based on its conversion rate. The conversion rate may well be high — for these users asked for advertisers by name, suggesting that the users are nearly ready to make purchases. But the traffic is still a bad deal for advertisers: By all rights, this is still traffic advertisers should have gotten for free.

This placement is all the worse for advertisers because IAC makes ad clicks excessively easy. Even a click 230 pixels to the right of a Verizon Wireless ad would be treated as a paid click “on” that ad. See video at 0:23, and accompanying screenshot, showing that the hyperlink area extends far beyond the text of the ad.

IAC may claim that users understand that, with MyWebSearch installed, the top-left box no longer allows direct navigations by domain name or URL. But IAC often sneaks its toolbars onto users’ computers in ways that don’t obtain meaningful consent: I previously demonstrated nonconsensual installations through security exploits and undisclosed bundles. IAC’s installations often target kids (as I have repeatedly documented). And even a direct installation from Smileycentral.com places a picture of the post-installation browser layout in a bottom-of-page image, below the fold on 800×600 screens and easily overlooked on larger screens. IAC’s use of the generic “My Web Search” label (just as in “My Documents”, “My Computer”, etc.) further compounds users’ sense that this box is part of Internet Explorer. Combining IAC’s user focus with its choice of labeling and positioning, IAC creates conditions in which users are bound to be confused.

Typosquatting: Cmcast.com, MediaLogik, and Thousands More Intercept Users’ Misspellings to Show Google Ads

Requesting Cmcast.com (s.i.c.) yields a page of Google ads.
Requesting cmcast.com (s.i.c.) yields a page of Google ads.

If no typosquatter had registered Cmcast.com, a user would have reached this page and reached Comcast without charge. If no typosquatter had registered cmcast.com, a user would have reached this page and found Comcast without charge.


PPC advertisers (e.g. Comcast)
money viewers
   Google   
money viewers
Typosquatters & brokers (e.g. MediaLogik)

The money trail – how funds flow from advertisers
to Google to typosquatters.

By paying partners to register typosquatting sites and to send the resulting clicks to Google’s ad platform, Google charges advertisers for traffic they would otherwise have received for free. Google charges advertisers as if these were genuine leads. Yet had typosquatters not registered these domains, ordinary web browsers would have assisted users in reaching advertisers’ sites without charge.

A staggering number of typosquatting web sites target users who misspell the domain names of famous web sites. For example, in my INTA presentation this spring, I showed 200+ typosquatting sites all targeting variations of “cartoonnetwork.com.” I’ll have further thoughts on typosquatting in a future article. But for present purposes, two facts are most important: Typosquatting sites are overwhelmingly funded through pay-per-click ads, and these days Google is the advertising provider of choice for most typosquatters. (My preliminary analysis indicates that Google funds more than 75% of those typosquatting sites that show Google ads.)

Consider a user who misspells comcast.com, instead typing cmcast.com (s.i.c.). Such a user will receive the first screen shown at right, presenting a series of pay-per-click links for Comcast. In all likelihood, such a user will then click a paid link to Comcast — meaning Comcast has to pay Google an advertising fee. Google in turn pays the typosquatter which had the foresight to register a domain. See the packet log.

(In some instances, traffic flows from a typosquatter to one or more brokers or intermediaries, then to Google and finally to the advertiser. As to cmcast.com: The packet log confirms that Google pays MediaLogik of Hong Kong. I cannot readily determine whether MediaLogik registered this domain for its own account, or whether MediaLogik brokers ads for some other registrant. Based on the domain’s hosting in the Bahamas, its monetization through a Hong Kong service provider, and its display of Google PPC ads, it is highly unlikely that this domain was authorized by Comcast.)

Although these typosquatting sites ultimately direct users where the users were trying to go, typosquatters leave targeted advertisers importantly worse-off. Had it not been for the typosquatter, the user would have received a standard browser page identifying the typo and, in general, referring the user to the requested site without charge. See the second screenshot at right, showing a request for cmcast.com in a standard installation of Internet Explorer 6. So, even if no typosquatter had registered cmcast.com, the user would still reach Comcast with equal ease; the typosquatter does not actually make it easier for the user to reach Comcast. But notice that a standard IE6 error page shows few ads — here, just one small ad, at far right — whereas the MediaLogik page showed all ads, front and center. So passing the user’s misspelled request to a standard IE landing page, rather than to a typosquatter, would spare Comcast an expensive Google pay-per-click fee.

Notably, typosquatting exactly fits the traffic pattern detailed in preceding examples. First, Google and its partners identify users who are already trying to reach a given advertiser. Then Google and its partners intercede — here, by registering a typosquatting domain. The user thus stumbles into Google’s ad listings and clicks through to the advertiser’s site — letting Google charge the advertiser a fee, even though the user would have reached the advertiser even without Google’s assistance.

Google Chrome Suggestions Divert Users from Direct Navigation to Search

Typing 'expedia' yields a suggestion that users search Google (simply by pressing 'Enter') rather than visiting Expedia.com directly (third entry -- requiring a mouse-click or multiple keystrokes). Meanwhile, the second link ('expedia/') yields an error -- discouraging future exploration of green direct-navigation listings.
Typing “expedia” yields a suggestion that users search Google (just press “Enter”) rather than visit Expedia.com directly (third entry — requiring a mouse-click or multiple keystrokes). The second link (“expedia/”) yields an error — discouraging future exploration of green direct-navigation listings.

As users type web addresses into Google’s Chrome web browser, Chrome’s “Omnibox” address bar suggests that users run searches instead of direct navigation. See the screenshot at right, showing the Omnibox’s suggestion after I typed “Expedia” and before I typed the final “.com”.

If a user accepts Chrome’s suggestion — by clicking the “Search for …” drop-down, or by merely pressing Enter — the user is taken to a page of Google search results for the specified term. See screenshot and screen-capture video. As usual, Google’s most prominent search result is an advertisement. If the user clicks the ad, the advertiser pays a pay-per-click fee — even though the user was nearly at the advertiser’s site, for free, before Chrome interceded with its “Search for…” suggestion.

To complete a direct navigation to the Expedia site, without passing through Google search results, a user must ignore Google’s suggestion and continue typing (“.com”), click the “expedia.com” entry (third on Google’s autocomplete list), or use the keyboard (down-down-enter) to navigate manually. In principle these steps are straightforward — just a few extra seconds. But by pushing default behavior from direct navigation to search, Google makes searches that much more frequent — yielding that many more ad-clicks, that much more revenue to Google, and that much more expense for advertisers.

Chrome further encourages searching by mixing useful autocomplete direct navigation listings (e.g. “www.expedia.com/”) with nonfunctional listings. Notice that the second entry on Chrome’s autocomplete drop-down is “expedia/” (s.i.c.) — not a valid domain name. If a user clicks that entry, the user suffers a delay followed by a DNS error — hardly a good user experience. (video) By placing this nonfunctional result prominently in the autocomplete drop-down, above the one working direct link to Expedia, and in the same distinctive green font as the working direct link, Chrome discourages users from exploring direct links. After all, if the prominently-listed “expedia/” link did not work, users are less likely to try the similar-looking link Google ranked lower.

Although competing browsers also perform searches if users enter malformed text into the Address Bar, competing browsers are less pushy in suggesting and encouraging searches in lieu of direct navigation. Chrome thus extends browser norms through its increasingly forceful suggestion of search, not direct navigation, as the default way to reach a site.

Beyond its effects on advertisers’ costs, Chrome’s Omnibox raises other serious concerns too. For example, Chrome transmits users’ every navigation keystroke to Google — including what users search for, what addresses users request by name, and what addresses users begin to request but ultimately decline to visit. CNET reports EFF and EPIC staff concerned about Omnibox’s privacy implications, and I share their discomfort.

Fair Play and a Way Forward

Notice similarities across all four examples:

  • In all four examples, Google and its partners intercede to divert traffic that, but for their intervention, would reach advertisers’ sites directly — without advertisers incurring any advertising expense.
  • In all four examples, Google and its partners ultimately pass the traffic back to the advertisers users were trying to reach — but only after collecting pay-per-click advertising fees.
  • In all four examples, if an advertiser attempts to measure its conversion rate, it will conclude that it receives traffic at attractive prices — at effective cost-per-acquisition consistent with its objectives. Standard conversion rate measurements will overlook the fact that, were it not for the intervention of Google and its partners, the advertiser would have received this traffic for free.

Not all advertisers measure conversion rates. For one, some advertisers may take Google at its word, rather than attempting detailed and rigorous monitoring of Google’s effectiveness. Furthermore, for some advertisers, conversion rates are unmeasurable. (Advertisers may have offline sales process, rather than online sales. Certain advertisers may seek brand awareness rather than immediate purchases.) But for advertisers that measure conversion rates and adjust bids accordingly, inflated conversion rates are of grave concern. In particular, inflated conversion rates yield records that seem to indicate that campaigns are working well and delivering fair value, when the truth might be exactly the opposite.

Aggrieved advertisers could sue Google over these placements. But Google imposes terms and conditions that discourage advertisers from pursuing such claims. Google’s Advertising Program Terms purport to “disclaim[] all warranties … [and] guarantees regarding positioning, levels, quality, or timing of costs per click, … clicks, conversions, or any other results” (internal numbering omitted). Google further claims that “Any refunds for suspected invalid impressions or clicks are within Google’s sole discretion.” Google also insists that advertisers’ sole remedy is advertising credits (never refunds), and Google says advertisers must complain within 60 days of a disputed charge (even if advertisers did not know about the improper charges because the charges were concealed through, e.g., inflated conversion rates). I doubt whether all these harsh provisions are enforceable. But if these provisions are enforceable, they would tightly limit advertisers’ ability to obtain redress from Google. Even if these provisions are unenforceable, their mere presence serves to discourage advertisers from filing complaints, whether informal (through Google’s customer service staff) or in court.

In a recent presentation defending its competitive posture and overall approach, Google claimed “advertisers pay what a click is worth to them” (slides 17-21). But does Google have the right to charge such a fee for all traffic, even the traffic advertisers receive without any Google assistance whatsoever? Few advertisers would tolerate such charges, if presented explicitly. Nor does Google ever seek permission to charge advertisers for their existing traffic. Yet in the future Google seems to envision, all manner of ruses intercede to claim Google delivered a customer, when in fact the customer reached the advertiser’s site without bona fide assistance from Google.

More generally, when Google and its partners overstate the effectiveness of the ads they deliver, advertisers cannot make well-informed decisions about which ads to buy or how much to be willing to pay. Danny Sullivan calls AdWords pricing a “black box,” and I certainly share that sentiment. But here, Google’s actions are even worse than opacity: By claiming to have delivered traffic advertisers would have received anyway, Google tricks advertisers into paying for that traffic — and even tricks advertisers into concluding, mistakenly, that the traffic is a good deal.

It’s also hard to reconcile Google’s WhenU and IAC/SmileyCentral placements with Google’s “Software Principles” requirements. With bundled installations disclosing WhenU’s presence only midway through installation, WhenU is anything but “upfront” (contrary to Google’s “upfront disclosure” section heading). Showing the generic “My Web Search” toolbar label after users requested the distinctively-named SmileyCentral, Ask’s offering similarly flunks Google’s call for “clear behavior.” I could continue. But even if these programs satisfied Google’s criteria, they still fall short of advertisers’ reasonable expectations — for they still intercede to claim traffic advertisers they did nothing to deliver.

Looking forward, I see two key principles. First, ad platforms must deliver genuine, incremental traffic — not just intercept and repackage traffic advertisers were already going to receive for free. Second, ad platforms need to stand behind their products — abandoning one-sided attempts to disclaim responsibility, and instead offering meaningful commitments to provide what they promise. In both these respects, Google currently falls short.

Windows Vista (teaching materials)

Edelman, Benjamin. “Windows Vista.” Harvard Business School Case 909-038, February 2009. (Revised December 2010.) (educator access at HBP. request a courtesy copy.)

Microsoft designs, modifies, publicizes, and distributes Windows Vista—against a backdrop of consumers already largely satisfied with their existing Windows XP systems. Microsoft must decide what features to include and what to drop, how to compete with its own installed base, and how to mobilize partners to offer Vista-compatible systems.

False and Deceptive Display Ads at Yahoo’s Right Media

Yahoo’s Right Media ad marketplace features widespread ads exactly designed to deceive. I present ten examples of these deceptive ads, and I critique their unwelcome characteristics. To estimate the prevalence of deceptive tactics, I examine Right Media’s own analysis ad characteristics — finding that by Right Media’s own admission, deceptive ads total 35% or more of Right Media’s advertising inventory.

Details:

False and Deceptive Display Ads at Yahoo’s Right Media

Restaurant Promotions in 2015 (teaching materials)

Edelman, Benjamin. “Restaurant Promotions in 2015.” Harvard Business School Case 909-034, January 2009. (Revised July 2015.) (educator access at HBP. request a courtesy copy.)

A variety of services offer consumers discounts when dining at participating restaurants. This case examines four such services: Entertainment Book, Restaurant.com, Rewards Network, and Groupon. Despite key functional similarities, each of the services chooses an importantly different approach–different pricing, different benefits to consumers, different benefits to restaurants, and different underlying technologies.

Teaching Materials

Online Restaurant Promotions – Teaching Note (HBP 909063)

Competition among Sponsored Search Services

Edelman, Benjamin. “Competition among Sponsored Search Services.” U.S. House of Representatives, Committee on the Judiciary, Task Force on Competition Policy and Antitrust Laws, 2008. (Hearing cancelled.) (Reprinted in Working Knowledge: Google-Yahoo Ad Deal is Bad for Online Advertising.)

Last month I was asked to testify to the United States House of Representatives Committee on the Judiciary Task Force on Competition Policy and Antitrust Laws about competition among paid search providers, particularly the proposed Google-Yahoo partnership.

At the last minute, the hearing was cancelled, and I won’t be able to testify at the rescheduled session. Rather than let my draft written statement languish, I’m taking this opportunity to post the prepared testimony I had planned to offer:

Competition among Sponsored Search Services.

PPC Platform Competition and Google’s "May Not Copy" Restriction

By all indications, Google AdWords features far more advertisers than competing PPC platforms such as Yahoo Search Marketing and Microsoft adCenter. (Consider: Search for “thinkpad x60 power supply” at Google, and there are six relevant ads from vendors who actually sell that product. Search at Yahoo Search or Microsoft Live Search, and there are various ads from indexers and aggregators, but only one or two from vendors directly selling the product. Other searches for offbeat, unusual or region-specific keywords show similar patterns.)

Why do more advertisers choose Google? Because more users search at Google, Google can offer wider ad distribution than any single competitor. So if an advertiser had to choose just one ad platform, Google would be the natural choice.

But in principle advertisers can easily use multiple ad platforms. Ads are trivially small plaintext data, and In principle ads can be copied from platform to platform without restriction. So why don’t more Google advertisers use Yahoo, adCenter, and others too?

One possible answer comes from a little-noticed Google AdWords API Terms & Conditions restriction which substantially hinders advertisers’ efforts to use multiple providers — exactly prohibiting software vendors from helping advertisers copy AdWords campaigns to competing platforms. In this article, I identify the restriction at issue, analyze its effects on advertisers and competing ad platforms, critique response from Google, and compare this restriction with Google’s commitment to “data portability” in other contexts.

The Restriction at Issue

To use the Google AdWords API, a software developer must accept Google’s AdWords API Terms & Conditions. The T&C’s include the following requirement:

“Any information collected from an input field used to collect AdWords API Campaign Management Data may be used only to manage and report on AdWords accounts. Similarly, any information or data used as AdWords API Campaign Management Data must have been collected from an input field used only to collect AdWords API Campaign Management Data. For example, the AdWords API Client may not offer a functionality that copies data from a non-AdWords account into an AdWords account or from an AdWords account to a non-AdWords account.” (emphasis added)

Sure enough, searching the web for commercial tools to synchronize PPC campaigns or to export data from Google to competing platforms, I found none.

The “May Not … Cop[y] Data” Prohibition: Effect on Advertisers

The quoted restriction prevents advertisers from easily exporting ads from Google to a competing paid search provider. Consider: The essence of an export procedure is to copy data from an AdWords account to a non-AdWords account — exactly what the restriction prohibits.

Indeed, available export procedures are strikingly complex. For example, to import a Google AdWords campaign into Microsoft adCenter, Microsoft offers a 17-step procedure (with some steps made more complicated by the presence of multiple sub-steps).

Microsoft’s procedure is necessarily convoluted because Google’s “may not … cop[y]” restriction prevents Microsoft, or any other vendor, from writing a tool that connects to the Google API, downloads an advertiser’s ads, and uploads those ads directly to, e.g., Microsoft adCenter. Instead, advertisers must download data manually, reformat it to match adCenter’s requirements, and upload it to Microsoft — just as Microsoft’s lengthy procedure specifies.

For many advertisers, Google’s restrictions on data export impose an ongoing burden even beyond the advertiser’s initial signup with a competing PPC provider. Consider an advertiser that changes its ads or keywords often — perhaps selling seasonal merchandise, or experimenting with alternative advertising strategies. Such an advertiser would typically prefer to make changes in one place, and have the changes automatically propagate to all the advertiser’s PPC platforms. If Google remains the advertiser’s primary PPC provider, the advertiser would probably want to make changes in Google’s interface, then have other PPC platforms optionally automatically copy those changes. But Google’s “may not … cop[y]” restriction applies equally to ongoing resynchronizations. If an advertiser made daily changes to its Google campaigns, it would have to daily repeat the manual export/import process — a task that would be both time-consuming and prone to error.

In short, the net effect of the quoted restriction is to reinforce the tendency of small to medium-sized advertisers to “single-home” — to use only Google AdWords, to the exclusion of competing platforms.

At their peril do advertisers rely solely on Google: If advertisers get stuck using only Google, for lack of any easy and efficient way to buy some traffic elsewhere, Google can charge prices higher than competing platforms. Of course Google can’t raise prices infinitely; at some point, advertisers would overcome the lock-in, accept manual export, and copy ads to competitors. But Google’s “may not copy” restriction increases the costs of multi-homing — letting Google charge that much more than competitors, without advertisers facing compelling incentives to look elsewhere.

The “May Not … Cop[y] Data” Prohibition: Effect on Competing Ad Platforms, on Publishers, and on Users

By encouraging small to medium-sized advertisers to advertise only with Google AdWords, Google’s API restriction reduces the number of advertisers using competing ad platforms. This harms competing platforms in two distinct ways. First, it reduces competitors’ coverage — preventing competitors from featuring relevant ads that pertain to obscure user searches. (Consider the power supply example from the first paragraph of this piece — better and more useful ads at Google.) With fewer relevant ads, the competing platform offers users an inferior service — inviting users to look elsewhere, and reducing the likelihood of a paid click that would earn the platform an advertising fee.

Second, by reducing the number of advertisers bidding for advertising positions at other platforms, the quoted provision dramatically reduces revenue at those platforms. My December 2006 Optimal Auction Design in a Multi-unit Environment estimates the revenue benefits of additional advertisers based on publicly-available data and estimates of market fundamentals. The intuition is straightforward: When many advertisers seek positions for a given search term, they must bid higher to avoid being outbid and receiving inferior listing position. Conversely, when only a few advertisers seek positions, prices can be strikingly low since even a low bid earns a prominent position.

Google’s API restriction also reduces the value of advertising inventory held by third-party publishers. Consider a publisher seeking to sell its sponsored search or other text ad inventory to a provider of sponsored search services. In general, Google can afford to pay more because Google’s revenue per search is higher than competitors’. But how much will Google offer? Google maximizes profits by narrowly outbidding competitors; anything higher is waste. So the weaker competitors become, the lower Google can bid — and the less revenue publishers receive for the traffic they sell. Google’s “may not copy” API restriction serves a role in weakening competing platforms — keeping advertisers using Google alone, and hence reducing competing ad platforms’ ability to pay for publishers’ inventory.

End users also suffer from Google’s restriction on copying ads. Were it not for Google’s restriction, more advertisers would sign up to use competing ad platforms — increasing the usefulness of Yahoo Search and Microsoft Live Search for the users who choose those services.

Google’s Response

I forwarded these concerns to Google in March, and I managed to get in touch with Doug Raymond, product manager for AdWords API. Doug offered three rationales for the restriction. The list below summarizes his arguments (black) and my responses (blue).

  • Google: The quoted provision only applies to third-party developers. Individual advertisers remain free to write software that exports their Google campaigns.
    • Small to medium-sized advertisers don’t want to be developers. Rather, they want to use code that others write. That’s exactly why the AdWords API offers a concept of developers, rather than requiring that every advertiser write its own code.
    • As a leading provider of centralized computing services, as distinguished from small programs individual users build themselves, Google well knows the benefits of rigorous design by competent professionals.
  • Google: Advertisers can extract their data in other ways, e.g. a comma-separated-value (CSV) file.
    • Manual export is convoluted, slow, and error-prone. API-based access would be faster, easier, and more reliable.
    • The existence of an inferior alternative does not justify imposing restrictions that prohibit superior implementations.
    • In other contexts (detailed below), Google has made strong requests for, and commitments to, data portability.
    • In other contexts, Google emphasizes the benefits of streamlined, automated data transfer — never viewing convoluted manual procedures as an acceptable alternative.
  • Google: Third-party developers ought not have free access to advertiser data.
    • Google’s AdWords API already offers an appropriate security model to limit developers to serving those AdWords advertisers that have specifically granted such permission. In short, a developer needs a password to access an advertiser’s account.

Google’s Position on Data Portability in Other Contexts

Google’s prohibition on AdWords API data export stands in sharp contrast to Google’s position on data portability in other contexts. Indeed, Google has previously taken a firm position in favor of data portability. Some specific examples: In a November 2006 interview at the Web 2.0 Summit, Schmidt specifically promised that “We [Google] would never trap user data.” Schmidt added that “If users can switch it keeps us honest.” Just last month, Google CEO Eric Schmidt called for open access to (and indexing of) social network data — telling IBM’s Business Partner Leadership Conference “People should be able to move from place to place, and their data is available everywhere” and “open is best for the consumer.” Well-known Google blogger Matt Cutts summarized Google’s commitment to data openness with the catchy title “Not trapping users’ data = GOOD” and a long list of Google products that support data export.

I credit that Schmidt’s statements refer to other kinds of data — search engines’ records of users’ search history, and a wide assortment of data held by social networks. But the same principles plainly apply to access to search ads: Just as consumers benefit from being able to move their data as they see fit, so too do advertisers benefit from flexibility.

Moreover, it strains credibility for Google to ask social networks to share their data with Google, while Google simultaneously imposes contractual roadblocks preventing others from accessing Google data.

Next Steps and Google’s Other Restrictions

Google already faces antitrust scrutiny for its striking growth and market share. In that context, it’s particularly hard to defend the restriction at issue — a barrier to competition, without any apparent pro-competitive purpose. Regulators might reasonably require that Google remove the quoted provision — letting third-party developers export and synchronize AdWords data if advertisers so desire. This would be a trivially straightforward requirement — just a sentence to be stricken from Google’s AdWords API T&C’s. Because Google’s existing APIs already provide the required data, Google would not need to add any new code or any new API functions.

Other AdWords API restrictions also deserve scrutiny. For example, Google insists that advertising tools collect AdWords instructions through separate fields not used for other ad platforms — blocking simplification via a single interface to streamline advertisers’ decisions. Google prohibits advertising tools from storing Google data in a single relational database along with data for other ad platforms — increasing the complexity of designing a system to manage campaigns on multiple platforms. And Google prohibits reports that compare Google ad performance data (e.g. costs and profits from advertising at Google) with data from other ad platforms — hindering advertisers’ efforts to evaluate competitors’ offerings. I gather Google defends these restrictions on the grounds that they purportedly prevent advertiser confusion. Perhaps — but their more obvious effect is to increase the costs and complexity of using competing ad platforms. Perhaps I’ll consider these restrictions in greater detail in a future article.

Meanwhile, I’m struck by Google’s calls for data portability in other contexts. With Google’s ongoing request that other companies provide data to Google, perhaps Google will return the favor by abandoning its “may not copy” restriction — ideally promptly and unilaterally, without requiring that regulators force Google’s hand.

Microsoft adCenter (teaching materials) with Peter Coles

Coles, Peter, and Benjamin Edelman. “Microsoft adCenter.” Harvard Business School Case 908-049, January 2008. (Revised February 2010.) (educator access at HBP. request a courtesy copy.)

Microsoft considers alternatives to expand its presence in online advertising, especially text-based pay-per-click advertising. Google dominates, and it is unclear how Microsoft can grow, despite considerable technical and financial resources. Microsoft considers a set of alternatives, each with clear benefits but also serious challenges.

Teaching Materials:

Microsoft adCenter (Teaching Note) – HBP 908062