The Right Remedies for Google’s AdWords API Restrictions

Last week the FTC closed its 21-month investigation of Google after Google made several small concessions, among them dropping certain restrictions on use of Google’s AdWords API — rules that previously limited how advertisers and tool-makers may copy advertisers’ own data from Google’s servers. Removing the restrictions is a step forward for advertisers and for competition. But the FTC could and should have demanded more from Google in order to address the harm resulting from seven years of these restrictions.

I first flagged Google’s AdWords API restrictions in my June 2008 senate testimony and in greater detail in PPC Platform Competition and Google’s “May Not Copy” Restriction. In short, the restrictions prohibited making and sharing tools to quickly copy and synchronize ad campaigns across multiple ad platforms — effectively compelling small to midsized advertisers to use Google only, for lack of tools to manage their campaigns on multiple platforms. Google enforces this prohibition with a system of tool passwords and audits — letting Google swiftly and completely disable any tool that Google deems impermissible. Indeed, any tool-maker found offering a noncompliant tool would immediately lose all access to Google’s AdWords API, as to all of its tool-using subscribers, a devastating blow that kept tool-makers under Google’s thumb.

As I pointed out, these AdWords API restrictions let Google charge prices higher than competing platforms: Thanks to these restrictions, a small to midsized advertiser would struggle to buy some placements from Yahoo or Microsoft, even if those vendors offered lower prices. Higher advertising costs directly harm advertisers, and higher prices get passed to consumers (according to the relative elasticity of supply and demand). I also pointed out harms to others in the advertising ecosystem: Competing ad platforms struggle to attract advertisers, hence showing less relevant advertising (discouraging users from clicking ads) and enjoying less auction pressure to push prices upward. Meanwhile, I noted, the AdWords API restrictions give Google that much more leverage in its negotiations with publishers: by weakening other ad platforms’ monetization, Google can more easily win deals for publishers’ inventory, granting publishers lesser compensation for the content they post.

Strikingly, Google has never seriously defended the AdWords API restrictions. In June 2008, Doug Raymond, Product Manager for AdWords API, argued that advertisers are free to export their data in other ways, e.g. as a CSV text file. But that far-inferior manual export is ad-hoc, time-consuming, and error-prone — a poor fit for high-priced online advertising. Indeed, this manual approach is a sharp contrast from a modern automation API, and a far cry from what Google offers in other contexts.

By all indications, competition regulators share my concerns. In a November 2010 press release, the European Commission flagged “restrictions on the portability of online advertising campaign data” as its fourth concern in reviewing Google’s conduct, a concern most recently reiterated in December 2012 remarks by EC Competition Commissioner Joaquín Almunia. Last week’s statement by FTC Chairman Leibowitz is in accord: “Some Commissioners were concerned by the tendency of Google’s restrictions to raise the costs of small businesses to use the power of internet search advertising to grow their businesses.”

So everyone but Google agrees that the AdWords API restrictions are improper, and even Google has little to say in its defense. Indeed, despite abandoning most other aspects of its investigation, the FTC did pursue this matter. But the FTC reports only that Google is to remove AdWords API restrictions and that, the FTC indicates, ends the FTC’s concern on this subject. I am surprised by such a narrow remedy. The AdWords API restrictions have been in place for more than five years. Were it not for these restrictions, advertisers for five years would have enjoyed lower prices. For five years, third-party publishers would have received higher payment for their ad space. For five years, consumers would have seen more relevant ads at competing ad platforms, perhaps helping to increase competitors’ market shares and put a check on Google’s dominance. Moreover, for five years competing ad platforms would have enjoyed higher advertising revenues and higher ad click-through rates. It’s all well and good for Google to remove the API restriction going forward. But that does nothing at all to address past harm to advertisers and others.

Three Appropriate Remedies

What remedies would be appropriate for Google’s seven years of improper AdWords API restrictions? Let me offer three suggestions:

First, after years of improper conduct in this area, Google should expect to pay monetary damages. Google’s AdWords API restrictions inflated the prices charged to advertisers. Google should disgorge these ill-gotten gains via pro-rata refunds to advertisers.

Second, Google’s changes should be formal binding commitments formalized in a consent agreement. The 1969 Report of the American Bar Association Commission to Study the Federal Trade Commission recognized that voluntary commitments were ineffective, and the FTC largely discontinued voluntary commitments after that report. Indeed, FTC Commissioner Rosch last week noted that the FTC’s voluntary commitment approach lets Google offer a statement of its current intent, which Google could reverse or alter at any time. Moreover, an order would have required the FTC to take a clearer position on whether Google’s conduct violated the law: An order would have required the FTC to file a complaint, which in turn requires a finding by the FTC that there is reason to believe a violation has occurred. This formality would offer a useful confirmation of the FTC’s view — either the FTC believes a violation occurred, or it does not, but the voluntary commitment process lets the FTC avoid a public statement on this subject. Finally, orders are also vetted with third parties to make sure they will be effective. Microsoft’s Dave Heiner immediately offered several gaps in the FTC’s approach on AdWords API restrictions. I would have offered additional feedback had I been asked.

Finally, the FTC’s investigation surely found documents or records confirming the intent and effect of Google’s AdWords API restrictions. The FTC should at least describe those documents — if not release them in full. Describing or releasing these documents would let concerned parties determine what private claims they may have against Google. If the documents confirm meritorious claims, victims can pursue these claims through private litigation (here too, as Commissioner Rosch suggested).

Google’s AdWords API restrictions were a direct assault on competition — indefensible rules serving only to hinder advertisers’ efforts to efficiently use competing search engines, without any plausible pro-competitive justification. On this clear-cut issue, the FTC should have pursued every remedy permissible under its authority. Fortunately it’s not too late for state attorneys general and the European Commission to insist on more.

A Holiday “Top 10”: Rogue Affiliates at Commission Junction and LinkShare with Wesley Brandi

Our automation continuously scours the web for rogue affiliates. In our query tool, we provide a basic sense of how much we’ve found. We have also written up scores of sample rogue affiliates, but the holiday season provides an impetus for more: Thanks to high online spending, affiliate fraud at this time of year is particularly profitable for perpetrators — and particularly costly to merchants.

In today’s article, we report the ten Commission Junction affiliates and ten LinkShare affiliates most often seen by our automation. Our findings:

Twenty Oft-Found Commission Junction and LinkShare Affiliate Violations

Pricing and Partnership at Zillow, Inc. (teaching materials) with Peter Coles

Coles, Peter, and Benjamin Edelman. “Pricing and Partnership at Zillow, Inc.” Harvard Business School Case 913-021, November 2012. (Revised March 2015.) (educator access at HBP. request a courtesy copy.)

As Zillow’s real estate search service gains user adoption, some real estate professionals question Zillow’s policies, fees, and power. Dissatisfied real estate professionals could remove listings from Zillow, reducing the service’s value to users. Should Zillow adjust its approach in order to address complaints?

Teaching Materials:

Pricing and Partnership at Zillow, Inc. – Teaching Note (HBP 914043)

Towards Efficiencies in Canadian Internet Traffic Exchange

Edelman, Benjamin, and Bill Woodcock. “Towards Efficiencies in Canadian Internet Traffic Exchange.” Canadian Internet Registration Authority, September 2012.

Canadian Internet access is heavily and unnecessarily dependent upon foreign infrastructure, especially U.S. infrastructure. This dependence imposes significant burdens upon Canadian Internet users:

  • Service prices are higher than would be the case if Canadian networks more densely interconnected domestically.
  • Network speed is slower than would be the case if Canadian networks more densely interconnected domestically.
  • When data en route from one Canadian network to another passes through other countries, the data is subject to examination by companies and government authorities in those countries. Canadian data-protection laws are understood not to protect data as it passes through other countries.

Despite these challenges, experience in other countries shows a clear way forward. By establishing more Internet exchange points (IXPs) within Canada, Canada can reduce the portion of network traffic that travels from one point in Canada, through the United States or other nations, and back to another point in Canada. The key benefits:

  • By reducing reliance on costly international data transit, additional IXPs will reduce networks’ ongoing operational costs. These cost savings will flow to Canadian Internet users, and unnecessary export of capital will be reduced.
  • By providing high-speed domestic links, additional IXPs will increase the amount of bandwidth available to Canadian users, mitigating networks’ bandwidth shortages and removing networks’ incentives to impose bandwidth throttling and usage caps.
  • By favoring shorter and more direct routes, additional IXPs will reduce network latency, improving the performance of new services like video and cloud-based applications.
  • By allowing Canadian data to remain in Canada as much as possible and as often as possible, additional IXPs will reduce the risk of Canadian data becoming subject to foreign laws and practices.
  • By increasing the richness and density of connections between Canadian networks, additional IXPs will increase the reliability of Internet access in Canada and its resilience to disaster and attack.

Affiliate Fraud Litigation Index

Some analysts view affiliate marketing as “fraud-proof” because affiliates are only paid a commission when a sale occurs. But affiliate marketing nonetheless gives rise to various disputes — typically, merchants alleging that affiliates claimed commission they had not properly earned. Most such disputes are resolved informally: merchants withhold amounts affiliates have purportedly earned but have not yet received. Occasionally, disputes end up in litigation with public availability of the details of alleged perpetrators, victims, amounts, and methods.

In today’s posting, I present known litigation in this area including case summaries and primary source documents:

Affiliate Fraud Litigation Index

Flash-Based Cookie-Stuffer Using Google AdSense to Claim Unearned Affiliate Commissions from Amazon with Wesley Brandi

Merchants face special challenges when operating large affiliate marketing programs: rogue affiliates can claim to refer users who would have purchased from those merchants anyway. In particular, rogue “cookie-stuffer” affiliates deposit cookies invisibly and unrequested — knowing that a portion of users will make purchases from large merchants in the subsequent days and weeks. This tactic is particularly effective in defrauding large merchants: the more popular a merchant becomes, the more users will happen to buy from that merchant within a given referral period.

To cookie-stuff at scale, an attacker needs a reliable and significant source of user traffic. In February we showed a rogue affiliate hacking forum sites to drop cookies when users merely browse forums. But that’s just one of many strategies. I previously found various cookie-stuffing on sites hoping to receive search traffic. In a 2009 complaint, eBay alleges that rogue affiliates used a banner ad network to deposit eBay affiliate cookies when users merely browsed web pages showing certain banner ads. See also my 2008 report of an affiliate using Yahoo’s Right Media ad network to deposit multiple affiliate cookies invisibly — defrauding security vendors McAfee and Symantec.

As the eBay litigation indicates, display advertising networks can be a mechanism for cookie-stuffing. Of course diligent ad networks inspect ads and refuse cookie-stuffers (among other forms of malvertising). So we were particularly surprised to see Google AdSense running ads that cookie-stuff Amazon.

The 'Review Different Headphones' ad actually drops Amazon Associates affiliate cookies.
This innocuous-looking banner ad sets Amazon Associates cookies invisibly.
The Imgwithsmiles attack

We have uncovered scores of web sites running the banner ad shown at right. On 40 sites, on various days from February 6 to May 2, our crawlers found this banner ad dropping Amazon Associates affiliate cookies automatically and invisibly. All 40 sites include display advertising from Google AdSense. Google returns a Flash ad from Imgwithsmiles. To an ordinary user, the ad looks completely innocuous — the unremarkable “review different headphones” image shown at right. However, the ad actually creates an invisible IMG (image) tag loading an Amazon Associates link and setting cookies accordingly. Here’s how:

First, the ad’s Flash code creates an invisible IMG tag (10×10 pixels) (yellow highlighting below) loading the URL http://imgwithsmiles.com/img/f/e.jpg (green).

function Stuff() {
  if (z < links.length) {
    txt.htmltext = links[z];
    z++;
    return(undefined);
  }
  clearinterval(timer);
}
links = new array();
links[0] = "<img src="http://imgwithsmiles.com/img/f/e.jpg" width="10" height="10"/>";z = 0;timer = setinterval(Stuff, 2000);

While /img/f/e.jpg features a .jpg extension consistent with a genuine image file, it is actually a redirect to an Amazon Associates link. See the three redirects preserved below (blue), including a tricky HTTPS redirect (orange) that would block many detection systems. Nonetheless, traffic ultimately ends up at Amazon with an Associates tag (red) specifying that affiliate charslibr-20 is to be paid for these referrals.

GET /img/f/e.jpg HTTP/1.0
Accept: */*
Accept-Language: en-US
Referer: http://pagead2.googlesyndication.com/pagead/imgad?id=CICAgICQvuXgahDQAhiYAjII3bQHU19r_Isx-flash-version: 10,3,183,7User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; ...)Host: imgwithsmiles.comConnection: Keep-AliveHTTP/1.1 302 Moved TemporarilyDate: Wed, 02 May 2012 19:56:59 GMTServer: Apache/2.2.21 (Unix) mod_ssl/2.2.21 OpenSSL/0.9.8e-fips-rhel5 mod_bwlimited/1.4
X-Powered-By: PHP/5.2.17
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Set-Cookie: PHPSESSID=174272468a212dd0862eabf8d956e4e0; path=/
Location: https://imgwithsmiles.com/img/kick/f/e.jpg
Content-Length: 0
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: text/html-

HTTPS redirect decoded via separate manual request
GET /img/kick/f/e.jpg HTTP/1.1 Accept: text/html, application/xhtml+xml, */* Accept-Language: en-US User-Agent: ... Accept-Encoding: gzip, deflate Host: imgwithsmiles.com Connection: Keep-AliveHTTP/1.1 302 Moved Temporarily Date: ... Server: Apache/2.2.21 (Unix) mod_ssl/2.2.21 OpenSSL/0.9.8e-fips-rhel5 mod_bwlimited/1.4 X-Powered-By: PHP/5.2.17 Location: http://imgwithsmiles.com/img/t/f/e.jpg Content-Length: 0 Connection: close Content-Type: text/html-GET /img/t/f/e.jpg HTTP/1.0 Accept: */* Accept-Language: en-US x-flash-version: 10,3,183,7 User-Agent: Mozilla/4.0 (compatible; ...) Connection: Keep-Alive Host: imgwithsmiles.com Cookie: PHPSESSID=174272468a212dd0862eabf8d956e4e0HTTP/1.1 302 Moved TemporarilyDate: Wed, 02 May 2012 19:56:59 GMT Server: Apache/2.2.21 (Unix) mod_ssl/2.2.21 OpenSSL/0.9.8e-fips-rhel5 mod_bwlimited/1.4 X-Powered-By: PHP/5.2.17 Expires: Thu, 19 Nov 1981 08:52:00 GMT Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Pragma: no-cache Location: http://www.amazon.com/gp/product/B002L3RREQ?ie=UTF8&tag=charslibr-20 Content-Length: 0 Keep-Alive: timeout=5, max=99 Connection: Keep-Alive Content-Type: text/html

If a user happens to make a purchase from Amazon within the subsequent 24 hours, Amazon will pay a commission to this affiliate — even though the affiliate did nothing at all to cause or encourage the user to make that purchase.

Does Amazon know?

The available information does not reveal whether or not Amazon knew about this affiliate’s practices. Nor can we easily determine whether, as of the May 2, 2012 observations presented above, this affiliate was still in good standing and receiving payment for the traffic it sent to Amazon.

On one hand, Amazon is diligent and technically sophisticated. Because Amazon runs one of the web’s largest affiliate programs, Amazon is necessarily familiar with affiliate fraud. And Amazon has ample incentive to catch affiliate fraud: Every dollar paid to fraudulent affiliates is money completely wasted, coming straight from the bottom line.

On the other hand, we have observed this same affiliate cheating Amazon for three months nonstop. All told, we’ve seen this affiliate rotating through 49 different Associates IDs. If Amazon had caught the affiliate, we would have expected the affiliate to shift away from any disabled affiliate accounts, most likely by shifting traffic to new accounts. Of the 28 Associates IDs we observed during February 2012, we still saw 6 in use during May 2012 (month-to-date) — suggesting that while Amazon may be catching some of the affiliate’s traffic, Amazon probably is not catching it all.

A further indication of the affiliate’s earnings comes from the affiliate’s willingness to incur out-of-pocket costs to buy media (AdSense placements from Google) with which to deliver Amazon cookies. As best we can tell, Amazon is the affiliate’s sole source of revenue. Meanwhile, the affiliate must pay Google for the display ad inventory the affiliate receives. These direct incremental costs give the affiliate a clear incentive to cease operation if it concludes that payment from Amazon will not be forthcoming. From the affiliate’s ongoing actions we can infer that the affiliate finds this scheme profitable — that its earnings to date have exceeded its expenses to date.

How profitable is this affiliate’s attack? Conservatively, suppose 40% of users are Amazon shoppers and make an average of four purchases from Amazon per year. Then 0.4*4/365=0.44% of users are likely to make purchases from Amazon in any given 24-hour period. Suppose the affiliate buys 1,000,000 CPM impressions from Google. Then the affiliate will enjoy commission on 0.44%*1,000,000=4,384 purchases. At an average purchase size of $30 and a 6.5% commission, this would be $8,547 of revenue per million cookie-stuffing incidents. How much would the affiliate have to pay Google for 1,000,000 CPM impressions? We’ve seen this affiliate on a variety of sites, but largely sites in moderate to low-priced verticals. At $2 CPM, the affiliate’s costs would be $2,000 — meaning the affiliate would still be slightly profitable even if Amazon caught 3/4 of its affiliate IDs before the first payment!

We alerted our contact at Amazon Associates to our observations. We will update this post with any information Amazon provides.

Earnings and Ratings at Google Answers

Edelman, Benjamin. “Earnings and Ratings at Google Answers.” Economic Inquiry 50, no. 2 (April 2012): 309-320. (draft as first circulated in 2004.)

I analyze all questions and answers from the inception of the Google Answers service through November 2003, and I find notable trends in answerer behavior: more experienced answerers provide answers with the characteristics askers most value, receiving higher ratings as a result. Answerer earnings increase in experience, consistent with learning on the job. Answerers who focus on particular question categories provide answers of higher quality but earn lower pay per hour (perhaps reflecting a lack of versatility). Answers provided during the business day receive higher payments per hour (a compensating differential for working when outside options are most attractive), but more experienced answerers tend to forego these opportunities.

Search My Logs of Affiliate Fraud

Since 2004, I’ve been tracking and reporting all manner of rogue affiliatesusing spyware and adware to cover competitors’ sites; using trickier spyware and adware to claim commission on merchants’ organic traffic; typosquatting; stuffing cookies through invisible IFRAME’s and IMG’s, banner ads, and even hacked forum sites; and the list goes on. I now have automation catching these practices in ever-increasing quantities.

While I’ve written up dozens of rogue affiliates on this site and in various presentations, today Wesley Brandi and I are introducing something better: query-based access to our records of affiliate fraud targeting top affiliate merchants. Enter a merchant’s domain name, and we’ll tell you how much affiliate fraud we’ve seen targeting that domain — handy for merchants wanting to check whether their program is clean, and for affiliates wanting to confirm the trustworthiness a program they’re considering promoting. We’re not currently posting details of the specific perpetrators, but we have affiliate ID numbers, domain names, and packet log proof on file for each violator, and we can provide these upon request.

Take a look:

Affiliate Fraud Information Lookup
(2015 update: service no longer operational)

Using Internet Data for Economic Research

Edelman, Benjamin. “Using Internet Data for Economic Research.” Journal of Economic Perspectives 26, no. 2 (Spring 2012): 189-206.

The data used by economists can be broadly divided into two categories. First, structured datasets arise when a government agency, trade association, or company can justify the expense of assembling records. The Internet has transformed how economists interact with these datasets by lowering the cost of storing, updating, distributing, finding, and retrieving this information. Second, some economic researchers affirmatively collect data of interest. Historically, assembling a dataset might involve delving through annual reports or archives that had not previously been organized into a format ready for research: in some cases, surveying stores, factories, consumers, or workers, or in other cases, carrying out an experiment. For researcher-collected data, the Internet opens exceptional possibilities both by increasing the amount of information available for researchers to gather and by lowering researchers’ costs of collecting information. In this paper, I explore the Internet’s new datasets, present methods for harnessing their wealth, and survey a sampling of the research questions these data help to answer.

Advertising Disclosures: Measuring Labeling Alternatives in Internet Search Engines

Edelman, Benjamin, and Duncan S. Gilchrist. “Advertising Disclosures: Measuring Labeling Alternatives in Internet Search Engines.” Information Economics and Policy 24, no. 1 (March 2012): 75-89.

In an online experiment, we measure users’ interactions with search engines, both in standard configurations and in modified versions with clearer labels identifying search engine advertisements. In particular, for a random subset of users, we change “Sponsored links” or “Ads” labels to instead read “Paid Advertisements.” Relative to users receiving the “Sponsored link” or “Ad” labels, users receiving the “Paid Advertisement” label click 25% and 27% fewer advertisements, respectively. Users seeing “Paid Advertisement” labels also correctly report that they click fewer advertisements, controlling for the number of advertisements they actually click. Results are most pronounced for commercial searches and for vulnerable users with low education and little online experience.