Measuring Bias in "Organic" Web Search - Benjamin Edelman and Benjamin Lockwood
| Yahoo | Bing | ||
|---|---|---|---|
| Top result | 6.3% | 10.0% | 37.5% |
| Top 3 results | 2.1% | 3.2% | 12.5% |
| First page | 6.9% | 3.1% | 11.6% |
| Yahoo | Bing | ||
|---|---|---|---|
| Top result | 13.5 | 10.0 | NA |
| (0.00) | (0.00) | ||
| Top 3 results | 1.82 | 1.86 | 14.72 |
| (0.06) | (0.07) | (0.00) | |
| First page | 2.27 | 3.22 | 6.89 |
| (0.00) | (0.00) | (0.00) |
Analysis of search bias is complicated by the growing trend to incorporate additional types of listings within search results pages. These panels of rich results are drawn from special-purpose search services (such as image search and news search) and appear among ordinary algorithmic search results. Best-known in this vein is Google's "universal search," but all leading search engines now include rich listings.
Figure 1 shows an example of universal search results from Google for the term "Justin Bieber" (a top trending search according to Yahoo Buzz). Within the first page of these results, Google shows three different universal search result boxes: news, images, and video. As is typical for universal search results, each box features multiple links to a specific kind of content (e.g. news stories or videos). The boxes also feature prominent links to the search service from which the content comes (Google News, Google Images, and Google Video). All told, the algorithmic result column measures 1100 pixels in height, of which 410 pixels (37%) are rich results.
The analysis in the body of our paper ignores universal search results. This omission yields an underestimation of search engine bias. In particular, universal search results disproportionately link to each search engine's own services. For example, in Figure 1, the top-most link in the results pane promotes a Google News search for "Justin Bieber" -- effectively a link to a Google service, but a link not considered in our analysis above. Similarly, in Figure 1, the Google Video universal search result box contains large links to two videos, both from Google's YouTube. Of course these additional services typically include further advertisements -- so retaining users on these properties yields more impressions for Google and greater advertising revenue.
In addition to providing universal search results for news, music, images, and videos, as shown at right, Google also provides rich results for numerous additional categories: books, box office movies, finance, health, maps, patents, products, and scholarly articles. Rich result types shown in underline distinctively link to other Google pages, while results in double-underline link solely to Google pages.
With rich results frequently in first position, Google presents its own material in first position more than any other search engine. See Table 10, top-left cell.
Table 10 uses the same approach as Table 3, but includes rich results. Since rich results tend to link to a search engine's own pages, the odds ratios in Table 10 are all larger than those shown in Table 3. The high percentages for Bing in Table 9, and Bing's high odds ratios in Table 10, are driven mainly by the fact that Bing displays geographic (map) rich results for many portal terms (such as "photos" and "translate"), while Google and Yahoo do not.