Google Discovery Violations in Epic Games v. Google and In Re Google Play Store Antitrust Litigation

This post is part of Revisiting Litigation Alleging Google Discovery Violations.

Epic Games, Inc. v. Google LLC / In Re Google Play Store Antitrust Litigation – docket.  3:20-cv-05671-JD (N.D. Cal.).  /and/ In Re Google Play Store ANtitrust Litigation – docket. 3:21-md-02981-JD.

Filed August 13, 2020.  First proceedings as to discovery violations: October 13, 2022.

Case allegation: Google acquired a monopoly in Android in-app payments, and Google unlawfully maintains a monopoly in Android mobile app distribution and payments.  Complaint.

Case disposition: Jury ruled in Epic’s favor.  Remedy on appeal to Ninth Circuit.

Plaintiffs’ Motion for Sanctions.

On the scope of deleted documents: “Google has destroyed—irretrievably—an unknown but undoubtedly significant number of communications by its employees about relevant business conversations, including on topics at the core of this litigation. Google permanently deletes Google Chats1 every 24 hours—and did so even after this litigation commenced, after Plaintiffs repeatedly inquired about why those chats were missing from Google’s productions, and after Plaintiffs submitted a proffer on this exact issue at the Court’s direction.”

On autodeletion: “Disabling an autodeletion function is universally understood to be one of the most basic and simple functions a party must do to preserve [Electronically Stored Information]” (quoting and citing DR Distribs).  “It is difficult to imagine a litigant better situated to prevent automatic deletion on its own platforms than Google. When Google, whose stated mission is to ‘organize the world’s information and make it accessible,’ irretrievably destroys information despite multiple warnings, its conduct is intentional.”

Google’s Opposition to Motion for Sanctions.  “Google took reasonable steps to preserve relevant ESI. … Rule 37(e) ‘does not call for perfection’.”  “Plaintiffs do not show they suffered any prejudice. … Plaintiffs fail to explain why any unique evidence regarding their claims would be found in chats rather than in the massive corpus of contracts, emails, presentations, strategy documents, and transactional data produced by Google.”

Partial transcript of January 12, 2023 hearing including hearing exhibits.   Additional transcript section.

Judge Donato on Google’s failing to preserve chats: “I think there’s little doubt on the evidence that we’ve heard so far that Chat, Google’s Chat function could, in fact, have contained evidence relevant … to this case. … Google did not systematically preserve those chats but, instead, left the preservation of chats to the discretion of each individual who received a hold notice. … Google never monitored the chats to see if relevant evidence was possibly being lost. … I’m concerned about all this for a variety of reasons [including] at our very first case management conference in October of 2020, Docket Number 45, Google represented to me that it had taken all appropriate steps to preserve all evidence relevant to the issues reasonably evident in this action. I’m finding that representation to the Court to be hard to square with what appears to have been failure to preserve the chats. … [I]f Google didn’t intend to preserve the chats, they should have told me about that in October of 2020.”

On remedy: “I’m not going to let Google get away with this is. There is going to be a substantial trial-related penalty”

Plaintiffs’ Reply ISO Motion for Sanctions.

On deletions generally: “Google’s argument that its efforts were reasonable is irreconcilable with the systematic and avoidable destruction of relevant Chats as well as its continued failure to explain why it did not suspend automatic deletion, including after being expressly put on notice.”

On prejudice: “Google’s contention that Plaintiffs have not suffered prejudice is refuted by evidence showing that Chats are just as substantive as (and often more candid than) email.”

On intent: “Google’s argument that it lacked the requisite intent ignores the facts that Google still, to this day, continues the wholesale destruction of Chats, that Google withheld information about the company’s destruction of Chats for months, and that Google’s custodians intentionally divert sensitive conversations to Chat to avoid discovery.”  “Google employees intentionally divert sensitive conversations to ‘history off’ Chats, and chide others for honesty in ‘history on’ Chats.”  “If Google truly believed that the wholesale destruction of all communications on one of the two communications platforms its employees routinely use was reasonable, it could have raised this position with Plaintiffs.  It did not.”  “Google’s conduct has the purpose of providing its employees a place to communicate free from discovery in litigation.”

Partial transcript of January 31, 2023 hearing.  Judge Donato: “It’s plain as day to any objectively reasonable lawyer, any objectively reasonable lawyer, that Chat is going to contain possibly relevant evidence.”

Plaintiffs’ Supplemental Brief on Google’s Chat Production.  “Google employees, including those in leadership roles, routinely opted to move from history-on rooms to history-off Chats to hold sensitive conversations, even though they knew they were subject to legal holds … even when discussing topics they knew were covered by the litigation holds in order to avoid leaving a record that could be produced in litigation..”  Provides specific examples of substantive discussions in chat.

As to the reason why Google employees use chat, quoting from a preserved message: “Historically (ha) [Google employees] have history off so that [they] can speak (more) freely.” “[H]istory is a liability.” “Dont editorialize/comment in this group chat because it is long-lived.”

When employees realized history was on, they joked about turning it off: “heeeey . . . also just realized our history is [on] . . . can we turn it off? haha.”

As to discovery violations by Google CEO Sundar Pichai: “In one Chat, Mr. Pichai began discussing a substantive topic, and then immediately wrote: “also can we change the setting of this group to history off. Then, nine seconds later, Mr. Pichai apparently attempted (unsuccessfully) to delete this incriminating message. When asked under oath about the attempted deletion of the message, Mr. Pichai had no explanation, testifying ‘I definitely don’t know’ and ‘I don’t recall.'” (citing Pichai deposition transcript)

Quotes from specific employees evading document retention obligations:

Bill Richardson: “Historically (ha) we have history off so that we can speak (more) freely.”

Margaret Lam and Ethan Young: “I talk about RSA related things all day and I don’t have history on for all my chats :)”   “we cannot delete it. I am also on multiple legal hold” “Ok maybe I take you off this convo”

Margaret Lam and Tim McDowell: “can I ask you to turn off history :)” “sure, I understand just feeling forgetful”

Margaret Lam: “do you mind turning history off”

Margaret Lam: “would it be too much to ask you to turn history off?” “lots of sensitivity with legal these days :)”

Margaret Lam and Shadia Walsh: “also just realized our history is [on]. can we turn it off? haha” “yes let’s turn it off”

Ambarish Kenghe: “Folks, *Please do consider if we nee to start a HISTORY OFF chat for this?* Threaded chats you can’t turn off history”

Talia Fernandes and Romin Malkan: “Since this is a sensitive discussion, I’m going to move it to a smaller room” “Group chat.. with the history off ;)”

Lyra Schramm: “should we have history off for this?”

Meg Campbell: “It needs to be history off”

Santiago Scully: “if super sensitive you need to use a GVC because  they could look at your recent ping history and that could go into court”

Divya Chandra to Otto Jan: “I am on legal hold.” “Prefer to keep chat history off.” “i’ll turn off” [chat ends]

Edvill@, Sherle@, and Tris Warkentin: “quick aside – do we want history on?” “NO” “History is on. I suggest everyone leave the room and create a new one with history off. I am happy to punt everyone out”

Cdimon@: “please do not share sensitive information here where possible. Rather, you can flag documents, email threads, etc. for
attention. Until we fix the room architecture, content here is searchable/discoverable within the corp.”

Marcel Folaron: “We can’t turn history on due to potential sensitive information being shared”

Nentl@ and Thao Nguyen: “I’d imagine history is intentionally off” “Yes, history is intentionally turned off for all our war rooms.”

Brandon Barbello and Arpit Midha: “history is a liability” “the more we chat the more threads we need and the more history on gets dangerous”

Stephan Somogyi: ” if we create a new room from scratch we can disable history. We should find a way to do that rather than continuing in perma-history mode”

Tian Lim: “not sure why history is on for our chat, but can you turn it off”

Tian Lim: “hey history is on in this chat, needs to be off”

Court’s Findings of Fact and Conclusions of Law re Chat Preservation.

“Google trains employees to ‘communicate with care’.”  (Training materials.)  “[I]nternal communications actively expressed concerns about the possibility of disclosure in litigation and the risks of preserving Chats” (quoting seven specific employee discussions on this subject, e.g. “should we have history off for this?”). Employees specifically sought to evade retention obligations: “please do not share sensitive information here where possible . . . Until we fix the room architecture, content here is searchable/discoverable.”

On efficacy of Google’s approach to production and alternatives:  “Google has the technical ability to set Chat history to ‘on’ as the default for all employees who are subject to a legal hold, but it chooses not to.”  “Google did not check to see if custodians were actually preserving relevant Chats as directed by the hold notice, and did nothing in the way of auditing or monitoring Chat preservation.”  “Overall, the record demonstrates that Google employees who received a litigation hold in this case were unable or unwilling to follow the Chat preservation instructions, and sometimes disregarded the instructions altogether.”  Quotes a two-page chat between Google employees, which one recognizes as requiring preservation, to which the other says he will “take you off” the discussion rather than proceed with preservation.  In another quoted chat, one employee said he was “on legal hold” but nonetheless “prefer[red] to keep chat history off”.

On scope and intent: Says Google “fell strikingly short” as to preservation, calls Google’s tactic “troubling” and a “worrisome” “substantial problem”, and says Google “did not reveal [its practices] with candor or directness to the Court or … plaintiffs.”  Criticizes Google’s efforts to “downplay the problem” including its “dismissive attitude ill tuned to the gravity of its conduct.”  Criticizes Google’s false claim that it was unable to change default chat history settings for individual employees, which the court says was “not truthful”.  Concludes that “Google did not take reasonable steps to preserve electronically stored information that should have been preserved.” As to “intentionality,” concludes “that Google intended to subvert the discovery process, and that Chat evidence was “lost with the intent to prevent its use in litigation” and “with the intent to deprive another party of the information’s use in the litigation.”  Rejects Google’s arguments about impact of the failure to preserve.

On remedy: “The determination of an appropriate non-monetary sanction requires further proceedings. The Court fully appreciates plaintiffs’ dilemma of trying to prove the contents of what Google has deleted. Even so, the principle of proportionality demands that the remedy fit the wrong, and the Court would like to see the state of play of the evidence at the end of fact discovery. At that time, plaintiffs will be better positioned to tell the Court what might have been lost in the Chat communications.”

On monetary sanctions: “[I]t is entirely appropriate for Google to cover plaintiffs’ reasonable attorneys’ fees and costs in bringing the Rule 37 motion, including the joint statement that preceded the motion and the evidentiary hearing and related events.”

Plaintiffs’ Proposed Remedy for Destruction of Evidence.

On retention: “The record contains example after example of Google employees beginning to discuss topics highly relevant to this litigation and then quickly agreeing to turn history off to trigger automatic deletion and shield further discussion from discovery. … These interrupted Chats are the tip of the iceberg and are powerful evidence of key information—which by any reasonable inference is information that Google thought would be harmful to its legal position if disclosed—being deliberately and permanently destroyed, and thus hidden from Plaintiffs and the Court.”  Says the destroyed information was “the most candid and unsanitized discussions regarding Google’s anticompetitive conduct.”  Quotes additional specific examples of employees turning off chat to avoid document preservation, including Google CEO Sundar Pichai.

On privilege Calls out “fake privilege” in which Google “encouraged” employees “to copy… counsel to apply attorney-client privilege” although not genuinely seeking legal advice.

On remedy: Seeks a permissive adverse inference, directing the jury that it “may” infer that Google destroyed evidence which would have been harmful to Google and helpful to Plaintiffs.

Google’s Opposition to Plaintiffs’ Proposed Remedy.  Criticizes Plaintiffs’ proposal as “severe.”  “Plaintiffs’ submission does not show that Google’s post-August 13, 2020 chat preservation practices deprived them of evidence that would have strengthened their case.”  “Plaintiffs offer nothing more than rank speculation that any missing chats would have been relevant—let alone helpful—to their case.”  “The vast majority of the conduct discussed in Plaintiffs’ motion took place before this lawsuit was filed—in many cases years before this lawsuit was filed—and therefore any chats on these topics would have been permissibly deleted pursuant to Google’s document retention policy long before Google’s preservation obligations arose.”  “Plaintiffs say that [certain] chats would have revealed Google’s subjective motivation for the challenged conduct, but the inquiry in this antitrust case focuses on effect, not intent.”

Plaintiffs’ Reply ISO Proposed Remedy.  “The time has come for Google to bear the consequences of its longstanding, deliberate policy of instructing its employees to conduct their most sensitive business communications in ‘history off’ Chats and then destroying those Chats despite pending litigation to which they were relevant, including this case and related investigations and regulatory actions.”  “Plaintiffs’ requested remedy is appropriately tailored to attempt to ameliorate the prejudice and harm resulting from Google’s calculated choice to evade discovery obligations and destroy evidence.”  “Plaintiffs have shown—as far as is possible without the benefit of the documents Google destroyed—that the thousands upon thousands of deleted Chats contained probative evidence that would have supported Plaintiffs’ case.”  “Google’s plea for ‘lesser measures’ fails because Google has not identified any ‘lesser measures’ that would be sufficient to redress Plaintiffs’ loss.”

Remedy in pretrial order.  Orders a permissive adverse inference jury instruction, consistent with Plaintiffs’ proposal.

Evidentiary hearing as to discovery violations summarized in Epic’s Opposition to Google’s Renewed Motion for Judgment as a Matter of Law under Rule 50(B) or for a New Trial under Rule 59.  “Trial testimony from Google witnesses … confirmed the destruction of hundreds of thousands of Chats.”  Judge called Kent Walker’s testimony “evasive” and “materially inconsistent with testimony given by Google’s witnesses”, saying his testimony “did not do anything to assuage [the Court’s concerns.”  Judge remarked that “a mandatory adverse inference instruction would be amply warranted.”  Judge continued: “this presents the most serious and disturbing evidence I have ever seen in my decade on the bench with respect to a party intentionally suppressing potentially relevant evidence in litigation” and remarked on “rampant and systemic culture of evidence suppression at Google.”  Judge said he intended to conduct his own independent investigation into “who is responsible within Google for tolerating this culture of suppression.”

Order re Google’s Renewed Motion for Judgment as Matter of Law or for New Trial in EPIC Case.

As to abuse of privilege designations: “[M]ore evidence emerged at trial of a frankly astonishing abuse of the attorney-client privilege designation to suppress discovery.  CEO Pichai testified that there were occasions when he ‘marked e-mails privileged, not because [he was] seeking legal advice but just to indicate that they were confidential,’ as he put it. He knew this was a misuse of the privilege.  Emily Garber, a Google in-house attorney, testified that there was a practice at Google of ‘loop[ing] in’ a lawyer based on a ‘misapprehension about the rules of privilege,’ and that Google employees ‘believed that including [an in-house lawyer] would make it more likely that the email would be considered privileged.’  Garber called this ‘fake privilege,’ a practice that she appears to have found amusing rather than something a lawyer should have put an immediate and full stop to.  On this record, there was no error in the Court’s evidentiary ruling that Epic could ‘present fake privilege’ and make arguments to the jury about it.”

As to adverse inference: “The Court determined after an evidentiary hearing held before trial that Google had willfully failed to preserve relevant Google Chat communications, and allowed employees at all levels to hide material evidence. The evidence presented at trial added more fuel to this fire. As discussed, Google in-house attorney Garber testified about the company practice of asserting a fake privilege to shield documents and communications from discovery. Other witnesses also amplified the seriousness and pervasiveness of Google’s preservation abuse.  For example, Google employee Margaret Lam, who worked on RSA issues, said in a Chat message that she didn’t have a specific document because ‘competition legal might not want us to have a doc like that at all :).’ She was a party to other Chats where, in a discussion about MADA, she asked to turn history off because of ‘legal sensitivity’; she requested to turn history off in a different conversation about RSAs, so there would be no ‘trail of us talking about waivers, etc.’”  “Overall, there was an abundance of pretrial and trial evidence demonstrating ‘an ingrained systemic culture of suppression of relevant evidence within Google.’”

As to the Court’s evidentiary hearing to evaluate a mandatory adverse inference about Google’s suppression of evidence: “The results of this hearing were disappointing. Google’s chief legal officer, Kent Walker, was the main witness. Despite the seriousness of these issues, and the likelihood that they could affect other litigation matters where Google is a party, Walker showed little awareness of the problems and had not investigated them in any way. Much of his testimony was in direct opposition to the facts established at the prior Google Chat hearing. Overall, Walker did nothing to assuage the Court’s concerns.” “Even so, the Court took the conservative approach of permitting the jury to make an adverse inference rather than requiring it to.” “Google’s complaints about the inference instruction are wholly misdirected. It has not provided anything close to a good reason to conclude otherwise.”

Add Viacom to the List of Google Discovery Victims

Three separate judges recently remarked on Google’s failure to preserve and produce documents relevant to antitrust claims against Google.  (More here.)  In response, critics asked the State Bar of California to investigate Google General Counsel Kent Walker for violating the California Professional Code of Conduct in counseling Google to destroy records to multiple ongoing lawsuits, including his 2008 memo instructing employees to “avoid inadvertent retention” of communications that might be “used against” Google in litigation.

From the timeline, one might think Google’s document preservation practices were perfect until the recent matters.  Note memo written in 2008 versus litigation in 2023-2024 questioning document retention.  If there were no discovery grievances during that fifteen-year period, maybe this is the first time Google failed to conduct itself properly during discovery – or at least the first time anyone figured out the problem and called Google out on it.  If so, maybe this is genuinely some kind of mistake, and maybe the sanction on Google should be correspondingly reduced.

Not so.  I have been following Google litigation for longer than most.  When I read the recent briefs and decisions about Google’s failure to preserve documents, I immediately thought of a different episode, a decade and a half ago, when Google failed to preserve documents relating to YouTube despite knowing about likely litigation.

Google’s failure to preserve documents relating to copyright infringement at YouTube

When Google considered acquiring YouTube, copyright concerns loomed large.  Analyzing a sample of videos from YouTube, Credit Suisse bankers found that more than 60% of YouTube video views were “premium” (their euphemism for copyright), and only 10% of those were licensed.  In this period, YouTube faced competition from, among others, Google Video—which took genuine steps to protect copyright.  Seeing YouTube traffic soar while they struggled, Google Video staff were unsparing in their criticism of YouTube’s approach to copyright, calling YouTube, “a rogue enabler of content theft” “completely sustained by pirated content” “trafficking mostly illegal content”.  (See also Google’s slides about copyright risks and rights-holder criticism.)  Concerned about a “precedent-setting lawsuit” about copyright, Google held back a 12.5% escrow from the YouTube acquisition price to cover possible copyright liability.  With Google’s own staff explicitly criticizing YouTube’s approach to copyright, and Google’s term sheet discussing copyright risks, copyright litigation was not just foreseeable; it was actually foreseen, then measured and priced.

Under longstanding law, when litigation is reasonably foreseeable, a party is obliged to preserve documents for use as evidence.  (See e.g. Silvestri v. General Motors, 271 F.3d 583, 590.)  From the start, copyright infringement at YouTube was far beyond that modest standard.  So from the first moment Google began to consider acquiring YouTube, Google should have preserved all relevant documents.  Certainly when the acquisition was completed, with copyright concerns still looming large and threats from rights-holders ongoing, Google was obliged to preserve.

Rather than preserve documents relating to copyright infringement at YouTube, Google did much the opposite.  The most brazen missing emails are those sent and received by then-CEO Eric Schmidt.  In a May 6, 2009 deposition, Schmidt indicated having about 30 computers (page 7 line 10) but configuring them “to not retain my e-mails unless asked specifically” (17:9-10).  Asked whether he deleted them manually versus through some sort of automation, Schmidt said “the answer would vary” (18:17-18).  He said he would “delete or otherwise cause the emails that I had read to go way as quickly as possible” (18:24 to 19:2).  Asked whether he began to preserve documents after a copyright lawsuit was filed alleging widespread infringement at YouTube, Schmidt said “I did change my practice after this lawsuit was filed and I was notified” but refused to reveal details which he indicated were attorney-client privileged.  Despite the possible change to Schmidt’s retention practice after the Viacom case was filed, searching his email for responsive documents yielded only 19.  (Hohengarten declaration ¶266)  Google’s acquisition of YouTube was its largest transaction to date.  It is inconceivable that the CEO of Google would have just 19 emails pertaining to its largest acquisition.  By his own admission, Schmidt “deleted or otherwise cause[d]” to be deleted the huge number of otherwise-responsive emails.  Thousands?  Tens of thousands?  Meanwhile, if he changed his document retention practice after the lawsuit was filed, why aren’t there more documents from that period?  He wouldn’t say, and it seems the world will never know.

In principle, excellent memory could stand in for the deleted emails.  The Google executive triumvirate—Schmidt, Page, and Brin—had no shortage of intellectual firepower, and arguably they’d be expected to have correspondingly excellent memories.  In fact, Business Insider specifically credits co-founder Larry Page with a “photographic memory.”  Yet in his deposition about the YouTube acquisition, Page said he could not remember YouTube’s copyright procedures, whether Google wanted an indemnification from YouTube for copyright liability, whether Google executives discussed infringing content at YouTube, whether he reviewed due diligence about the acquisition of YouTube, whether he attended the board meeting when the Google board approved acquiring YouTube, or even whether he was in favor of the purchase or against it.  To these questions among many others, Page remarked “I don’t recall” and the like, over and over — by my count, 132 times.  See the questions and verbatim quotes of Page’s answers.

(Incidentally, even Page’s deposition transcript almost didn’t see the light of day.  Initially, this deposition was filed under seal in Viacom v. Google litigation as Hohengarten declaration part 11 (docket #222) attachment #20, redacted in full at Google’s request.  See also docket #247, ordering that redaction.  But in rechecking the docket incidental to this article, I found later docket #324, which provides the transcript.  Last month I uploaded these files to Courtlistener for public archival and review.  Page’s deposition is Attachments 5, 6, and 7.  Until I uploaded Page’s deposition transcript to Courtlistener, these files lingered in PACER – unindexable by web search, available only for those who knew exactly where to look, knew how to log in, and were willing to pay per-page access fees.)

The memory of YouTube co-founder Chad Hurley was equally poor, but newly revealed in Attachments 2-4 in the same docket #324.  Hurley had no difficulty recalling most aspects of product strategy, negotiation strategy, and distribution strategy.  However he was unable to recall multiple subjects that were sensitive in light of the litigation: the basis of Google’s valuation of YouTube (part 1 pages 29-32), entire movies uploaded to YouTube (part 2 pages 12-16), copyright enforcement policies (part 2 pages 19-20 and 26-30), and why YouTube removed ads from Watch Pages (part 2 pages 23-24)—Hurley said he couldn’t recall.

Separately, Viacom calls out Google’s failure to produce relevant documents within its possession.  For example, on March 22, 2006, YouTube founder Jawed Karim presented to the YouTube board of directors:

As of today episodes and clips of the following well-known shows can still be found: Family Guy, South Park, MTV Cribs, Daily Show, Reno 911, Dave Chapelle. This content is an easy target for critics who claim that copyrighted content is entirely responsible for YouTube’s popularity. Although YouTube is not legally required to monitor content (as we have explained in the press) and complies with DMCA takedown requests, we would benefit from preemptively removing content that is blatantly illegal and likely to attract criticism.

Having acquired YouTube, Google should have received the materials previously presented to YouTube’s board.  Yet Google failed to produce this document to Viacom in litigation.  Viacom only received it when Karim, to his credit, produced it after he left YouTube.

Viacom’s Motion for Summary Judgment criticizes Schmidt’s failure to preserve relevant documents, as well as what it calls witnesses’ “serial amnesia” and Google’s other failures to produce.  But Viacom never filed a motion seeking sanctions for spoliation.  The court ultimately granted Google’s motion for summary judgment without ruling on Google’s alleged failure to preserve, witnesses too conveniently forgetful, or any other discovery violation.

Rereading Google’s deficient discovery in the Viacom’s matter, in conjunction with 2023-2024 proceedings about Google discovery violations, to me the picture became increasingly clear.  Google’s decision to evade discovery obligations isn’t a new problem, and it isn’t really a change.  Rather, this problem has been lingering for years, and has surely harmed scores of other litigants—whose ability to prove their cases against Google was stymied by Google falling short of both the productions and the candor required by court rules.  For broader thoughts and more on the other discovery disputes, see Revisiting litigation alleging Google discovery violations.

Google Discovery Violations in Viacom v. YouTube

This post is part of Revisiting Litigation Alleging Google Discovery Violations.

Viacom International, Inc. v. YouTube, Inc. – docket.  1:07-CV-02103-LLS (S.D.N.Y.).

Filed March 13, 2007.  First filing as to discovery violations: March 18, 2010.

Case allegation: “YouTube’s website purports to be a forum for users to share their own original ‘user generated’ video content. In reality, however, a vast amount of that content consists of infringing copies of Plaintiffs’ copyrighted works.”  Complaint.

Case disposition: Summary judgment granted for defendant based on the safe-harbor provisions of the Digital Millenium Copyright Act.  Viacom appealed.  The parties announced a settlement in March 2014 and announced that no money changed hands.

Viacom’s Motion for Summary Judgment at heading “Defendants Cannot Walk Away from Their Contemporaneous Internal Documents.”

As to YouTube’s knowledge of copyright infringement: “The internal emails and memoranda of YouTube’s founders and Google’s senior executives discussed above make a compelling and indisputable record of Defendants’ intent to use infringing videos clips to build the YouTube business.”  Viacom’s Motion, pages 5 to through 21, offer a dozen quotes supporting this contention, such as “we’re hosting copyrighted content” and “we need views, [but] I’m a little concerned with the recent Supreme Court ruling on copyright content.”  “We’re going to have a tough time defending the fact that we’re not liable for the copyrighted material on the site because we didn’t put it up when one of the co-founders is blatantly stealing content from other sites and trying to get everyone to see it.”

As to YouTube founder documents: “Defendants fail[ed] to preserve and produce many key documents.”  “Almost none of … the internal emails and memoranda of YouTube’s founders … were produced by Google or YouTube, which claims they were all lost.”  “Chad Hurley, a founder and YouTube’s Chief Executive from its inception to today, revealed for the first time [at] his deposition that he ‘lost all’ of his YouTube emails for the key time period of this case.”

As to documents from Google CEO Eric Schmidt: He “claims to use and email from ‘probably 30’ different computers … [y]et [his] search for responsive materials [pertaining to YouTube’s policies and practices and Google’s acquisition] ‘yielded 19 documents.’  Schmidt explained: ‘[i]t has been my practice for 30 years to not retain my emails unless asked specifically. … ‘It was my practice to delete or otherwise cause the emails that I had read to go away as quickly as possible’.”

On witness memories: Viacom remarks on “the ostensible memory failures of their key executives when deposed.”  YouTube founder Chad Hurley “developed serial amnesia” in his deposition.  Google go-founder Larry Page “essentially disclaimed memory on any topic relevant to this litigation, even including, for example, whether he was in favor of Google’s acquisition of YouTube, even though it was Google’s largest corporate transaction to date and viewed as transformative to its business.”   (See excerpted deposition transcript.)  “This Court can decide whether these key executives and witnesses behaved with the level of candor and respect for the legal process that this Court has a right to expect from senior executives of important public companies.”

Viacom never filed a motion for sanctions for spoliation.

***

Deposition of Google then-CEO Eric Schmidt including his explanation why he had just 19 emails pertaining to YouTube (“[i]t has been my practice to not keep my e-mails”) as well as claiming not to remember numerous facts about YouTube and Google’s strategy in video.

Deposition of Google co-founder Larry Page with analysis of his recollection.  By my count, Page said he did remember when asked about 132 aspects of Google and YouTube practices.

Deposition of YouTube co-founder Chad Hurley (parts 1, 2, 3).  Hurley had no difficulty recalling most aspects of product strategy, negotiation strategy, distribution strategy, and other topics.  However he was unable to recall multiple subjects that were sensitive in light of the litigation: the basis of Google’s valuation of YouTube (part 1 pages 29-32), instances of entire movies uploaded to YouTube (part 2 pages 12-16), copyright enforcement policies (part 2 pages 19-20 and 26-30), and why YouTube removed ads from Watch Pages (part 2 pages 23-24).

Google’s Opposition to Plaintiffs’ Motions for Partial Summary Judgment.  Makes no mention of Viacom’s allegations of documents inexplicably lost, of emails deleted, or witness’ memories.

Order of June 23, 2010 granted Google’s motion for summary judgment without ruling on any of the discovery issues raised in Viacom’s motion.

Eric Schmidt’s Missing Emails and Memory Problems

In Viacom v. Google, Viacom took the deposition of Google then-CEO Eric Schmidt.  In sworn testimony, Page explained why he had just 19 emails pertaining to YouTube, then Google’s largest acquisition.  (Source: Hohengarten declaration ¶266)  Schmidt also repeatedly reported  not remembering events and facts relating to YouTube.

Viacom remarked in associated briefing: “This Court can decide whether these key executives and witnesses behaved with the level of candor and respect for the legal process that this Court has a right to expect from senior executives of important public companies.”

Schmidt’s testimony (as excerpted by Viacom).

As to document retention: “[Some] people over time either delete or lose some of that e-mail. It has been my practice for 30 years to not retain my e-mails unless asked specifically.”  “[I]t has been my practice to not keep my e-mails.”  “Q: And is this on some sort of automatic system where they are deleted in the ordinary course over some ordinary period of time?  A: Depending on the e-mail system and the company and so forth, the answer would vary.”  As to document retention while at Google: “It was my practice to delete or otherwise cause the e-mails that I had read to go away as quickly as possible.  Q: Within days?  A: Yes.”

Separately, Schmidt claimed he didn’t remember multiple aspects of Google’s strategy in video.  A representative example:

Q: [Y]ou are aware, I assume, that the acquisition agreement contains an indemnification provision relating to copyright lawsuits?  … A: Yes. Q: And was that discussed by the board …?  A: Yes.  Q: And do you remember that discussion, sir?  A: No.”

Other subjects Schmidt didn’t remember.

Larry Page’s Bad Memory of YouTube and Google Video

In Viacom v. Google copyright litigation, Viacom took the deposition of Google co-founder Larry Page.  In a sworn deposition, Page reported remembering very little about YouTube and Google video.

Viacom remarked in associated briefing: “This Court can decide whether these key executives and witnesses behaved with the level of candor and respect for the legal process that this Court has a right to expect from senior executives of important public companies.”

Page’s full deposition is available in parts 1, 2, 3.

In the list below, I summarize the substantive questions asked of Page, and quote his verbatim response to each.  By my count, there were 132 distinct instances in which Page could not recall the answer to a question.  (In this  count and list, I excluded questions about whether Page received documents, excluded questions about why the reasons why he doesn’t remember, and excluded or grouped most repeat questions.)  From my line-by-line review of Page’s deposition:

The 2008 Walker Memo

In recent competition proceedings against Google, multiple courts have discussed whether Google staff were forthright in their remarks (not to mention whether the company preserved and produced documents in accordance with court rules).  See Google Discovery Violations.  Discussions often turn back to a memo sent by Google General Counsel Kent Walker as well as Bill Coughran (then SVP of Engineering at Google).  In the dockets for these cases, this document appears in unformatted monospace with broken spacing between paragraphs, and in image form without indexing or searchability.  I am therefore reposting it here in plain text.

Sent: September 16, 2008 2:09 PM
Subject: Business communications in a complicated world
Confidential/Please Do Not forward

Googlers –

As you know, Google continues to be in the midst of several significant legal and regulatory matters, including government reviews of our deal with Yahoo!, various copyright, patent, and trademark lawsuits, and lots of other claims. Given our continuing commitment to developing revolutionary products and doing disruptive things, we’re going to keep facing these kinds of challenges. So we’ve got two requests of you and one change to announce.

First, please write carefully and thoughtfully. We’re an email and instant-messaging culture. we conduct much of our work online. We believe that information is good. But anything you write can become subject to review in legal discovery, misconstrued, or taken out of context, and may be used against you or us in ways you wouldn’t expect. Writing stuff that’s sarcastic, speculative, or not fully informed inevitably creates problems in litigation. In your communications, please avoid stating legal conclusions. Speculation about whether something might breach a complex contract, or whether it might violate a law somewhere in the world, is often wrong and rarely helpful. So please do think twice before you write about hot topics, don’t comment before you have all the facts, and direct questions regarding continuing litigation holds and any legal and/or regulatory matters involving Google to the friendly (albeit lawyerly) folks at [redacted]@google.com

Second, remember that these same rules apply not just to Gmail but also to Google Talk and all other forms of electronic communication (for example wiki’s, doc’s, spreadsheets, etc.). We end up reviewing millions of pages of these communications as part of producing documents in regulatory and litigation matters — and we’re working together to streamline and simplify that process.

To help avoid inadvertent retention of instant messages, we have decided to make “off the record” the Google corporate default setting for Google Talk. We’ll also be providing this option to our Google Apps enterprise customers. You should see this new default setting taking effect over the next few days. You will still be able to save Talk conversations that are useful to you — but please remember that “on the record” conversations become part of your (more or less) permanent record and are added to Google’s long-term document storehouse. If you’ve received notice that you’re subject to a litigation hold, and you must chat regarding matters covered by that hold, please make sure that those chats are “on the record”.

Finally, remember that even when you’re “off the record”, your chat partner may be recording the conversation, so always take care with what you write. Thanks for your help and understanding on this. Let one of us know if you have any questions.

Bill Coughran

Kent Walker

See also required “Communicate with Care” training.

See also Antitrust Basics for Search Team (March 2011), recommending word choice for Google employees discussing competition matters.

On geofencing at Polymarket

Last week prediction/betting platform Polymarket was in the news for a Justice Department raid, arising out of Polymarket allegedly accepting trades from US-based traders.  Suppose we stipulate that Polymarket betting is illegal for US users, and Polymarket must keep US users out, to the very best of its ability, to avoid liability under US law.  How exactly would it do so?

This question is familiar for me because my first service as a litigation expert, in February 2000, covered a surprisingly similar subject.  There, Canadian video streamer iCraveTV wanted (or purported to want) its service to be available to Canadians but specifically not to Americans.  I say “purported” because iCraveTV didn’t try very hard, and Americans could access the video easily — as I showed in two declarations as well as oral testimony.  In a 2001 regulatory comment, I pointed out that when there’s something valuable on the Internet that motivated users want to access, users have multiple methods to get the desired access.  My bottom line was if some body of material is so sensitive that a significant country rightly and properly disallows it, by far the easiest approach is to keep that content off the Internet altogether — as, to be sure, an injunction did to iCraveTV.

For Polymarket, the situation is somewhat different.  It seems many people agree that Polymarket is properly allowed in other countries, yet properly disallowed in the US. Fine: reasonable people, and reasonable countries, can disagree.  So, the engineering requirement is a system that robustly separates US users from international users — a geofence.  Importantly, the geofence must be strong enough that even motivated US users can’t climb over.  “Bob the Builder” fans, rejoice: Can we build it?  Yes we can!

Here’s how I’d approach robust geofencing in a scenario like Polymarket, where users register and provide substantial information.

1. Block all users whose network connection indicates a location in the United States.  Check a user’s IP address via standard geolocation services.  Easy enough.

2. Block users based on their registered physical addresses.  In any registration form that requests a street address, or any correspondence or other procedure that requests a street address, a US address is a clear indication of a US affiliation.  Gold standard here is to check that the address is truthful — not just that it exists, but that a person can receive a one-time PIN sent by mail to this address.

3. Block users based on their phone number.  A US phone number similarly indicates a US affiliation.  Check that the user really has this number via a one-time PIN sent by voice or SMS.

4. Block users based on payment mechanisms linked to their account.  Some might expect all Polymarket users to pay with a privacy-protecting payment mechanism such as crypto.  But in fact Polymarket supports debit and credit cards as well as bank transfer.   A payment instrument associated with a US financial institution indicates a US affiliation.

5. Block users based on geolocation in a desktop web browser.  Web pages can request a user’s geolocation using the W3C geolocation API.  Most web browsers ask their users if they want to share location.  If a user does share, and if the location says US, proceed no further.  It may seem implausible that users would voluntarily disclose, but one mistaken click could reveal — and a diligent site should at least try.  Maybe a site should require a user to grant geolocation permission in order to proceed — everyone has to be somewhere, and Polymarket could demand that users reveal.  On this theory, a failed geolocation API request would itself prevent login.

6. Block users based on geolocation in a mobile app.  In a mobile app, it’s much easier to seek a user’s geolocation — a common permission that users are accustomed to granting.  It might seem illogical for a user on a desktop device to have to switch to mobile just to confirm geolocation, but many sites require a switch to mobile for some aspect of security such as a secure photo upload, so this isn’t out of the question.  As usual, users can override device geolocation, but this requires increasing technical skill.

7. Detect proxy servers and VPNs.  A savvy user can use a proxy sever or virtual private network to bounce traffic through a server in a different country, then browse “from” there, with the server relaying requests and responses back and forth.  At first this might seem unworkable for a service like Polymarket: How would they know which IP addresses are used by proxies and VPNs?  But actually they have multiple reasonable paths:

  • The natural starting point is to ask the largest proxy and VPN makers to share their lists of IP addresses.  They may refuse, but the mere act of asking shows an attempt.
  • Test the largest proxy networks and VPNs to find representative IP address ranges (or pay a specialist to do so).  Hands-on testing also creates an opportunity to check whether there’s something unusual about their traffic (such as reverse DNS or a distinctive protocol-level header) that actually gives them away.  (In my testing, this happens surprisingly often.)
  • Look for implausible patterns in user IP address logins.  If a user is purportedly in Frankfurt at 8:00 and Seoul at 8:30, maybe the user is actually in neither place — and is actually bouncing back and forth via proxy or VPN.
  • Use these learnings to find others.  If a user is logging in from an IP address widely used by other users who bounce back and forth, the user is probably on a proxy or VPN.

The logical final step is to require affirmative proof of nexus with a country where Polymarket’s service is lawful.  When creating an account with Wise, I was impressed by their multiple methods of verification — photo ID, proof address, even uploading a picture showing face and ID together.  These must hinder users’ account creation — every step and every click cause users to drop off.  Despite that cost, such methods provide particularly strong proof of a user’s nationality.  And if Wise can do it in their highly-regulated sector (money transmission), it’s hard to see why Polymarket should have lower standards.

I take no position on the wisdom of laws disallowing Polymarket and kin.  And reasonable people may disagree about which of the tactics above should be required — how much a site like Polymarket can be required to inconvenience some of its users, in order to keep US users out.  Fair questions!  But to the basic adversarial question, I answer decidedly in the affirmative: A motivated site operator can keep out most US users, and can make sure that even those who sneak through end up feeling uncomfortable.

A separate challenge is the prospect of a site going through the motions rather than making a good-faith effort (not to mention investing in genuine innovation in this area).  Certainly sites have every reason to tread lightly: Every user means growth, and every user contributes positive expected profit.  So turning away more users means correspondingly weaker economic results.  These factors create a direct economic incentive to look the other way.  Meanwhile, a high-functioning compliance team would need real resources including talented engineers and data scientists.  If I were evaluating whether a site truly did everything possible, I’d want to see real resources invested, multiple methods tried and compared, and multiple imperfect methods used in combination in order to increase overall effectiveness.

Impact of GitHub Copilot on code quality

Jared Bauer summarizes results of a study I suggested this spring.  202 developers were randomly assigned GitHub Copilot, while the others were instructed not to use AI tools.  The participants were asked to complete a coding task.  Developers with GitHub Copilot had 56% greater likelihood of passing all unit tests.  Other developers evaluated code to assess quality and readability.  Code from developers with GitHub Copilot was rated better on readability, maintainability, and conciseness.  All these differences were statistically significant.

What I’m working on

Many of my recent projects are confidential, and it’s not easy to provide public write-ups.  But I can summarize in general terms.  Selected recent matters:

  • Evaluating alternative remedies for proven violations of competition law.
  • Online forensics to determine whether a given publisher/partner is sending legitimate traffic versus malware, invisible traffic, fake clicks, and the like.
  • Measuring the incrementality of ad campaigns to distinguish the genuine incremental benefit, versus the sales that would have occurred anyway.
  • Estimating the market value of IPv4 addresses, and evaluating the impact of rules and restrictions on their transfer.
  • Exploring Excel data glitches including how data can become corrupted inadvertently, and what can be learned from internal Excel data structures.

I’m enjoying combining software engineering, law, economics, and (often) a bit of gumshoe work.  And it’s a delight to always be learning!

My next chapter

I am delighted to announce that I’m returning to more frequent writing on this site.  Closely related, I’ve resumed multiple projects to hold tech goliaths accountable.  Expect future writings and projects exploring all manner of online malfeasance.

Last month I began to serve as an advisor at Geradin Partners, a European law firm best known for its leadership in matters adverse to big tech.  Fully two decades ago, I was already flagging tensions between Main Street and Silicon Valley.  Those ideas took off slowly in the US, but in Europe they moved faster, in no small part thanks to the attorneys now at Geradin Partners.  I’m looking forward to working with them, and their clients, for all manner of projects with a locus in Europe.

More announcements to follow as to other affiliations.