Author: Benjamin Edelman

Posted on

Measuring the Impact of AI on Information Worker Productivity with Donald Ngwe and Sida Peng

Measuring the Impact of AI on Information Worker Productivity. SSRN Working Paper 4648686. With Donald Ngwe and Sida Peng.

This paper reports the results of two randomized controlled trials evaluating the performance and user satisfaction of a new AI product in the context of common information worker tasks. We designed workplace scenarios to test common information worker tasks: retrieving information from files, emails, and calendar; catching up after a missed online meeting; and drafting prose. We assigned these tasks to 310 subjects tasked to find relevant information, answer multiple choice questions about what they found, and write marketing content. In both studies, users with the AI tool were statistically significantly faster, a difference that holds both on its own and when controlling for accuracy/quality. Furthermore, users who tried the AI tool reported higher willingness to pay relative to users who merely heard about it but didn’t get to try it, indicating that the product exceeded expectations.

(Also summarized in What Can Copilot’s Earliest Users Teach Us About Generative AI at Work? at “A day in the life” and “The strain of searching.” Also summarized in AI and Productivity Report at “Copilot Common Tasks Study” and “Copilot Information Retrieval Study.”)

Posted on

Edelman v. Harvard

For 11 years, I was a faculty member at Harvard Business School. I met and exceeded the school’s high standards for research and teaching. I loved my work and won high praise from colleagues and students for my contributions to multiple academic fields, for my teaching, and for my service to the school. I looked forward to continuing my work at HBS for the foreseeable future.

My promotion to tenure was derailed by improper disciplinary proceedings — a kangaroo court that ran roughshod over the governing rules. For example, the rules require the disciplinary committee to share with me (and readers of its report) “the evidence gathered.” Far from providing evidence, the 2017 report attached zero emails, zero transcripts of interviews, and zero other documents. Instead of providing evidence, the committee offered mere summaries of twelve anonymous interview remarks, with both names and contexts intentionally removed, in brazen violation of the requirement to provide “the evidence gathered.” That’s not justice, and it’s clearly not permitted under the applicable rules.

I’m suing Harvard to insist that these proceedings be corrected, in conformance with the rules. I’m not perfect — who is? — but if the proceedings follow the rules, I will clear my name of the incorrect charges, and my candidacy can then be evaluated on its merits.

Case web site including complaint and other documents

Posted on

Alaska Airlines – missing baggage fee disclosures

Complaint. Answer. Reply. Surreply.

Status: Pending.

Summary: Governing regulation requires an airline to provide the exact price for a passenger’s first and second checked bag within the text of an eticket confirmation email, but Alaska did not do so. Furthermore, the regulation requires bag allowance and price information in a booking summary page, but again Alaska did not. Meanwhile Alaska’s Manage Trip page provided an incorrect statement of baggage benefits and fees.

Posted on

American Airlines – price advertising violations (2022)

Complaint. Answer. Reply. Surreply.

Status: Pending.

Summary: The American Airlines Business Extra site misrepresented carrier surcharges as “tax” in violation of governing regulation and prior DOT consent decrees. Furthermore, the site listed “approx” charges rather than the exact amount to be paid. And contrary to governing regulation, the site entirely omitted carrier surcharges from initial fare quotes.

Posted on

Multinationals in the Digital Economy

The Brookings Institution‘s Global Goliaths: Multinational Corporations in the 21st Century Economy includes my chapter Multinationals in the Digital Economy. The lead paragraph:

Modern digital services largely come from multinational corporations (MNCs) whose size and scope are unprecedented.  It has not always been this way.  Just a few decades ago, users typically turned to local firms for most kinds of information technology (IT).  And, historically, software was known for its low barriers to entry and the quick rise of startups and small firms.  This chapter examines the forces contributing to the rise of digital MNCs, as well as the challenges they face. 

Posted on

Revisiting Barlow’s Misplaced Optimism

Revisiting Barlow’s Misplaced Optimism, Symposium for John Perry Barlow, 18 Duke L. & Tech. Rev. 97.

As part of Duke Technology Law Review‘s Symposium for John Perry Barlow, I reflected on the perspective of early Internet luminary John Perry Barlow, the vision he offered, and what I see as the most promising sources of accountability for online behavior. My piece begins:

Barlow’s A Declaration of the Independence of Cyberspace calls for a “civilization of the mind in cyberspace,” and he says it will be “more humane and fair” than what governments have created. Barlow’s vision is unapologetically optimistic, easily embraced by anyone who longs for better times to come.  Yet twenty years later, it’s easy to see some important respects in which reality fell short of his vision.  Alongside the Internet’s many pluses are clickbait, scams, hacks, and all manner of privacy violations.  Ten thousand hours of cat videos may be delightful, but they’re no civilization of the mind.  With a bit of hindsight, Barlow’s techno-utopianism looks as stilted as other utopianism—and equally far removed from reality.

Beyond being overly optimistic about how perfectly the ‘net would unfold, Barlow was also needlessly skeptical of plausible institutions to bring improvements.  He writes: “The only law that all our constituent cultures would generally recognize is the Golden Rule.” But the moral suasion—and practical effectiveness—of the Golden Rule presupposes participants of roughly equal power and status.  It is no small feat to meaningfully consider what Joe User might want from Mega Social Network if the tables were turned and Joe owned the goliath.  As a practical matter, any claim a user has against a goliath requires state institutions to adjudicate and enforce.  When Barlow wrote A Declaration of the Independence of Cyberspace, tech goliaths were much smaller.  Plus, the Internet’s early users were in a certain sense more sophisticated than the mainstream users who eventually joined.  So the gap from little to big was much narrower then, arguably making governments less important in that era.  But as the big get bigger and as the Internet attracts average users who lack the special sophistication of early adopters, governments play key roles—adjudicating disputes, enforcing contracts and beyond.