How MIT Dragged Uber Through Public Relations Hell

On March 1, a prestigious MIT research center published a Research Brief on a study of rideshare driver compensation, led by a respected Stanford University researcher. The MIT report asserted that half of Uber and Lyft drivers earn no more than $3.37 per hour, after accounting for vehicle ownership and operating costs. The study further concluded that 74% of rideshare drivers earn less than the minimum wage in their state, 30% are actually losing money and 74% of whatever driver profits there are could go untaxed, if drivers claim the standard IRS deduction for vehicle operating costs.

After MIT published its Research Brief, the press was quick to pounce, with brutal, indignation-laden headlines:

• MIT study shows how much driving for Uber or Lyft sucks – TechCrunch
• Uber's New CEO Faces New PR Disaster: MIT Study Reveals Drivers Make Less Than $3.37 an Hour – Inc.com
• Seriously, how are Uber and Lyft drivers surviving? – MIT Technology Review
  
  

Uber quickly published a thoughtful response, rebutting the MIT Research Brief by identifying a key error in its underlying methodology. In response, the MIT team leader, Stephen Zoepf, tweeted a mea culpa, blaming his error on confusing wording in the survey provided by a third party, and on the absence of more complete and transparent data from Uber and Lyft. After recalculating driver earnings using two revised methods, the MIT study author found rideshare driver hourly earnings to be in the range of $8.55 - $10 per hour (similar to many prior published studies), with “only” 41% - 54% of drivers making less than minimum wage in their state.
Ordinarily, this story would end here, with little additional fanfare. Stephen Zoepf isn’t the first university researcher to make an error in publishing preliminary sponsored research findings, and to his credit, Zoepf promptly issued a retraction, if not an apology, after being advised of his mistake.

But this story shouldn’t end here as it provides a useful parable on the powerful forces that promote the creation and viral dissemination of inaccurate information, aka fake news. The insidious dynamics behind this saga need to be told in three parts.

1. The root cause of the errors in the original MIT report was not simply a case of misinterpreting a confusing survey question as its author avers, but reflects a preventable lapse in research integrity, which ignored several red flags that should have signaled errors in the newsworthy but flawed findings before they were published.
2. The press was extremely eager to not only report, but to editorialize on the moral and societal implications of the MIT Research Brief before taking the time to review and vet the full study report or to elicit dissenting views.
3. Uber’s already fragile reputation took a serious hit, and has continued to be harmed even after the original version of the MIT report was retracted.
   
   

MIT’s Epic Fail
The mistake in the MIT Research Brief can be traced to how MIT researchers improperly interpreted responses from 1,121 Uber and Lyft drivers surveyed in January, 2017. In explaining his faux pas, the MIT team lead  benignly states, “I can see how the question on revenue might have been interpreted differently by respondents.” But when over 1,000 respondents interpret a survey question one way, and an experienced MIT research team sees the survey responses in an entirely different light, there has to be a reason.

It’s fair to assume that Uber and Lyft respondents answered the critical survey question driving MIT’s analysis of driver earnings as it was originally posed:“How much money do you make in the average month? Combine the income from all your on-demand activities.”

But the MIT study team inexplicably changed the wording and hence the meaning of this key survey question in their analysis. In their full study report, the MIT researchers state: “Responses to Question 14 ‘How much money do you make in the average month?’ determined monthly revenue for each driver.” By dropping the second part of Question 14, clarifying its reference to only earnings from on-demand activities, the MIT researchers erroneously were led to reduce driver earnings for the 83% of respondents who reported having other sources of income. As a result, MIT’s calculations of gross and net driver earnings were understated by more than a factor of two.

In response to my email to the lead study author, which mirrored Uber’s critique of MIT's study methodology, Stephen Zoepf told me “I’m re-running the analysis this weekend using Uber’s more optimistic assumptions.” But I don’t see this as a case of being more or less optimistic, but simply being accurate, consistent and applying common sense in conducting research to high academic standards.

My bigger concern with this research project – its epic fail – was the failure to recognize obvious red flags or to vet the results with knowledgeable professionals who could have quickly identified methodological errors, prior to publication. But instead, the project team charged ahead to publish its preliminary research findings, ignoring:

• MIT's reported findings on driver earnings were more than 50% lower than the comparable figures reported directly in the very survey used to drive their analysis.
• The MIT team actually used a second methodology to calculate hourly earnings for the small number of respondents that did not respond to the the monthly income question on the survey. Had MIT compared results from their two methodologies for the majority of respondents who did answer all driver income questions, they would have discovered major internal inconsistencies in their published results.
• The same survey that was used to drive MIT’s estimates of punitive driver earnings also reported that only 36% of Uber drivers somewhat or strongly disagreed with the statement: “I am satisfied with my experience driving for Uber.” If driver earnings were really as low as MIT calculated, the team should have expected to find lower driver satisfaction.
• MIT’s reported findings were starkly inconsistent with multiple prior published studies on rideshare driver earnings, all of which reported gross and/or net earnings at least twice as high.
• The researchers did not vet their surprising research findings with Harry Campbell – author of therideshareguy.com, a widely read blog on the rideshare sector – who provided the underlying survey data for the MIT analysis. Campbell told me shortly after seeing widespread press reports, that it was a shame that he wasn’t given a chance to preview MIT's findings prior to publication, because he would have immediately raised a red flag with the reported results.
  
  

Why would an experienced research team fail to detect an error that was so glaringly obvious to others who operate in or follow the rideshare business closely? Perhaps it was caused by a dose of self-serving bias, confirmation bias, ideology, or the excitement of racing to publication with a study that promised to be highly newsworthy. Anyone can make an honest error in conducting research on complex business issues, but failing to do even nominal stress testing or vetting of results is a grave mistake.


The Role of the Press
There are immense pressures on news organizations to avoid getting left behind in reporting breaking news, and this story offered the promise of immense “eye candy” appeal. After all, the study came from a highly trusted source, and presented alarmingly negative findings on a widely known and frequently maligned company.

Shortly after the MIT Center for Energy and Environmental Research (CEEPR) published its Research Brief, the story went viral, precipitating a newsroom arms race to get the shockingly low estimates of rideshare driver earnings out to the public.
When I last checked, a Google search on “MIT study Uber driver earnings” generated over 700,000 search results, undoubtedly exposing the bogus study results to tens of millions of viewers.

It was naïve for the study authors and its sponsor to not recognize the dangers of publishing such a potentially explosive and unvetted study as a working draft to solicit feedback on the approach and findings. The press predictably treated the results as gospel (without the benefit of reading the full report), and broadly condemned Uber’s (and Lyft’s) perceived abusive business practices.

Some news organizations did publish a brief follow-up on Uber’s rebuttal and the MIT team leader’s ensuing retraction, but with far lower impact and reader interest. The damage was done.


Uber’ Public Relations Hell
The public relations harm to Uber as a result of MIT’s initial study release is self evident. But even with Uber’s prompt rebuttal and the study author’s retraction, Uber’s reputation is bound to suffer in the weeks ahead.

One widely read business publication jumped on the initial release of the MIT study findings by headlining: “Uber’s New CEO Faces New PR Disaster.” But when Uber CEO Dara Khosrowshahi tweeted a testy rejection of the flawed MIT study results, the same publication followed with the story: “With a Single, Insulting Tweet, Uber's CEO Just Destroyed Months of Hard Work.” Opprobrium throughout the news cycle on this story has fallen mostly on Uber, rather than  on the instigators of this bogus news firestorm.

Stephen Zoepf’s revised study results are continuing to create negative press for Uber. For example, NPR recently reported that “in each new calculation method a significant percentage of drivers are still earning less than minimum wage in their state — 54 percent in the first case or 41 percent in the second case, according to Zoepf.”

While this statement is factually accurate, it fails to characterize MIT's research findings in proper context. Gig economy jobs provide considerable flexibility that is highly valued by drivers. Comparing Uber and Lyft driver compensation to minimum wage rates is mixing apples and oranges: part time/flexible work vs. full time/inflexible employment.

The MIT study correctly observed that fewer than 20% of drivers  rely exclusively on ridesharing services for their earned income. For most drivers, ridesharing is hard work that doesn't pay particularly well, resulting in extremely high turnover rates.  On the other hand, for many drivers, ridesharing provides a valued supplement to household income with considerable scheduling flexibility and frictionless entry/exit.  The value drivers associate with flexible, part time scheduling helps explain why less than one-third of rideshare drivers are dissatisfied with their work experience with Uber and Lyft, despite seemingly low hourly pay. 

The market will ultimately decide whether rideshare driver earnings are adequate to fuel Uber and Lyft's profit and growth aspirations. Time will tell. In the meantime, when MIT releases its revised full study report, it would be helpful for the authors to present a balanced, context-sensitive view on driver earnings that hopefully can be accurately reported by the press.

​
Lessons Learned
There are no singular villains or heroes in this story. The creators, disseminators and avid news readers drawn to the flawed MIT study each had reasons to behave as they did. Guarding against the epidemic of fake news that surrounds us will require all stakeholders to exercise better judgment in avoiding the fool’s race to spread and consume news with more shock value than validity.
​

• Outstanding academic institutions like MIT and Stanford carry an extra burden for integrity and professionalism, given the deep trust accorded to their published research.
• News organizations should avoid blindly accepting and editorializing on unverified press releases until the results can be vetted and verified. It is particularly dismaying that some news organizations that levied the harshest criticism of Uber and Lyft in their initial rush to judgment, have yet to publish the retraction by the MIT study author.
• News consumers need to be vigilant in recognizing their own confirmation biases. We’re stuck in a world of information overload, so individuals ultimately need to exercise their own editorial judgment in deciding how much and how soon to believe what we read.

_________________________________________
Disclaimers:

1. I have three degrees from MIT, and hold the institution in the highest regard
2. I have no connection with Uber or Lyft, and have no reason or desire to promote or defend their business practices