After both liberal and conservative media outlets ran away with conflicting studies on Trump rallies and hate crimes, is anyone telling the truth?

Three Texas professors from the University of North Texas and Texas A&M-Commerce wrote a paper claiming that Trump rallies increased hate crimes by 226 percent. However, the academic paper was only a draft; it hadn’t been published and hadn’t even gone through the peer review process, yet the trio’s claim was picked up by several media outlets, including the Washington Post. Now another study, also not yet peer-reviewed nor published in a journal, by two Harvard economics Ph.D. students suggests any Trump effect on hate crimes was also seen at Hillary Clinton rallies. That study was also then picked up and run away with by conservative media outlets, but is either study right?

Harvard and Texas Studies on Hate Crime

The hallmark of the scientific world is the ability to reproduce the same result, and that is exactly what two Harvard economics Ph.D. students set out to do. In an article for Reason Magazine, Matthew Lilley and Brian Wheaton explained how, though they didn’t have access to the original study’s data and code, they recollected the variables mentioned in the Texas study and performed the same analysis, but also analyzed Clinton rallies’ effect on hate crimes — something the original professors had not included.

After analyzing the data, the Harvard duo actually found that Clinton rallies had an even greater impact on hate incidents – though they did not include a statistical figure in the Reason article. (Lilley and Wheaton published their data here and here).

Wheaton and Lilly also argued that the Texas study failed to include a statistical control for population size, writing:

“Both of these results rely on comparing counties with rallies to other counties without them. This produces a glaring problem. Politicians tend to hold political rallies near where large numbers of people live. And in places with more people, the raw number of crimes is generally mechanically higher. Simply put, no one should be surprised that Orange County, California (population 3.19 million) was home to both more reported hate incidents (5) and Trump rallies (2) than Orange County, Indiana (population 19,840, which had zero of each).”

However, while the Texas study did mention that their study suggested counties hosting a Trump rally had more hate crimes than counties not holding a rally, the Texas study also looked at incident rates of hate crimes within counties both before and after a Trump rally – a statistical analysis not subject to the flaws Wheaton and Lilly mention.

The Texas study writes:

“We begin by examining if the occurrence of a Trump rally impacts extremist and antiSemitic incidents at the county-level. The results indicate when Trump held a political rally within a county the incidence of hate incidents significantly increased. Indeed, the predicted incident rate ratio is statistically significant and positive, which substantially indicates that counties hosting a Trump rally experienced increased rates of extremists and anti-Semitic incidents after the rally when compared to before the rally. Further, the results indicate there were heighten incidents targeting minorities in counties hosting a Trump rally compared to counties that did not host Trump rallies.”

As a result, both so-called “liberal” and “conservative” media outlets have run with headlines touting the questionable results of the study of their choice. So as Wheaton and Lilly said, the Washington Post, CNN, et. al. ran with premature headlines claiming Trump’s rallies increased hate crimes by 226%, so too have conservative outlets run prematurely with headlines celebrating how the Harvard study debunked the Texas study.

What’s Really Going on With the Two Studies?

First, both studies clearly need to be peer-reviewed. Secondly, if the Texas study was performed correctly and found an increased rate in hate crimes after a Trump rally in a defined geographical area then the study also needs to look at whether Clinton rallies had the same effect, as the Harvard study alleges.

Third and perhaps most importantly, what are both studies measuring? The Texas study and thus the Harvard study used data based on hate crimes as reported by the American Defamation League’s heat map which tracks incidents of extremism or anti-Semitism. But included in that data is a very wide range of activities. Of the 3,991 incidents that the ADL recorded between 2018 and 2019, 2,114 are categorized as “white supremacist propaganda” which at a cursory glance seems to include mostly incidents of distributing flyers. As an article on the blog Memepoliceman.com mentions, using the FBI’s database of Hate Crimes would be a more accurate representation of actual hate crimes.

It’s also important to look at what does a 226% increase actually mean, especially when looking at reports of low statistical figures. Using the FBI’s hate crime data, Memepoliceman writes:

“Of the 223 cities and towns Trump had rallies in, over 1/3 (86) reported zero hate crimes for all of 2016. Over half (127) reported one or less. Considering the relatively low frequency of these crimes, statistics for a given area can be wildly misleading.

“For instance, Tampa, which Trump visited more than any other city, had hate crimes rise by a whopping 300%! But this is because it went from 1 incident in 2015 to 4 in 2016. Meanwhile, hate crimes in Las Vegas, where Trump visited 4 times, plummeted by 40% (48 in 2015 to 29 in 2016).”

Ultimately, the point is how wildly misleading polls and studies can be and how quickly an unproven study with results that make a great headline can go viral, in both liberal and conservative media.

Credible researchers dutifully practice the scientific method and make every attempt to be unbiased. Peer review is the gold standard of the scientific review; all credible journals use it. Yet even the best scientist can fall victim to allowing ideology to interfere and thus, the peer-review process is needed.

In an ideal world a neutral press corps would have vetted both articles, and once realizing they were unpublished and not peer-reviewed, would have left them alone. But even if the Texas article is eventually peer-reviewed and their findings that Trump rallies increased hate crimes 226% is accurate, running that statistical result alone without an equal comparison to Clinton rallies and any idea of what a 226% increase translates to in terms of numbers of incidents would have been just as misleading.

But Americans don’t live in a world with a neutral press, mostly because neutrality doesn’t sell and doesn’t result in clicks. Most media outlets are actually entertainment outlets and their primary purpose is to earn revenue by getting more eyeballs on their content, and what sells is sensationalized titles and stories designed to get you mad at the “other side” or “other political party.”