An Inconvenient Graph

Or: How to Manipulate a Time Series if You Don’t Like What it Shows

A version of the original Hockey Stick Graph by Klaus Bittermann, based on the original work of Mann, Bradley, and Hughes’s 1999 paper “Northern hemisphere temperatures during the past millennium: Inferences, uncertainties, and limitations.”

In recent decades, the world has been getting warmer. This change in climate has and will continue to disrupt how we live our lives, occasionally catastrophically. There is broad scientific consensus that this change is linked to human activity, specifically the release of greenhouse gases, dwarfing the contribution of other factors like solar cycles, volcanic activity, or orbital perturbations. Climate scientists have known about this for a while. Heck, oil companies have known about this for a while, and have been positioning themselves to take advantage of the shift away from fossil fuels, future-proofing their oil tankers and oil rigs to account for a world with higher sea levels and more extreme weather, all while muddying the waters for everybody else.

As Al Gore pointed out, all of this is pretty inconvenient! It’s a (potentially existential!) risk to human civilization that would require immediate and radical changes in how we consume resources, what resources we consume, and the global balance of this consumption. It would be a big hassle. But if you look at charts like the hockey stick graph above, it seems pretty inescapable that warming is happening. So, in motivated reasoning land, what can you do about this data?

One tactic is to just ignore data and go with your gut. For instance, there is a genre of political cartoon so cliché that there’s a Tumblr dedicated to collecting dozens and dozens of examples: “if global warming is real, how come it’s cold sometimes?” It only takes me a second or two to look at a line chart, but it might take me hours to shovel my sidewalk, and which one am I gonna remember, huh? But it’s a sort of reflex action kind of denial-ism I think, the death spasms of a genre so rote and moribund that if somebody told me they got their political opinions from newspaper editorial cartoons I would give them a nickel and tell them to catch a news reel before the picture show to get more up to date.

There’s also the move I alluded to earlier, to say that, yeah, sure, warming is happening, but it’s not greenhouse gases causing it. One of the things that I really like about the Bloomberg chart I linked to earlier (and I’m not alone in my appreciation) is that it really sort of forces people to try to set out what their alternative hypotheses are and quantify them. “Okay, you say it’s this other thing. What other thing, and how does it work? Can we measure it, and does it explain the trend we’re seeing?” It’s very easy to throw doubt on existing hypotheses, especially if you believe your enemies are shadowy cabals of one sort or another, but it’s harder to set out an alternative hypothesis that equally explains the facts.

But that, too, I think is a bit too easy. I am concerned, in this post, with ways to accept the truth of the data in hockey stick graph while simultaneously not accepting the message or trend in the hockey stick graph. That is, the person who would say “yeah, sure, there are some good models of past temperature” but who would not take the obvious (to me, at least) next step of “whoa, looks like the world is warming!”

“How to futz with a line chart” might not be the most interesting problem in the rhetoric of climate change, but it is the one closest to my area of expertise, and I think also provides a couple of interesting object lessons. For one, I hope it dissuades people from the idea that just shoving data in front of people, even important data, is sufficient for persuasion, but also against the idea that there’s a clear and unambiguous logical necessity that follows from a dataset to a single conclusion. MIT’s work on patterns of COVID visualization use should have put paid to both of those hypotheses, but they still seem to be perniciously grasping on to our conception of what data visualization is or could be. The other goal is, by playing a bit of devil’s advocate here, to show how the “tricks” being played in most of these cases are part of decisions that we have to make each and every time we make a chart, rather than weird outlier cases from incompetents or criminals, and to speculate about the potential analytical or rhetorical goals of each. That is, instead of immediately tagging @graphcrimes or whoever whenever we see a chart that sucks, to recognize the persuasive work these charts (and our own) are trying to do, and think about how that work was or was not successful.

To be more candid, I don’t think any of these people who made these charts did so because they were ignorant of “best charting practices” or whatever: I think they did so for specific purposes. They might not be good purposes (“trolling people,” say) but there’s intentionality there all the same. You’re not going to “fix” the impetuses behind these charts by sending the author a d3 tutorial or something, in the same way that grammar lessons don’t stop people from lying. You’re going to have to out-argue, out-persuade, or otherwise out-perform your rhetorical opponents.

Y-Axis Shenanigans

The National Review’s “only #climatechage chart you need to see.” A screenshot here because they deleted the original tweet, the cowards.

This example is so annoying that we wrote a whole paper about it, with experiments and everything. The Washington Post wrote an entire article about how it was misleading. I’ve gotten so much mileage in terms of talk slides and lecture examples out of this shitpost of a chart.

But the general idea is that you futz with the y-axis of the hockey stick to make it more of a… I guess a yard stick? Something flat, in any event. The original trend is there, I swear, but is very difficult to see. One of the reasons why the usual hockey stick graph uses “temperature anomaly” rather than raw temperature is precisely to avoid this kind of thing.

The usual reason why this particular manipulation draws so much ire is that 1 or 2 or 3 degree changes in global average temperature are a big deal: they can be the difference between having to spend a little more on infrastructure versus, say, Miami being totally underwater. “Raw” temperature is less important here than an indexed increase in temperature (we care that it’s warming, not just that it’s warm). So starting the Y-axis at 0 degrees Fahrenheit (well, -10, but…) is “technically correct” in chart design land but is misleading in practical terms, since it hides the trend and presents the data in the wrong “output space,” if you like. As I often present in my (somewhat tongue in cheek) annotation below, if the global average temperature is below freezing, or in the upper 90s, we’ve got pretty big problems, not just as a civilization but as a species.

My manipulation of the chart above, with some helpful skull-based annotations showing that if the global average temperature is below freezing, I think we’re in big trouble, civilizationally speaking.

When I show this chart to people in talks or lectures, I almost always get a comment like “why didn’t they plot the Y-Axis in Kelvin, ha ha.” And it’s true that this would make the hockey stick even flatter. But I think that’s somewhat of a misunderstanding of the purpose of the manipulation here. I mean, if their objective was just to draw a flat line, why use any real data whatsoever? I’ll discuss potential motivations below, but I think the choice of Y-Axis here is anything but arbitrary: it’s to assert “hey, this increase isn’t a big deal:” to communicate the hockey stick in (misleadingly, in my opinion) “practical” terms.

For instance, I might not have a strong idea in my head of what an increase of global average temperature of 0.5 degrees Celsius means. What does that do to sea level, for instance, or to extreme weather events, or to biodiversity? I’d have to consults with experts and use predictive models and stuff. But I could say, “oh, looks like average temperature has been around 55–60 degrees Fahrenheit for more than a century: that’s not so bad, I set my thermostat higher than that.” The choice of Y-Axis, in other words, is an intentional choice by the chart designers to determine how big of a deal an effect is determined to be, in both visual and numeric space.

I should note here that all designers of visualizations make this choice of what scale of effect sizes matter, either intentionally or just by implicitly accepting software defaults or the units provided to them in the data sets they work with. Beyond the issue of visual truncation of the axis of a chart, which I’m sure I’ve written about too much already, there’s the question of the units you use to express the data. Do you talk about stocks in terms of price per share, or in terms of percentage of value lost or gained? Do you talk about year-over-year gains, or absolute gains? Have you ever succumbed to the American temptation to describe things as so-many football fields away, rather than miles or kilometers? Described how expensive something is in terms of how many meals or concert tickets or or fighter jets something is? Congratulations, you have manipulated perceived effect size from some default. My issue with the National Review is not that they did this manipulation, but that they chose a manipulation that was shitty rather than one driven by a good-faith interpretation of what’s important to see in the time series.

X-Axis Shenanigans

Chart from a NOAA article by Michon Scott and Rebecca Lindsey’s article on how warm the earth has been “lately.” The hockey stick is in there, I promise, but compressed in a narrow range (the last 0.001% or so of the x-axis range, I believe). As an aside, the denser sampling rate for more recent data makes the recent past look more uncertain and variable (to me, at least) than the distant past, when, in fact, the opposite is the case.

If the message of the example in the previous section is “what’s a few degrees of warming, between friends?,” there’s a related move that goes something like “oh, a bit of human-released carbon dioxide? Call me when you’ve got real environmental change going on. Heard of the great oxidation event?” I’ve usually seen it connected to some naturalistic fallacy of the form of claiming that the Earth’s climate has swung wildly in the past, and so recent warming is just par for the course. The chart directly above (or one similar to it, showing at least several millennia if not eons) is often presented as part of such arguments, perhaps even to illustrate that the climate right now is actually much cooler than it has been, say, pre-ice age, and so we might just be reverting to the mean, or experiencing another regular upswing. There are several immediate objections: for instance, showing there’s something special about recent climate change (e.g., that it is happening much more rapidly than past changes, which the chart makes it impossible to detect) or pointing out that, hey, our current global human civilization probably wouldn’t be doing so hot back in the Paleocene either, so knowing that we’re heading back to that is still a crisis we’d need to manage. But you’re still playing the game of having to communicate what kinds of scales and effect sizes are relevant in both cases.

From a visual design standpoint this is all conceptually similar to my previous example: compress the actual effect people care about into a narrow space (except here in X rather than Y), but there are a couple of crucial differences. The first is that the Y-axis manipulation just adds empty space, whereas here we’re adding in data (it’s just data of dubious relevance). The second difference is that the first example was created de novo to produce the visual effect in question, but here it’s a re-appropriation of a chart meant for a different purpose.

I guarantee that the authors of this chart did not intend for it to support the message “hey, global warming is no big deal; Earth’s temperature changes all the time, and by way more than just a couple of degrees Celsius” and would probably be pretty peeved to know that they’ve been included in this particular rogue’s gallery; the article the chart was originally attached to is very specifically part of a larger project about contextualizing the current era of climate change with Earth’s past. And yet, it is comparatively easy to repurpose the chart in service of a different rhetorical goal. The article in which this chart appears has 19 references, 5 figures, and even then is only part one of two of a series of blog posts, the second part containing even more references, figures, and paragraphs. Yet the most common context I see it is in a tweet, unattributed (or sometimes just attributed to “NASA” or “NOAA” or “the government”). I believe that charts are “portable” in a way that allows for loss of context, nuance, or “capture” for other rhetorical goals (extended diatribe on “data visualization as Deleuzean war machine” excised for space and reader interest). This capacity for capture also shows the impossibility of creating a “neutral” or “default” chart that “just shows the data” and doesn’t consider audience and message: the choice of X-Axis range that is “misleading” or “inappropriate” for global warming debates is a perfectly natural one for the original rhetorical project, but would be perhaps misleading for others, even closely related projects (for instance, 65 millions years is a nice post-dinosaur chunk of time, but it’s pocket change compared to the overall age of the earth and the temperature fluctuations since the planet accreted from solar debris).

So no matter who you are, you have to decide not just what output variables your audience should care about, and in what quantities, but also the contexts (temporal or otherwise) in which you should care those variables. You also have to think about how your chart might spread beyond your intended audience, or its intended purpose, and design accordingly.

Cherry-Picking

There are several dozen tweets that are like this one: picking a pairs of months and drawing (or remarking upon) a negative temperature differential. I’ll spare you all of the rest of those.

The original draft of this post contained something like three or four charts from this guy, but I decided to try to be a bit more expansive (and to give him fewer clicks if possible). You have to be taken aback by the chutzpah, if nothing else (although I will speculate what I think he and people like him are doing and why they are doing it at the end of this post). But hopefully you get what’s going on in this particular chart: December 1987 was, in fact, warmer on average than December 2022. So how do you explain that, huh? Really this is just “if there’s global warming, how come it’s cold sometimes” with a little perfunctory data visualization fig leaf, but I do admire, in a way, the sheer contempt involved in showing a line chart with a solid upwards trend and then a single pair of points to claim that the trend doesn’t exist.

In line with “why not use Kelvin to hide the trend?” above, a natural question is “why show the whole graph at all, and not just the two points, since the full graph undermines the message?” That’s where I think things get more interesting. For one, charts are an important part of the bullshitting toolkit: they look “science-y” and “data-y” in a way that just the sentence “it was colder in December 2022 than December 1987” doesn’t, or at least doesn’t seem to.

But I think the second component is that people get hung up on the cherry-picking so much that they ignore the other argument that’s being made here in the text immediately before the chart: that, if CO2 is causing warming, how come there’s not an obvious linear connection between CO2 and temperature. CO2 doubled, we’re told, but there’s no doubling in temperature (of course, we’re not shown the graph of CO2 concentrations, because then you could readily gauge how those variables are linked). In short, if you are swayed by the cherrypicked points, then he’s got you. But if you aren’t, then he can say “yeah, it’s warming. But how come it’s not warming more, huh?” To refute that component, you’d probably have to bring in more data that is likely at different levels of temporal granularity, uncertainty, and variability, talk about lagging indicators, discuss the distinction of weather versus climate, sea ice and non-linear responses, and a whole host of things that can’t be solved by just pointing out that the time series in question is going up. Here’s a dozen-or-so page lesson plan from the EPA that goes through how you establish that link, for instance; it should take your students about an hour. Here’s a shorter document from Skeptical Science, but it still has lots of hyperlinks out to other material you’d need to read.

Brandolini’s Law, as always, is in effect: it takes more energy to refute the bullshit than to generate it. And this particular example has the benefit that there’s a bit of “obvious” bullshit to go after that means that the deeper, more insidious bullshit can escape unharmed.

The Staircase of Denial

The headlining chart of Christopher Monckton’s description of the “pause” or “hiatus” in global warming for the past 8 years or so. As an exercise, look at the chart that adorns landing page of the National Space Science & Technology Center’s climate data page (the source of the data in this graph) to get a heads up on why this particular regression line might be misleading.

This last technique combines the previous two into a new, exciting form: you futz with the X-Axis by cherry-picking a specific temporal range, and then rely on the data in this range to tell the story you want it to. Okay, sure, maybe you’re less convinced by the fact that two random months years apart don’t show warming, but what if I tell you that the last 8 years, as a whole, shows a slight cooling trend?

What gives away the game here is this hyper-specific choice of duration (“no warming for the last X years”). As we’ve seen by now, climate is not a smooth and steady phenomena; there’s variability, and particular outlying warmer or colder years occur even within a period of otherwise obvious warming, even before bringing in periodic cycles of other sorts of phenomena. But climate (as distinguished from weather) is also a long-term thing, so yearly temperatures within a particular span of time are all relatively similar. The net effect is that if you choose a window that includes one of those outlier hotter years and fit a line to it, you can pretty reliably generate lines of best fit with negative or at least very small slopes, and then you can say “hey, no warming [since the last hotter year].” When you can’t say that anymore, you just reset the “window” and start again, all while the overall series keeps ticking upward. Here’s an animated GIF showing this procedure in action:

The “Staircase of Denial” from Ken Rice et al. Would you have figured out the “trick” in the original graph in this section, if I hadn’t shown you this?

I hadn’t been exposed to “The Staircase of Denial” as a term for this manipulation, but now I’ve started seeing theses staircases everywhere. When we see a visualization, we very rarely get to see lots of models: we just get to see the particular model that the designer decided to show us (the exception here of “bi-temporal” or “hedgehog” plots are also some of my favorite things to talk about, but I’ll spare you that particular digression). So that means the designer has a lot of power in what sort of takeaways are more or less obvious in the chart. As with the choice of axes bounds above, it’s also one you don’t get to refrain from making: different choices of time scale would have produced different linear fits, different forms of regression would likewise produce different predictions for future behavior, and it’s not clear to me that there’s an “obvious” or “correct” choice to display here. I might even argue that this would be a case for eschewing the trend line entirely and relying on “regression by eye,” but then we’re in for a whole host of other design questions and audience assumptions.

Why Do All This?

Okay, so if the hockey stick chart is pretty solid evidence for recent warming, and most of the charts that attempt to show otherwise have issues that can be pointed out in a paragraph or two with relatively little effort (when I show some of these examples to classes, I think the longest pause I’ve gotten before the critiques come in has been 15 seconds or so), why do all of these people bother? Are they genuinely deluded? Do they have a fetish for being yelled at on the internet? I mean, maybe, but I don’t think that’s really at the core of why these “dunkable” manipulations of the hockey stick graph are created or shared. I think there are more pertinent rhetorical goals at play. In particular:

1. Undermining authorities. A commonality of all the examples I showed here are that they rely on data from groups like NOAA or NASA, or otherwise reuse or repurpose charts from authority figures. That’s not an accident: they could have taken data from their backyard, or just made up data entirely. But these charts are often shared with captions of the form “even NASA’s own data say global warming is a hoax!” or the like. By piggybacking on these sources, you lend yourself some of their credibility. Most nefariously, even if your manipulations are detected, you can still play the card that “both sides” have to “spin” the data to “build a narrative” (which is even true, after all: data don’t speak for themselves), and so you’ve leveled the playing field against organizations that would otherwise be considered unassailable.
2. Gish galloping. The term Gish galloping comes from early internet debates around creationism, where the idea is that you just sort of bury people in large numbers of low-quality arguments. If your opponent fails to address all of these arguments, then you can declare victory. This is connected to Brandolini’s law that I mentioned above about how creating bullshit is so much easier than refuting bullshit. If you throw enough charts at people, one of them is bound to “hit” with somebody. And if it takes several thousand words to explain what’s wrong, or if the debunker is dragged into explaining complexities of statistical modeling or methodologies while all you had to do was draw a line on one of their charts, you’ve already won. Nobody’s gonna read all that.
3. Building audiences. If there were anybody who had a hobby where they collected misleading climate change charts, it would be me, but I assure you I don’t go out of my way to do so. These examples all arrived to me relatively “organically,” by which I mean that somebody I know retweeted the person in question in order to dunk on them. So the charts might have larger reach because they are so easily “debunked.” Steve Milloy (the guy from my 1987–2022 cherry picking example) continually harps on how one of his graph tweets got over 13 million views (usually a tweet or two before complaining about being censored, which is also part of this gimmick). If the charts in question were more nuanced, less combative, they would, I think, have less reach. It’s feeding the trolls, in other words. And you only need to sow doubt in a few of those millions of views to get a return on your investment.
4. Reinforcing the in-group. This is somewhat conspiratorial, but I’ve always heard that the cults and religious sects that do door to door or sidewalk proselytizing do so for two reasons. Sure, you might get one or two converts who are questioning their faith and were genuinely interested in what you are pitching, but you also get a bunch of people who will tell you to fuck off, or otherwise be rude or mean to you. And there’s nothing for building in-group solidarity like spending lots of time with all of your in-group, who are all nice to you, and contrasting that with the time spent with the out-group, who consistently are not nice to you at all. Provoking rude responses, in other words, might be part of the point, as a way of making your particular clique adhere closer together. If all of these authority figures spend so much energy debunking you, you must be on to something, right? Or at least be some sort of rebel. I would also not underestimate the way that these sort of large-circulation posts act as rallying points in general: if you look at the comments, it’s often full of people just giving each other metaphorical high fives and laying out their own (often totally disconnected) grievances to a (largely already receptive) audience.
5. Filtering. I also wonder if there’s something here like how phishing attacks will often include obvious spelling errors and so on as a way to weed out potential prospects for fraud: if you’re careless or clueless enough to miss the errors, then you’re more likely to fall for their scam. Similarly, the people with PhDs. dunking on charts and citing their own books at each other are probably not the target audience for the people I’ve alluded to above: making “obvious” chart errors might be a similar sort of way to filter the audience down to people who are more likely to be receptive anyway. You are not designing a chart to withstand intense scrutiny, after all. If it survives as a contextless jpeg, or as a half-remembered anecdote, or as a nebulous feeling of doubt somewhere, then you’ve accomplished quite a bit of persuasive work.

Wrap Up

I will leave you all here with a few lessons:

1. There is no chart or data that is so “clear” that it cannot be manipulated, misinterpreted, or otherwise futzed with. You don’t get a pass on having to persuade your audience just because the data are “on your side.”
2. Likewise, every chart is built with assumptions about its audience and what they might care about. These assumptions can be incorrect, stale, or otherwise inappropriate. So have some intentionality when you make your design choices (and don’t assume that the only people making these choices are villainous propagandists).
3. You are not going to be able to solve the messy, very human issues of misinformation or persuasion by the having some Willy Wonka style “good egg” detector that puts “good charts” in one bucket, and “misleading charts” in another. “Good” charts can be used for “bad” ends, and vice versa.
4. You don’t automatically “win” just because you’ve pointed out something wrong in a chart. There are lots of rhetorical goals that a chart designer can have that will survive you saying mean things about them. Some might even be counting on precisely your reaction. Similarly (and somewhat ironically) there’s also a thing called the “fallacy fallacy”: the false assumption that the conclusion of an argument that contains a fallacy must therefore be false. Your job as a critical thinker doesn’t end just because you’ve pointed out that somebody is being manipulative with their data: you have to lay out a positive case as well.