A little number sense can provide perspective on the COVID-19 risks

Model of the COVID-19 virus

Given the growing deluge of horrifying stories and images surrounding the global COVID-19 pandemic over the past few weeks, it is easy to find yourself reacting in one of two ways, both of which are potentially dangerous and neither of which is justified by the data.

First, when faced with daily reports of increasing numbers of deaths, you might think the odds are so stacked against you, there is little point in taking the recommended steps to protect yourself from infection. Alternatively, you could find yourself paralyzed with anxiety, afraid to do anything, and sink into a nihilistic depression.

Neither is justified by the actual data. In fact, what the data indicate are:

  • The odds are in favor of you avoiding infection if you follow the recommended preventative measures (avoiding contact with or close proximity to others, regular washing of hands, not touching your face, and a few others);
  • Even if you are infected, the odds favor you not having to be hospitalized;
  • The odds on you dying from COVID-19 infection are very low.

But just how good are those odds really? More specifically, how do they stack up against the risks associated with other things we encounter and do in life?

To allay my own initial anxiety, I did a few quick calculations, based on the data available in the press, and found that, while the risk of death is certainly much higher than that of, say, dying in an airplane crash, you are much more likely to die of heart disease than from the novel coronavirus.

How much more likely? Numerically, the risk is a whopping 28 times higher. But be careful. Whenever you see risk comparisons like that, you need to pay attention to what is being compared with what. On their own, the numbers tell you nothing; absent a proper context, that factor of 28 is meaningless.

In fact, that caveat about context can be significant whenever numbers are quoted; it is particularly so when a number is given to compare risks.

In the case of COVID-19, I went into the details of my calculation in a post in my personal blog profkeithdevlin.org. I leave you to follow the link and read what I said there. The mathematics is not hard; indeed, it is just basic arithmetic, along the lines of some of the more recent preceding posts in the SUMOP blog. The important thing is to keep in mind what the numbers refer to. The ultimate goal is not to produce a number — an answer; rather, it is to understand the relative risks.

“The Dumbest Moment in the History of Television”? Not so fast

A segment about the 2020  presidential election primaries on MSNBC News on March 5 caused a flood of comments on Twitter. Here is the tweet that brought the segment into the Twittersphere and started all the ruckus: 

Both the guest, a member of the New York Times Editorial Board, and host Brian Williams failed to notice how absurd was the arithmetical claim being made. 

If this is the first time you have seen this segment, it likely flew by too quickly to register. Here (right) is the original tweet that started it all.

Curious as to who put out the tweet, I checked her profile. (See below.) [Normally I anonymize tweets, even though they are public, but on this case the tweet was shown on live national television.] She starts out with the claim that she is bad at math. I have no idea whether she is or she isn’t. But that tweet does not show she is. It actually suggests she may be a better mathematical thinker than many – read on.

Many of the comments on Twitter lamented the poor arithmetical skills of the tweeter and the two media figures on the show. In fact, the story went well beyond Twitter. The next day, the Washington Post, no less, devoted an entire article to the gaffe.

The Big Lead took another swipe the same day, pointing out that the tweeter, Mekita Rivas, is a freelance writer and Washington Post contributor, and noting that the math disclaimer on her Twitter bio was a result of her oh-so-public gaffe.

But she was by no means alone. The error was repeated, or at least overlooked, by a whole host of smart media folk: Mara Gay, the New York Times editorial board member who brought it up in her on-camera conversation, the MSNBC graphics department, several producers, host Brian Williams himself, and likely more.

The episode is reminiscent of the numerical error made by a famous New York Times commentator that I wrote about in a post a few days ago (March 3). To be sure, both episodes highlight the clear need, in today’s data saturated and data-driven world, for all citizens to have a reasonably well developed number sense.

For this problem is not, as many critics of Ms Rivas claimed, that she cannot do basic arithmetic. I would not be surprised  if she were no worse than most other people, and besides arithmetic skill became in-principle obsolete with the introduction of the electronic calculator in the 1960s, and in every practical sense became really obsolete when we entered an era where we all carry around an arithmetic powerhouse in our smartphone. [There is educational value from teaching basic arithmetic, but that is a separate issue.]

What is most definitely not obsolete, however, and is in fact, a crucial human skill in today’s world, is number sense. What was worrying about the whole Twitter–MSNBC episode is that none of those involved recognized instinctively that the claim in the original tweet was absurd.

The two follow-up news articles I just referred to delve into the absurdity of the error, which comes down to the difference between $1M+ and $1.52.

But this is where it gets interesting. What is it about the statement in the Rivas tweet that led a whole host of smart professionals to not see the error? What led them to not feel in their bones that the amount every American actually would receive from Bloomberg would be “something a bit north of a dollar-fifty” and not a sum “in excess of a million dollars.” This is not a question of calculating; it’s not poor arithmetic. It’s something else, and it’s far more dangerous. Doing arithmetic is something our iPhone can do, quickly, error-free, and with more numbers, and bigger numbers, than we computationally-puny humans can comfortably handle. Understanding numbers, on the other hand, is very much people stuff. Our devices understand nothing.

If a whole group of smart people are so quantitatively illiterate (and that’s what we are talking about) that they don’t instinctively see the Rivas error , how can we as a society make life-critical decisions such as assessing our personal risk in the coronavirus outbreak or the degree to which we should take a candidate’s climate change policies into consideration when deciding who to vote for.

This is why I, and many others in mathematics education, are putting such stress on the acquisition of number sense. Indeed, an article I wrote for the Huffington Post on January 1, 2017 carried the headline Number Sense: the most important mathematical concept in 21st Century K-12 education. I was not exaggerating.

Many of the videos and blogposts on (and referred to on) this website focus on number sense, and discuss how best to ensure no future citizen graduates from high school without adequate number sense. (The Common Core State Standards are designed to achieve that goal, though teaching practices often seem to miss that point, sometimes as a result of inappropriate administrative pressure coming from poorly informed politicians.)

What interested me in particular about the MSNBC example was the nature of the error. It’s similar to the example I discuss at the end of the first of the Blue Notebook videos on this site, in that the way the proposition is worded triggers learned arithmetic skills (more accurately, and appropriately derisive, “test-taking tricks”) that in general are reliable.

Here is the Rivas argument, spelled out:

1. FACT : Bloomberg spent $500 million on TV ads.

2. FACT: The US population is 327 million.

3. (FALSE) ARGUMENT: We are talking millions here. If you take the whole amount and divide it up among the population, everyone gets 500 divided by 327 (millions). Good heavens, that’s more than one million each!

Rivas is doing two smart things here – smart in the sense that, in general, they lead to the correct answer quickly with the least effort, which is what she (and you) likely needed to be able to do to pass timed math tests.

1. First, she says, everything is about millions, so we can forget those six zeros at the end of each number. [GOOD MOVE]

2. Then she says, we have to divide to see how much each person would get. That’s 500 divided by 327, which is around 1.5 (or at least more than 1). [GOOD MOVE]

3. Then finally she remembers everything is in millions. So it’s really $1.5M (or more than $1M). [EXCELLENT. SHE REMEMBERED THE SIMPLIFICATION AND PUT THE ZEROES BACK IN]

On its own, the idea behind each step is fine, indeed can be useful – in the right circumstances. But not, unfortunately, in this coupling! [I’m not saying she went through these steps consciously doing each one. Rather, she was surely applying a heuristic she had acquired with practice in order to pass math in school.]

The trouble is, if someone leaves school having mastered a bunch of heuristics to pass timed math tests – which is how many students get through math – but has not been taught how to think intelligently about numbers (and thereby develop number sense), then they are prone to be tripped up in this way.

Not convinced? Check out the example toward the end of that first Blue Notepad video. It’s a bit more subtle than the MSNBC example I am discussing here; in fact, more than half the people in every audience I have given that example to (under time pressure) get it wrong. The odds are, you would have too. But the overall message about math education is the same.

Ms. Rivas should take that disclaimer off her Twitter bio.

But maybe replace it by one that says, “I need to improve my number sense.” That’s a motto that – by my observation of the news media, social media, and society in general – would well serve the majority of people, including many who are good at “getting the right answer.”

Don’t be a sucker. Number sense can help you out at the county fair

I was wandering aimlessly round a county fair not long ago, and came across a stall with a game I remember playing as a teenager, many years ago back in the UK – and losing. [The image here is not the one I saw; I just pulled it from the Web for illustration.]

You pay an entry fee and are given three darts. The goal is (and this may vary, though likely not by much) that if two of your three darts lands in a card, you win a small prize, and if all three do you win a more substantial prize.

I remember that as a teenager I was surprised, and disappointed, when none of my three throws ended up in a card. Not one. After all, at first glance it looks as though the cards occupy most of the board, so the odds should be in your favor.

Understanding what is going on provided yet another great example of number sense, the topic I focused on in my previous two posts. Let’s do some quick calculations. As usual, they don’t have to be accurate; we can make simplifications.

There are 4 rows of 5 cards, so 20 cards in all. According to a quick google search, playing-cards typically measure around 2.5 in by 3.5 in. Based on that information, a brief glance at the photo shows that the cards are placed on the board with a vertical and horizontal separation of roughly 1.25 in, with an outer border roughly the same size. So we can conclude that the dimensions of the board are

[5 x 2.5 + 6 x 1.25 = 12.5 + 7. 5 = 20] in  wide  by  [4 x 3.5 + 5 x 1.25 = 14 + 6.25 = 20.25] in high.

Which means the total target area is 20 x 20.25 = 405 sq in.

The area of a single card is 2.5 x 3. 5 = 8.75 sq in, so the total area occupied by the cards is 

[20 x 8.75 = 175] sq in.

Hence, the proportion of the target area occupied by a card is 175/405, or approximately 0.43. 

That’s less than half, so the odds are against you on each throw. Assuming your throwing is random, which for most of us is will be, your throws will fail to land on card 57% of the time.

The probability of getting all three darts to land on cards to win a “substantial” prize is 0.433, which is approximately 0.08. You will win a real prize only 8 times in 100 attempts.

In fact, the odds are against you even winning a small prize, since the probability of two darts landing on a card is 0.432, or approximately 0.2, so you fail to win anything at all on 4 of every 5 attempts.

So the game is fine for a bit of fun, where you play maybe one or two rounds. Especially if the stall is there to raise money for a worthy charity. But for most of us, this is definitely not financially wise. 

As for skilled darts players, in a commercial setting, the stallholder will surely stop anyone playing once they recognize the contestant’s skill (as happens in casinos with blackjack), so even if you play well, the odds are still heavily stacked against you making a killing. 

Number sense and the news media

This tweet caught my eye a few weeks ago. It was from a well-known political commentator for a major US newspaper, who I follow. I am obscuring his name since his identity is not relevant to this post. What does bother me, is that an influential commentator with a large pulpit can display such a lack of number sense. (I still enjoy reading him; he has a good grasp of many things. But not where numbers are concerned.)

In fact, the tweet did more than catch my eye; it jumped right off the screen and nigh on blinded me. Regardless of whether the initial assumption is correct or not, the conclusion is so obviously wrong on simple numerical grounds.

As usual, a very simple, made-up example is all it takes to highlight the fallacy. Also as usual in such situations, virtually no mathematics is involved here, and no arithmetic beyond very simple stuff with whole numbers. Number sense is not arithmetic. The goal is to get a quick numerical sense of the issue

Assume that a worker making $25,000 a year in 2010 saw their wages rise by 5% over the ten-year period to today. They would now be making $25,000 + $1,250 = $26,250. Compare that with their boss, who made $100,000 a year in 2010 but whose income rose only by 2.5%. In 2020, they bring in $100,000 + $2,500 =  $102,500. Their pay increased at half the rate  of their low-paid employee, but that meant their income rose by twice as much.

As with the example of the “danger of drinking wine” I wrote about last time, it all depends on the base figures on which the percentages are calculated.

How close to reality is my made-up-on-the-spot, number sense example? I googled “rise in wages”, and immediately found myself looking at a recent report from the Brookings Institute that gave me all I need. (I chose a page from a source I knew to be reliable.) For the low-paid worker, my example was not too far off.  From 2010 to 2018, the bottom 10th percentile of hourly wages grew 5.1%. But I was way off with my number for the boss. In fact, the entire top 90th percentile of workers saw their income go up by an average of 7.4%.

What this tells us is that the opinion writer not only has poor number sense, he is not even able to get his underlying numerical facts right. Of course, you might say the writer set out to deliberately mislead. That’s possible, I suppose, but I doubt it. His article was in a major national newspaper that has fact-checkers, so both he and the publisher knew that his claims could easily be checked. To continue doing his job, readers have to trust him. Much more likely, I think, the issue was innumeracy. Like many people I know, when faced with figures, both the writer and his editor likely found their eyes glazing over. 

Yet to anyone with number sense, the writer’s tweet jumps off the page as being absurdly wrong. What the actual numbers are does not matter. Even if his assumption about the relative rises had been correct, his conclusion would be false, as my initial made-up example showed.

In fact, my one use of google showed that even his starting point was off-the-charts, factually wrong: wages of higher-paid workers had risen a lot faster than for those at the bottom.

The sad fact is that the majority of people in the media have very poor educational backgrounds in mathematics, science, or engineering. Numbers are alien to them.

Given the huge significance of numerical data in today’s world, that tells us that our educational system needs to change. Arguing about the pros and cons of teaching Algebra 2 or Calculus seems totally misplaced, when we are producing so many people who lack basic numeracy.

In the meantime, if we cannot rely on the media to get the numbers right—by which I mean, ballpark, number-sense, appreciate-it-when-you-see-it right—then it is up to we citizens to protect ourselves from being misled.

And, for those of us in the education business, to make sure our students can be both good consumers of quantitative information and good communicators of numerical data.

I had a scare. A simple application of number sense dispelled my worry

A family member alerted me recently to the Medium article with the above headline. Knowing I have a long-adopted European lifestyle habit of having a glass (sometimes two) of wine with a meal, she was concerned about what seemed to be a significant health risk.

The article led off with this paragraph:

It was so comforting to think that a daily glass of wine or a stiff drink packed health benefits, warding off disease and extending life. But a distillation of the latest research reveals a far more sobering truth: Considering all the potential benefits and risks, some researchers now question whether any amount of alcohol can be considered good for you.

As a scientist, I am always open to being convinced by hard data. So I read on. The article rapidly became more alarmist:

For years, moderate drinking — typically defined as one drink (such as regular beer or a glass of wine) per day for women and up to two for men — had been billed as a way to reduce the risk of stroke, in which a vessel carrying blood to the brain bursts or is clotted. But a study earlier this year, involving more than 500,000 men and women in China and published in the Lancet, refutes that claim. “There are no protective effects of moderate alcohol intake against stroke,” says one of the study’s co-authors, University of Oxford professor Zhengming Chen. “Even moderate alcohol consumption increases the chances of having a stroke.”.

Wow!

While not for a moment doubting the validity of the science and the conclusions the researchers stated, I nevertheless wondered if there were really any cause for alarm. Just how significant is the risk I am being exposed to as I sip my Pinot Noir? After all, from a scientific standpoint, a well-established increased risk of 0.5% can merit publication in a professional journal, but most of us would discount such a low increase when it comes to deciding to give up an activity that gives us a great deal of pleasure. Maybe the article writer was being unjustifiably alarmist.

In the case of moderate wine drinking, there is certainly a well-established correlation between countries where that is common practice and countries with longer life-expectancy—though as is often the case, establishing any causation is a tricky challenge. (I tend to think that the pleasure I get from a glass of good Pinot, together with the relaxed sensation is produces, has a net beneficial effect on my overall health. That’s a plausible assumption, but did the new study show that effect was outweighed by negative consequences?)

In fact, the article quoted far more alarmist figures:

The study, published in the Lancet, found a “strong association between alcohol consumption and the risk of cancer, injuries, and infectious diseases.” Among other findings, just one drink daily was linked to a 13% increased risk of breast cancer, 17% increased risk of esophageal cancer, and 13% higher risk of cirrhosis of the liver.

Those percentages definitely grab the attention. But again, while not for a moment disputing them as valid scientific findings, I wondered how significant they are in making a life decision. 

Time to do some quick calculations.

According to the article, 88,000 people in the United States die each year from alcohol-related causes. Let’s assume the bulk of those are adults (i.e., legal drinkers). The total population of people over 21 in the United States is about 200,000,000. So according to the figure quoted, the proportion of American adults who die from alcohol each year is 88 out of 200,000.

To simplify the calculation, let’s assume the situation is worse, and 100 out of 200,000 die each year. That’s 10 people in every 20,000. I dropped down to 20,000 because that’s something we can visualize, as many stadiums and arenas have capacities of around that number. A quick Google search revealed that the Amway Center in Orlando, Florida has a stated capacity of exactly 20,000.

The Amway Center in Orlando, Florida, seating capacity 20,000

I can now get a visual representation of the annual risk of death from drinking alcohol as 10 people in the crowded stadium. (Other well-known examples I came up with are Madison Square Garden in New York City, with a capacity just over 20,000, and Meadowlands Arena in New Jersey that comes in just under 20,000. Imagine ten individuals in either of those facilities.)

Personally, I don’t find that particularly scary. If it were not for the degree to which a glass of wine with my meal gives me significant pleasure, it would absolutely make sense to avoid the risk of being among those ten people in the stadium. But in my case, I’ll take that risk. Not least because I suspect that most of those deaths are from people who drink a lot more than I do. (The writer of the buzzkill article responsibly alludes to the far greater risks associated with excessive alcohol consumption and binge drinking, where to my mind the data cannot be dismissed.)

But the focus of the article was not on the base-level risks, rather the increased risks of one drink each day. That has me right in the crosshairs. What is the significance of that particular risk?

Again, let’s look at a scenario worse than the one reported, to make the math simpler. Assume there is an increased risk of 20% — bigger than those reported increases of 13% and 17%. That would mean 12 people in the Amway Center; 2 up from 10. Now I can visualize, and hence understand, my increased risk as follows. (I’m assuming I were not already in the danger group, but become so as a result of my daily glass of wine. Again, my goal is not to produce accurate risk data, but to visualize, and hence understand, the data presented. I am making simplifications that, if anything, create a picture showing greater risks than the actual data indicate.)

In any one year, 10 of those people in Amway Center will die as a result of alcohol. If you don’t drink, you will not be one of those 10. The increased risk as a result of having one drink daily enlarges that danger group of 10 people to a group of 12 people. At least to my eyes, the difference between 10 people and 12 people in that massive Center is nothing like enough to convince me to give up my daily glass of wine. I’ll accept the risk.

In my scenario, those two additional people in the audience are the ones who get there as a result of one glass of wine a day. Becoming one of those two individuals in the Amway Center are the ones the buzzkill article is warning us about.

Just to be doubly sure I’m being sensible, however—after all, I have a vested interest in convincing myself my wine habit is not unwise—I decided to google common causes of accidents and deaths. 

According to my search, taking a shower turns out to be a dangerous activity. (For my entry “accidents in the bathroom”, Google returned over 22 million hits.) But what are the figures? In the U.S., roughly one person per day dies in the shower, mostly adults. In an adult population of around 200 million, that’s a low risk. But that’s the risk of death. Statistically of more significance, around 250,000 people aged 16-or-older per year have an accident in the bathroom that requires a visit to the emergency room (with 14% requiring hospitalization). That represents 25 people in Amway Center. We all accept the risk of being one of those 25 every time we step into the bathroom.  

None of this is to say we should ignore the valuable evidence science provides when it comes to risks. On the contrary, I went through the above exercise because I habitually try to maximize my length and quality of life. Every action we take carries risks—including inaction. We just have to make wise decisions that balance the pluses and minuses of everything we do. 

To do that, it helps to visualize the risks in some real-world scenario we are familiar with. I tend to go for movie theaters, sports stadia, and the like. It usually doesn’t require any complicated calculations; it’s number sense rather than arithmetic. Grab a few relevant figures from the Web and make gross simplifications that if, anything, make things worse than they really are. 

In the situations described in the article, that “20%” figure refers to a 20% increase in size of the risk group. But if that risk group is already small, the increased risk group will still be small. In the case of moderate alcohol consumption, it corresponds in going from ten people in the vast Amway Center to being twelve in Amway Center.  

But the question I asked—and everyone faces questions like this all the time—is this: Will I accept the risk if there is a personal cost to me. Once I had a clear way to visualize that risk, I found the decision easy. Pass the Pinot!

That’s not an endorsement or a recommendation. It’s my personal decision arrived at from a good understanding of the risks, based on hard data. (Hence no photo of a glass of wine.) Comparable data on risks persuades me to always fasten my seat belt in a car and never go out on my bike without a helmet. Why take a risk—however small—when there is no personal cost to avoiding it?

And that’s what this post is about: Applying number sense to make sense of numerical data in order to reach personally significant decisions. Once I had a good image in my mind to visualize the risks, I found the decision was easy.

But that was not the focus of the buzzkill article, where I found the complete absence of important contextual information to help readers understand what the data actually represents resulted in a story that I think is unduly alarmist. To repeat a recurring SUMOP theme, number sense is a critical mental tool in today’s data-rich world.

Using math to solve a novel real-world problem is hard. Cognitive science explains why

We’ve all been there. Someone shows us a method to solve a particular kind of problem and helps us as we try to go through the steps ourselves. Eventually, we feel confident we know how the method works. Then our instructor says, “Here is a similar problem. Try this one completely on your own.” 

And we have no idea how to begin!

The instructor told us it is similar to the one we just saw. But we can’t quite see how it is similar.

Clearly, I’m not talking about elementary-grade worksheets designed to provide repetitive practice at one specific mathematical operation, such as addition or multiplication. Though even there, beginners can experience the same feeling of not knowing how to proceed, even if just one or two specific numbers change. 

More significantly, however, instructors are almost certainly familiar with the situation where students in, say, a physics class are unable to solve a problem the requires the very linear algebra techniques they just applied to pass a test in the math class. Simply being in a different class can make a big difference.

The problem is even more pronounced when it comes to using mathematics techniques to solve real-world problems. Even when we are told explicitly that the same technique just used successfully to solve one problem can be used to solve the new one, fewer than 10% of us can actually do that. 

This is not a sign of intellectual weakness. It’s a built-in feature of the human brain that cognitive scientists have known about and studied for many years. The good news is, it can be overcome. Those same cognitive science studies show us what we need to do to overcome the difficulty.

The starting point is the recognition that the evolution by natural selection of the human brain equipped it to learn naturally from experiences, to better ensure its survival, or to perform in a more self-advantageous fashion, when next faced with a similar experience. Learning through experience in this fashion is automatic, and tends to be robust, but is heavily dependent on the particular circumstances in which that learning occurred. It does not take much variation in the circumstances to render what was learned ineffective. 

Let me give you an example. Imagine you are a doctor faced with a patient who has a malignant tumor in their stomach. It is impossible to operate on the patient, but unless the tumor is destroyed the patient will die. There is a kind of ray that can be used to destroy the tumor. If the rays reach the tumor all at once at a sufficiently high intensity, the tumor will be destroyed. Unfortunately, at this intensity the healthy tissue that the rays pass through on the way to the tumor will also be destroyed. At lower intensities the rays are harmless to healthy tissue, but they will not affect the tumor either. What type of procedure might be used to destroy the tumor with the rays, and at the same time avoid destroying the healthy tissue?

You might want to think about this for a while before reading on. Fewer than 10% of subjects are able to solve it at first encounter.

Have you solved it?

Are you one of that ten percent?

Okay, let’s move on.

The solution is to direct multiple low-intensity rays toward the tumor simultaneously from different directions, so that the healthy tissue will be left unharmed by the low-intensity rays passing through it, but when all the low-intensity rays converge at the tumor they will combine to destroy it.

Easy, no? At least, it’s easy once you are shown how to solve it!

Here is a second problem. A General wishes to capture a fortress located in the center of a country. There are many paths radiating outward from the fortress. All have been mined so that while small groups of men can pass over the paths safely, any large force will detonate the mines. A full-scale direct attack is therefore impossible. What does the General decide to do?

Again, you might want to think about this before proceeding.

This too is solved by at most 10% of subjects when they first meet it.

How did you do with this one?

The General’s solution is to divide his army into small groups, send each group along a different path, and have the groups converge simultaneously on the fortress.

This is, of course, logically the same problem as the cancer treatment, and the same “hub and converging spokes” solution works in both cases. Except that there is no “of course” about it. A substantial proportion of people fail to recognize the similarity, even when presented with one problem right after the other, as I just did here. 

Specifically, a number of studies have shown that, whereas only 10% of people solve the radiation problem at first encounter, if they are first shown the General’s problem and its solution and then presented with the radiation problem, about 30% are able to solve the radiation problem. They see the similarity. That’s up from the 10% of subjects who can solve the radiation problem in the absence of any priming, but the do not see a connection between the two problems.

Notice that, in this two-problem scenario, where the subjects are not told to look for a similarity, fewer than a third of them are able to spontaneously notice it. Moreover, this disparity arises despite the fact that, knowing they are subjects in a psychology experiment, one might expect that all subjects would consider how the first part might be related to the second. But they do not make that connection.

On the other hand, if a group is shown the two problems one after the other, and told to look for a similarity, then around 75% of them can solve the second problem. (But fully one quarter still cannot!) 

In both study cases with the two problems, a substantial number of subjects see one as a problem about warfare and the other as a problem about medical treatment. They do not see an underlying logical structure that is common to both. That’s a significant finding, from which we can learn a lot. 

Cognitive scientists use the term “inflexible knowledge” to refer to knowledge that is closely tied to the surface structure of the scenario in which it was acquired. It takes time, effort, and exposure to two or more different representations of what is in actuality the same problem in order to recognize the underlying structure that is the key to making that knowledge flexible—something to be applied to any novel scenario having the same underlying structure. 

In the case of the radiation problem, 90% of people see it purely in terms of radiation—essentially, a physics problem, unconnected to the General’s problem (a problem about military strategies). 

Yet those two instances of inflexible knowledge become mere applications of a more general strategy of a hub-and-spokes wheel, where activity that follows the spokes from the circumference inwards converges into a single combined action at the hub—once you have acquired that wheel concept. When you have, it’s easy to recognize new instances where it can be applied. Being on an abstract level, your knowledge is flexible. But getting to that more abstract, flexible-knowledge structure is not automatic. It requires work. In the case of the hub-and-spokes solution, it takes exposure to a third problem scenario before most people “get” the trick. (See Note 1 at the end.)

Here’s why this is relevant to mathematics. In order to solve mathematical problems (or, more generally, to solve problems using mathematical thinking), you generally need to identify the underlying logical structure, so you can apply (possibly with some adaptation) a mathematical solution you or someone else found for a structurally similar problem. That means digging down beneath the surface features of the problem to uncover the logical structure.

To be sure, once you have mastered—by which I mean fully understood—a particular mathematical concept, structure, or method, it becomes a lot easier to see it as the underlying structure (if indeed it is). For example, if you understand linear algebra, you will be able to identify many problems where it can be applied, in math, in physics, in economics, or whatever. Quite simply, your (flexible) knowledge of the (abstract) method makes it possible for you to recognize when it may be of use.

Here’s the educational rub. Actionable, flexible knowledge cannot be taught, as a set of rules. It has to be acquired, through a process of struggling with a number of variants until the crucial underlying structure becomes apparent. There is no fast shortcut.

How do you recognize an underlying abstract structure or achieve understanding of an abstract method in the first place?  It seems important to make those connections between the abstract-and-general, to the concrete-and-particular. We learn best from concrete examples we experience, and the mind’s natural inclination is to store knowledge in a fashion closely bound up with the scenarios in which it was first acquired. Overcoming that constraint to learning to recognize abstract structural or logical patterns requires effort, and often the study of more than just a couple of examples. You need several examples and you need to go deep into them.

To sum up, in order to develop the flexible thinking required to tackle novel problems using mathematics—regardless of where those problems come from, what specific mathematical concepts they may embed, and what specific techniques their solution may involve—what is required is experience working in depth on a small number of topics, each of which can be represented and approached from at least two, and ideally more, different perspectives.

NOTE 1: For a more expansive discussion of these issues, with more examples, see my December 1 Devlin’s Angle post for the Mathematical Association of America.

NOTE 2: For anyone curious as to why the brain works the way it does, and why it finds some things particularly difficult to master, particularly the recognition of abstract structure, check out my book The Math Gene, published in 2001, where I draw on a wide range of results from several scientific disciplines in an attempt to shed light on just how the human brain does mathematics, how it acquired the capacity to do so, and why it finds abstraction so challenging.

The crucial importance of digital literacy in 21st Century math education

A common theme among the articles and videos on this website is the regular use of online resources in developing and using mathematics. The image shown here (which you will find in several of my videos and linked-articles on the site) presents some of the digital tools professional mathematicians use most commonly when working on a problem.

This particular list is based on my own experience over several decades working on problems for industry and various branches of the US government, and in conducting various academic research projects, but also includes two tools (MATLAB and the graphing calculator) that I do not use but many other mathematicians do, which I list as a nod towards others’ toolboxes. I typically use those tools in the order they appear in the display, reading left-to-right.

Whenever I show this image to an audience, I inevitably remark that the use of, in particular, Google and Youtube requires a degree of sophistication in order to (1) find the most useful sites and (2) assess the reliability of the information provided on those sites. Item (1) requires sufficient knowledge to enter keywords and phrases that pull up useful resources; item (2) depends on a skillset generally referred to as digital literacy.

Given the central role of digital tools and online resources in contemporary mathematical praxis, these two skillsets are absolutely critical components in 21st Century mathematical praxis. That means they have to be part of all students’ mathematics education.

The first skillset, being able to make effective use of search engines to navigate to relevant information and online resources, has to be provided in the mathematics class. It is only by having a good, broad overview of a sufficiently large part of the range of mathematical concepts, facts, methods, procedures, and tools that are available, that a student can select keywords and phrases to conduct a productive search. Such an overview can be acquired only by experience, over several years.

Since there is no longer any need for a student to spend many hours practicing the hand execution of mechanical procedures for solving math problems in order to achieve accuracy and speed, the vast amounts of time freed up from what used to be a massive time sink, can be devoted to working on a wide variety of different kinds of problem.

[CAVEAT: Whether an individual teacher has the freedom to follow this strategy is another matter. Sensible implementation of mathematics education along the lines of the Common Core should, in theory, make this possible; indeed, the CCSSM were developed — as standards, not a curriculum — to facilitate this. But I frequently see complaints that various curricula and local school districts still insist on a lot of rote practice of skills that will never be used. Other than use my bully pulpit to try to change that, as I not infrequently do, I cannot remove that obstacle, I’m afraid.]

Turning to the second skillset, assessing the reliability of the information provided on a Web resource, in today’s world that needs to be a major component of almost every classroom subject. 

In the case of Wikipedia, which is high on my list of mathematical tools, for post-secondary mathematics it is an efficient and highly reliable resource to find out about any particular mathematical definition, concept, or technique — its reliability being a consequence of the fact that only knowledgeable professional mathematicians are able to contribute at that level. Unfortunately, the same cannot be said for K-12 mathematics. 

For example, a quick look as I was writing this post showed that the Wikipedia entry for multiplication is highly misleading. In fact, it used to be plain wrong until I wrote a series of articles for the Mathematical Association of America a few years ago. [See the section on Multiplication in this compilation post.] However, while the current entry is not exactly wrong, its misleading nature is a pedagogic disaster in the making. It therefore provides a good example of why the wise teacher or student should use Wikipedia with extreme caution as a resource for K-12 mathematics.

Ditto for Google, Youtube, and any other online resource. “Buyer beware” needs to be the guiding principle.

Unfortunately, a recent report from Stanford’s Graduate School of Education (see momentarily) indicates that for the most part, America’s school system is doing a terrible job in making sure students acquire the digital literacy skills that are of such critical importance to everyone in today’s world.

Note: I am not pointing a finger at any one person or any one group here. It is the education system that’s failing our students. And not just at K-12 level. Higher education too needs to do a lot more to ensure all students acquire the digital literacy that is now an essential life skill.

The Stanford report focuses on Civics, a domain where the very functioning of democracy and government requires high levels of digital literacy, as highlighted by the massive growth of “fake news” in the period leading up to the 2016 U.S. election, and subsequently. But the basic principles of digital literacy apply equally to mathematics and pretty well any discipline where online resources are used (which is pretty well any discipline, of course). So the Stanford report provides an excellent pointer to what needs to be done in all school subjects, including mathematics.

The report is readily available online: Students’ Civic Online Reasoning, by Joel Breakstone, Mark Smith, & Sam Wineburg, The Stanford History Education Group, 14 November, 2019. (Accessing it provides an easy first exercise in applying the reliability assessing skills the report points to!)

While I recommend you read the whole thing (there is a PDF download option), it is 49 pages in length (including many charts), so let me provide here a brief summary of the parts particularly pertinent to the use of online resources in mathematics.

First, a bit of general background to the new report. In November 2016, the Stanford History Education Group released a study showing that young people lacked basic skills of digital evaluation. In the years since then, a whole host of efforts—including legislative initiatives in 18 states—have tried to address this problem. Between June 2018 and May 2019, the Stanford team conducted a new, deeper assessment on a national sample of 3,446 students, chosen to match the demographic profile of high school students across the United States.

The six exercises in the assessment gauged students’ ability to evaluate digital sources on the open internet. The results should be a wake-up call for the nation. The researchers summed up the results they found in a single, dramatic sentence: “The results—if they can be summarized in a word—are troubling.”

The nation’s high school students are, it seems, hopelessly ill-equipped to use the Internet as a source for information. The report cites the following examples:

• Fifty-two percent of students believed a grainy video claiming to show ballot stuffing in the 2016 Democratic primaries (the video was actually shot in Russia) constituted “strong evidence” of voter fraud in the U.S. Among more than 3,000 responses, only three students tracked down the source of the video, even though a quick search turns up a variety of articles exposing the ruse.

• Two-thirds of students couldn’t tell the difference between news stories and ads (set off by the words “Sponsored Content”) on Slate’s homepage.

• Ninety-six percent of students did not consider why ties between a climate change website and the fossil fuel industry might lessen that website’s credibility. Instead of investigating who was behind the site, students focused on superficial markers of credibility: the site’s aesthetics, its top-level domain, or how it portrayed itself on the About page.

The assessment questions used by the Stanford team were developed after first looking at the ways three groups of Internet users evaluated a series of unfamiliar websites: Stanford freshmen, university professors from four different institutions, and fact checkers from some of the country’s leading news outlets.

Of particular relevance, the fact checkers’ approach differed markedly from the undergraduates and the professors.

When fact checkers landed on an unknown website, they immediately left it, and opened new browser tabs to search for information about the trustworthiness of the original source. (The researchers refer to this approach as lateral reading.)

In contrast, both the students and the academics typically read vertically, spending minutes examining the original site’s prose, references, About page, and top-level domain (e.g., .com versus .org). Yet these features are all easy to manipulate. 

Fact checkers’ first action upon landing on an unfamiliar site is, then, to leave it. The result of this seemingly paradoxical behavior is that they read less, learn more, and reach better conclusions in less time. Their initial goal is to answer three questions about the resource: (1) Who is behind the information? (2) What is the evidence? (3) What do other sources say? 

While the value of the fact-checkers’ approach is critical in navigating today’s online sources of news and current affairs, the approach is no less critical in using online resources in the STEM disciplines.

For instance, imagine the initial stage of collecting data for a mathematical analysis of problems about climate change, the effectiveness of vaccines, or diet — all topics that students find highly engaging, and which thus provide excellent projects to learn about a wide variety of mathematical techniques. In all three cases, there is no shortage of deliberate misinformation that must be filtered out. 

Here then, is a summary of the six assessment tasks the Stanford team developed, listed with the fact-checkers initial questions being addressed in each case, together with (in italics) the specific example task given to the students:

  • Evaluating Video Evidence (What’s the evidence? Who’s behind the information? Evaluate whether a video posted on Facebook is good evidence of voter fraud.)
  • Webpage Comparison (Who’s behind the information? Explain which of two websites is a better source of information on gun control.) 
  • Article Evaluation (What do other sources say? Who’s behind the information? Using any online sources, explain whether a website is a reliable source of information about global warming.)
  • Claims on Social Media 1 (What’s the evidence? Who’s behind the information? Explain why a social media post is a useful source of information about background checks for firearms.)
  • Claims on Social Media 2 (What’s the evidence? Who’s behind the information? Explain how a social media post about background checks might not be a useful source of information.)
  • Homepage Analysis (Who’s behind the information? Explain whether tiles on the homepage of a website are advertisements or news stories.)

The remainder of this post focuses on the study’s results of particular relevance to mathematics education. It is taken, with minimal editing, directly from the original report.

Overall, the students in the high schools study struggled on all of the tasks. At least two-thirds of student responses were at the “Beginning” level for each of the six tasks. On four of the six tasks, over 90% of students received no credit at all. Out of all of the student responses to the six tasks, fewer than 3% earned full credit.

Claims on Social Media question 1 had the lowest proportion of Mastery responses, with fewer than 1% of students demonstrating a strong understanding of the COR competencies measured by the task.

Evaluating Evidence had the highest proportion, with 8.7% earning full credit. 

The Website Evaluation task (which is of particular significance for the use of online resources in mathematics) had the highest proportion of “Beginning” scores, with 96.8% of students earning no points. The question assessed whether students could engage in lateral reading—that is, leaving a site to investigate whether it is a trustworthy source of information. Students were provided a link to the homepage of CO2 Science (co2science.org), an organization whose About page states that their mission is to “disseminate factual reports and sound commentary” on the effects of carbon dioxide on the environment. Students were asked whether this page is a reliable source of information. Screen prompts reminded students that they were allowed to search online to answer the question. The few students who earned a Mastery score used the open internet to discover that CO2 Science is run by the Center for the Study of Carbon Dioxide and Global Change, a climate change denial organization funded by fossil fuel companies, including ExxonMobil. The Center for the Study of Carbon Dioxide and Global Change also has strong ties to the American Legislative Exchange Council, an organization that opposes legislative efforts to limit fossil fuel use.

A student from a rural district in Oregon wrote: “I do not believe this is a reliable source of information about global warming because even though the company is a nonprofit organization, it receives much of its funding from the “largest U.S. oil company–ExxonMobil–and the largest U.S. coal mining company–Peabody Energy” (Greenpeace). Moreover, Craig Idso, the founding chairman of the Center for the Study of Carbon Dioxide and Global Change, was also a consultant for Peabody Energy. It is no wonder this organization advocates for unrestricted carbon dioxide levels; these claims are in the best interest of the Center for the Study of Carbon Dioxide and Global Change as well as the oil and mining companies that sponsor it.”

Another student from suburban Oklahoma responded: “No, it is not a reliable source because it has ties to large companies that want to purposefully mislead people when it comes to climate change. According to USA TODAY, Exxon has sponsored this nonprofit to pump out misleading information on climate change. According to the Seattle Post-Intelligencer, many of their scientists also have ties with energy lobbyists.”

Both students adeptly searched for information on the internet about who was behind the site. Both concluded that the site was unreliable because of its ties to fossil fuel interests.

Unfortunately, responses like these were exceedingly rare. Fewer than two percent of students received a “Mastery” score. Over 96% of student responses were categorized as “Beginning”. Instead of leaving the site, these students were drawn to features of the site itself, such as its top-level domain (.org), the recency of its updates, the presence or absence of ads, and the quantity of information it included (e.g., graphs and infographics). 

A student from suburban New Jersey wrote: “This page is a reliable source to obtain information from. You see in the URL that it ends in .org as opposed to .com.” This student was seemingly unaware that .org is an “open” domain; any individual or group can register a .org domain without attesting to their motives. A student from the urban South was taken in by CO2 Science’s About page: “The ‘about us’ tab does show that the organization is a non-profit dedicated to simply providing the research and stating what it may represent. In their position paper on CO2, they provide evidence for both sides and state matters in a scientific manner. Therefore, I would say they are an unbiased and reliable source.”As the Stanford team note in their report, “Accepting at face value how an unknown group describes itself is a dangerous way to make judgments online.”

In fact, students often displayed a confusion about the meaning of top-level domains such as .org. While there are many .org’s that work for social betterment, the domain is a favorite for political lobby groups and groups that cast themselves as grassroots efforts but which are actually backed by powerful political or commercial interests). For-profit companies can also be listed as .org’s. Craigslist, a corporation with an estimated $1 billion in revenue in 2018, is registered as craigslist.org. Nor is nonprofit status a dependable marker of an organization’s credibility. Obtaining recognition by the IRS as a public charity is an extremely easy thing to do. Of the 101,962 applications the IRS received in 2015, 95,372 were granted tax-deductible status—an approval rate of 94%.

Food for thought, don’t you agree?

For a discussion of the other items in the study, I’ll refer you to the report itself. 

Let me end by re-iterating that the specific findings I listed above are all highly relevant to developing good skillsets for using online resources in mathematics.