The Internet is awash in urban legends and outright lies. It’s also the best fact-checking tool in history, if you know how and where to look. (More)
Clueless, Part III – Finding Clues (Non-Cynical Saturday)
This week Morning Feature looked at cluelessness and its causes. Thursday we considered common ignorance. Yesterday we examined political ignorance. Today we conclude with how to open our own and others’ eyes.
Banning Religious Broadcasting
A friend invited me to visit his church during my first year of law school. My former church was less a religious body than a conservative political action group. They asked me to leave when I disputed their claim that protecting LGBTs in the county’s civil rights ordinance would force them to hire a gay pastor. The county ordinance specifically exempted churches.
Not to worry, my friend said. His church didn’t talk about politics. I said okay.
Good morning … please give a warm welcome … and before we start, did you know there’s a bill in Congress to ban religious broadcasting? I have a flyer right here. Please write to your congressman and senators….
I’d heard the story before. Many times. It began circulating in the mid-1970s, after the FCC rejected a petition to limit religious groups’ use of channels reserved for educational programming. The FCC rejected the petition on First Amendment grounds, and that should have been that. But the FCC has received over 30 million pieces of mail in the decades since, demanding they stop their plan to ban religious broadcasts. House and Senate members get letters as well, as the story sometimes morphs into a bill proposed in Congress.
“Has anyone verified this?” I asked. “It sounds like a hoax that’s been making the rounds for years. It would take only a few minutes to check out at a law library.”
Healthy Skepticism
Skepticism has acquired a bad reputation over the years. For many, it’s become synonymous with cynicism: disbelieve everything. But cynicism is, as Brooks Jackson and Katherine Hall Jamieson wrote in unSpun, “another form of gullibility.” It’s just as irrational to disbelieve everything as it is to believe everything.
Skeptics look for evidence, and rigorous skeptics look for a particular kind of evidence. Philosopher Karl Popper called that evidence “risky predictions,” and proposed seven guidelines for examining a theory:
- It is easy to obtain confirmations, or verifications, for nearly every theory — if we look for confirmations.
- Confirmations should count only if they are the result of risky predictions; that is to say, if, unenlightened by the theory in question, we should have expected an event which was incompatible with the theory — an event which would have refuted the theory.
- Every “good” scientific theory is a prohibition: it forbids certain things to happen. The more a theory forbids, the better it is.
- A theory which is not refutable by any conceivable event is non-scientific. Irrefutability is not a virtue of a theory (as people often think) but a vice.
- Every genuine test of a theory is an attempt to falsify it, or to refute it. Testability is falsifiability; but there are degrees of testability: some theories are more testable, more exposed to refutation, than others; they take, as it were, greater risks.
- Confirming evidence should not count except when it is the result of a genuine test of the theory; and this means that it can be presented as a serious but unsuccessful attempt to falsify the theory. (I now speak in such cases of “corroborating evidence.”)
- Some genuinely testable theories, when found to be false, are still upheld by their admirers — for example by introducing ad hoc some auxiliary assumption, or by reinterpreting the theory ad hoc in such a way that it escapes refutation. Such a procedure is always possible, but it rescues the theory from refutation only at the price of destroying, or at least lowering, its scientific status.
In the case of the bill to ban religious broadcasting, my “risky prediction” was that any such bill could be found in a search at a law library. If a thorough search finds no such bill, then the flyer is a hoax. That’s a bit complicated, because the provision might be in an amendment to some other bill. But with computer search tools, it’s not difficult to do.
The Best Fact-Checking Tool Ever
As Jackson and Jamieson write in unSpun, the Internet is the best fact-checking tool in history … if you know how and where to look. The Annenberg Public Policy Center run FactCheck.org, and that is one good fact-checking reference. You may not always agree with them, but they are transparent and list their sources for every story. That transparency is essential in online fact-checking. If there’s no source for a claim, be very wary.
But merely giving a source is not enough. I often find circular sourcing: A cites B, B cites C, and C cites a previous article by A. That is especially common in blogs and opinion columns, but you may find it in news stories as well. While circular sourcing proves A, B, and C agree, it doesn’t prove their claim is true.
Instead, Jackson and Jamieson suggest you look for reputable sources: official government statistics and transcripts, peer-reviewed academic journals, and media outlets with reputations for solid reporting. Remember that even reputable sources make mistakes, especially when a story is first unfolding. Don’t disbelieve breaking news stories outright, but mentally insert the words “so far as we know” after every fact claim. The story will probably change as more information emerges.
Jackson and Jamieson also say to be wary of blog sites, and I agree. At BPI we tell our writers to check their facts – especially the ones they think they already know – and cite their sources. While our writers can’t cite sources for personal anecdotes, remember those are anecdotes and “the plural of anecdote is not data.” Oh, and don’t be so sure Roger Brinner said that.
How Can You Know?
Jackson and Jamieson offer these rules for testing facts:
- You Can’t Be Completely Certain – The universe is messy and surprising, so be skeptical if someone claims complete certainty.
- You Can Be Certain Enough – What are the consequences of believing or disbelieving this claim, if you’re wrong? If the consequences are trivial, you don’t need much evidence. If the consequences are severe, be more skeptical.
- Look For General Agreement Among Experts – That doesn’t guarantee they’re right, but it’s more likely.
- Check Primary Sources – Be wary of second-party summaries, headlines, quotes, and video snips. Look for the original material.
- Know What Counts – When you see statistics, check out what they counted and how they counted. When you see polls, find the original polling site and read the questions that were asked.
- Know Who’s Talking – When you see studies or reports, check out the researchers or groups involved. Go to their website and find out who they are, and who funds them. “Non-partisan” does not mean “has no political agenda.”
- Seeing Shouldn’t Necessarily Be Believing – We all misinterpret what we see at times, and memory is very fallible.
- Cross-check Everything That Matters – This ties into #2 above. The more severe the consequences of being wrong, the less you should rely on a single source, even if that source is reputable.
- Be Skeptical, Not Cynical – Appeals to cynicism are common, and often come as claims of Secret Knowledge: “Don’t believe what They say. I have the real truth.” If you can’t verify it with other sources, be wary.
I’ll add a tenth rule: “I Don’t Know” Is Not Failure – There’s an old law school joke:
Q: What’s the difference between a first-year law student and an experienced attorney?
A: Ask a legal question, and the law student will answer “I don’t know.” The attorney will answer “I need to research that.”
In fact, they’re both giving the same answer. And often that’s the best answer we can give.
+++++
Happy Saturday!
Excellent, Crissie, and perfect for non-cynical Saturday!
I like the argument that being cynical is another form of gullibility. I think it is often used as an excuse as well – an excuse to not take action on something because “what is the use?”
Cynicism is not only an excuse, Glenda. It’s sometimes a strategy. For example, Republicans oppose government solutions as a matter of ideology. They often set out to make government programs fail, so they can then say “See? Government can’t work!” (I’ll share some documented examples in next week’s series.)
That strategy works two ways: (1) the program under attack may be repealed or allowed to wither; and, (2) more ordinary Americans think “what’s the use?” and don’t bother to be politically active. That leaves the wealthy – who are politically active – an even more dominant voice.
Good morning! ::hugggggs::
“Consider the source” comes to mind. I honestly hadn’t thought of fact checking in terms of trying to refute things. That’s an interesting method, and I can see how it works.
Sadly most of us don’t have the time to check into most things. I know I don’t, so I’m probably running around with a whole boatload of misconceptions. Guess I need to rely on Factcheck more often, since it’s gotten so I don’t know what to believe about what I read a lot of places.
All too often I rely on a gut-check: is this good for people? And I could be seriously wrong about that first impression.
I’m going to be referring to this article often.
That gut-check – “Is this good for people?” – is actually a good place to start, winterbanyan. It goes to point #2 in Jackson and Jamieson’s checklist: what are the consequences if I’m wrong? That question should set your standard of evidence.
Trying to refute a claim can be as simple as: “If this were true, I should find sources who have no stake in whether I believe it.” If the only sources making a claim have a stake in whether you believe it … ask yourself: “Does this seem too good (for them) to be true?”
Journalists call that the “sniff test,” and you should use it even if – indeed especially if – the claim is also good for you. If it seems “too good to be true,” it’s usually false.
Good morning! ::hugggggs::
The word “theory” has its own problems. Non-academicians use it when they have a reasonable notion of why something is the way it seems to them. It implies something is well thought out and tested. Academicians and especially researchers in the hard sciences have a very specific meaning which Jackson and Jamieson hint at with their checklist of questions. A real scientific theory is the result of lots of critical thinking, experiments and analysis.
A few of my PhD friends get sputtering mad when a preacher rails against evolution as, “It’s just a theory.”
When I am talking with someone, my favorite questions are, “How do you know that?” and “How do you know that is true.” It helps me figure out what kind of discussion we are having.
There is a long continuum with belief (in the faith based or religious sense) on the one end and scientific thinking (in the rigorous scientific method sense) on the other end.
“I believe” has as many problems of carrying a host of meanings as “I have a theory” does.
Better to say: I have a good working hypothesis. 😉
That’s a great point, addisnana. People often discredit scientific explanations as “just a theory.” Well yes, but in science a “theory” is not idle speculation. As Hopper noted, a scientific theory makes “risky predictions,” and scientific testing sets out to refute those predictions.
While not every public policy claim turns on science, we can and should expect evidence. If a claim cannot possibly be disproved – if any possible facts can be “made to fit” the claim – be very skeptical. Such claims don’t allow you to learn from experience … and that’s a dangerous place to be.
Good morning! ::hugggggs::
A few months ago i received an email from an OFAer who was passing along an email that he received from a friend. The gist of the email was that congress people and their staffs were getting unbeleivable perks that the common man would never receive. Among other things, the email claimed that congress staffers were getting free tuition, room and board, etc.
I wrote back and said, “John, this is a tea party inspired effort.” John writes back: ‘Mike, if I didn’t know you…, and went on to say that his friend is no tea partier and that I was missing the populist boat. This is what the common man reacts to.”
A few more exchanges and I finally go to factcheck.org and sure enough, the whole campaign was a tea party effort and the facts were wrong. I send John a note with a link to factcheck. He sheepishly writes back and says that he can’t beleive his friend could be so wrong and apologizes to me.
I applaud you on setting your friend straight using facts and I applaud his apology. But, when lying becomes the strategy of choice on the other side, I think we have to do more than simply call out each lie individually as it arises.
We need to point out a pattern of lying as a way to confuse and/or rile the populace. If we don’t take some sort of proactive approach to this we will be chasing down individual lies ad infinitum.
Yes, addisnana, and the Medicare episode gives us that opportunity, don’t you think.
Totally agree!
Thank you for that excellent example, Mike. You brought it home to Fred Whispering. I intended to do that in the article, but I ran out of space. We need to be Fred’s fact-checkers … and that’s why we need to learn how to fact-check. If we do fact-check well, and if we Fred Whisper courteously, Fred will listen.
Good morning! ::hugggggs::
Thank you for the tenth rule.
Next step is to design standardized tests that include the Tenth Rule in their grading scheme.
I like that very much, Jim. Very much indeed. “I don’t know” seems to be one of the statements we are most afraid to make, yet it’s most often probably the truest one we can make.
Thank you, Jim. I use the Tenth Rule a lot, and I’m always a bit taken aback when someone replies: “But you must have an opinion!”
I usually say something like: “Actually I just don’t know enough to say, one way or the other. I’ll check it out and get back to you, though.”
The most common response: “But what do you think?”
Sometimes I can explain what I would look for and where I would look. But sometimes I don’t know enough about the topic to even say where I’d start looking. I’m okay with that … but many people aren’t.
Good morning! ::hugggggs::
“I don’t know” is a great tool for keeping political discussions from degenerating into an unseemly contest of dubious assertions.
If we’re willing to say “I don’t know” when it’s applicable, that demonstrates honesty and provides a basis for trust. It provides precedent and cover for the other parties in the discussion to admit that maybe they don’t really know either. It prompts questions like “well, what DO we know” and “how could we learn more.” It establishes the conversation as a shared project to arrive at a better approximation of the truth, rather than an adversarial face-off where winning is more important than truth.
Outstanding diary today, Crissie. One of your best.
Thank you, DBunn. And yes, refusing to say “I don’t know” can turn any discussion into an “unseemly contest of dubious assertions.”
Part of the problem is another heuristic: we tend to equate confidence with trustworthiness. Again, that mental shortcut works well enough and often enough to be useful. Experts are usually more confident about their fields of expertise than non-experts. Even if the expert doesn’t know the answer to a question off-hand, the expert can usually confidently explain how to find the answer.
Most of us want others to trust us, and “I don’t know” seems like it will undercut that. But so will getting caught out in an bold but false claim. Someone who always sounds confident and never says “I don’t know” either knows everything or is willing to bluff when he/she doesn’t know. Which is more likely?
Good afternoon! ::hugggggs::
In line with the theme of the week, one of my favorites is, “I have no clue.” This becomes much easier to admit as one ages.
This post is a good one to bookmark as it has a lot of excellent lists. Thank you for giving us the tools to be better informed as news consumers and opinion writers.
+++
This is sad:
Not because 30 million people were duped into thinking this was a problem (or 10 million people were duped 3 times each) but because it morphed into a bill proposed by Congress. We have dumbed down our Congress to the point where they are fact challenged … or, cynically, they are doing it to appeal to an easily duped base.
+++
And thank you for this: “the plural of anecdote is not data.” It is an excellent comeback to a lot of situations. Expect to see it again. 😉
Thank you, and I apologize as I wasn’t clear on this:
The story sometimes morphs, but no such bill was proposed. Both the continuing story about the FCC rule – which was rejected back in 1975 – and the stories about bills to ban religious broadcasting are myths.
“The plural of anecdote is not data” is an old and wise aphorism. It is most often attributed to Roger Brinner, but I could not confirm that he was the first to say it, or indeed that he ever said it.
Good morning! ::hugggggs::
I was duped again!