The Cult of Evidence and the Importance of Source Quality

The discussion of new affirmatives and Scott’s most recent post about the SPS article controversy intersect at the issue of how we are teaching students to evaluate evidence. I will write more about this over the coming days, but I wanted to chime in with a few thoughts about this meta-issue before discussing more about new affirmatives or about the SPS article controversy in particular.

My agreement with Roy’s initial post was not intended as an indictment of new affirmatives. Instead, I think the proliferation of poor-quality new affirmatives at season-ending tournaments reveals something important about the state of our activity. In particular, the following questions come to mind:

  1. What does it say about the way we are teaching our students that breaking new affirmatives is seen as so strategic at end-of-the-year championships? Why is it that students feel that they have a better chance of winning when they break even a poor-quality new affirmative than they do when reading one of their existing affirmatives?

  2. Does this represent a positive or negative trend? What should we be doing to nudge the competitive advantage toward a style of debate that rewards engagement with the topic literature and the opposition’s arguments more than evasion and trickery?

I don’t think it was Roy’s intention to “call out” those teams that consistently broke new affirmatives at this year’s TOC or to discourage teams from reading new affirmatives in the future. As I have written, there are certainly strategic benefits to breaking new cases and it is good to encourage students to invest the effort required to write a new case and prepare to defend it.

But why is it that so many students feel that breaking a new affirmative is so strategic? Scott does a good job of assessing the typical negative strategy: most 1NCs are terrible, abusive, or not about the case while only a few are well-researched and specific. This is something we should be ashamed of, but these negative strategies win debates.

Why? For the same reason that poor-quality new affirmatives are perceived as strategic: because the quality of evidence comparison and the level of scrutiny applied to evidence in most debates is exceedingly low. Even if they are initially taught to question the sources of their opponents’ evidence and to make arguments about sources in the round, debaters quickly learn that this discussion of evidence quality will not be rewarded nearly enough to justify the time investment.

Judges routinely resolve these debates with a kneejerk appeal to “evaluate the warrants”: “the neg wins that the aff’s solvency author is a quack with no qualifications who is writing on a blog, but if that’s the case, the neg should be able to beat the warrants, and they don’t have any cards that disprove the aff’s author.” In an attempt to ensure that no legitimate sources are excluded, most judges have taken an extreme position, whether implicitly or explicitly: so long as it is published, all sources have equal credibility—only the warrants of the evidence presented to support an argument matter, not its publisher.

This is an intuitively appealing position: learning to keep an open mind, after all, is one of the primary benefits of our activity. Nonetheless, there is a difference between keeping an open mind and refusing to critically evaluate sources of evidence. Students that participate in policy debate—especially at the national level—learn research skills that put their non-debate peers to shame. But in an important way, debate students are being taught to be poor consumers of information. One of the first things that middle school students learn when writing their first research paper is that the source matters. There is a wealth of information on the internet about the evaluation of information sources, and teaching students how to think critically about the information they consume is one of the most important jobs of our education system. At this point, almost anyone can find almost anything on the internet; very few people, however, can effectively separate the wheat from the chaff. Our goal as educators should be for debaters to be part of that latter group.

In the context of the SPS article controversy, Scott writes:

If you are aff you should have a solid defense of your case that comes from peer reviewed journals and is written by qualified authors. You should also be able to explain why evidence that does not meet rigorous academic standards should be discarded- if you can’t you will lose to way more University Wire/Sac bee cards than cards written by other participants. Part of the reason this is a problem is because of the delcining standards of what constitutes evidence, the “cult of evidence” that thinks any card auto beats an analytic, and because debaters are taught to just read cards and not critically think and deconstruct arguments ( a definate failing on the part of coaches).

This is all true. The implication, however, is not that we should ignore practices that we find unethical simply because we have done a poor job of teaching our students to critically evaluate evidence. (I will discuss more about the issue of coach-generated evidence in a future post.)

The “cult of evidence” results in a feeling on the part of debaters that they must have a card about every possible argument. “What if they say XYZ and I don’t have a card?!?,” the 2A will gasp before a big debate. The concern, it is important to note, is not about the opposition having a good argument that the affirmative is unprepared to answer. A well-prepared affirmative team that has read the books, journals, and think tank articles about their case should be prepared to defend it even if the negative makes an argument for which the affirmative does not have specific evidence. But because we are teaching our students that evidence is evidence and that you’ve gotta have a card, we are fostering a culture that devalues smart analysis and incentivizes a race to the edges of the topic. Why know more than your opponents when you can just find something they don’t know about?

Great debates do not occur when teams avoid engaging their opponents. As educators, we should do all that we can to ensure that more great debates occur. A necessary part of that process is to nudge the dominant argument culture away from “the cult of evidence” and toward “the cult of intelligence”. It’s cool to know a lot about the topic, and we should never let our students forget that.

There is nothing wrong with competition; it is, in fact, the reason that we all work so hard at something as educational as debate. My claim is not that we need to suppress our competitive impulses. Instead, we need to be cognizant of the practices our competitive impulses are pushing us toward and question whether they are practices in which we should take pride. If “being strategic” currently results in un-educational debates, then we need to consider how to alter teams’ strategic calculations in ways that result in more educational debates.

The solution to poor-quality new affirmatives, in other words, is not to insist that teams stop reading poor-quality new affirmatives: it is to change the way the game is played in ways that remove the incentive for teams to read those affirmatives. In my opinion, this means doing more to emphasize critical analysis of the sources that are cited as evidence in debates.

5 thoughts on “The Cult of Evidence and the Importance of Source Quality

  1. Ryan Ricard

    So how do we make decisions between bad ev and no ev? I think there’s a disconnect between debateworld and reality when it comes to what to do when faced with a lack of good evidence.

    Stupid contrived example: If I worked for the FDA, I’d never say “well, the only study about this drug being safe was really shoddy. Poor methodology, not a big enough sample, possible bias. But there are no studies that say it isn’t safe. Approved”

    When faced with a lack of appropriate evidence, a good scientist would say “go get more evidence”. Unfortunately, this isn’t possible in a debate round, so I suppose the only tool we have left is analytical reasoning. We need a better way of articulating that notion, that “bad evidence” is a priori better than “no evidence”

    Two other quick thoughts:
    -Public shaming (of ev, not debaters) is good. See a crappy card? Grab the cite and throw it up on the web. Talk about how bad it is.
    -Maybe we don’t require enough citation information to distinguish crap from good. If I had two cards, one from “Weekly World News” and one from “Newsweek”, and I didn’t have prior knowledge about those two publications, how would I know which is celeb gossip and which is legit news? Of course, this is even worse for smaller fields like alt energy.

  2. Roy Levkovitz Post author

    Here is some additional proof we need to be weary of what we’re cutting. Here is an article about wikipedia and newspapers using it

    Irish student’s Wikipedia hoax dupes newspapers
    http://www.google.com/hostednews/afp/article/ALeqM5gQV2LU_QhL5w_BcPY5B6pvuUUMGg

    DUBLIN (AFP) — An Irish student’s fake quote on the Wikipedia online encyclopaedia has been used in newspaper obituaries around the world, the Irish Times reported.

    The quote was attributed to French composer Maurice Jarre who died in March.

    Shane Fitzgerald, 22, a final-year student studying sociology and economics at University College Dublin, told the newspaper he placed the quote on the website as an experiment when doing research on globalisation.

    He quoted Oscar-winning composer Jarre as saying, “One could say my life itself has been one long soundtrack. Music was my life, music brought me to life, and music is how I will be remembered long after I leave this life.

    “When I die there will be a final waltz playing in my head, that only I can hear.”

    The quote was posted on Wikipedia shortly after Jarre’s death and later appeared in obituaries in major British, Indian and Australian newspapers.

    Fitzgerald told the newspaper he picked Wikipedia because it was something a lot of journalists look at and it can be edited by anyone.

    While he was wary about the ethical implications of using someone’s death as a social experiment, he had carefully generated the quote so as not to distort or taint Jarre’s life, he said.

    Fitzgerald said he was shocked by the result of his experiment.

    “I didn’t expect it to go that far. I expected it to be in blogs and sites, but on mainstream quality papers? I was very surprised about,” he said.

    He said the hoax remained undiscovered for weeks until he e-mailed the newspapers that had been deceived to tell them that they had published an inaccurate quote.

    The Irish Times said that despite some newspapers removing the quote from their websites or carrying a correction and the fact that it had been dropped by Wikipedia, it remained intact on dozens of blogs, websites and newspapers.

  3. Pingback: The 3NR » Exploiting Inefficiencies: Moneyball and Opportunities For Innovation in High School Policy Debate

  4. Pingback: Academic Debate And The Ethical Use of Evidence — PFDebate Blog

  5. Pingback: The 3NR » Nudging Evidence Analysis In The Right Direction: The Case For Reading Author Qualifications Aloud In High School Policy Debate

Comments are closed.