Paul Strait of the University of Southern California recently authored an interesting post on the CEDA forum about the time it takes judges to make their decisions. As discussed in a previous column, this is a hot topic in the college community because the average length of decisions at that level is forcing tournaments to consider reductions in the number of preliminary rounds offered in order to prevent marathon tournament schedules. Paul’s contention is that we need to foreground consideration of judging methodologies in order to determine what contributes to lengthy decisions and what effect this has on the quality of decisions.

Most judging philosophies are a collection of banal “dont change how you debate on my account” comments (which, incidentally, are question begging for so many reasons) along with some belief statements about various theory issues. Very few philosophies explain the process or method that the judge plans on using—especially those belonging to young judges. I think this speaks to the fact that there is very little emphasis in the community on consciously developing and then following a judging methodology. I think if people did think about this process more, and put that process into their judging philosophy, inexperienced judges would be more likely to have a strategy to follow after the 2AR finishes. More importantly, the methods of other judges could be inspected and discussed — we would have some data to inform our conversation. The best we can do now is “well, if you do x, you should do y instead,” which is only but so useful.

Paul outlines a number of questions that can help judges develop and describe their judging methodology.

How much of your decision do you prepare before the debate is over (mentally and/or written down)?

How do you decide which part of the debate to evaluate first?

Under what conditions will you call for a card?

What will you do with those cards besides read them (compare with the flow, write down warrants, write down citations, write down some kind of evaluative comment, look at un-underlined parts, reread the tag and other stuff on the page, etc.)?

Do you read the card first and then ask yourself questions about it? or do you have a specific question in mind every time you start reading a new card?

What kinds of questions will you spend the bulk of your deciding time attempting to resolve?

Do you generally decide who you will vote for at the very end, or do you decide earlier and then spend the rest of your time making your decision comprehensive / preparing for questions from the losing team?

Are you mostly an information gatherer or do you argue with yourself back and forth in your head (and if so, how do you conduct such arguments)

What if anything do you do to ‘double-check’ that you aren’t missing anything?

What conditions have to be satisfied for you to have confidence that your decision is sufficiently ‘careful’?

This is a fascinating battery of questions. I will describe my answers to each of them in hopes of beginning a conversation among high school coaches and judges about judging methodologies. I encourage other judges to share their responses to these questions and to pose new ones that they think might help illuminate a given judge’s methodology.

How much of your decision do you prepare before the debate is over (mentally and/or written down)?

Almost all of it. I actively think about the debate as it is unfolding and I usually have a mental checklist of things that the 2NR and 2AR need to do in order to win. I place a lot of emphasis on the “thesis positions” of each team, or their overall narrative or vision of the debate. I actively attempt to prevent myself from allowing the micro issues in the debate to overshadow the macro issues; when that becomes necessary, I tend to take much more time resolving the debate.

How do you decide which part of the debate to evaluate first?

By feel—I try to determine which part of the debate is most relevant to the resolution of the two teams’ thesis positions. This can depend a lot on the type of debate that occurred: it can be anything from “does the counterplan solve the case?” to “are the negative’s interpretation and the affirmative’s counter-interpretation mutually exclusive?”. Lately, I have consciously thought more about how to approach my decision—in a close debate (or a debate that I perceive as close), I will first map out the way I plan to make my decision and then determine based on that plan which issue to resolve first.

Under what conditions will you call for a card?

This also depends on the issue but for the most part I will ask for evidence under three circumstances:

1. A piece of evidence is contested either explicitly (“their evidence is bad”) or implicitly (“prefer our evidence”).

2. A piece of evidence is uncontested but relevant to a broader question. Most frequently, this occurs when one team concedes an argument and I must determine the relative weight of that concession compared with another argument. I do this less now than I used to because I tend to find that most evidence is implicitly contested even when “dropped”. If I decide that the evidence really isn’t contested, then I typically do not bother reading it except to provide commentary (see #3).

3. A piece of evidence is contested or uncontested that I want to explain or discuss with one or both teams. This is usually a situation where a card could have been used differently or where one or both teams did not understand something about the issue it was discussing. This circumstance does not influence my decision, only my post-round commentary.

What will you do with those cards besides read them (compare with the flow, write down warrants, write down citations, write down some kind of evaluative comment, look at un-underlined parts, reread the tag and other stuff on the page, etc.)?

I first identify the placement of the evidence within the narrative structure of the debate (in which speech was the evidence originally presented, to what extent was the evidence discussed in later speeches, how much emphasis was placed on the evidence by the team that advanced it, how much emphasis was placed on the evidence by the team that responded to it, etc.). I often will quickly read most of the evidence that was read on an issue (an entire advantage or disadvantage shell, for example) to remind myself of where that particular debate began. Once I refresh that context, I begin carefully examining the pieces of evidence that I felt were crucial to each side’s thesis position. For those cards, I read the un-underlined portions as well as the citation/qualifications. I type notes in a text document about each card and about each issue as a whole. The degree to which I consider the “undebated” portions of evidence (un-underlined, citation, qualifications, etc.) varies based on the quality of the debating and my overall feel for the debate. I attempt as much as possible to reflect with my decision a faithful reading of the debate that occurred, but I am definitely more willing than most judges to scrutinize micro-level arguments and filter them through a lens that I develop based on the way I resolve the macro-level issues.

Do you read the card first and then ask yourself questions about it? or do you have a specific question in mind every time you start reading a new card?

In most cases, I have a specific question in mind when I am reading a given piece of evidence. As I review it, other questions are often raised and I subsequently resolve those as well. In some percentage of debates (probably 33% or less), I resolve an important issue by reading the evidence on both sides and scrutinizing it piece-by-piece; this usually occurs in very good debates that are close and focused on an in-depth issue.

What kinds of questions will you spend the bulk of your deciding time attempting to resolve? Do you generally decide who you will vote for at the very end, or do you decide earlier and then spend the rest of your time making your decision comprehensive / preparing for questions from the losing team?

I think these two questions go together. As discussed previously, I concentrate mostly on resolving the two teams’ competing thesis positions. I almost always decide debates immediately (or more accurately as the 2NR and 2AR are unfolding) and spend the rest of my decision time confirming my initial impression and subjecting it to rigorous scrutiny. As I have described before,

[I] actively consider my decision as the debate is unfolding and then come to a contingent decision immediately after the debate has ended. In about 90% of the rounds I judge, this contingent decision is confirmed through the subsequent process of deliberation. In the other 10%, I ultimately decide that my initial decision was incorrect—usually based upon unexpected differences in evidence quality or because of an argument interaction that I had not initially grasped.

Are you mostly an information gatherer or do you argue with yourself back and forth in your head (and if so, how do you conduct such arguments)?

I play devil’s advocate and attempt to prove to myself that my initial decision is wrong. That process often requires information gathering but that is rarely my intention when reviewing evidence.

What if anything do you do to ‘double-check’ that you aren’t missing anything?

In close debates, I copy-and-paste the final rebuttal of the team that I am going to vote against into a fresh document and delete each argument one-by-one as I determine that I have sufficiently evaluated it. In debates that aren’t as close, I do the same thing but less rigorously.

What conditions have to be satisfied for you to have confidence that your decision is sufficiently ‘careful’?

This is something that I have attempted to formalize: I require three conditions to be met before turning in my ballot.

1. Have I given both teams a fair hearing by considering their overall narrative as well as the evidence they have advanced to support it? For me, this often requires evaluating the debate twice: first through the lens of the affirmative’s narrative and then through the lens of the negative’s narrative. When I stop myself and go back to be more careful, it is almost always because I decided that I was too quick to discount one team’s thesis position and therefore evaluated one or more micro-level arguments without giving one side enough credit.

2. Would I be comfortable with my decision as a universal norm? In other words, would I be satisfied if all judges resolved this debate in the same way that I resolved it? If my students were on the receiving end of this decision from another judge, would I be satisfied with that judge’s decision? This is a feel thing—if I am satisfied that I have met the “fair hearing” standard, I am almost always comfortable with imagining my decision as a universal norm. Occasionally, however, this standard forces me to revisit the decision and more thoroughly consider whether I am being faithful to the narrative that developed in the round.

3. Am I prepared to defend my decision and provide educationally enriching commentary to the students on both teams? This sometimes requires taking a few more minutes to outline my decision and to construct a bullet pointed list of suggestions for each team/debater. I have found that I can do this more quickly by working on this outline and especially the suggestions during the course of the debate.