The Iceberg, Method

Like many writing instructors, I teach Hemingway’s “iceberg method” to students. I make sure to remind them that Hemingway was not an “academic” writer but a novelist and writer of short stories but, still, I tell them, he was adamant that writers must know what they’re talking about. If you’re going to write a novel about war, or bullfighting, or love, you better have some knowledge of the subject to start with. Indeed, he would argue, you should have some experience with these things. And, while academic knowledge is not always based on direct, personal experience, there is one section of a typical research paper that can apply Hemingway’s method almost directly. This, it turns out, is the methods section.

Let’s quickly recap what Hemingway meant. In fact, let us let Papa himself explain it.

If a writer of prose knows enough of what he is writing about he may omit things that he knows and the reader, if the writer is writing truly enough, will have a feeling of those things as strongly as though the writer had stated them. The dignity of movement of an ice-berg is due to only one-eighth of it being above water. A writer who omits things because he does not know them only makes hollow places in his writing.

Ernest hemingway, death in the afternoon

How does this apply to writing your methods section? Well, imagine writing the full story, in every detail, of how you gathered your data and analyzed it. This would be more Proust than Hemingway, I guess, and it’s not something you’d ever actually hand in for a grade or submit for publication. But imagine making it so complete that the reader could, not only replicate something like your study, but would in effect be doing the very same study if they did everything you tell them. In fact, imagine giving them instructions for having the very same experience that you did. Imagine putting in physical descriptions of the locations in which you interviewed your subjects or the office where you came up with your survey questions. Describe the bus ride to the field location where you carried out your observations, even what you had for lunch. Put in all your subjective judgments and perceptions, everything you thought and felt. Tell the reader where you are in doubt that you did it right, and which parts you thought at the time were just brilliant. Tell them what your enjoyed and what you suffered through, what gave you pleasure and what gave you pain. Capture every nuance of the process.

Now, think of your reader as someone who has done a similar study. What can you leave out of your account and let this reader fill in with their own experience? What do you have to keep in your account so that the reader can feel that you actually did all the things you leave out, because that’s what the reader would have done to have the experiences you describe. What parts are irrelevant to whether or not your data is of high quality because the reader assumes that you did them in the proper way (and you did them in that proper way, of course). What lengthy descriptions in our imaginary first draft can be summarized in a single bit of jargon (“semi-structured interview”, “control group”, “coding scheme”)? What is the simplest possible statement of your method to a reader who understands your methodology?

We an easily imagine that this version would be (less than) one-eight as long as the “Proustian” draft. That is, you can leave a great deal under the surface and the reader will still know everything they need to know to replicate your study, or at least enough to trust your data. “A writer’s problem does not change,” Hemingway said. “He himself changes, but his problem remains the same. It is always how to write truly and, having found what is true, to project it in such a way that it becomes a part of the experience of the person who reads it.” In research writing, this art of writing “truly enough” begins with an honest statement of your method that your reader understands. Face that problem squarely and you are well on your way.

Five Easy Paragraphs

This post is not about the five-paragraph essay.

This semester, I’ve decided to offer a course for faculty and doctoral students about how to write the ideal introduction and conclusion of a paper or, perhaps more precisely, the ideal paper‘s introduction and conclusion. I know that no paper, introduction, or conclusion is ideal, but it can be a good exercise to give yourself conditions that let you approximate an ideal. In my course, then, I will assume that the participants can take five deliberate moments to compose the first three and the last two paragraphs of a (real or imagined) paper that presents one of their research results. Since I’m assuming they have such a result, the hard part (doing the actual research) is presumably behind them. We will even leave the struggle of writing the whole paper on the side; it is only the introduction and conclusion that will interest us. We will focus on five easy paragraphs that can be written in two and a half hours, or five half-hour writing moments at the start of the day for a week.

The first paragraph of your introduction should evoke a world. This is the first minute of your reader’s attention and it should set the scene for the result you will present in the third. It is the world you share with your reader, so it should be familiar but also interesting. This is not where you tell your reader that the UFOs are real and the Andromedans walk among us. That would be “out there”. But, if your analysis does indeed suggest that we are being visited by beings from other planets, then this opening paragraph may be where you say that the Pentagon has acknowledged that they have some video that is very difficult to explain. That, after all, has been reported in the New York Times. That is, while your description of the world in this paragraph should be uncontroversial, you should use what you know about the subject to make it interesting.

The second paragraph should invoke a science. Please remember that this is your science and the science of your reader. It is not “their” science, i.e., that of a community that is foreign to you and unfamiliar to your reader. You are not introducing your reader to a new body of knowledge, you are reminding your reader of the state of the conversation in their own discipline, which is also yours. As a rule, I suggest you characterize it either as a consensus or a controversy; your field may be founded on a broad agreement or, at the moment, engaged in a foundational dispute. It may be a bit of both, but you should write this paragraph with a focus on one or the other. Describe your field as a conversation between named scholars who are aware of each other’s work. Keep this paragraph human and concrete, not conceptual and abstract. You are describing a community of people who are brought together by a shared interest in a set of ideas.

The last paragraph of your introduction should now propose a thesis of your own. We could also say that it should state your conclusion. “This paper shows that…” is a great way to start this paragraph, followed by a straightforward statement of your results. That statement should be theoretically significant and empirically true. That is, its meaning should be depend on your theory but its truth should depend on the analysis of your data. Importantly, this paragraph does not claim that your result is correct, only that you paper will show that it is. That means you will elaborate on exactly how it will show it, and the ideal way to do this is to write two sentences about your methods (“It is based on…”), three sentences about your analysis (“The findings indicate…”), and two sentences about your discussion (“This has implications for…”). Including the first sentence about the paper, that’s eight sentences in all. They are doing a lot of work, but they get it done in under a minute of the reader’s attention.

These first three paragraphs can be imagined as three concentric circles. In the middle, there is your object; around this, there is your paper, then your science, and then the world. “The world,” said Wittgenstein, “is everything that is that is the case,” but you are interested in some particular set of facts, and you have constructed these facts using the theories and methods of your science. You live in a society, done a scientific study, and gotten a result that you are presenting in a paper — the paper you are here introducing to a reader who also does science and lives in the world.

The first paragraph of your conclusion should restate your thesis. Remember that it’s been about thirty-five paragraphs (a good half hour of reading time) since we last read it in its plainly stated form. Your key sentence for this paragraph could simply be whatever you wrote after “This paper shows that” in paragraph three. But, to keep this interesting also for you, I recommend that you find a way of rephrasing it so that it says the same thing but, this time, for a reader who has just read your whole argument (background, theory, method, analysis, discussion, and all). The rest of the paragraph should support the conclusion. Write the strongest, least apologetic statement of the reasons you think your conclusion is true. Assume the reader is familiar with your methods and with your data and already understands the limitations of your study. Say it the way you say it to yourself, fully understanding what you mean. Your reader is ready and deserves to be addressed seriously.

The last paragraph of should reimagine your world or your science in the light of your results. Keep in mind that there will ideally be some interesting tension between your theory and your analysis, which your discussion has dealt with the implications of. The first two paragraphs of your introduction evoked a world and invoked a science that were both innocent of your results. Now that we know what your study shows, how should we either either see the world or do our science going forward? Maybe we should prepare for one or another crisis waiting us in the future; maybe we should stop being worried about one. Maybe we should resolve the standing controversy in our field and replace it with a consensus of a kind you specify; maybe the years of peaceful consensus are over and it is time to debate the issues vigorously in our discipline. Maybe we need some specific new policy; maybe we need to conduct more research. Whatever you do, try to leave the reader with a clear view from exactly the point that you now yourself proceed.

These five paragraphs are of course easier to write about than to actually write. But I do want to insist that they are easier to write than what they are about is to know. Writing them is therefore a good test of whether you’re ready to write a paper. Take 5 x 30 minutes to find out whether you have something to say. That’s what my little course this spring will show how to do.

How They Must Write: Saving the Five-Paragraph Essay and Other Contingencies

John Warner’s Why They Can’t Write: Killing the Five-Paragraph Essay and Other Necessities was published at the end of 2018 and was widely celebrated by my fellow writing instructors. Initially provoked by its title, however, I found the book less engaging than I had expected; and my exchanges with its author on Twitter were also not as productive as I had hoped. It turned out that he had much bigger fish to fry than the simple writing assignment he had used as a hook and, after a year of trying to debate various issues, we finally gave up and left each other in peace. Then, in early December of last year, Ian Bogost, whose work I’ve long admired, cited Warner’s book approvingly in the Atlantic on the occasion of the release of ChatGPT, which, argued Bogost, will make traditional school papers a thing of the past, where he thinks they belong. They bore him. “Imagine worrying about the fate of take-home essay exams,” he balked as if looking directly at me, and then added, as if winking to Warner, “a stupid format that everyone hates but nobody has the courage to kill.” Ouch!

I’ve been thinking about this over the break and it seems fitting to start the new year with some reflections on the “freakout” that ended the last one. I use that word advisedly. When Open AI released its new toy, and everyone, not least me, began to play with it to see what it could do, Warner published a post called “Freaking Out About ChatGPT,” in which he assured us that it was “not a threat to anything truly important when it comes to student learning,” even though it could produce a passable five-paragraph essay on most topics without any learning on the part of the student who used it. This, Warner suggested, said more about “the kind of work we allow to stand in for student proficiency” than it does about the about the efficiency of modern technology. Like Bogost, he thinks five-paragraph essays are worthless. The fact that an algorithm can write one in a matter of seconds, he suggests, only proves how utterly unimpressive the performance is. Those who are “freaking out” (I count myself among them) are merely too committed to pedagogies that he abandoned years ago. Or, as Bogost implies, we’re either too stupid or too cowardly. Double ouch!

Fortunately, I am not entirely alone. At the end of November, just before ChatGPT was released, I had the pleasure of a long conversation with four people who have thought as least as much as I have about these things. Using Charles Knight’s presentation about the future of essay exams as a keynote, he, Anna Mills, Marc Watkins, Anette Vee and I had stimulating exchange of views and experiences over Zoom that did a great deal to prepare me for the events of December, and which you can view on my YouTube channel. Charles has since cited Warner’s work, again approvingly, in his entertaining invocation of Star Trek’s conception of the relationship of humanity to technology, and Warner invited Marc to contribute a guest post to his blog at Inside Higher Ed. So there is promise of a lot of interesting discussion going foward.

However we may feel about it, I agree with Stephen Marche that these recent developments show that artificial intelligence is likely to be highly disruptive to higher education. My intitial reaction was indeed to think that, as he put it, “the college essay is dead,” but I have since grown a little more hopeful that it can survive, much like the book survived the invention of the printing press and the wheel survived the invention of the automobile. Perhaps more precisely, I think it will survive like thinking survived the invention of the book and running survived the invention of the wheel. But it will force us to rethink writing pedagogy and examination in ways that doesn’t just encourage, but actually requires, students to write sentences and paragraphs. Unlike Warner and Bogost, I’m not happy to let the five-paragraph essay die. Indeed, unlike them, I am not bored when a student (I mean an actual flesh and blood, heart and brain student, a human mind in a human body) produces one. I think we have forgotten what an accomplishment a series of coherent prose paragraphs actually is.

Let us consider the infamously “formulaic” writing task that Warner would have us kill. When I teach it, I sometimes offer a very specific recipe as a start. Since a paragraph consists of at least six sentences, it can be useful to imagine the problem as that of writing thirty sentences. The first two will describe the state of the world, the next two will describe the state of field, and the final two sentences of the first paragraph will describe the essay itself. “I will here argue that…” the fifth sentence might begin, followed by a statement of the essay’s conclusion, which can be reused as the key sentence of the last paragraph. The sixth sentence will outline the three main points of the argument.

I know, I know, you’re bored already. Bear with me, it gets worse!

Those three main points in the sixth sentence provide the content of the key sentences of the second, third, and fourth paragraphs. These can be imagined as the seventh, thirteenth, and nineteenth sentences; and the key sentence of the last paragraph (the conclusion I already mentioned) can become the twenty-fifth sentence; i.e., they could be the first sentences of each paragaph. But it should be pointed out to the students that the key sentence of the first paragraph is actually its fifth sentence (“I will here argue that…”) which is motivated, situated and elaborated before and after. Similar decisions can be made with the others, though putting them at the beginning can ease the reading and result in a perfectly fine text. The essential thing is that there be a key sentence, plus five sentences that support, defend, or elaborate it, in each of the three body paragraphs.

The final paragraph can provide a summary and perspective. One of the sentences (the key sentence) will state the conclusion (perhaps reusing, but ideally rewording, the claim that followed the “that” in the fifth sentence.) Three of them will restate (again, ideally, rephrasing) the key sentences of the body paragraphs, and two of them will redescribe the state of the world or the state of the field in the light of the essay’s conclusion. (Note that in the introductory paragraph the world and the field were described before the conclusion was even mentioned.)

If you’re still with me, you should be able imagine a structure of sentences that is indeed formulaic:

(2 + 2 + 2) + 3 x (1 + 5) + (1 + 3 + 2)

This does not, of course, describe every (perhaps not even any) essay that has ever been written. It merely sets up an exercise. And here’s the kicker: I believe that anyone who claims to know anything in an academic setting is implicitly claiming to be able to write one of these essays about it. Any scholar worth their salt can write thirty coherent sentences that situate their knowledge claims both in a world of common concern and a field of academic inquiry, telling us what they think, and how they know, and why it matters. In every assignment like this, therefore, students are demonstrating their (always partial, ever imperfect) mastery of the scholarhip in their discipline.

Obviously, we should never hold students strictly to any particular formula and deduct marks for every departure from it. But we must continue to insist on the competence implicit in this simple assignment, among both students and scholars. A human being who can produce a text like it in under three hours is doing something that should impress us, even if the subject matter is less than mindblowing. In a class of thirty students, we should be able to distinguish very articulate and knowledgeable students from merely passable ones even if they all stick to the same formula. (If we reward their ventures beyond it, it will be even easier.) The fact that a machine can make one (and I have no doubt that a machine soon will) should not diminish the achievement of an eighteen-year-old who organizes 30 sentences around the the claim that Hamlet’s love for his mother was complicated. Just as we are not unimpressed with an eight-year-old who solves the equation

x = (2 + 2 + 2) + 3 x (1 + 5) + (1 + 3 + 2)

just because a machine can produce the same result in less than a second or because the answer depends on “conventions” like the order of operations. (For good measure, I gave this problem to ChatGPT. It took two tries to get it right. But a calculator can of course do it easily.) That this competence is “formulaic” does not mean it is trivial, and there is no reason that an account of Hamlet’s relationship to his mother must be boring just because it is expressed in orderly prose that follows a preset pattern. Must I paint you a picture? Shall I compare thy paragraph to a sonnet? Shall I compare ChatGPT to a hundred thousand billion sonnets?

I do believe that the advanced state of artificial intelligence constitutes something of an emergency for higher education. If I’m not quite (or just no longer) literally freaking out about it, it is because I believe we have long had a contingency plan in place, albeit precisely the one that Warner and Bogost propose to “kill”. What really worries me now is that they will persuade us (and our students) that a passable 1000-word college essay should no longer be considered an accomplishment. Like I say, this would be like being bored by eight-year-olds mastering addition and multiplication problems. The ability to write coherent prose is not trivial, and not valuable just for its output. A human body that can write even a very conventional essay should be celebrated.

The five-paragraph essay forces students to be both knowledgeable and articulate about an assigned topic “on the spot” and therefore gives them a straighforward opportunity to demonstrate to us whether they have learned what we have tried to teach them. If they know they actually have to a write a good one to a prompt they don’t know in advance (but of course on a topic that is relevant to the class they are taking) they have a reason, not just to learn the material, but to keep their thinking and prose in good shape. Perhaps this will even motivate them to write their own take-home essays themselves just for practice! We just have to put them in a room for three hours without the internet at regular intervals, with a substantial grade at stake, and have them show us what they can do. If we take the time to talk to them about it afterwards, all the better. But we must, in any case, insist that they develop the ability to compose a coherent essay. If there’s any hope, it lies in the prose.

Happy New Year!

Grading the Moment

Many years ago, while I was still a PhD student, a mentor of mine said something very wise about examination. “An exam is an opportunity for the student to demonstrate their knowledge and intelligence to the teacher,” he said, “not for the teacher to expose the student’s stupidity and ignorance.” This is something that both student and teacher do well to remember and, as much as possible, ensure is true. After all, we’re all more stupid and ignorant than we’re intelligent and knowledgeable (on most things, most of the time, we’re not the leading scholar in the field) and an exam is intended to test only what we have most recently (been) taught in a particular discipline. If both teacher and student approach it with the right attitude, it can be one final learning experience in semester that was, hopefully, full of them.

In my last post I described what I believe to be an almost ideal exam at the end of an ordinary one-semester course (e.g., about Hamlet) during which students were asked to read and understand a variety of materials and write a series of essays about them. In my fantasy of the course, they have also been strongly encouraged to compose their paragraphs very deliberately. The learning objectives are therefore quite clear. By the end, they should be able produce an articulate and knowledgeable essay that answers a question that naturally arises from the course content. For the exam, they’re given three hours to write such an essay and are then examined orally on that basis for 20 minutes. In this post I want to say a few words about how to grade the performance.

Ideally, you will have divided your class into groups of eight students to be examined on separate days. (Less ideal conditions can work too.) In the morning they show up as a group and are given a prompt (a different prompt for each group must be designed). After three hours, they hand in their papers. Then, in the afternoon, they’re each given a half-hour slot for an oral exam.

In the first ten minutes (before they enter the room) you read their essay. (They are of course encouraged to do the same.) Since it’s is no longer than 1000 words, you should be able to get through it in that time and even form some thoughts about the competence of the student, both as a writer and as a scholar of the subject. Most importantly, it should give you some ideas for an interesting conversation with the student. If it doesn’t, that’s already a weakness of the student’s performance. Note down whatever reactions of this kind you have, then invite the student in.

Now, talk to the student about the ideas in the paper for twenty minutes. Do not evaluate the paper itself as a piece of writing; don’t talk to them about the paper and the rhetorical choices they made in it. Talk to them about the ideas they expressed. If you think they got something wrong, feel free to explore that with them and give them a chance to correct it. But what you mainly want to do is to poke around looking for the all the things they know and are able to talk intelligently about. If you think they can handle it (which is already promising for their grade) then challenge them directly on points you disagree with (especially if you won’t hold it against them when they stand their ground). Good students should expect to have a spirited (but never heated) debate during this examination. Average students should expect to have a conversation about a topic that both participants are familiar with. Poor students should expect to have a boring conversation about something they know and care little about.

Keep some notes to remind you of anything remarkable that happened during the exam. When the twenty minutes are up, thank them and bid them a good day. Score their perfomance on a scale from 0 to 100. Your criteria are entirely up to you to decide. The point is just that there has to be such a thing as a good, moderate, and bad performance. Now quickly reflect on your first impressions of the paper. Then give the paper a score as well, which may be higher or lower than the oral score. Did the conversation go exactly as the paper got you to expect. Did it go much better? Or did it go worse? The paper score will reflect the discrepancy. Note it down and put the paper, your notes, and the scores, on the “examined” pile. Next student!

When all the exams have been completed (which may take several days), you’ve got a pile of papers, notes from the oral exams, and numerical scores summarizing your impressions. At this point you could simply average the two scores and translate them into letter grades on whatever scale you already use. If you have time, and think it’s a good idea, you can also reread all the papers one more time and give a final grade in the light of the whole experience.

If you’re allowed to do so (in Denmark we are not) I would encourage you to “normalize” the grades — that is, distribute As, Bs, Cs, Ds and Fs on a curve. Let the students know that their challenge for those 3 hours + 20 minutes is to be as articulate and knowledgeable as they perhaps will ever be on the subject. Let them know that any training they do (whether in writing or conversation) will probably benefit them. It’s the students who are best prepared and in best shape for this exam that will get the top grades. All the effort they make, both in the moment and throughout the course, will be rewarded, because it will make them stand out relative to their peers, i.e., the community of those “other knowledgeable people” that their own knowledge is part of.

That’s it. That’s my suggestion for how to grade the ideal exam. Your comments are more than welcome. From the middle of next week, I’m going to take a break from blogging and tweeting until the new year. I will return with a post reflecting on Ian Bogost‘s and Marc Watkin‘s optimism about the, let’s call it, “creative destruction” of the college essay by artificial intelligence. (See also Stephen Marche and my conversation with Charles Knight, Anna Mills, Annette Vee and Marc Watkins.) My approach to writing instruction and examination does, indeed, seem a bit endangered by ongoing technological developments. But I’m not ready to give up yet. Let’s see what 2023 brings!

Examining the Moment

Suppose you are trying to teach your students to compose themselves in the manner I’ve been suggesting over the previous three posts. And suppose you’re not just trying to teach them to write in general, but you are trying to impart knowledge to them as well. You may be teaching them anything from organization theory to Elizabethan drama, and along the way you’ve been assigning essays that, you tell them, should consist of a series of coherent prose paragraphs that together present an argument. You’ve been encouraging them to think of a paragraph as a composition of at least six sentences and at most two-hundred words — about half a page — that says one thing they know (about the content of the course, of course) and supports, elaborates, or defends it for the purpose of discussing it with another knowledgeable person. That person, you have been telling them, is best thought of as a fellow student in your class, a peer. You and your students are, in this sense, now on “the same page”.

In this post, I will suggest a form of examination that I consider essentially ideal, even if we had no worries about plagiarism or artifical intelligence, but one that the increasingly sophisticated technologies in this area now make virtually necessary. That is, I’m hopeful that the fact that the take-home assignment no longer constitutes a serious test of the student’s knowledge of a subject or ability to write about it will force us to adopt a form of testing that was always much more serious. In fact, it’s one that I suggested a few years ago might go some way towards reducing the problem of impostor syndrome later on in an academic career. If done right, the competent student will literally feel like they have passed through an impostor filter. I will desribe the exam here and then discuss it in greater detail in a later post.

I’m assuming that the exam comes at the end of a one-semester course. You will have taught the course content as you think best and, like I say, encouraged them to write in orderly writing moments as I suggest. You will also have told them about the structure of the final exam, which will have a significant influence on their final grade. The exam that I will now describe will hopefully have impressed the importance of training on them; they will want to have their minds and their prose “in good shape” to be able to perform the task that is coming.

The exam has two parts.

The first part of the exam is a 3-hour, 1000-word (max.) invigilated essay. Under controlled exam conditions, using only materials and equipment you provide to them, and giving them no connnection to the internet or each other, the students are to respond to a prompt that you give them. You may give them three prompts and let them choose one, but you should not let them write about anything they want. They need to show you that they can quickly recognize what you are asking them to do (which should of course be something that is reasonable to ask of a student that has dutifully followed your class). Then, given those three hours they should be able to compose at least five coherent paragraphs that consititutes an intelligent, knowledgeable response to your prompt.

The second part of the exam is a 20-minute oral examination based on the submitted essay. You might be able to examine ten to fifteen students in a day in the manner I propose, depending on your stamina. Each session begins with you taking ten minutes to read the 3-page essay the student has written. They then enter the room and you discuss it with them for twenty minutes. That’s it. You make some notes, score them, and assign grades on the basis of your notes and scores when you’ve been through all the students. If you’re allowed to do so, I suggest giving the grades on a curve. More on this in the next post.

The test will work best with as little time as possible between the written and oral exams. You can expect students to spend the time between them reading up on the material they will be discussing, but you don’t want the whole oral exam to consist of them correcting themselves on matters they have since learned more about (and will forget again in due course). With a class of twenty-students you can get through this over two or three days. Out of fairness you don’t want some students to have a lot more time to prepare for the oral exam than others.

But there’s a simpler and less exhausting way through this, actually. If you’re willing to write a lot of prompts, you can examine an arbitrary amount of students in an arbitrary amount of days, spread out over an abitrarily long period. Simply split the students into groups of 8, assign them exam dates where their group writes in the morning and is then examined orally after lunch (between, say, 13:00 and 17:00). You read eight essays and have eight more or less engaging conversations with your students on some number of afternoons during the exam period.

Like I say, I will offer some reflections on this in a later post. I’m sure the idea itself provokes reactions, positive and negative, in many of my readers. I would love to hear about them in the comments.