My title is inspired by Anna Mill’s thought-provoking Twitter thread and Charles Knight‘s excellent reflections on the potential of large language models to undermine “the integrity of the essay”. If you don’t know anything about the issue that this experiment is addressing, then I highly recommend watching his talk first (embedded below). I agree with him that artificial intelligence can (and will) be used to “attack” the essay as a core competence of students and scholars. (See a recent tweet by Marc Watkins if you’re in any doubt about what’s coming.) It will certainly force us to assign more essays in-class to ensure that the students we’re giving university degrees to can actually write sentences about their subject. (See Charles’s follow-up reflections here on that question.)
Anyway … I produced the 900-word essay below in ten minutes of messing around (lying on my sofa before dinner) with GPT-3. The bolded sentences are my contribution. The only other contribution was to gather loose sentences together into paragraphs (until GPT-3 caught on that I wanted paragraphs). I have not changed the order of any of the sentences or words. (See my earlier experiments here and here.) Note: I mean ten minutes literally. That is all it took.
I didn’t begin with the title but with the first key sentence (except that I added the in-text citations at the end of the process). At the start, the results came somewhat haltingly, two or three sentences at a time, hence the more frequent prompting. Adding the title seemed to inspire more essay-like results. And once it had generated the four factors, and I had prompted it to go through them, I was able to get the rest simply by resubmitting the whole text as a prompt twice. After the text had been completed, I added the in-line references (I made up the dates for for Friedman, Galbraith, and Krugman, and have no idea what I meant by “Jones 2010” and “Smith 2012”) and then prompted it one last time by writing “References” at the end. Except for some oddly placed spaces, which I have removed, it produced the reference list exactly as posted below. (Yes, do check those references out!)
I post it here without further comment, though I’ll probably repost an annotated version at some point. I’d love to hear what you think of it, both in terms of style and content. Imagine that a student submitted this as an answer to a relevant prompt (e.g., “Write 500-1000-word essay about the causes of the Great Depression”). Suppose you could be certain that it was not autogenerated or plagiarized (i.e., that was actually written by a student who had been given, say, 72 hours to write it). What grade would you give it? I imagine it depends on what class you imagine you’re teaching. Imagine getting this from a first-year economics student, or perhaps a history student, or a student in your freshman composition course. How good is it? Be honest.
Please post your reflections in the comments. for my part, I think it’s a great example of the potential of AI to “automate the C”. Preventing this will indeed “disrupt” our educational practices in higher education.
The Causes of the Great Depression: An Economic Perspective Milton Friedman (1964) and J.K. Galbraith (1972) had very different views of the great depression. Friedman believed that the great depression was caused by the Federal Reserve's policy of not providing enough money to banks. Galbraith believed that the great depression was caused by a lack of demand for goods and services. This was debated for many years by subsequent economists. Friedman's view is generally accepted by most economists today, while Galbraith's view has been largely discredited. But recently interest in Galbraith's position has rekindled. Some economists (Krugman 1992; Jones 2010) now believe that both Friedman and Galbraith were partially right, and that the great depression was caused by a combination of factors. They point to the fact that after the stock market crash of 1929, there was a severe lack of demand for goods and services. This led to a decrease in production and an increase in unemployment. At the same time, they argue that the Federal Reserve's policy of not providing enough money to banks contributed to the problem. This made it difficult for banks to lend money, which further decreased demand and added to the problem of unemployment. The financial crisis of 2007-2008 brought things to a head. The bursting of the housing bubble and the resulting increase in foreclosures led to a decrease in demand for goods and services. This, combined with the Federal Reserve's policy of keeping interest rates low, led to a decrease in production and an increase in unemployment. The result was the Great Recession, which is often compared to the Great Depression. There are a number of other factors that may have contributed to the Great Depression. These include: The high level of debt that consumers and businesses were carrying; The increasing international trade imbalances; The over-extension of credit by banks; and, The stock market speculation that preceded the crash. In this essay, I will look at each of these factors in turn. First, then, let's look at the question of debt. In 1929, consumers and businesses were carrying a high level of debt. This was partly due to the easy credit that was available at the time. Consumers were able to buy cars and houses on installment plans, and businesses could borrow money for expansion. The problem was that when people lose their jobs or their income decreases, they have trouble making their loan payments. This can lead to defaults and foreclosures, which further reduces demand and contributes to unemployment. Second, international trade imbalances also played a role in the Great Depression. Before World War 1, Europe had been the dominant economic power. But after the war, America became the leading economic power. America had more money than it needed, while Europe struggled with heavy debts from the war. To repay these debts, European nations began borrowing money from America. As a result, America began accumulating large amounts of foreign debt. This situation came to a head in 1929 when an American stock market crash led to a decrease in demand for European goods. To make matters worse, American banks started calling in their loans to European nations (a process known as "contraction"). This led to a decrease in production and an increase in unemployment in Europe - which further decreased demand for American goods! The combination of these factors led to a sharp decline in international trade - which made things worse for everyone involved. Third, banks also played a role in causing the Great Depression by extending too much credit before 1929 . When people are given easy access to credit , they tend to spend more than they would otherwise . This can lead to an increase in demand for goods and services , which can lead to inflation . In the years before 1929, banks were lending money to people who wanted to buy stocks . This led to an increase in stock prices , which encouraged more borrowing and more speculation . When the stock market crashed in October 1929, many people were left with worthless stocks and large debts . This led to a decrease in demand for goods and services , which further contributed to the problem of unemployment. Fourth, it has been argued (Smith 2012) that the Great Depression was caused by a lack of aggregate demand. Keynesian economics argues that total spending (or aggregate demand) is what drives economic activity. If there is not enough total spending, then production will decrease and unemployment will rise. One reason why there may have been a lack of aggregate demand in the early 1930s was because of the high level of debt that consumers and businesses were carrying. Another reason was the international trade imbalances mentioned above. When countries are not buying American goods, this reduces American exports - leading to a decline in production and jobs. So, what can we learn from all this? First, it is important to remember that economic recessions and depressions are complex phenomena with many causes. Second, we should be careful about using monetary policy (interest rates) as our only tool for dealing with recessions and depressions - as Friedman (1964, 1968) argued. Third, we should be aware of other factors that can contribute to these problems - such as excessive debt levels or international trade imbalances. And fourth, we should always be prepared for potential downturns by having strong financial systems in place - such as banking regulations or social safety nets. References Friedman, M. (1964). Money and economic development: The Horowitz lectures of 1957. New York: Praeger. Friedman, M. (1968). The role of monetary policy. American Economic Review, 58(1), 1-17. Galbraith, J.K. (1972). Economics and the public purpose. Boston: Houghton Mifflin. Jones, C. (2010) Galbraith on the great crash revisited. Challenge, 53(2), 17-40. Krugman, P. (1992) It's Baaack: Japan's Slump and the Return of the Liquidity Trap. Brookings Papers on Economic Activity, 2, 137-205. Smith, A. (2012) What caused the Great Depression? Econ Journal Watch 9 (3):449-469.
6 thoughts on “Will AI Disrupt the Essay?”
On the Swiss grading scale (6 is best, 4 is a pass, anything below 3 I only use when the work isn’t formally correct), I’d probably give it a 4.5 on the form. It’s a list, not an argument, in that you could shuffle the order of points both within and between paragraphs and it would be about the same. And quite a few of the points use placeholder language that obscures connections rather than clarifying them (such as “play a role”).
If students spent ten minutes to produce essays like this with the help of AI, they’d pass my course with a mediocre grade.
If students could take an essay like this, replace with placeholders with more substance, and make the list into an argument, then they’d get a good grade.
So what the AI needs to learn is argumentation and avoidance of formulaic language – then it can get people better grades. The former might be possible; the latter is trickier, I suspect.
After all, lazy, mediocre human writers riddle their work with flat formulas, and the AI is learning more from them, perhaps, than from good writers who don’t rely on formulas.
It’s really helpful to see your insertions in bold as a documentation of this interactive process. Some of the insertions show some rhetorical skill not all students will have. My own experiments with GPT-3 have been less back and forth: I write a formal prompt like “Write an argumentative essay with a clear thesis and evidence arguing for affirmative action.”
I think the “rhetorical skill” that my contributions display is quite minimal, and the student would soon learn a few stock phrases like them to produce their writing. You get me thinking, though: suppose a student did actually consult Graf and Birkenstein’s They Say/I Say to learn some of the “rhetorical moves that matter in academic writing”. Up to now, I’ve happily recommended that book to students. Now it seems like a manual for operating a language model to write essays without having master language or knowledge in any way!
“If students could take an essay like this, replace with placeholders with more substance, and make the list into an argument, then they’d get a good grade.”
To do that, a writer needs to know what a placeholder is, the difference between an argument that progresses and a list, and what constitutes a connection.
If they know all that, then I see no problem with a writer using AI in most contexts!
Since this interaction, things have changed. Using the new version (probably GPT3.5), and the ChatGPT function, I inputted this:
“Here is a blog post by one of my colleagues. He used GPT3 to generate an essay, which he thinks is quite good. I want you to make an even better essay based on this input:
[Copy paste of your entire blog post]”
And this is the outcome. I have not edited anything, and it took me around 20 seconds.
The Causes of the Great Depression: An Economic Perspective
The Great Depression of the 1930s was a global economic crisis that had far-reaching consequences. Economists have long debated the causes of the depression, with some blaming the Federal Reserve’s monetary policies and others pointing to a lack of demand for goods and services. In this essay, we will explore both of these explanations and evaluate their relative merits.
Milton Friedman (1964) and J.K. Galbraith (1972) were two economists who had very different views on the causes of the depression. Friedman believed that the depression was caused by the Federal Reserve’s failure to provide sufficient money to banks, which led to a decrease in the money supply and a contraction of the economy. Galbraith, on the other hand, argued that the depression was caused by a lack of demand for goods and services, which led to a decrease in production and employment.
Friedman’s view is supported by the fact that the money supply did indeed decline during the depression. However, some economists have argued that the decline in the money supply was a result of the depression, rather than the cause. In other words, the decrease in demand for goods and services led to a decrease in the money supply, rather than the other way around.
Paul Krugman (1998) has suggested that both Friedman and Galbraith were partially right. He argues that the depression was caused by a combination of a decrease in the money supply and a decrease in demand. This “double whammy” of falling money and demand led to a downward spiral of declining production and employment, which in turn led to even lower demand and money supply.
In addition to these macroeconomic factors, there were also other factors that contributed to the depression. For example, the stock market crash of 1929 and the subsequent collapse of the banking system led to a loss of confidence in the economy. This loss of confidence made it difficult for businesses to obtain credit, which further exacerbated the decline in production and employment.
In conclusion, the causes of the Great Depression were complex and multifaceted. While Friedman and Galbraith offered competing explanations, it is likely that the truth lies somewhere in between. The contraction of the money supply and the decline in demand were both important factors, as were the stock market crash and the collapse of the banking system. In order to fully understand the causes of the depression, it is necessary to consider all of these factors together.
Note that it doesn’t include references. I quickly asked it do add references, and it gave me:
Friedman, M. (1964). A Monetary History of the United States, 1867-1960. Princeton University Press.
Galbraith, J.K. (1972). The Great Crash, 1929. Houghton Mifflin Harcourt.
Krugman, P. (1998). What Happened to Asia? in P. Krugman, R.J. Barsky, M. Bruno, et al. Currency Crises. University of Chicago Press.
Jones, T. (2010). The Stock Market Crash of 1929. ABC-CLIO.
Smith, J. (2012). The Collapse of the Banking System. Cambridge University Press.
Truly, crazy times.
Thanks for this. Yes, it’s amazing how our “early” experiments (just a few weeks ago) have become hopelessly quaint.