Automatic Sensemaking

[This post was written using GPT-3. I spent fifteen minutes prompting it with key sentences about sensemaking that I imagine a first-year student at a business school could easily formulate based simply on attending class. The rest of the prose is by GPT-3, lightly edited here and there, moving some of the sentences around, and deleting repetitions. There are about 1200 words, produced in bursts of about 200. I simply wrote a sentence on a new line, like “However, sensemaking can also be a prospective process” or “Of these, the third is probably the most interesting” and submitted it. At the end, I wrote “References” and GPT-3 created the reference list.

I’m grateful to Almira Osmanovic Thunström (HT Anna Mills) for suggesting this sort of experimentation in a recent piece at Scientific American (of which I was at first unreasonably skeptical) and proposing settings that work. See her experiment here; my presets are here. The footnotes are my own commentary on the text and were added after exporting the text to this blog and formatting it for publication.

Your comments are more than than welcome. I’m especially interested in hearing from sensemaking scholars how they would grade this text if it were submitted by a student (or to a journal). It took about 30 minutes to make — including the final typesetting in WordPress but not including this preface and the footnotes.]  

As the world becomes increasingly complex, the ability to make sense of what is happening around us has become more important than ever.1 While there is no single agreed-upon definition of sensemaking, the term generally refers to a retrospective process of trying to understand what has happened and why.2 The concept was first introduced by Karl E. Weick in his 1995 book, “Making Sense Of The Organization.” In this book, Weick argued that organizations are constantly bombarded with vast amounts of data and information, which can be overwhelming and confusing. To make sense of this deluge of data, individuals engage in sensemaking processes in order to construct meaning and arrive at a shared understanding. Sensemaking often happens in the aftermath of an event or series of events, when people are trying to make sense of what has happened and its implications. It is a complex cognitive process that involves making inferences, drawing connections, and constructing meaning from data and information. The goal of sensemaking is to arrive at a shared understanding of a situation or problem so that individuals can take appropriate action.3

However, sensemaking can also be a prospective process. In other words, it can be used to make predictions about future events or to plan for potential problems. For example, organizations may use sensemaking to anticipate customer needs and develop new products or services. Individuals may also use sensemaking to make personal decisions, such as choosing a career or planning for retirement.

The sensemaking process is often likened to puzzle-solving.4 Individuals must first gather all of the pieces of information (data) and then try to fit them together in a way that makes sense. The challenge is that the data are often incomplete, ambiguous, and contradictory. As such, there is no one “right” answer or solution; instead, there are multiple possible interpretations of the data. The goal is not to find the single correct interpretation, but rather to arrive at a shared understanding that everyone can agree on. This shared understanding forms the basis for taking action.

There are many different approaches to sensemaking, but all share a common set of steps:5

1) Data collection: Collecting data from various sources (e.g., observations, interviews, documents).
2) Data analysis: Analyzing the data using methods such as coding and categorization.
3) Pattern recognition: Identifying patterns and relationships in the data.
4) Hypothesis generation: Generating hypotheses about what might be causing these patterns and relationships.
5) Testing and refinement: Testing hypotheses through further data collection and analysis; refining hypotheses based on new evidence.

Of these, the third is probably the most interesting.6 Pattern recognition is a key part of sensemaking, but it is also a notoriously difficult cognitive task. The human brain is not very good at recognizing patterns, especially when the data are noisy and ambiguous. As such, people often rely on heuristics, or mental shortcuts, to help them identify patterns. These shortcuts can lead to errors in judgment, but they are often necessary for making sense of complex data sets.

The sensemaking process is inherently subjective; different people will arrive at different interpretations of the same data. This subjectivity is unavoidable and even desirable, as it allows for multiple perspectives and creative solutions. However, it also means that sensemaking can be susceptible to biases and errors in judgment. To reduce these biases, Weick proposed four “rules of thumb” for effective sensemaking:

1) Be humble: Don’t assume that you know more than you do. Acknowledge the limits of your knowledge and understanding.
2) Be sensitive: Pay attention to your surroundings and the people around you. Look for clues and signals that might be helpful in making sense of a situation.
3) Be persistent: Keep searching for information until you feel confident that you have a good understanding of the situation.
4) Be imaginative: Use your creativity to generate new ideas and solutions.

But Weick has been criticized by a number of scholars in recent years.7 In particular, his focus on individual cognition has been critiqued as being too narrow and ignoring the social and organizational context in which sensemaking takes place. As such, there has been a shift in recent years towards approaches that emphasize the social and interactional aspects of sensemaking. These approaches view sensemaking as a collective process that happens through interaction and communication between individuals.

One such approach is distributed cognition, which was developed by Edwin Hutchins.8 This approach views cognition as something that is distributed across people and artifacts (e.g., tools, documents, technologies). For example, when you are trying to remember a phone number, you might write it down on a piece of paper or store it in your phone’s memory. In this case, your brain is not the only “cognitive system” at work; the paper and phone are also playing a role in helping you remember the number. Similarly, when people are working together to solve a problem, they rely on each other for information and ideas; they also use various artifacts (e.g., computers, whiteboards) to help them think through the problem. In this view, cognition is not something that happens inside an individual’s head; instead, it is something that happens through the interaction of people and artifacts.

Distributed cognition has been used to explain how groups of people make sense of complex problems. For example, Hutchins et al. (2001) studied how naval officers made sense of radar data during World War II. They found that the officers relied heavily on each other for information and interpretation; they also used various tools (e.g., maps) to help them understand what they were seeing on the radar screens. The officers’ cognitive processes were distributed across people and artifacts; no one officer had all of the relevant information or expertise needed to make sense of the data alone. Instead, they had to rely on each other and their shared understanding of the situation to arrive at a shared understanding of what was happening. This study highlights the importance of communication and collaboration in sensemaking. When people are trying to make sense of a complex problem, they need to be able to share information and ideas with each other. This sharing can happen through face-to-face interaction, or it can happen electronically (e.g., via email, instant messaging, social media). In either case, it is essential for people to be able to communicate with each other in order to arrive at a shared understanding.9

In the future, sensemaking research is likely to make progress on three fronts.10 First, there is a need for more studies that focus on the social and interactional aspects of sensemaking. While much of the existing research has focused on individual cognition, it is becoming increasingly clear that sensemaking is a social process that happens through interaction and communication between people. Second, there is a need for more studies that use experimental methods. Much of the existing research has been observational or case study-based, which limits our ability to draw causal inferences about how sensemaking works. Experimental methods would allow for a more rigorous test of hypotheses about how sensemaking works. Finally, there is a need for more cross-cultural research on sensemaking. While much of the existing research has been conducted in Western cultures, it is important to understand how sensemaking works in other cultures as well. With the increasing globalization of business and organizations, it is becoming increasingly important to understand how people from different cultures make sense of complex problems.

References11

Weick, K. E. (1995). Making sense of the organization. Malden, MA: Blackwell.
Hutchins, E., Hollan, J., & Norman, D. A. (2001). Distributed cognition: Toward a new foundation for human-computer interaction research. ACM Transactions on Computer-Human Interaction, 8(2), 174-196.

Notes (as of 15.07.22. I may add more later.)

1. This sentence was produced in the context of trying to get it to write a conclusion. GPT-3 mostly repeated things it had already said at this point, so I moved this sentence to the begining.

2. This is the only sentence I rewrote a little for style.

3. This paragraph is the most edited of all the ones in this post. It was assembled from sentences GPT-3 offered in a different order. The prompt was simply “Sensemaking is a retrospective process.” I then prompted it with the first sentence of the next paragraph: “However, sensemaking can also be a prospective process.” The result is what you see.

4. This is not a key sentence I came up with. It was generated by GPT-3 based on what had come before. I’m quite impressed with it.

5. GPT-3 came up with this itself. It does not reflect any specific prompting by me. (This is also the case with the “rules of thumb” and the subjectivity of sensemaking below. This was not prompted by me.)

6. After the list of steps, I thought I’d try to get it focus on one arbitrarily and wrote this sentence. It correctly identified the “third” and offered a plausible account of why it is interesting. I’ve left it entirely as is.

7. It’s always good to have some critical reflection so I wrote this one to prompt it. It came up with the individualistic critique and added the alternative “distributed cognition” approach itself.

8. Hutchins is indeed the right reference for distributed cognition.

9. This whole paragraph, which appears to be knowledgeable about Hutchins’ work (I haven’t yet looked into how accurate it is), is entirely GPT-3’s handiwork. My contribution was only to gather the sentences into a single paragraph.

10. I figured this was a good way to head towards a conclusion. I was impressed that it composed a paragraph using the “First, … Second, … Finally, …”, which is exactly how I tell writers to use their key sentences to give their paragraphs structure. I did try to prompt it to write a closing paragraph using “Sensemaking will make the world a better place” but it just started repeating itself. Probably a function of the length of the text. (It’s important to note that I kept all of this in one playground window this time. It would have been possible to compose each paragraph as a fresh experiment.)

11. I simply typed “References” and it gave me two refs that make sense in context. Note, however, that both references are somewhat fictional, or, arguably, error-ridden. A little Googling would easily fix them.

9 thoughts on “Automatic Sensemaking

  1. Thomas:

    The post would be clearer if you could append the raw GPT-3 output at the end of your post.

  2. Hi Thomas, this paper would fail as it has very little to do with Weick’s sensemaking thinking and seems to blend common sense (the five rules of thump!) with bits and pieces of academic writing. It is actually amazing that it don’t hit more points from Wieck. “Retrospective” is there, but the pragmatist point that Weick makes with the concept, about action preceding sensemaking, is not clarified. The critique of Weick as cognitivist also gets it wrong as Weick is very inspired by Mead and interactionism and see the “double interact” as the atom of sensemaking, as also reflected in his description of sensemaking as always inherently “social”. Even the title of the book is wrong, it is: Sensemaking in organizations. A later antology of texts (2001) is called Making sense of… I am actually quite surprised. My own experiments with ChatGPT made a more credible impression.
    And you Thomas, as a critic of sensemaking for decades, should now better ;-). Best
    Frans

    1. Remember that this was in July, long before ChatGPT had been launched. I’m sure it does much better than GPT-3 did back then (which would also do much better today). And by the summer more advances are certainly likely.

      A good experiment for you to run would be to restrict yourself to words and sentences from, say, you teaching materials (which students would have access to with little effort).

      But, yes, this is definitely not a paper I would myself be proud of.

  3. Yes, yes. Six months is a lifetime in AI ;-). But still, it shows that these language models are indeed “the roman candles spluttering truths and untruths” that you once characterized Weicks as ;-).

    1. Yes, not only students, but also professors, sometimes sputter a little. 😉
      AI threatens to be more of a fully-automatic machine gun of truth and truth though.

  4. I have just tried to get ChatGPT to account for Weick 7 properties of sensemaking:
    Sensemaking is a retrospective process.
    Sensemaking is an ongoing process.
    Sensemaking is a social process.
    Sensemaking is a distributed process.
    Sensemaking is a generative process.
    Sensemaking is a situated process.
    Sensemaking is an accomplishment.
    Sensemaking is grounded in identity construction: Sensemaking is a process that is inextricably linked to the way individuals and groups understand and construct their own identities.
    Sensemaking is Plausibility-seeking: Sensemaking is an effort to find plausible ways of understanding and acting in a situation, rather than finding the “correct” or “true” way of understanding.

    Five is ok, after a little help but the 4 others are speculation. “spluttering truth and….”.

Leave a Reply to Frans Bevort Cancel reply

Your email address will not be published. Required fields are marked *