The Two Narrative Metrics

An essay on the And Frequency (AF) and Narrative Index (NI). Originally published as Appendix 2 in "The Narrative Gym for Politics."

QUANTIFYING NARRATIVE STRENGTH

The idea of putting a number on “How powerful is your story?” sounds somewhat absurd, given how much is involved in telling a story, BUT …

There’s that word BUT.

We know the word BUT is powerful and is the central element of narrative, so why wouldn’t a text with more BUTs be more powerful than a text that is devoid of BUTs and saturated with ANDs?

Over the past decade, I’ve analyzed thousands of speeches, debate performances, articles, novels—pretty much any text I could get my hands on. The patterns are real. You can see a lot of the data in my 2019 book Narrative Is Everything.

Here, in more practical form, are the two (and only two) narrative metrics I’ve found to be useful, intuitive, robust, accurate, and meaningful.

1) THE “AND FREQUENCY” (A.F.)

Yeah, I know, A.F. stands for something else to most people (see the Urban Dictionary definition), but defined as the “And Frequency” (AF), it turns out to be a powerful and fundamental reflection of how we communicate.

I learned about it from the 2015 study by the Stanford Literary Lab titled, “Bankspeak: The Language of World Bank Reports, 1946-2012.” It’s extremely simple—just the percentage of total words in a text that are the word AND. That’s all.

The most fascinating thing reported in that document (at least to me) is that there exists an established optimum for well-edited texts, which is 2.5%. This means that of all the words in a well-edited document, 2.5% are the word AND.

As far as the AF goes, that’s all you really need to know and to aspire to—have AND be 2.5% of your total words.

I was stunned when I read this. And skeptical. But twice I’ve worked with assistants to analyze ten articles each from The New Yorker, The New York Times, and The Atlantic. The results were convincing—both times the average was right around 2.5%, with almost none scoring above 3.0%.

The whole story with the World Bank is that originally, when it was founded, their annual reports scored just above 2.5 for the AF. But over the decades, the scores drifted up to above 3.0, then above 4.0, until by 2012 and onward, they score consistently over 5.0.

This means the use of the word AND in the reports has doubled. And, along with it, emerged the term “bankspeak,” in reference to how unreadable the reports have become.

HOW TO USE THE AF

So this is the first simple metric for any policy statements you produce. You don’t want to be speaking “bankspeak” to your voters. To use the AF: Just put the text into your word processor, search the word AND by putting a space on either side of it (you don’t want to count words like sand, or hand), and then divide it by the total number of words in the text.

Try it out on a few documents. You’ll be amazed. Just like the World Bank reports, well-edited texts score very close to 2.5. When the scores get higher, the narrative strength declines. Once you’re above 4.0, you’re almost certainly boring. Beyond that, you enter the unreadable realm of government reports, which sail above 5.0, sometimes past 6.0, and I’ve even found ones that scored over 7.0.

You don’t want to bore. Try to keep your AF scores close to 2.5%.

2) THE NARRATIVE INDEX

This second index gets to the real heart of narrative strength because it involves BUT.

In 2015, I came to a simple realization: If you followed “The Rule of Replacing” (replacing the word AND with the word BUT to strengthen the narrative structure of a text), you would produce a metric. The transformed text would have a higher ratio of the word BUT to AND than the original text. The higher ratio would reflect stronger narrative content and thus, all else equal, more powerful communication.

If you multiply the ratio by 100, you get a whole number instead of a fraction. This is what I’ve termed THE NARRATIVE INDEX.

THE NARRATIVE INDEX = (BUTs/ANDs) X 100

In early 2016, I tried hard to call attention to the fact that the speeches of the Republican Party presidential nominee had an average Narrative Index of 29, versus his opponent’s, which were averaging less than half that at 14. I got nowhere. Typical of the responses was when the editors of the 538 Blog said, “We’re intrigued but are skeptical of something so simple.”

We all know how that election turned out. The candidate with the higher Narrative Index won. That was followed by four years of him proving himself to be (like him or not) a powerful communicator.

It’s a half decade later, and the pattern is real. Here’s an iconic initial data set to illustrate it.

THE LINCOLN-DOUGLAS DEBATES OF 1858

In the summer of 1858, incumbent Illinois Senator Stephen Douglas engaged in a series of seven debates with challenger Abraham Lincoln. The debates were transcribed and can be found online. The texts for each man’s performance average around 10,000 words, which makes for solid sample sizes (I’m generally leery of texts that are fewer than 1,000 words.)

FIGURE 1. NARRATIVE INDEX SCORES FOR THE 1858 LINCOLN-DOUGLAS DEBATES. Lincoln’s scores were greater for every debate than Douglas’s. The bars on the mean values represent 95% confidence intervals....

You can see in Figure 1 that Lincoln’s Narrative Index for every debate was well above the scores of Douglas. Lincoln’s overall average was double Douglas’s.

More specifically, if you look at samples of the text, you can see the pattern. Lots of ANDs, almost no BUTs for Douglas (Figure 2), versus numerous BUTs for Lincoln (Figure 3).

FIGURE 2. SAMPLE OF TEXT FROM STEPHEN DOUGLAS. The word AND is abundant, but there are almost no BUTs.

FIGURE 3. SAMPLE OF TEXT FROM LINCOLN. Lots of ANDs, but also plenty of BUTs.

In addition to the metrics, there’s even in-person corroboration of this difference in narrative structure—from newspapers that reported on the debates. One article said, “Lincoln’s words did not flow in a rushing, unbroken stream like Douglas.”

That makes sense. When you’re communicating via the AAA structure, all you’re doing is spewing out disconnected statements of this AND this AND this AND this … you don’t need to slow down and think because there’s no real structure to what you’re assembling. You can spew words like “a rushing, unbroken stream.”

But when you’re tapping into the power of the ABT, the elements are connected, and that takes time. As a result, Lincoln spoke more slowly and thoughtfully.

Were the debates worth it for Abe? Did he win the 1858 election? No. But there was widespread agreement that he won the debates. By the last few, Douglas was losing steam, people were getting bored with him, and everyone could sense a great voice emerging with Lincoln.

PUTTING THE NARRATIVE INDEX TO WORK IN YOUR CAMPAIGN

Just as you measured the AF for a text, you should also calculate the Narrative Index. Here’s a rough guide for evaluating the scores of both metrics:

“AND” FREQUENCY:

• Over 5.0 - Deadly boring

• Over 4.0 - Pretty dull

• Over 3.0 - Adequate

• Around 2.5 - Optimal

NARRATIVE INDEX:

• Over 30 - Probably fiercely argued

• 20’s - Solid narrative strength

• 10’s - Average for most politicians

• Under 10 - Snoozer

• Under 5 - Equipment-maintenance manuals

FIGURE 4. Narrative Index scores for four politicians, one comedian, and a set of equipment-maintenance manuals.

A FINAL NOTE: NO JOKE, HIRE A GOOD COMIC WRITER

You can see the whole spectrum of narrative strength in Figure 4. At one end, we have complete narrative absence in the form of equipment maintenance manuals we found on the internet. They have virtually no BUTs—why would they?

Then there’s Stephen Douglas, spewing his statements, followed by Hillary Clinton, the policy wonk, itemizing her issues. Lincoln begins to deliver strong narrative content, but even he was exceeded by the free-wheeling rage and bile of Donald Trump, who managed to ABT his singular narrative of American greatness all the way to over 90% support among his party.

That is your whole spectrum of narrative strength in communication, but there’s one last thing to note, on the highest end of the spectrum: It’s the narrative strength of good comic writers, which is not a quirk—it’s a real pattern.

Every candidate should hire a talented comic writer, NOT to write jokes, but to implement strong narrative structure. In Narrative Is Everything, Randy listed the Narrative Index scores for the scripts of comedy specials from the best stand-up comics. Their scores are almost always above 20, frequently above 30, and comedian Michelle Wolf set the record of 47 with her legendary comic assault at the White House Correspondents’ Dinner in 2018.

The average score of 33 for Bill Maher is based on nearly 100 of his ending monologues for his HBO show Real Time. Each one is only about a thousand words, which as a sample size is just barely sufficient, but taken together as replicates, this produces a very solid data set.

Try it for yourself, as I (the ultimate narrative nerd) do every week. Just count the BUTs and ANDs in real time as he delivers the end monologue. You’ll be amazed at the frequency of BUTs. You’ll feel the entire narrative structure of the piece as he lays out a few silly pieces of current news events, but then gets down to work, often with the BUT/BECAUSE couplet (“… BUT we think these events are meaningless BECAUSE of our failure to …”). The team of comic writers he employs are major veterans who are able to wield the word BUT with the skill of ninja masters.

More than probably anyone else in our society, good comic writers know narrative structure. Every politician should yearn to hold the interest of their audience as well as a good comic writer does.

Update: For a deep dive into AF and NI, read my latest book Lincoln But Trump: Narrative Metrics and Similarities Between Heroes and Villains.