There have been two numerical analyses of
The Dark Tower (TDT). The first -- and the only one whose details have been published -- was by Carla Faust Jones (“The Literary Detective Computer Analysis of Stylistic Differences Between ‘The Dark Tower’ and C. S. Lewis’ ‘Deep Space Trilogy’,”
Mythlore, Spring 1989). Jones used a computer program to perform a widely-recognized, albeit crude, form of stylistic analysis of the first 16,336 characters of TDT and of comparable text samples from the openings of the three space-trilogy books. The method involved counting and comparing frequencies (numbers of appearances) of single letters and letter pairs (bigrams) in the text.
Jones found that the quantitative stylistic differences between TDT and the other three space-trilogy books were greater than those between the three trilogy books. In other words, by these standards or metrics applied to this sample of text, TDT was stylistically distinct. That was not a judgement but a numerical fact arising from a reproducible calculation.
The other analysis, which was of much shorter fragments of TDT and the space trilogy books, was performed by A. Q. Morton using his controversial “cusum” (cumulative sum) technique. The Morton analysis is highly questionable. Not only have its details never been published, but the cusum technique has been tested and found wanting by the literary computing expert community: it is used only by a dissident handful of analysts and is widely considered useless. I have myself read Morton’s book and implemented his techniques by hand and by computer. They are full of opportunities for self-deceptive manipulation of the text and over-interpretation of the results, and I decided for myself, even before reading expert opinions in the journal
Literary and Linguistic Computing, that it was useless.
The
Christianity Today article referenced above is a hatchet job on Lindskoog. It does not even mention the Jones computer analysis, which is the one that Lindskoog attributed her first suspicions of the Dark Tower’s authenticity to and which is the only analysis yet published using a recognized, objective technique (letter and bigram frequencies). While it is correct, in my opinin, to dismiss the Morton cusum analysis -- although
Christianity Today justifies its dismissal only by vague mockery (“This type of style analysis has been used to prove that Shakespeare did not write his plays”) --- to dismiss the cusum results without even mentioning the Jones results, which are prominently discussed in Lindskoog’s books, is yellow journalism.
So, in short, thus far we have one serious, checkable analysis and one bogus analysis. It is regrettable that Lindskoog was taken in by the Morton analysis, which would have been better off undone, but the Jones analysis is a serious piece of work.
Software such as Jones used has long been available only to experts who write the computer code themselves. Her code is simply not publicly available and may no longer even exist. Searching for Jones on the internet turns up no contact information (perhaps a marital name-change). Recently, however, a prototype authorship-attribution program has been made freely available by literary-analysis expert Patrick Juola (see ), with funding from the National Science Foundation. Anybody who has the patience to prepare careful text samples, figure out how to run a Java program, and do the numbers, can now download this program and perform authorship-attribution studies (with limitations). I have done so, or begun to. The obvious place to start is with Jones’s results. Are they reproducible? Does she get it basically right or was it all a mirage from the get-go?
These are my results, below. The first diagram is a two-dimensional plot of Jones’s published 1989 numbers. The vertical axis is text-to-text distance (dissimilarity) for character bigrams (letter pairs). The horizontal axis is textual dissimilarity for single characters. Textual dissimlarities for TDT and the space-trilogy books (three pairs of figures) are plotted as green squares; dissimilarities between the space-trilogy books (also three pairs of figures) are plotted as red stars. The greater mutual similarity of the space-trilogy books manifests as clustering of the red stars apart from the green squares:
This is just a handy way of visualizing Jones’s old results.
Now here, below, are the results of two analysis I did recently using JGAAP on the same text samples that Jones used (which I obtained by scanning in the book pages and doing optical character recognition on the scans to produce text-file samples). The first shows textual dissimilarity measured by “histogram distance,” a metric calculated internally by the JGAAP program but generally similar to Jones’s distance measure:
The next shows dissimilarities calculated by JGAAP using an alternative statistical standard, Kolmogorov-Smirnov distance (K-S Distance):
As you can see, clustering is visible in both fresh analyses. So, Jones had it basically right. Indeed, by the K-S distance criterion, the clusters are even more distinct than in Jones’s analysis.
What does this mean? It means that there is, as far as these textual samples and difference metrics go, we have at least one reproducible observation: TDT is stylistically distinct from the Space Trilogy. It does not prove that TDT was forged. As Jones herself was careful to point out, there are many possible explanations for such a difference -- though TDT authorship by someone other than Lewis is one of those possible explanations. I draw no conclusions at this point and don’t think anyone could.
It also means, incidentally, that
Christianity Today’s sneering dismissal of the textual-analysis evidence can itself be dismissed. Such evidence does exist -- though not from Morton’s cusum method. Jones’s results are essentially reproducible, as I have shown -- using completely different software.
Clearly, more work begs to be done. Most pressingly, what about taking text samples of similar length from later in the Trilogy books and TDT? Does one find a consistent stylistic difference across samples? And what about other Lewis writings: does his style (as measured by reproducible, objective metrics) vary between fiction and nonfiction, in the same period as these other sample texts (late 30s, early 40s)? And what about analysis of the other texts that Lindskoog argued had doubtful provenance? (We can dismiss A. Q. Morton’s rarely-cited analysis of “Christian Reunion,” which claimed that “a large section” of that esay “is not by C. S. Lewis anad is by some person who cannot be distinguished [stylistically] from Hooper.” His method is simply no good and no conclusion can be drawn here until better analyses are done.)
I have the tools to answer these questions, but it is time-consuming, especially the preparation of clean text samples and the recording and plotting of test results, but am keenly interested in doing the work. Just writing up this precis has taken too much of my morning. . . Anyway, I hope this answers the original question. Stay tuned to the Wardrobe for my next round of results, when they are ready.
Sincerely,
Larry Gilman
