Posts in category: DH

Two years ago, I designed and taught a graduate seminar on approaches to Literary Data. I was invited to contribute an essay on the course to Debates in the Digital Humanities. The essay was some time in the writing, and it will be some time longer in the publishing: the next DDH volume is now due to appear at the start of 2018. But by kind permission of the editors, I am able to share a preprint of my essay: Teaching Quantitative Methods: What Makes It Hard (in Literary Studies). If you quote it, please cite its forthcoming publication.

The essay explains the rationale of the course, which combined a practicum in computing with literary data using the R language with theories of literary data from structuralism to the present. My own evaluation of the course is quite mixed, and I offer my materials and my experience not as a model but as evidence for an argument about the conditions of possibility for a successful quantitative methods pedagogy in literary studies. Pedagogy, in this case, also raises serious questions for research; and I also hint at what I take to be the conditions for fruitful quantitative methodology tout court.

I couldn’t have wished for better students—that condition of possibility is indeed already realized. The major lessons I draw are (this is from the essay):

1. Cultivating technical facility with computer tools—including programming languages—should receive less attention than methodologies for analyzing quantitative or aggregative evidence. Despite the widespread DH interest in the former, it has little scholarly use without the latter.

2. Studying method requires pedagogically suitable material for study, but good teaching datasets do not exist. It will require communal effort to create them on the basis of existing research.

3. Following the “theory” model, DH has typically been inserted into curricula as a single-semester course. Yet as a training in method, the analysis of aggregate data will undoubtedly require more time, and a different rationale, than that offered by what Gerald Graff calls “the field-coverage principle” in the curriculum.

Some more remarks on the essay and the course follow after the jump.

My department asked me to meet with our first-year Ph.D. students to talk about “Using Digital Media” in their scholarly lives. It was interesting to try to sketch an outline of what I thought a starting English Ph.D. student should hear on the topic, and I thought I’d post a copy of that outline here. There’s also a pdf version. I also thought there are one or two people who might be amused by my sketch of research in digital humanities.

I recently picked up the first of Henning Mankell’s Wallander novels, Faceless Killers. I loved the Kenneth Branagh TV adaptations of these mysteries but had been saving up the pleasure of the novels themselves. I have the special talent of forgetting the resolution of pretty much any mystery I read, so I have no trouble enjoying the suspense a second time around. What I do not enjoy, however, is reading an excellent novel in a poor text.

This post is a discussion and partial replication of Ted Underwood and Jordan Sellers’s fascinating essay “How Quickly Do Literary Standards Change?” (available in preprint and discussed on Ted’s blog). Hats off to Ted and Jordan, who have contributed something really remarkable here, not just in their provocative arguments but in the data they have made usable by others. It’s one thing to circulate data and code so that someone can in principle re-run your scripts—though that is already a lot—and quite another to make the data accessible enough for fresh analyses. The latter is a very demanding standard, too demanding for everyone to meet, I think. But it is what is required to let others build directly on your results. Even more importantly, it’s what’s needed to make research results pedagogically available. As I argue in an essay I’m working on now, any quantitative methods pedagogy should—must—lean heavily on the results of research. In the immortal words of Miriam Posner, “It’s just awful trying to find a humanities dataset”: one of the best ways to address this challenge would be to make good research data available in recirculable, easily accessible form.

So consider this post partly pedagogical in intent: I want to show that Ted and Jordan’s replication repository is already an excellent dataset and could be the basis for a lesson in automatic classification. What I want to emphasize here is that their work allows us to breeze right past the data-wrangling and straight into the analytical substance. This may not be entirely obvious from their python code, so I’m going to try to make it clearer by doing the whole thing in R instead.

Rather than give all the technical detail, I’ll only show the R code where it makes a point about the analysis. The full R markdown source for this post is in this repository. Feel free to skim right past the R listings anyway. This is a literary theory and data analysis post that happens to have some software details in it, not a programming post. Here we go…

I’m speaking at NYU this Thursday afternoon, at the invitation of the NYU Digital Experiments group:

Corpus or Field? A Challenge for Quantitative Methods

One of the most promising prospects for quantitative methods in literary studies is that of rigorous and empirically wide-ranging accounts of the relations between literature and society. Yet the boundary between textual interpretation and a sociological analysis of literature has proven surprisingly hard to cross. In this talk, I retrace some sociological traditions of quantitative textual study, from postwar content analyses of political opinion to contemporary field theory, and I argue that they offer literary scholars alternatives to the doxa of “reading” that dominates and limits methodological discussion in our discipline. The sociological traditions turn us from corpus to field, from text collections to social spaces of symbolic competition and collaboration. I will discuss (and exemplify) the many challenges and pitfalls of this shift, technical and conceptual, in my own attempts to quantify the changing status of “reading” in the history of literary scholarship.

If you’re around, the talk is in Bobst Library (2nd fl., Avery Fisher Center, East Room) at 4:30. Come accuse me of naïve positivism, I dare you.

Pierre Bourdieu, pioneer of quantitative literary analysis:

Pour vérifier la correspondance entre l’espace des positions et l’espace des prises de position, nous avons recensé 537 textes de 510 auteurs publiés par les éditeurs retenus dans notre étude, qui ont été traduits en français entre juillet 1995 et juillet 1996 et retenu, pour chacun des titres, les variables suivantes : genre (roman, nouvelle, récit, conte), éditeur d’origine et et d’arrivée, langue d’origine (pour l’anglais on a distingué entre «anglais» et «américain»), nom du traducteur, nom et sexe de l’auteur, année de parution de l’édition originale, de la traduction française (1995 ou 1996), jugements de la critique, prix, nombre de pages, nombre total d’auteurs étrangers publiées par l’éditeur concerné, nombre d’auteurs ayant la même langue d’origine nationale. L’immensité des recherches nécessaires pour le mener à bien nous a conduits à abandonner ce projet.

In order to verify the correspondence between the space of positions and the space of position-takings, we took a census [survey?] of 537 texts by 510 authors translated into French between July 1995 and July 1996 which were published by the publishers in our study, considering the following variables: genre (roman, nouvelle, récit, conte [novel, short story, (non-fiction) narrative, (fantastic) tale]), original publisher and French publisher, original language (for English we distinguished between “English” and “American”), name of translator, name and sex of the author, year of publication of the original edition, of the French translation (1995 or 1996), critics’ judgments, prizes, number of pages, total number of foreign authors published by the publisher in question, number of authors having the same original national language. The immensity of the research necessary to carry this project off led us to abandon it.

("Une révolution conservatrice dans l’édition," Actes de la recherche en sciences sociales 126, no. 1 [1999]: 3–28; here 18n31.)

Nothing beside remains….

(thanks to @rania_tn on twitter for help translating genre terms; I alone am responsible for errors in translation above)

It’s gratifying, and a little frightening, when someone else uses your own code. Jonathan Goodwin has built on my dfrtopics and dfr-browser code-blobs to produce a fascinating visualization of topics in fiction 1920–1922, derived by modeling the genre-specific word frequencies data set from HathiTrust. He’s given a nice description of his process as well. In the process, Jonathan revealed some unnecessarily restrictive assumptions built into my code. He solved the problem by modifying my code: all praise to him! But then I felt bad and wanted to make it possible for others to go further without having to dig into the Area X that is my code. So I made a few adjustments to my new version of dfrtopics. Here are some notes on using the updated version to cope with the issues Jonathan found in processing and modeling the Hathi data.

I have spent a lot of time experimenting with and exploring topic models of text. Aside from an article, some blog posts, and a bunch of strongly held opinions, that time also produced quite a few lines of computer code for handling topic models from MALLET. I started out with a big file of R functions, then escalated to a folder full of R functions. The organization got ever more byzantine, even more so when I collaborated with others. Finally I bit the bullet and, following the gospel of Wickham, converted my pile o’ scripts into an R package, called dfrtopics because I was making models of data from JSTOR’s Data for Research. There it has sat, on github, accumulating bits now and then, plus some function documentation written in a fit of compulsion, but really not in a form that anyone but I could use (and, as time went on, becoming hard for me to use too). My website has had a note promising a tutorial demonstration of how to use my package for nigh-on two years, but no demonstration demonstrated itself.

The package was hard to use and document because of the messy and ad hoc way I represented pieces of the topic model. A hierarchical model is not easy to wrap your mind around, and different questions require different slices of the model to answer. And all the mess of code passing around random collections of data frames, lists, and who knows what else seemed like fertile ground for errors and glitches, even when the whole thing seemed more or less to do what I wanted most of the time.

So: in a questionable expenditure of energy, I’ve spent a few days applying some polish to the package. Herewith dfrtopics version 0.2. Install it from github with devtools::install_github("agoldst/dfrtopics").

Three things are new from a potential user’s perspective—and my hope is that the idea of a potential user is slightly less far-fetched than before. First, there is now an introductory tutorial in the form of a package vignette. Second, the whole package has been rewritten around the idea that a topic model is an object, stored in a single R variable. Third, I have tried to make everything as modular as possible, so that the usefulness of the package is not restricted to MALLET models of DfR data. If you have other textual data in wordcount-form—oh, let’s say, 180,000 18th-through-early-20th-century volumes—you might write up some variant file-loading code if necessary, then use the package functions to model those texts with MALLET. And if you have wordcount data that you want to analyze in R in some other way than with MALLET, there are still, I hope, some useful things here for converting those wordcounts into data frames or term-document matrices.

If we want to do sociology of literature, let’s get away from texts for a bit.

One of the most promising things about the current interest in quantitative methods for literary study is that it offers us some alternatives to reading as a method. There are important questions about literature—above all, about literature as a social and historical system—that cannot be answered with the tools of the expert textual interpreter. Such questions are better answered, I believe, in closer collaboration with our disciplinary kindred in the social sciences.

Thanks to a new special issue of Cultural Sociology focused on literature, we have a chance to look at some concerete examples of what sociological approaches to literary problems currently look like. In this post I’ll discuss the excellent essay there by Gisèle Sapiro, Translation and Symbolic Capital in the Era of Globalization: French Literature in the United States. Sapiro, whose work I have been following for a while, works in the tradition of the sociology of fields, and has written on both twentieth-century literary history and contemporary world literature. This latest essay is particularly fun to think with, because I have my greedy paws on some of the same data Sapiro uses, so it will be possible to look in some detail at the sort of evidence and the sort of analysis her approach entails—and, perhaps, to think how to extend it.

Here are some tentative notes on methodological problems in quantitative approaches to literary history that I’ve been worrying over. I’m moved to post these because of two great discussions of quantitative literary studies by sociologists, a review by Ben Merriman of recent work by Franco Moretti and Matt Jockers, and a blog post by Tressie McMillan Cottom on quantitative text analysis across the disciplines. Monomaniacal proselytizer for closer relations between sociology and literary study that I am, first I say: go read those! They’re really great! Also, I’m leaping onto this bandwagon. Mostly I want to indicate how this very useful pressure from sociologists might help literary scholarship to refine its methods and clarify its research program. I’m afraid this is all very drafty, in the form of broad assertions with basically no citations. Blogging!