(See the introduction to and explanation of this series of posts.)

Many kinds of political science data are messy. Demographers get to aggregate, e.g., births and deaths: pretty clear individual-level facts, even if accurate counting is often hard. Economists often study phenomena that are quantitative from the outset; anything expressed in prices conveniently comes with numbers already attached. That’s not to deny that there are major difficulties and hurdles involves in data collection and construction in those fields; it’s just to emphasize how constitutively messy many kinds of political science data are by contrast. (Not all, of course. Election results also show up with numbers attached at the outset.)

One of the most important datasets in the parts of political science I follow is the Minorities At Risk project, headed by Ted Robert Gurr. By making quantitative cross-national comparative work feasible, it has really invigorated the field of ethnic politics, and a great deal of productive use has been made of the data.

But counting ethnic minorities, or politically salient ethnic minorities, is one of those messy projects. Ethnic identities are themselves results of political projects of identity-formation; where to draw the lines between groups is always contested. The criteria for demarcation vary dramatically from place to place and from time to time: language (oral or written), religion, skin color, ancestry (real or imagined), history in a place, and so on.

The MAR project seeks to include all groups over a minimum size that are politically salient in one of two ways: discriminated against in official policy, or the object of substantial political mobilization (viz. through a political party that campaigns for the group’s autonomy or secession). This has involved many people over twenty years making judgment calls about whether, and whom, to include. Ethnic groups don’t come into the world with numbers attached; and neither discrimination nor mobilization is a matter of plain fact the way birth and death are.

The literature Simon Hug summarizes and builds on (and to which he has contributed) in “The Use and Misuse of the ‘Minorities at Risk’ Project” how much, and when, to worry about these problems. Mere mistakes (or differences of judgment) about what to include, mere measurement error, doesn’t by itself much impair the use of the data. More problematic is the selection criteria themselves: discrimination and mobilization. Hug emphasizes that many of the kinds of studies of ethnic politics in which people want to engage seek to explain something about state policy toward minorities or about minority political mobilization (or some other phenomenon closely related to one of these). Using the MAR data for that raises the problem of selection on the dependent variable: minorities only register in the dataset at all because they were identified in ways that draw on the dependent variable of interest.

Nothing much to say about this very helpful article. It incidentally helped me catch up on a fair amount of the ethnic politics literature that has come out since I last read regularly in the field (including a lot of work by both my friend and former colleague Stephen Saideman and friend-of-BHL Jason Sorens). It’s the kind of work that really needs to be done regularly. The messiness of political science data often results in problems of selection on the dependent variable; we study wars by looking at cases of wars, and revolutions by looking at countries that had revolutions, right? (Wrong, but unavoidably tempting, and often very hard not to do.) Endogeneity is a constant problem as well. Political science hasn’t followed the lead of (large parts of) economics down the rabbit hole of paralysis about this kind of thing: the insistence that, without a good instrumental variable, a randomized controlled trial, or an exogenous shock that provides a quasi-experiment, there’s just nothing that can be done or known, and so we should only study the kinds of things that lend themselves to one of those solutions. Instead we proceed as best we can, trying to quantify and study the things that we want to study– but periodically noting the limitations of what we can do with the data we have, and (as Hug notes) trying to improve them, or supplement them with different kinds of data to help solve different kinds of problems.

Next article: Gillian Hadfield and Barry R. Weingast,“Microfoundations of the Rule of Law.”