Skip to main content

A Response to Tyler

By Kitty Calavita and Valerie Jenness, University of California, Irvine

We appreciate Tom Tyler’s close read of our article, “It Depends on the Outcome’: Prisoners, Grievances, and Perceptions of Justice,” and his pinpointing of issues of concern to him. We would like to respond here to just a few points.

Most importantly, it is a mischaracterization to say, as Tyler does, that we argue “that perceptions of procedural justice do not matter in the prison context.” As we say in the abstract (“These findings do not refute the importance of procedural justice” [p. 41]) and throughout the paper (e.g., “Nor do we argue that procedural justice is not important to these prisoners” [p. 43]), procedural justice is important. It matters, both as a moral principle and as a concrete reality for people involuntarily interacting with the long arm of the state, whether in a police stop or in a prison cell. The prisoners we interviewed clearly want to be treated fairly and with respect; nevertheless, the marker for them is in no small part whether the officials grant their grievance.

Our quantitative and qualitative findings reveal that in the context of grievances prisoners often are hard-pressed to make a clear distinction between procedural and substantive justice. Consider this typical example: A prisoner writes in a grievance that he has been denied medical attention for difficulties breathing and he requests once again to see a doctor. His grievance is denied. The prisoner takes this to mean that he has been disrespected and distrusted (i.e., feigning his medical need), and that the procedure must have been unfair. Is his dissatisfaction based on his perception of disrespect and unfairness (indicating an absence of procedural justice) or is it based on the fact that he was denied? It is both, with the outcome leading the prisoner to infer a lack of procedural justice. Our findings suggest not that procedural justice is unimportant—far from it. Instead, they show: 1) that institutional context matters in how it is perceived; and 2) that the distinction between procedural and substantive justice may not always be as clear-cut as is sometimes assumed.

Relatedly, as Tyler points out, and as we note in the article, the three models we present at the end of our empirical analysis (p. 66) reveal that procedural justice is important to prisoners’ satisfaction. It is, however, the complete set of data we present here, including the lengthy qualitative analysis that precedes and informs these models, that leads us to conclude that for these men in these prisons outcome in large part drives their understanding of process. Corroborating our qualitative findings, the statistical analysis presented on p. 59, for example, reveals that when we examined granted grievances separately the modal pattern is satisfaction with both process and outcome; and when we examined those that were not granted, the modal pattern is dissatisfaction with both process and outcome. This is consistent with the qualitative data from our face-to-face interviews with 120 prisoners randomly selected from three prisons in California.

Mixed methods, such as those we used here, are increasingly heralded in the social sciences—at least in some circles—as a way to develop rich, informative accounts of the social world (Small 2011). Unfortunately, in his assessment of our work, Tyler ignored our qualitative data (and much of our quantitative data), which were critical to the analysis, as the qualitative and quantitative data complement and reinforce each other, each helping make sense of the other. Far from being based on mere “intuition,” our conclusions are based on the scrutiny of a wide array of data and the pattern in which they converge.

We spent hundreds of hours listening to how prisoners think about, orient to, engage with, and experience the grievance system and many more hours reading official grievances submitted by hundreds of prisoners in their effort to contest their conditions of confinement in California prisons. Time and time again, when we asked the men in our study whether they were satisfied with how their grievance was managed, they replied with some version of, “No, it didn’t get granted.” When asked more generally whether prisoners were satisfied, one prisoner summarized what we heard from so many, “It depends on the outcome.”

It was because of these consistent and insistent prisoner voices expressing the importance of outcomes in determining their satisfaction, that we examined further the tangled relationships between and among objective measures of process (such as whether the prisoner had a hearing), prisoners’ subjective satisfaction with the process of the dispute, and their satisfaction with the outcome. Ultimately, it was clear to us that for these prisoners perceived procedural justice is so intertwined with substantive justice that there is no bright line separating them, nor a way to easily disentangle them.

On the issue of “appropriate operationalization”, Tyler argues that we have not used his measures of procedural justice and therefore “cannot know if a failure to replicate is due to the original theory being wrong…”. To be clear: this is not a replication study, and we never implied that it was. Our goal was not to “replicate” any of Tyler and colleagues’ studies—which, as he argues, would have required using identical operationalizations. Instead, our article examines whether the procedural justice framework holds up in a different setting and institutional context. 

If one wants to test the broad applicability of a theory it can be useful to see if it holds up in different populations and using different indicators. Let’s say you want to see if Gottfredson and Hirschi’s self-control theory of crime, one of the most tested theories of crime in the last thirty years, is confirmed in a population of white-collar business people. In that case, you might operationalize self-control in a way relevant to that context and that population, which may be quite different from those used by researchers studying delinquency among youth. (For an extensive discussion of the myriad measures used to test the robustness of self-control theory, see Tittle et al., 2003; see also Gottfredson, 2017). Studies using different methods, under different research conditions, and using a variety of indicators are in fact critical to fortifying, elaborating, and/or qualifying a general theory.

Tyler also raises the issue of “a broader literature review.” As he notes, we do not spend much time reviewing the literature on how perceptions of procedural justice influence legitimacy, compliance, or future violence. This is because it is not relevant to our study. We were concerned with perceptions of justice among prisoners and how they are formed (especially in the grievance process), not with how those perceptions then influence future behavior. While this is an important avenue of study, it was not our focus. Given the space limitations of the journal, we not surprisingly confined ourselves mostly to literature relevant to our study.

The most important takeaway from our article is not that procedural justice doesn’t matter; it’s that context matters in how it is understood and in how important outcomes are to disputants’ satisfaction. We argue more specifically that outcomes may be particularly important in the context of prison because of the high stakes of the prison experience. Statistical tests of prisoner grievances with very high stakes (i.e., medical and disciplinary grievances) reveal that in these high-stakes grievances outcome is even more important than in prisoner grievances with less at stake (i.e., other types of grievances). This is consistent with our hypothesis that the generally high stakes of prison help explain the dominance of outcomes in disputants’ satisfaction.

Exposing the dominance of outcomes in these prisoners’ perceptions of justice is not to say that procedural justice is not important to them. Clearly, it is. What we hope to contribute with this article is that context shapes how these disputants interpret and weigh procedural and substantive justice, just as it shapes so much else of social life.

Gottfredson, Michael R. 2017. “Self-Control Theory of Crime.” Oxford Research Encyclopedia of Criminology. DOI DOI: 10.1093/acrefore/9780190264079.013.252

Small, Mario Luis. 2011. “How to Conduct a Mixed Methods Study: Recent Trends in a Rapidly Growing Literature.” Annual Review of Sociology 37: 57–86.

Tittle, Charles R., David A. Ward, and Harold G. Grasmick. “Self-Control and Crime/Deviance: Cognitive Vs. Behavioral Measures.” Journal of Quantitative Criminology 19: 333-365.

Popular posts from this blog

Europeanization or National Specificity? Legal Approaches to Sexual Harassment in France, 2002–2012

By Abigail Saguy, UCLA

Sexual harassment represents a massive problem for working women worldwide. A recent social media campaign has brought increased awareness to this fact. In late 2017—after three-dozen women accused Hollywood producer Harvey Weinstein of sexual harassment, assault, or rape—millions of women posted “Me Too” on Twitter, Snapchat, Facebook, and other social media platforms. Taking inspiration from African American activist Tarana Burke—who, in 2007, started an offline “Me Too” campaign to let sex abuse survivors know that they were not alone—actress Alyssa Milano launched this online Me Too campaign to shift the focus from Weinstein to victims. She hoped this would “give people a sense of the magnitude of the problem.”[1] While some posted simply, “Me Too,” others provided wrenching detail about abuse they had sometimes never before shared publicly. In France, a similar social media campaign flourished, under the hashtag “balance ton porc,” loosely translated as “sq…

Comment: Making valid claims in social science research: A comment on Jenness and Calavita

By Tom Tyler, Yale Law School

I am writing to comment on several methodological issues raised by the article by Valerie Jenness and Kitty Calavita, entitled “It depends on the outcome”: Prisoners, grievances, and perceptions of justice”. I am pleased that the methodology blog for Law and Society Review has been created and provides a forum to discuss research design issues. I will address three aspects of the study: operationalization of the variables; statistical analysis; and inclusiveness of the literature review.

The Jenness/Calavita paper studies California prisons using data collected through interviews with prisoners. The paper says that it tests the perceptual procedural justice model, in particular there are frequent references to the Tyler model, in a prison setting. The study concludes that “prisoners privilege the actual outcome of disputes as their barometer of justice” showing “the dominance of substantive outcomes” (from the abstract)”.

I agree with Jenness and Cala…