Visiting Eastern State Penitentiary, we saw exhibits outlining the sentencing disparities across various demographics in the modern United States prison system. Specifically people of color tend to receive harsher sentences than whites who commit the same crime. We were interested in researching whether the same sentencing disparities existed in the United States in the mid-nineteenth century.
Basing our project off of datasets in ESP’s admissions books from the period of about 1840-1860, we initially sought to find admissions data from other prisons in the US from the same time period in order to see if disparities occurred nationwide, but we were unable to find any comparable data. However, the ESP dataset was large enough for us to use by itself in order to analyze whether sentencing disparities occurred at the prison.
Initial Datasets
For the data, we used three admissions books from ESP provided by the American Philosophical Society. We were able to combine the 3 available datasets with relevant data to produce a single dataset of 1729 prisoners. Furthermore, we found that 1235 of the 1729 prisoners had an ethnicity labeled. Given our research question, those 1235 entries were the data we would use.
Cleaning the Data
In order to perform our analysis, we realized that we would need to do some cleaning of the admissions book spreadsheets, since many columns in the files had data that was not in a homogeneous format, as described above. We used a Python script to clean the data by reading all the data in from the three files, combining it into one dataset while fixing problems in columns of interest, and outputting this dataset to be used later.
To deal with the disparities in how ethnicities were reported across the admissions books, we first dropped any entries which did not have a reported ethnicity. Then, to fix the outdated terminology used to describe ethnicity, we created a column where all people with a labeled ethnicity were labeled as either ‘White’, ‘Biracial’, or ‘Black’.
To deal with sentences which were reported in different time formats, such as months, years, or weeks, we created a column which contains the original sentence converted to days. Finally, we created a column which sorts the original offense into one of 8 categories: ‘Counterfeitting/Forgery’, ‘Murder’, ‘Assault’, ‘Horse Crime’, ‘Burglary’, ‘Larceny/Robbery’, ‘Arson’, ‘Manslaughter’. Crimes which did not fit in any of these categories were dropped from the analysis.
Analyzing the Data
We created another Python script to perform our analysis. First, we loaded the cleaned dataset and calculated the average sentence length in days for a crime for each of the three ethnicity groups, as described and visualized below:
We noticed from this information that assault, murder, and burglary have the widest sentencing disparities:
We then calculate the percent difference in the average sentence length for each crime for both Biracial and Black inmates compared to the average length of a White sentence for the same crime, displayed and visualized below:
Finally, we performed a statistical t-test to determine the statistical significance of Ethnicity in the sentence length against an assumption that there is no sentencing disparity between white inmates, and inmates of color. Based on the t-test, we found that the results for biracial inmates who committed assault, black and biracial inmates who where charged with burglary, and black inmates who were charged with horse crimes or murder where statistically significant as compared to their white inmate counterparts.