A comparison of old and corrected reports on the SAT scores of incoming Claremont McKenna students shows that the reported score changes entailed systematic manipulation of the vast majority of the data points that CMC had reported.
This manipulation, which was announced Monday by President Pamela Gann in a schoolwide email, led to Vice President and Dean of Admission and Financial Aid Richard Vos’s resignation.
The new, corrected numbers demonstrate a roughly flat trajectory of SAT scores in both the Critical Reading and Math sections. The old, manipulated numbers showed a clear upward trajectory in both mean and median scores.
Critical Reading scores were artificially increased by an average of more than 17 points, a greater amount than Math scores, which were increased by an average of only 10.5 points.
The manipulation hid year-to-year drops in SAT scores as well. While the manipulated numbers showed CMC’s median Math section SAT score to be constant at 700 from 2004 to 2007, in reality, the median value had dropped to 680 from 700 in 2006 and 2007.
The score manipulations hid the fact that the freshmen admitted in 2011 — the class of 2015 — had mean Critical Reading scores that were the lowest since the class admitted in 2007. Their mean had been boosted by 23 points. The mean Math score of the freshmen admitted in 2007 — the class of 2011 — had been boosted 28 points.
Gann’s email said that reported scores were “generally inflated by an average of 10-20 points each,” without distinguishing median and mean scores. No median scores were artificially increased by more than 20 points, though median scores are necessarily multiples of 10.
But those current freshmen whose Critical Reading scores were so low may be heartened to learn that their Math scores had been manipulated only very slightly: one Math score between 500 and 540 was reported as being between 550 and 590. This manipulation did not change the mean or median.
The data manipulation also had the effect of hiding students’ scores which were extremely low. Since 2005, six students had Math scores below 500 but only one had been reported. Three of these six were admitted in 2008 or later. Five students had Critical Reading scores below 500 and only one was reported, though no students with Critical Reading scores below 500 began attending CMC after 2006.
CMC’s Office of Institutional Research publishes a “factbook,” which is not publicly available, that shows SAT scores by each incoming freshman class. The factbook document attributes the SAT data to the CMC Admissions Office. The document was updated Monday to replace the old, manipulated numbers with corrected numbers. The Port Side used this data for its analysis.
That factbook lists the number of incoming students with SAT scores within 50 point ranges on the Critical Reading and Math scores, as well as mean and median results.
In order to achieve median scores that appeared higher than they actually were, students’ scores were moved from lower ranges to higher ranges. Some scores were also fabricated out of thin air, as in 2009’s numbers, by lowering the total number of “missing” scores. Update 2/1/12: [Corrected date] At CMC, applicants are required to submit either SAT or ACT scores. If a student’s SAT scores are “missing,” then, according to CMC spokesman Max Benavidez, the school does not have their SAT scores in any format. Presumably, these students submitted ACT scores. Though 76 students out of the 282 who matriculated in 2009 did not report SAT scores, only 48 missing scores were reported in the original manipulated numbers. In another year, low scores where not reported by increasing the number of missing scores, as in 2005.
More than three quarters of non-zero data points that CMC had reported were manipulated.
CMC also publishes its statistics in a Common Data Set format; the common data sets for 2004-2005 school year through the current year were updated with corrected statistics on the afternoon of Friday, January 27, 2012, as shown by file metadata.
This article will be updated with more analysis, graphics and links to raw data over the course of the day.
Here is the data comparing results. Blue cells were manipulated downwards; yellow cells upwards.
Critical Reading, 2004-2011
Tables split into Math/Critical Reading, updated 3:30pm 1/31/12. Google Docs by Jonathan O. Hirsch.
The viewers below display the PDF versions of the factbook.
Original, manipulated data
New, corrected data