This white paper considers the benefits and limitations of multiple measures of college and career preparedness. The previous papers in this series evaluated measures against a framework consisting of 10 criteria. This white paper uses theory and practice related to measures of college and career preparedness as its frame of reference and primary organizing structure. The paper also considers the historical context for how accountability measures have been used in American education. The paper explores a set of criteria to consider when designing a multiple-measures system, then examines current multiple-measures systems used across the nation, and summarizes emerging themes from the literature on multiple measures for accountability of college and career preparedness. The paper concludes with a consideration of cutting-edge concepts and uses of multiple measures to ascertain college and career preparedness.
From the Conclusion: New Conceptions of Accountability
Conley and Darling-Hammond (2013) endorse uses of multiple measures that “contribute to a comprehensive picture of the quality of learning in classrooms, schools, schools systems, and states.” They recommend supplementing standardized test cut scores with additional data to reduce the likelihood of misclassification, yielding student “profiles of information for evaluating and conveying insights” rather than a single benchmark score. This information can be aggregated upward to reach accountability decisions about individual schools and the system as a whole.
It is important to note that adopting a multiple-measures system does not automatically enhance an accountability system’s quality or the quality of inferences drawn from it. However, Chester (2005) showed that combining performance indicators, a performance index, a growth calculation, and AYP status enabled Ohio to classify schools as Excellent, Effective, engaged in Continuous Improvement, on Academic Watch, or in Academic Emergency, resulting in an expanded definition of effectiveness, better identification of school levels, and a higher degree of confidence in the accountability system’s legitimacy.
Ultimately, a state’s decision to adopt multiple measures of college and career preparedness depends upon how it wants to manage an array of factors including stakeholder collaboration, design method, breadth of coverage, measurement and reporting method, data integration, ability to make comparisons, and level of stakes. Such determinations are not easily made because they rely upon complex and often competing values. Adding to the challenge is the fact that the use of multiple measures for accountability purposes has been on hold for well over a decade and is only now again beginning to be viewed as a viable option for state policy. The potential for a highly valid accountability system that influences practice in positive ways and that has strong practitioner buy-in and ownership is one of the key factors that makes it worthwhile to consider the technical challenges of a multiple-measures approach to a college and career indicator.
Authors: David Conley, Michael Thier, Paul Beach, Sarah Collins Lench, Kristine Chadwick
