National Bias and Sensitivity Review/Evaluation of Assessments

National Bias and Sensitivity Review/Evaluation of Assessments

Date(s)

March 8, 2017

Status

Activity is Ongoing
Scholarly Activity Role: 
Evaluator
Organization Name: 
Measured Progress, Inc.
Role or Title within the Organization: 
Contracted Evaluation Consultant

During Year 2 as a Contracted Bias and Sensitivity Review Facilitator, Dr. Johnnie McKinley convened a 20-member National Panel to evaluate and make recommendations on bias-free and sensitive implementation of third through eighth grade assessment products developed by content experts that will be used in 10 states by Measured Progress, Inc. McKinley continued to use the services of the Outreach Coordinator, two Co-Facilitators, and 20 Panelists from national, community, business, civic, ethnic-focused, service, and K-20 education organizations to serve on Bias and Sensitivity Review Panels from 2017 - 2020. During the fall 2017 and winter 2018 in-person and virtual Bias and Sensitivity Review, McKinley reviewed the salient content of the April 2017 educational modules to ensure the 20-person Panel's capacity to effectively conduct valid and reliable Bias and Sensitivity Reviews with high fidelity. 

The intent of the quarterly Virtual Reviews through 2020 is to minimize impacts on students’ performance on Reading, Writing, Language Arts, and Mathematics assessments from factors unrelated to the content being assessed. Biasing factors may include physical requirements (e.g., interpretation of a visual stimulus, manipulation of a computer touch pad or mouse), language syntax, vocabulary, unfamiliar contexts, geographical region, gender, and so forth. Sensitivity refers to factors that may negatively affect individuals or groups of students due to the sensitive nature of the content (e.g., stereotyping, violence, or war).

After Panelists reviewed passages/stimuli and items/performance tasks and noted any perceived issues on Record Sheets, they discussed their perceptions as a facilitated group. Dr. McKinley and the Co-facilitators recorded their responses and Dr. McKinley developed a report to Measured Progress describing the descriptive statistics on outcomes of items accepted as written, recommended for revision, and rejected for violating established Bias and Sensitivity Considerations (first developed by the State of New York). At the end of the virtual events, McKinley led the Panelists to consider and report out a qualitative and quantitative evaluation of their learnings and capacity to conduct Bias and Sensitivity Reviews with reliability, validity and fidelity.