A new look at how to evaluate teachers

August 14, 2013
Contact:
  • umichnews@umich.edu

A teacher engages with students in a classroom. Image credit: U-M Measures of Effective Teaching Project.A teacher engages with students in a classroom. Image credit: U-M Measures of Effective Teaching Project.ANN ARBOR—One of the most extensive collections of videos showing teachers at work in the classroom will be available this fall at the University of Michigan to help researchers learn more about how to evaluate good teaching and how to develop excellent teachers.

The videos of 3,000 teacher volunteers from around the country are a crucial piece of the Measures of Effective Teaching project, a multi-year study that set out to determine how to identify and advance strong teaching. MET is funded by the Bill and Melinda Gates Foundation.

MET used three kinds of measures to evaluate teaching—classroom observations, student surveys and student achievement gains. Typically for classroom observations, one or two observers mark items on a checklist while watching a teacher at work. A key innovation of the MET project was to use video cameras in place of human observers; teachers set up the cameras and did the recording themselves.

Now U-M’s Inter-university Consortium for Political and Social Science Research (ICPSR), part of the Institute for Social Research (ISR), has compiled the videos and the quantitative research that accompanies them into a collection known as the MET Longitudinal Database.

“The ability to gather huge quantities of observational data on classrooms was a real advance,” said Brian Rowan, the Burke A. Hinsdale Collegiate Professor of Education at U-M’s School of Education, professor of sociology and research professor at ISR. “Now we have instruments that can show teachers, ‘Okay, this is where you need to improve.'”

Using videos allows for a richer approach to evaluating what teachers do, Rowan said. For example, researchers can “double code” a video after the fact, marking good and bad examples from different areas of teaching, such as activities and content, and then link it to other data such as student performance.

“We think they’ll have a big impact on teacher education,” Rowan said.

Another major use of the videos will be to train observers in how to do effective evaluations. In fact, while the initial purpose of the videos was to evaluate teachers, researchers are already proposing a broad range of projects that will draw on the collection.

The MET Longitudinal Database will be kept in a virtual data enclave at ICPSR in order to protect the identities of participants, Rowan said. Researchers who want to work with the data will need special clearance.

Ten early-career researchers already have received access to the database—and $25,000 each—after winning a grants competition earlier this year sponsored by ISR, ICPSR and the National Academy of Education. See a list of winners at http://bit.ly/17M3wi9.

In addition to the original restricted research data, U-M will soon offer more publicly accessible videos. The Gates Foundation provided funding to follow 396 teachers—most of them participants in the original MET study—during the 2011-12 and 2012-13 school years.

The videos in this MET-Extension study will be available in October through the School of Education’s Brandon Center Digital Archive. The student and teacher participants gave consent for the broader education community to view their classroom sessions.

“We’re building a set of tools that people can use to search, clip, analyze, play back and work in groups with the videos,” Rowan said.

Users can stream or use virtual clips of the MET-Extension videos, but can’t download them to their desktops. The videos aren’t scored with quality ratings, but are searchable by practices and content, according to Lesli Scott, U-M’s project director for MET-Extension activities.

Scott said there could be considerable demand for the videos, particularly among organizations that want to include them in professional development products or evaluation tools. More than 30 states now require annual teacher evaluations that include observations of teaching.

 

Related Links:

 

Established in 1949, the University of Michigan Institute for Social Research is the world’s largest academic social science survey and research organization, and a world leader in developing and applying social science methodology, and in educating researchers and students from around the world. ISR conducts some of the most widely cited studies in the nation, including the Thomson Reuters/University of Michigan Surveys of Consumers, the American National Election Studies, the Monitoring the Future Study, the Panel Study of Income Dynamics, the Health and Retirement Study, the Columbia County Longitudinal Study and the National Survey of Black Americans. ISR researchers also collaborate with social scientists in more than 60 nations on the World Values Surveys and other projects, and the institute has established formal ties with universities in Poland, China and South Africa. ISR is also home to the Inter-University Consortium for Political and Social Research, the world’s largest digital social science data archive. For more information, visit the ISR website at www.isr.umich.edu.