Teacher Quality and Preparation

Three organizations recently released new education rankings of states. Education Week’s Quality Counts is a comprehensive analysis of states’ education policies and student outcomes, conducted by the Editorial Projects in Education Research Center.

This week's National Journal Education Experts blog asks about the big takeaways from the Bill and Melinda Gates Foundation's MET study on effective teaching. My colleagues Lisa Guernsey, director of the Early Education Initiative, and Anne Hyslop, education policy analyst, weighed in. 

What can you do with $45 million and three years? Well, if you’re the Bill & Melinda Gates Foundation, you can confirm, empirically, what educators have always known implicitly: great teaching matters, it can be measured, and it improves student learning.[1]

That was one of the many findings released last week in the final report from the MET Project (Measures of Effective Teaching). MET has generated buzz in education and popular media alike, so I won’t provide a full synopsis here. For a basic summary, check out the Washington Post or Huffington Post rundown; for more thoughtful commentary, turn to posts from Chad Aldeman, Andy Smarick, Rick Hess, Marty West, and National Journal Experts Blog. Instead, I want to call attention to two big takeaways from the MET Project.

What teacher evaluations measure is just as important as how they measure it.

Much has been made of the finding that classroom observations are the worst predictor of student learning, compared to state test scores and student surveys. Some have questioned whether observations are worth the significant time and personnel costs involved to do them well. Tim Daly of TNTP even claimed that MET shows “the way that most teachers have been evaluated forever is completely unreliable.”

It’s easy to jump to that conclusion: MET used proven, high-quality observation tools, and observers were trained and certified on their knowledge of them. This isn’t the case with many of the classroom observations used across the country.  Still, observations are a critical component of teacher evaluations, particularly for those in the early grades and in untested subjects. And using observations typically receives greater support from educators compared to test scores. Finally, MET’s research found that although classroom observations didn’t improve the predictive power of the evaluation measure, they did improve its reliability – or stability – from year to year. 

Test scores also don’t have the same diagnostic power as classroom observations: as Amanda Ripley put it, “test scores can reveal when kids are not learning; they can’t reveal why.” Observations can provide teachers with valuable, timely, and clear feedback on their practice. Given their complexity and the timing of state testing, value-added measures are far less teacher-friendly – not to mention, limited in scope. Surely, great teaching involves much more than improving student scores on multiple-choice tests in two subjects.

To this end, it’s laudable that MET’s researchers also used higher-order tests (the SAT 9 Open-Ended Reading Assessment and the Balanced Assessment in Mathematics) to measure student learning. In some states, these assessments are more similar to the Common Core assessments they will offer in 2014-15. Presumably, states should want teacher evaluations that not only function well with today’s tests, but also those of the future.

Still, the tests MET used only consider English Language Arts and math skills. If the ultimate goal of evaluations is to measure whether teachers create learning environments where students achieve a broader set of outcomes (say, the knowledge, skills, and attributes it takes to be college- and career-ready), then there is still a long way to go in developing these systems. In 2014, many states will be simultaneously implementing new teacher evaluations and the Common Core assessments. But the best evaluation systems today do a far better job identifying teachers that improve student learning via state test scores than teachers that improve college and career readiness. MET’s findings suggest that states should carefully consider whether their evaluation systems are measuring the teacher attributes needed to meet the Common Core’s objectives.

How teacher evaluations are used is just as important as what they measure.

Part of the demand for research like the MET Project comes from the push to use teacher evaluation systems to make human resources decisions. Hiring, retention, placement, compensation, and tenure can all be affected. Some of the push can be attributed directly to the Obama administration: developing and using teacher evaluation systems like the ones in the MET study for HR decisions was a major component of both Race to the Top and the No Child Left Behind waivers.

But there is still uncertainty surrounding teacher evaluation systems; the MET Project doesn’t provide a definitive roadmap or specific policies for states and districts looking to measure effective teaching. Many of its findings are ambiguous (with the exception that value-added measures must account for students’ prior test scores). The MET report is inconclusive when it comes to:

• whether student demographics should be included as a control in value-added models;
• precisely how to weight each component within a composite effectiveness measure: value-added data, student-perception surveys, and classroom observations;[2]
• whether measures like the Content Knowledge for Teaching (CKT) tests or subject-based classroom observation tools could be useful additions to composite measures of teacher quality; and
• who should observe teachers, how long these observations should last, and how many observations should occur each year.[3]

The teacher quality measures MET suggests are “better on virtually every dimension than the measures in use now.” But does that mean similar teacher evaluation systems should be used as the deciding factor for whether a teacher is fired? Or promoted? Or receives a pay increase?

Thorny questions, indeed. Yes, the new measures of effective teaching are promising, compared to most old-school teacher evaluation systems where nearly every teacher rated ‘satisfactory.’ But given MET’s lingering questions and inevitable measurement error in these measures of effectiveness, wouldn’t it make more sense to continue developing and refining teacher evaluation systems without rushing to use them for high-stakes decisions? Especially since most schools lack the capacity and resources to implement evaluations of the rigor and quality that the MET study used? States and districts should consider using the results from teacher evaluations in a more diagnostic manner: why not make these measures of effective teaching the first step in the process of providing professional development, determining who receives pay increases or tenure, and making decisions about hiring or firing – rather than the final step?

Everyone from U.S. News to David Letterman knows that a surefire way to get attention is to produce a ranking. Few – including my colleagues – can resist their clear-cut simplicity (or the opportunity to be judgmental). Add a controversial figure like former DCPS Chancellor Michelle Rhee to the mix, and it’s no surprise that StudentsFirst’s State Policy Report Card has received so much attention.

Love or hate Rhee, these rankings matter – at least to policymakers, the media, and wonks. Two-thirds of states couldn’t muster a ‘C’ on their report card, which examined twenty-four policies across three categories: elevating teaching, empowering parents, and improving school governance and spending. Only two states, Florida and Louisiana, scraped by with a ‘B-’ average. Now, state policymakers are using the rankings to tout their (relative) success, while others are vowing to use the Report Card as a roadmap for education reform in their states.

Many, including the American Federation of Teachers, have criticized StudentsFirst for excluding student achievement from the grades (yes, you read that correctly: the AFT wants to use student test scores) and focusing only on policy. While the omission is worth noting, it shouldn’t diminish the credibility of the StudentsFirst grades outright. Policy choices matter. They create incentives, signal what’s important, and enable or prevent certain actions at the local level. And it’s appropriate for an organization centered around a policy agenda, like StudentsFirst, to produce rankings focused on whether states have enacted those policies. That said, the only thing I learned from this report card was which states had adopted Michelle Rhee’s favored education reforms.

Those policies, and the theory of action accompanying them, are where I have issues with StudentsFirst. Often, they rewarded states for arbitrary, highly specific choices, without the rationale to support such specificity. For example, StudentsFirst calls for 50 percent of teacher and principal evaluations to be based on value-added data, despite inconclusive research. Why insist on 50 percent for an ‘A’ grade, particularly when the MET project found that "there is a range of reasonable weights" for reliable, composite measures of teacher quality?

Similarly, StudentsFirst rewarded states with A-F school grades to help parents better understand school quality. But if states used alternative ways of labeling schools, like a five-star system, they could not earn higher than a ‘D’ on that policy from StudentsFirst. What is included in school accountability systems is surely as important as how schools are labeled, and there is no reason to think a five-star system couldn’t be as successful as an A-F one.

More troubling, StudentsFirst frequently rewarded states with high marks for enacting the most severe version of reform – even if the policies are new, untested ideas like parent trigger laws. This may not be the wisest choice for state policymakers, as divisive reform agendas are often bogged down in criticism: too extreme, too dogmatic, hastily conceived, and poorly communicated. See: Tom Luna in Idaho, Tony Bennett in Indiana, and yes, even Michelle Rhee in DCPS.

Take the case with parent information about teachers’ effectiveness: to earn an ‘A’ for this policy, StudentsFirst requires parental consent to place students with teachers labeled ineffective. But with something so controversial, it may make more sense to start first with parental notification. This would be a huge policy shift for many states, but warrants only a ‘C’ grade. Further, before you have parental notification about ineffective teaching, states probably want to pilot and refine their teacher evaluation systems so that they are accurate and fair – building buy-in from educators. But without parental notification on the books now, StudentsFirst deems states as failing. Why penalize policymakers for making thoughtful, logical choices about the sequencing and implementation of reform? 

Change doesn’t come easy, and it often requires patience and compromise – rather than bulldozing. Policy choices are critical, but building trust and respect among stakeholders can make or break whether those policies take root. In the case of DCPS, many of Rhee’s reforms have been sustained, albeit under the radar by her successor, Kaya Henderson. It’s unclear whether Indiana and Idaho will see similar results. In any case, it would be nice if StudentsFirst could also reward states for taking less severe – but perhaps more inclusive – action to elevate teaching, empower parents, and improve school governance.

Education reform is divisive enough without encouraging all-out aggression. This may not be the Michelle Rhee-way, but it can be an equally effective way to enact and sustain the kinds of reforms Rhee would like to see.

Conventional wisdom may say that the federal government should make way for states in education reform, but a week ago, voters didn’t seem to agree. In Idaho, voters rejected merit pay for teachers, limits on unions’ collective bargaining powers, and an expensive contract to provide one-to-one mobile computing devices to students and teachers. South Dakotans voted down similar measures to eliminate tenure and adopt merit pay. And in perhaps the biggest upset, Glenda Ritz, a teacher, defeated Indiana’s incumbent Superintendent for Public Instruction, Tony Bennett, an education reform superstar. Ritz received strong support from unions and Tea Party conservatives alike, who opposed Bennett’s initiatives to adopt the Common Core standards and assessments, develop an A-F school grading system, evaluate teachers based in part on standardized tests, takeover failing schools, and implement school vouchers. The only education initiatives that fared well were charter school­s, with both Georgia and Washington approving ballot initiatives.

If last week’s elections tell us anything about the fate of education policy over the next four years, it’s that parents – and the public as a whole – have little faith in education policy. And who can blame them? Which specific policies, from the federal government or from states, have improved their child’s experience in the classroom over the last decade?

Sure, there are better data than ever before about schools, teachers, and students. But it’s not always shared with parents in a compelling, personalized way. Yes, states adopted more rigorous standards for all students, not just those expected to succeed. But the general public is, in most cases, completely unaware of the standards and why they matter. And they’re definitely turned off by the idea that, even if students learn the same standards, there are different expectations for their performance based on race. Finally, new technologies have emerged that can engage students in learning experiences in remarkable ways. But they are overshadowed by standardized tests of little value that appear to take time away from real learning.

More so than public opinion toward unions or Common Core, last week’s election results appear to demonstrate the extent of public frustration with testing and “teaching to the test” in particular. In the No Child Left Behind era, teachers and school leaders have often felt battered, rather than empowered, by reform, and their views have seeped into the general public. From the Atlanta cheating scandal to Pineapple-Gate, testing is viewed – at best – as a necessary evil with enough influence already over the education system. Parents and educators (and voters) don’t necessarily take issue with merit pay or evaluations in their own right, but rather with the fact that these judgments will be based on test scores.

Education policy tends to involve rules, requirements, sanctions, and other left-brained tools and structures. However, these tend to conflict with the critical thinking, analytical, innovative right-brained skills and attitudes widely perceived as critical for success in the hyper-connected, 21st century world. Based on the last decade, government bureaucracies and structures – no matter how well-intentioned – often appear ill-fit to create educational settings where this kind of learning occurs.

So some parents have turned to charter schools, hoping that schools outside the authority of districts and states could break the hold of testing and create more productive, engaging school environments. But while growing numbers embrace the idea of charters, the same cannot be said for other mainstream education reform ideas like school takeovers or closures, teacher evaluations, and changes in HR policies like merit pay or tenure reform, based primarily on student test scores. For most parents, returning autonomy and authority to educators is a far more appealing proposition than increasing the significance of standardized test scores. While over 70 percent of the public has trust and confidence in America’s teachers, the opposite is true for government – and it showed in last week’s polls.

As for education policy wonks, this election should serve as a warning. Advocates and policymakers must do a better job of making the case for standards, tests, data, and accountability policies enacted by states and the federal government. One way to do this is by improving them – creating more sophisticated assessments, using and valuing achievement data beyond test scores, and improving how parents and the public access information about school quality. But maybe it’s also time to ditch the rule book and add some right-brained thinking to education reform.

When Michelle Rhee was chancellor of DCPS, one of her chief accomplishments was negotiating a new contract with the teachers union that included a new teacher evaluation system. The system, called IMPACT, was designed to keep good teachers in the classroom through incentives like merit pay and weed out the bad by giving the district the power to fire teachers who were repeatedly ranked at the bottom.

IMPACT rates teachers on a variety of metrics, from their students' test scores to classroom observations. It has been both controversial and held up by education reformers as a model for how other districts could begin evaluating teachers in a holistic way. In some ways, the methods for observing teachers are similar to those of the Classroom Assessment Scoring System (CLASS), the Danielson Framework for Teaching and other evaluation systems that are catching on in the early childhood world in that it both evaluates teachers and gives them opportunities for feedback and mentoring. 

DC has been using this system since 2009, so two school years have passed since it began. This month, The New Teacher Project released a report that addresses important questions about how the new teacher evaluation system is playing out. In this podcast, Dan Weisberg of The New Teacher Project and Anne Hyslop of the New America Foundation discuss the new report and what it says about the future of the teaching workforce. Maggie Severns hosts.

Click here to listen to the podcast. You can also subscribe to our podcasts in iTunes, and download previous podcasts from our online archive.

There was a lot of education-related activity in the states this election season, from a ballot initiative in San Antonio that will raise the sales tax to help pay for pre-K, to the race for Indiana superintendent, where controversial incumbent Tony Bennett lost to challenger Glenda Ritz, who reportedly got more votes in the race than Governor Pence did.

We put together this interactive map to help readers peruse some state-by-state results of key races that will affect early education in the states in coming years. Scroll over states to find out more about who ran, who won and where there could be big policy changes afoot.

State-by-State Results of Key Early Ed-Related Races and Ballot Initiatives

A special thanks to Megan Carolan of NIEER, who contributed research to this map.

Update: The Washington gubernatorial election was called for Inslee (D). Voters in Washington state also approved Initiative 1240 to allow charter schools.

On this week’s podcast, Laura Bornfreund, Senior Policy Analyst for the Early Education Initiative, and Albert Wat, Senior Policy Analyst for Education at the National Governor’s Association, discuss states that have enacted literacy laws that include a requirement to retain third graders who do not pass the state’s reading test. Also on this podcast, Clare McCann, Program Associate for the Education Policy Program, highlights the Washington gubernatorial race. This is the second of three installments about key state and Congressional elections.

States are starting to enact laws that retain third graders who are unable to score at proficient levels on states’ reading assessments. In 2002, under the leadership of Governor Jeb Bush, Florida instituted one of the first such policies as part of its comprehensive reading law. And that is key – the retention policy is just a piece of the law. Florida has made significant investments in literacy, PreK-12, and has seen improvements in children’s reading proficiency since the law’s implementation.

Apps on iPads are dominating the ed-tech conversation these days, but last week I had an opportunity to move beyond a trumpeting of the touchscreen and examine how online, digital media could change the early ed workforce. In a presentation for a meeting of the Council of Chief State School Officers in Indianapolis, I talked to early childhood specialists in state education agencies about some untapped areas for enhancing training and forming partnerships among educators , including librarians, via digital technology.