Extra Credit

The Quick and the Ed has the latest on 2 important new reports. Ed Trust exposes how little states pay attention to graduation rates, and The National Center for Education Statistics has a massive statistical analysis on the background of who goes into teaching and what happens once they get there.

In light of my recent exchange with Joe Williams on merit pay, I cannot omit this latest from Ed Week.

Strategies for luring more students and working adults into math and science teaching have proved as popular among elected officials as financial incentives, which try to make one of the least appealing aspects of the job—low pay—a little less daunting.

Lawmakers on Capitol Hill are considering a number of bills that would expand existing incentives, such as scholarships and loan forgiveness for aspiring educators, and create new monetary inducements. Dozens of states, meanwhile, already offer their own incentives for teachers in subjects with shortages, including mathematics and science.

But those who have studied financial incentives say evidence is scant that they are attracting substantial numbers of college students and career-changers to math and science teaching, despite years of investments in those programs.

Opinions vary on why incentives have not shown greater results. Some believe the money available is relatively insignificant when weighed against potential job candidates’ worries about poor salaries and working conditions. Others say the hodgepodge of federal, state, and local incentives is so fragmented that few potential teachers are aware of what’s available.

“There’s been virtually no research on how effective [these] options are,” said Dan Goldhaber, a research professor at the Center on Reinventing Public Education, based at the University of Washington in Seattle. “We ought to be making decisions about these programs based on something more than what might be effective, and instead base it on empirical evidence.”

I hate it when common-sense doesn’t pan out. Next thing you know we’ll find out that paying kids for better test scores doesn’t work either.

Joe Williams and EducationGuru have thoughtful responses to the questions I raised about merit pay plans (here and here). Joe’s bottom line is that merit pay right now would be “a total disaster” in most places because of implementation problems. But he supports, as I do, economic incentives to lure the best teachers into the schools that most need them.

EducationGuru’s detailed plan ends up lending support to Joe’s conclusion–when you read what she has to offer, you come away convinced that it is worthy, expensive, and hard to implement well.

In light of all of this, I am left wondering why so many education reformers get excited if a politician mentions merit pay? If Joe is correct, it is not going to be done well anytime soon at scale. Maybe we should move on to more feasible, higher-impact items.

According to Mark Fisher, Michelle Rhee in DC is committed to thinking broadly about what would constitute success in school. He says she will resist defining success simply by looking at test scores in reading and math. Maybe we’ll even get arts programs in our schools! Like Fisher, I pray this is the case.

On the federal level, George Miller is making similar noises about NCLB, but it is too early to tell exactly what Miller has in mind. Like Ross Wiener of Ed Trust says in this WPost article, the devil is in the details on this one. And as Kevin Carey notes, Miller’s comments are a really big deal.

The bottom line is that reading and math tests have a hugely important role to play, but the evidence has simply become overwhelming that our current focus means 1) other subjects get less attention, and 2) schools–not knowing how else to raise scores–resort to drill and test prep in place of rich, engaging instruction.

Regarding drill and kill, I agree with Amy Wilkins and the folks at Ed Trust who argue that the best schools raise test scores while providing excellent teaching, and don’t need to teach to the test. (Amy said in a recent radio interview that in good schools the test is like a “gnat on the windshield” while driving down the road of education.) But that exposes one of the great ironies of No Child Left Behind–the very schools that we already knew did not provide high quality teaching are somehow expected to become places of rigorous instruction once we measure the results. But teaching kids who struggle academically is really hard stuff, and it is evident to anybody who spends time in urban schools that while some teachers are lazy, others simply do not know what to do to get better outcomes.

The key then is measuring a wide variety of outcomes (including reading and math, but not only them), measuring them in ways that are authentic and valid (some will disagree, but I think tests are ok, if they are good ones), and providing schools and teachers with the supports they need to develop good curricula and instructional practice. This last one is the hardest, and the one where federal and state policies have so far had the least impact.

I’ve said that I’m drawn to the idea of merit pay, but the details seem really hard to get right, as the Working Group on Teacher Quality has recently argued (pdf). Maybe so hard that it isn’t worth it. I’m not sure, and I don’t want to give up on something that seems, in the abstract, to have common sense going for it. But most of what I hear from the gung-ho merit pay crowd avoids the tough questions and instead simply asserts that people who who support merit pay, like Bloomberg (and maybe Obama) are courageous, and those who oppose it are pandering and stuck in liberal orthodoxy. But this seems like an example of an area where the details really matter.

So my question for Whitney Tilson, Joe Williams, Andy Rotherham, Edspresso etc., is . . . if you were a non-pandering reform-oriented superintendent, what exactly would your merit pay proposal be? Specifically, here’s 5 questions to start with:

1) Would you propose using value-added assessment, and what would you do if you were in one of the overwhelming majority of districts that don’t have the data systems to support that?

2) Do you endorse what Aspire schools do, and include school-wide measures of achievement and parent satisfaction surveys? Or would you base the merit pay solely on test scores tied to an individual teacher’s classroom?

3) How much weight, if any, would you give to the judgment of principals above and beyond standardized measures? Would there be any appeal process for teachers who felt they had been judged unfairly?

4) What about the areas that aren’t routinely tested? Are those teachers eligible for merit pay, and if so, who decides and on what basis?

5) Finally, if we accept as we must, that doing this right will cost more money (not the pay itself, but the investment in the assessment tools), how much should we be willing to pay?

This last one matters a lot since smart, not-stuck-in-liberal-orthodoxy school leaders like Emily Lawson, who are actually trying to implement merit pay, have argued that good merit pay plans are 1) costly to implement, and 2) would rank relatively low on her list of priorities for improving teacher quality.

My feelings about NCLB are quite complicated, because I’ve seen some of its benefits and harms first hand. I just got around to reading a Nation piece from May, which remains timely. The authors are critical of the law, but for the most part want to retain (while improving on) some of its core features.

Linda Darling-Hammond says, and I’ve seen how this happens,

Perhaps the most adverse unintended consequence of NCLB is that it creates incentives for schools to rid themselves of students who are not doing well, producing higher scores at the expense of vulnerable students’ education.

The impact is most severe on mission-driven charter schools. It is getting harder and harder to convince people to start schools serving the neediest students. It is one of the reasons you hear about so few great charter high schools. Nobody wants to start a school serving kids who arrive in the 9th grade 3 or 4 years behind.

And she has this proposal, which seems like a great one. I hope the Obama campaign takes it up; he could add merit and performance pay as his fourth prong, in line with his speech at the NEA convention.

A Marshall Plan for Teaching could insure that all students are taught by well-qualified teachers within the next five years through a federal policy that (1) recruits new teachers using service scholarships that underwrite their preparation for high-need fields and locations and adds incentives for expert veteran teachers to teach in high-need schools; (2) strengthens teachers’ preparation through support for professional development schools, like teaching hospitals, which offer top-quality urban teacher residencies to candidates who will stay in high-need districts; and (3) improves teacher retention and effectiveness by insuring that novices have mentoring support during their early years, when 30 percent of them drop out.

Last week I posted this comment after attending a Center for American Progress panel on teacher compensation practices. One of the panelists, Nancy Van Meter of the American Federation of Teachers, had this to say in response:

Before we even get to the question of your question “whether merit pay offers the biggest bang for the buck” — I think we need to ask how much bang for the buck it gets period. Unfortunately, I was very disappointed that the paper we discussed provided no evidence on whether the plans succeeded in attracting and retaining teachers, nor did it provide evidence that it resulted in improved student achievement.

Frankly, I was surprised by the paper’s implicit assumption that public schools can and should draw lessons from the compensation systems of privates and charters…that privates and charters are more successful than publics in raising student achievement and recruiting and retaining more effective, qualified teachers. In fact, as I explained, the evidence from the most recent research shows just the opposite — on average. I don’t dispute that there are high performers in every sector — public, private and charters. I strongly believe we can learn a lot from successful charters and privates — but based on the evidence presented in this paper, not in this area.

As I said at the event, there are scores of public schools where performance pay and skills-and-knowledge-based pay are being developed and implemented — some poorly (see Houston) and some well (see Denver, Cincinnati, Douglas County, CO). Public school districts are more likely to look at what other similar districts are doing successfully than to look at the private Salem Academy or High Tech High charter school because districts have to grapple with issues of scale and replicability in ways that don’t confront private schools or small chains of charter schools.

I think Center for American Progress would do a service for all if in a future paper it explores the best models of redesigned compensation systems regardless of what sector they come from– private, charter, public. As you point out, James, it’s not clear that any of the mechanisms are in place to evaluate the effect of the examples presented in the CAP paper. It will be unfortunate (though not unsurprising) if the push for performance pay/merit pay/etc continues with no evidence from any plans showing improved teacher recruitment and retention; improved instruction or gains in student achievement to back up claims.

For more on how hard it is to do merit pay in such a way that all stake-holders perceive as fair, check out this article (subscription required) from Education Week on the-fall out from the Houston initiative.

The debates over teacher pay are heating up again. Jay Greene and Marcus Winters recently published a paper arguing that teachers make more than many people assume.

I have always found the teacher pay debate fairly silly from where I sit. Does anybody seriously dispute that the compensation levels for teachers impacts the overall quality of those who enter the profession? I’ve known something was odd about this issue since the time a fellow law professor told me that K-12 teachers were overpaid. This fellow made more than K-12 teachers, taught less, and had virtually total control over the hours in his day.

But if anyone needs evidence that how much money we pay teachers has an influence on who enters the profession, and how long they stay, just go to any law school today. At Georgetown Law School I teach 8-10 students a year who are former TFAers and loved teaching. But they came to law school because as soon as they graduate, they could get jobs making 3-4x their teaching salary.

Fortunately, some more complex questions about the relationship between pay and teacher quality are being raised today. The Center for American Progress has recently published a couple of papers on this topic, including one that was released yesterday, Teacher Compensation in Charter and Public Schools, by Julie Kowal, Emily Hassel and Bryan Hassel. (The other recent paper from CAP is Teacher Pay Reforms: The Political Implications of Recent Research, by Dan Goldhaber).

I attended the panel presentation for the release of the Kowal/Hassel paper, and found the paper to be a useful primer on how some charter schools are trying to use teacher pay as a way to improve recruitment and retention, and motivate current teachers. Aspire Schools, for example, bases teacher pay raises on 3 criteria: 1/3 principal evaluation, 1/3 parental satisfaction survey results, and 1/3 school-wide academic performance. Fellow panelist Emily Lawson reported that D.C. Prep, which she founded, uses a complex model developed by the Teacher Advancement Program.

Lawson also pointed out, quite correctly, that it is extremely hard to put a high-quality system of merit pay into practice. (And she also made clear that it was expensive, even if we only consider the extra time and personnel required to make fair and appropriate decisions about who should get what raise). In my work at the Maya Angelou Public Charter School, I have struggled over these questions of implementation. Value-added evaluations are the only way to go, but they are not easy to do. And it gets yet more challenging for teachers and school staff who are not in math and english, the subjects most commonly tested.

Other interesting ideas floated in the Kowal/Hassel paper involved non-financial rewards which schools used to compensate and motivate teachers. KIPP gives teachers a travel certificate after four years of service “to let them go sit on a beach somewhere and rest,” High Tech High gives them a $2,000 debit card at the start of the year to purchase instructional materials, and Aspire Schools helps pay the student loans for its new teachers who enter the profession with student loan debt.

Good ideas, all. But the discussion of merit pay and other ways to modify the traditional salary scale leave one central question unanswered. Is merit pay the most effective way for a school or a school district to spend resources in pursuit of recruiting, motivating, and retaining effective teachers?

The theory behind merit pay is that it will promote student achievement by producing better teachers and teaching in the classroom. And as a matter of common sense, that seems right. But there are other things that schools and districts can also do to recruit, motivate and retain teachers that also seem promising, as a matter of common sense. For example, many young teachers feel alone and lost, and as I have discussed before, some believe the answer is improved professional development and mentoring programs.

Other teachers say that student mis-behavior makes teaching challenging and sometimes demoralizing, so perhaps smaller classes and additional school staff devoted to climate and behavior would make teaching more attractive. Lawson, for her part, suggested that although she is implementing merit pay at her school, she would rank merit pay as less important to recruiting and retaining teachers than creating an effective learning climate.

The essential question is whether, against the alternatives, investing additional resources in merit pay offers the biggest bang for the buck? Amazingly, it appears nobody really knows. Hassel said as much at the panel. More depressingly, the panel admitted that the current experiments with merit pay are not likely to yield reliable research results that will tell us how merit pay compares to other reforms in effectiveness.

The AFT’s Nancy Van Meter made an important point as well: merit pay tied to a principal evaluation (which most schemes are) only works to the extent that every school as a fair and qualified principal, trained to make these judgments. All the panelists agreed this was currently lacking in many of our schools.

I arrived at the panel in favor of trying to come up with good approaches to merit pay, including pay raises that are tied to individual and school-wide student achievement gains. And I’m still there. But those who would tinker must be candid about the fact that we are just making educated guesses. We really don’t know that this change will have a greater influence on teacher quality than would some other potential reforms.

Leo Casey, on EdWize, says my recent post regarding teacher quality shows that I have a “blindspot” on the question. Casey and I agree on many things. We agree on the importance of improving teacher quality. We agree that the Urban Academy program is a model of effective teacher mentoring and professional development. Casey takes me to task, however, for suggesting that the program will be hard to implement in the great number of schools that already face retention issues. In those schools, I suggest, existing resources will not allow for the Urban Academy-style investment in mentoring new teachers. To this, Casey responds:

It’s interesting that Forman, who on many counts is a radical critic of established power relations, accepts as fixed and given the very terms of the problem charter schools face in developing and retaining quality teaching. Teaching is an extraordinarily demanding craft. Under optimum conditions of solid preparation, quality mentoring, good professional development and an safe, secure school in which to learn, it takes a novice teacher a minimum of three years to master the fundamental skills of teaching. If a school is turning over its staff every three years, it simply can not develop a meaningful cohort of educational professionals who know their craft well. Accept that state of affairs, and you accept that the school will not have quality educators. Fixing the retention crisis is a pre-condition for the promotion of teacher quality. This axiom is true for district public schools, where one in every two new teachers is gone by the fifth year of teaching, and it is doubly true for charter schools, which have a much higher rate of turnover.

I accept this criticism. Casey is correct that I was too quick to accept the status quo as a given. My response came from the fact that I sit in two places at the same time. On the one hand, I teach education law and policy and write on education issues, and I try to see the world as it needs to be. On the other, I am a practitioner. I co-founded a charter school that recruits students who are years behind in school (including many from the juvenile justice system), and I work against great obstacles to prepare those schools for success in college, meaningful work, and life. When I am in that place, I often feel terribly constrained by existing resources and institutions. And I often feel compelled to try to make incremental change within a fundamentally unjust status quo. When I am in that place, facing resource constraints, hounded by day to day budget limitations, and trying to make the most of every penny, thinking of scaling the Urban Academy seems hopelessly utopian. But in the end, Casey, and Terry, are right. There is not really a choice. To accept the current state of teacher retention in many urban schools is to accept that we will never fix the schools. And my love for our kids and what they deserve means I cannot accept that.