Questions on Testing

National Research Council Study Reaffirms Limits of Testing,
Adds to Concerns Raised by Prior Study

The first major independent study of D.C. school reform has concluded that rising standardized test scores are of limited value in determining whether students are actually learning more, as reported last month by the Washington Post. The study from the National Research Council states that "Looking at test scores should be only a first step -- not an endpoint -- in considering questions about student achievement, or even more broadly, about student learning." The report is the first in a series of evaluations required by the 2007 law that placed the D.C. public schools under mayoral control. The report recognizes that the city has made "a good faith effort" to implement the Public Education Reform Amendment Act, but notes that determining the impact on student achievement will take further study.  Researchers did conclude that the school department must develop a more sophisticated capacity to track individual students who move from traditional public schools to charter schools, or in some cases drop out of the system entirely. "In the meantime, naive aggregate comparison of test scores among race-ethnic groups in the District should be interpreted critically and cautiously," the study said.
See the report: http://www.nap.edu/catalog.php?record_id=13114

The NRC reports has further deepened concerned among educators who began to ring alarm bells concerning a government study released in November 2009 which found that problematic educational practices were occurring more frequently in high-poverty and high-minority schools across the country. That report, by the Government Accountability Office (GAO), was requested to examine teaching practices related to the No Child Left Behind education law and, in particular, techniques being used to prepare students to meet state standards and raise scores on state standardized tests.

One legislator expressed that the GAO report, “ reaffirms my concern that the No Child Left Behind Law’s one-size-fits-all approach and heavy focus on high-stakes testing is causing problems in schools, particularly schools that serve our most disadvantaged students.  The study found that problematic teaching practices like teaching to the test and spending more time on test preparation are happening more frequently in high-poverty and high-minority schools, many of which already have less access to high-quality teachers and resources than more affluent schools.” Two key reforms were suggested by the legislative panel who reviewed the results of that GAO study: supporting the development of higher quality tests and ensuring students and schools are measured by more than test scores, but neither has been enacted.

Read the GAO report highlights at http://www.gao.gov/highlights/d1018high.pdf

Oregon Small Schools

Oregon Small Schools Initiative

Nearly a decade ago, the small schools movement in Oregon was limited to small rural high schools and a sprinkling of charter and magnet schools that had sprung up across the state. Never in Oregon’s history had there been a statewide effort to intentionally create small high schools whose focus was meeting the needs of all of their students.

The Oregon Small Schools Initiative—the largest investment in high school reform in the state’s history—has produced a cohort of 34 small schools that is closing the achievement gap between minority and low-income students and their majority peers and changing the lives of the more than 25,000 students and 500 staff who have participated in this work.

In the past seven years the Oregon Small Schools Initiative has made an indelible mark on the landscape of education. An analysis by ECONorthwest revealed that student achievement levels are notably improved, often dramatically and especially so for historically disadvantaged students. ECONorthwest also concluded that the small schools model does not necessarily require a significantly greater investment per student than would a traditional high school serving the same students.

Read the full report: http://www.e3smallschools.org/

New Teacher Training

NCATE Report on Teacher Preparation: Developmental sciences critical to student achievement

Teacher-education programs must include the basics of developmental science in their training programs, according to a report released recently by a panel convened by the National Council for Accreditation of Teachers Education (NCATE). The panel, co-chaired by Dr. James P. Comer, found that too few teachers enter the profession with a firm grasp of the importance of developmental sciences on student learning. “Teachers cannot improve learning if they don’t know how to help address the social, emotional, and cognitive needs of children and adolescents,” said Dr. Comer, founder of the Yale Child Study Center School Development Program, which implements developmental science interventions, including the well-known Comer School Development Program.

Dr. Comer told a news conference in Washington, D.C., that students attending underperforming schools were very often not born into networks that provide the attachments, bonding and linguistic skills critical for learning. But when teachers and administrators address developmental needs, academic achievement often increases, he said.  “Most teachers and administrators are not prepared, no fault of their own, to create environments in schools that compensate for underdevelopment,” Dr. Comer said.

A meta-analysis of 213 school programs, for example, found that programs such as the Comer School Development Program led, on average, to an 11 percentile-point gain in student achievement. The study, by J.A. Durlak, appears in the January 2011 issue of Child Development. The report by NCATE, the professional accrediting organization for schools, colleges, and departments of education, also says little effort has been made to ground school reform in the developmental sciences, which include cognitive science, neuroscience, and the science of child and adolescent development. The report calls on states and the federal government to enact policy changes to address problem – and to do so quickly. “Developmental science is not ‘fluff’ that can be considered optional or an add-on to what schools do or how educators are prepared,” said panel co-chair Dr. Robert Pianta, a prominent psychologist who is the dean of the Curry School of Education at the University of Virginia he says. “If we don’t act now to integrate development sciences knowledge into preparation programs, we may lose another generation of learners.”

The report, The Road Less Traveled: How the Developmental Sciences Can Prepare Educators to Improve Student Achievement: Policy Recommendations, was prepared by a multi-disciplinary panel of experts, including some of the nation’s most prominent educators, psychologists, and authorities on young people from related disciplines. Highlights of the findings included:

• Educator-preparation programs fall short of providing adequate training in the developmental sciences, including cognitive science and the science of child and adolescent development.
• Programs need to do a better job of integrating the behavioral sciences with best practices in classrooms and communities.
• Policymakers must take into account the importance of child and adolescent development as they design new ways to assess student and teacher performance. Developmental science is a crucial factor in low-performing schools, where students often need of developmental supports to improve achievement.

Alan Dichter Memoire

The Question

There’s nothing quite like the right question…

Looking back, I can honestly say it was the most powerful question I ever heard asked in a staff meeting; the distilled essence of a probing question.  It took about five years of very hard work to answer it and, in the process, teaching and learning at our school were transformed. Let me tell you about it.

Download Satellite 
Academy HS document (.doc)

It was 1991 and “Graduation by Exhibition” was something that many high schools in the Coalition of Essential Schools were tackling.  The staff of our small New York City high school was diligently working on clarifying their expectations for graduation exhibition portfolios.  We’d been at it months. The sheets of newsprint were saved and re-posted at each meeting --“What do we want students to know and be able to do when they graduate, and how will they demonstrate it?”  A fine question, but not THE question. That would come later.

The lists got longer and longer with every session, of course. There were just so many worthy things that couldn’t possibly be left out. One could feel very proud of the group’s work – they basically had it all up there.  Of course no one, not for one minute, actually thought that any student could really know and would be able to do it all well by the time they graduated– it was a wish list and everyone knew it.  Many of the items had been talked about, off and on, for years and nothing much was ever really done about them. I remember asking if the list was “what students needed to know and be able to do in order to graduate” or was a list of “wouldn’t it be nice if…”

But editing the list continued to be difficult, because everything was worthy, despite the fact that everyone could see the totality of skills and content was quite unrealistic.  And we had had no discussion at all about what and how we might need to teach in order to get to many of the things on the “wish” list.  Our discussions so easily and quickly went from energizing to deflating.  What were we really doing here?

And then came THE question. No one actually remembers who asked it, but everyone remembered hearing it:  “Maybe instead of asking ourselves ‘what do students need to know and be able to do in order to graduate,’ we should first ask ourselves, ‘What do students need to know and be able to do to succeed in this school?’”…. A collective pause… “What do students need to know and be able to do to succeed in this school?”… Hmm…  “And let’s assume for a minute that we will have to teach them to do these things (which we can’t assume they already know how to do).” Another pause.  “And let’s also assume we want them to collect and organize evidence from our classes.  Wouldn’t it be worthy to identify a few things we can agree that students need to know and be able to do to succeed in our classes, and then go about teaching them and having them collect evidence?  The students would learn worthy things. We’d learn a great deal also wouldn’t we?  Isn’t that what we should be doing after all?”  Hmm…

The conversation began to accelerate. Energy began to flow again. What do students need to know and be able to do to succeed in this school?  Revise their work!  Yes, indeed.  Do we teach that? Or do we simply ask for it, and the kids who can already do it succeed and those who can’t? Well, we ask them again nicely and give them more time.  And if we intend for it to be more than just making it neat and correcting their spelling, we also need to be clear about exactly what we want, and we need to give them feedback connected clearly to those expectations.  And students will need to know those expectations explicitly. Who could argue against that?  So, what if the ability to revise your work in connection to commonly understood expectations based on various forms of feedback was something we thought our students need to be able to do to succeed in this school?  Well, that means… now we have to teach them to do it and we have to provide the kind of assignments that allow them to produce the evidence we want to see.  It was a classic form of “Understanding by Design”.

You could see the wheels turning.  Oh, my!  So many things will need to change in my class:  fewer essays, if multiple drafts are expected; the students will need to actually be able to use any rubric we devise; my feedback needs to be more precise.  And if we want peer feedback on first drafts, there’s even much more to do.  But I have my own way of doing things and my favorite assignments which the students like, but which may not actually produce much evidence of anything. Hmm…

It became clear that agreeing about what we wanted students to know and be able to do to succeed in our school was in many ways more daunting than agreeing on a list of graduation expectations.  It was all about defining our priorities in teaching, and it forced a discussion which demanded a pedagogical coherence across the school far more powerful than anyone could have predicted.

Over the five subsequent years, the list of “know and be able to do to succeed in the school” grew modestly but thoughtfully.  We didn’t add anything until we felt we had made sufficient progress on what was already on the table – until we were assured that we could teach students to do it and they could in turn produce the evidence --taking notes (a fascinating discussion, since all agreed it had to be more than copying off the board); “close” reading (strategies for  accessing and making meaning of various texts); working intensely in groups; discussion & presentation; independent work; and eventually, numeracy across the curriculum.

In particular, I remember one conversation that happened about two years into the endeavor.  The staff had kept writing samples from a large group of students when they entered the school, and had also collected additional samples from the same students about a year and half later.  And, let me also say that this was a great staff – smart, committed, everything you could possible want.  The room was silent as they looked at the work.  Finally Susan spoke, “I think no one in this room actually thought our students could do this well.  We only thought we had high standards.  We really didn’t.  We had no idea what our students could do if we worked together.”

And we had really just begun.  I learned a great deal about the dynamics of staff setting, and then achieving, high expectations for students.  It’s not just an intellectual exercise. I’ve come to believe that it may have more to do with just how much individual teachers believe in themselves.  It wasn’t until the (very talented) staff got beyond doing great individually and began to work in an interdependent fashion that they were able to actualize some of the more ambitious goals.  Being in each other’s classroom became routine and it became necessary. Our faculty named the approach “Learning to Learn”, a name that meant something powerful to us and which eventually came to permeate every aspect of the school.  The challenge of figuring out the graduation requirements by exhibition ultimately became easier as we found that these simple constructs made it easier for teachers to collaborate in order to help students develop authentic evidence of learning.

Over time the things that had absorbed big chunks of meeting time (planning events, prioritizing this or that, programming and operational decisions, etc.) began to fade away and were handled “off-line.”  The really important work – the work that HAD to be done together—took their place: collaborative problem solving, examining student work, committee work on revising rubrics, learning literacy strategies, etc.  Time was precious and could not be wasted.  Creative scheduling ensured that teams of teachers had planning periods at a common time and staff grew skilled in using their time effectively.

It took those five years before we felt that we finally had a reasonable answer to The Question, and the assurance that it led to authentic students learning.  And in the process we all became better educators than we probably ever imagined we might become.

So, go ahead and try it.  Ask yourselves what your students need to know and be able to do to succeed in your school.  Be prepared for some very difficult conversations.  If you are able to stick with it, things will never quite be the same again.

Alan Dichter

2010

Alan Dichter was Principal of   Satellite Academy in New York, a long-time  CES member school, followed by positions as Director of New School Development, Director of the Executive Leadership Academy, Deputy Superintendent for New Schools and  Leadership Development. He has  also served as Co-Director of National School Reform Faculty at New York  University.  He retired from the NY DOE in 2006.

Ventures at Fenway HS

CNN’s recent feature on Fenway High School’s award-winning Ventures Program prompted us to track down its Director, Amy Carrier. We wanted to know how this program continues to engage Boston’s diverse professional community in setting real-world standards for Fenway students. Amy was interviewed by Craig Levis, ERC associate.

CL- Amy, I understand that you and Fenway Ventures were recently featured on CNN. First of all, congratulations, and secondly, why do you surmise they were interested in covering something like Ventures?

AC- Thanks for asking me to comment.  I think the producers of Chalk Talk, the program that featured my interview, are looking for examples of what’s new and different, programs that are really “working” in our public schools.  The Ventures program is an example of a best practice in education that has been shown to make a difference in preparing students for success in a 21st century workplace.

CL- How would you describe the Ventures program at Fenway, how long has it been in the school, and why do you think it has been successful with and for students?

AC- Fenway was forward-thinking in this arena when it began a version of the program over a dozen years ago.  Students partnered with external organizations to do business planning and have a real-world example of how their creative thinking could make a difference to an existing institution.  Since that time, the program has grown and changed – responding and flexing to the trends of an ever-changing business world.  Today, 120 students each year are required to take the course in eleventh and twelfth grades where they learn entrepreneurship, write business plans, explore careers, practice professionalism skills and learn financial literacy lessons ranging from savings and interest to car loans and insurance.

I’ve seen the far-reaching success as my students become more confident presenters in exhibitions in school and as they develop the kind of savvy it takes to communicate with local professionals about their own areas of career interest when they interview for their six-week internships - the capstone of the program.  I’ve also seen the long-term impact of this curriculum as my students graduate, move on to college and report back that a professor was impressed with how much they were able to contribute to a business course, or how their ability to organize their goals and communicate with business people sealed the deal on a highly competitive co-op or a coveted job in the university president’s office.

CL- What have you learned from and about the expectations of the people and institutions that host Fenway students in interns?

AC- First, let me say that year after year, internship supervisors and mentors that come into the classroom and report over and over just how impressed they are with Fenway’s students who are poised, confident and professional, surprising and exciting these folks all at once.  I host a great event in May, the culmination of all 70 Fenway graduating senior internships.  Our workplace mentors go on stage to speak about their experiences, and they talk about their pleasant surprise with the skill level of Fenway’s interns and about a level of maturity and responsibility that often exceeds that of their college interns!  The mentors who work so closely with Fenway’s students express that students must be prepared to think on their feet, have the confidence to speak up and take charge – all necessities in a fast-paced work environment.

Of course, there are challenges, but because Fenway’s students enter their internships with a year of preparation in our Ventures classrooms, those challenges are more easily addressed and provide another “teachable moment” for a student who deserves to learn an important lesson while still being supported, and better prepared for the world of work – not once he’s out in the world with no one to guide him, or worse, without the care and concern of a teacher or mentor who wants him to learn and grow.

CL- What are the important things a Fenway student learns in the course of his/her Ventures sequence?

AC- There are so many important experiences.  The financial literacy lessons provide a foundation from which a young person can start out in life with knowledge of just how smart choices or mistakes will impact the future.  The constant practice of “SBE” – standard business English and etiquette-- in the classroom, along with practice in stand-up presentations, allows students to feel comfortable entering and “fitting into” the work environment.  Beyond these, the time spent exploring, trying on (through job shadows and interviews) and thinking about potential careers is a very important step in preparing our students for successful futures.  We simply can’t expect our young people to figure these things out by trial and error once they have graduated from high school.  In the end, no matter what path my students choose, they have built a foundation of skills and practice that graduates them into the competitive 21st century world of work having a least considered, observed and reflected upon just what that means.

CL- How has the Ventures program evolved since you’ve been involved with it? And have you heard from other sites interested in Ventures?

AC- I have heard from other schools, teachers and even politicians who want this kind of program in their own schools.  Every adult I’ve ever described the program to says – without fail – that they wish they had this program when they were in high school (and I’m one of those adults!).  Most people see it as a no-brainer and then they want to figure out how to make it happen.

And in terms of how the program has evolved since I’ve been at Fenway – let me start by saying that one of the things I love most about directing and teaching the Ventures program is the flexibility I have with curriculum and the topics I cover in class.  The world, the economy, workplace trends – they all change so rapidly that the curriculum must adapt to meet those changes.  Every year there are new hot-button issues.  In 2006, I talked about Enron but today, my students wouldn’t know what Enron was.  Unfortunately, there is no shortage of examples in our society that become teaching moments for students.  Last year, I taught about the foreclosure crisis which affected students sitting in my classroom.  This year, I used the example of a Kim Kardashian credit card which has consumer advocate groups up in arms.  Students get excited when they hear terms or stories that are familiar – it makes the curriculum engaging.  When I teach entrepreneurship, I empower the students to identify and solve a problem or need that they see in their lives and then write a business plan around it.  The ownership they take of their own ideas is something no one can take away.  When the idea is their own – when the topic of a lesson on interest rates reminds them of a television jingle or gets them to tune in because their neighbor’s home is boarded up due to foreclosure – kids want to learn.  I just love that I get to teach what kids are already thinking about and then empower them to make a difference themselves.

CL- Final question, Amy, what would you say to other schools, perhaps not as innovative or high-performing as Fenway right now, about how they might benefit or learn from including a program like Ventures?

AC- There is no question in my mind that including a program or class like Ventures will benefit every single student who takes it.  I don’t necessarily believe that everyone who works in a school would even tie this kind of learning to innovation or performance.  Certainly there are priorities that must be set and goals that must be achieved in our public schools – but at a very basic level, we all know that we are trying to graduate better citizens.  When we think about our students as citizens of our own society, we cannot avoid thinking about the basic skills that you and I must have – as citizens in a society – to be responsible and successful.  If an educator has even considered teaching anything that I teach in the Ventures program – or, I would argue, if they have even read this far in our interview, I would say that they have what it takes to make a small change now – and maybe a larger change later – to include some of all of this kind of curriculum.  I think it’s our responsibility to our students and while it’s not a walk in the park, it is just the right thing to do.  At Fenway, we teach our students how to do the right thing.  I think that providing all students with the experience of an internship in a local business, teaching all students how to speak for themselves, perform well at an interview, avoid mistakes that will damage their credit and basically become citizens who don’t fall through the cracks, well, *that* is the right thing to do.

Forum ESEA proposal

As leaders and individuals with decades of experience in improving public education at all levels, the Conveners of The Forum for Education and Democracy have viewed the debates over the Elementary and Secondary Education Act (ESEA) as offering the opportunity to finally get it right.  The Forum asserts that, after a decade of tinkering around the edges and avoiding the hard questions, the “No Child Left Behind” version of ESEA did little to improve schools. In fact, they posit, our schools look much like they did when the act was passed – and many think that the attempts to implement NCLB have hindered school improvement efforts going on prior to its passage.

Inspiration, hunger: these are the qualities that drive good schools. The best we educational planners can do is to create the most likely conditions for them to flourish, and then get out of their way.
– Ted Sizer

Read the report: http://www.forumforeducation.org/files/u48/FED_Short_Paper_on_ESEA.pdf

Wayne Ogden Interview

Schools principals and administrators have seldom felt more pressure to improve teaching and learning than in our current environment. Wayne Ogden, former superintendent and principal, is widely known and esteemed for his work on the mentoring of new leaders and his skills and commitment to the practice of instructional leadership. He was interviewed for this edition by Dr. Larry Myatt.

LM- Wayne, “The Skillful Leader”, which you co-authored more than a decade ago, remains an incredibly popular and useful tool for school leaders. Can you say why you think that is the case?

WO-  Larry, my co-authors I have been surprised and pleased that The Skillful Leader: Confronting Mediocre Teaching has remained relevant and helpful. I believe that its success and staying power comes because school principals find it immediately useful in their supervisory efforts by providing a clear focus --to provide students with better instruction.  Interestingly, we predicted that mediocre instruction would persist unless supervisors and teachers jointly understand and pay rapt attention to the connection between their efforts and student learning.  Only in the last few years has this topic actually made it onto the national radar, driven in large part by the “all-kids agenda” and the ossification of so much of the school model.

LM- What’s new in terms of the context of instructional practice? What dilemmas or challenges should we be paying attention to?

WO-  What’s new in some contexts is the challenge of linking our traditional supervisory practices to the demands that student performance data be included in the comprehensive performance appraisal of our teachers and principals. Policy makers, school board members, and school reform advocates all seem to be demanding accountability measures that recognize the direct connection between teacher performance and student learning. But, I do not know that anyone has figured out yet how to understand and manage the many complexities of the process.

LM- So, what are the implications for talent recognition, development, retention, etc.? And can we find time to do those things seriously, including deep mentoring, with the current distractions, budget constraints, and fewer staff?

WO- The implications are huge and made enormously more difficult by our struggling economy. While I’ll happily leave the economic dilemma to the folks in Washington, DC, I do have some thoughts about reshaping how aspiring teachers and principals come through the training and talent pipeline. And, while I do think it will require some open minds as well as open wallets to accomplish some of what I suggest, I don’t think this is rocket science. The folks who long ago decided how best to train our nation’s physicians and plumbers concluded that disciplined academic study needed to be combined with extensive supervised apprenticeship experiences in order to produce capable beginning physicians and plumbers who could successfully “hit the ground running” in their professions. Yet, despite all the other changes in public school education, we continue to think we can prepare the majority of our beginning teachers by grafting on an undergraduate degree a three-month student teaching experience much of which is spent merely watching.

And on the leadership side, a high percentage of aspiring principals can now get their administrative license by adding on some graduate school credits and an administrative apprenticeship that is even shorter and less intense than that of our student teachers. This type of teacher and administrator preparation has never been adequate, and in today’s complex world of public education it is virtually useless.

Aspiring teachers and administrators both need year-long, supervised apprenticeships in places that combine the status and prestige of our nation’s best teaching hospitals. Master teachers and principals must closely monitor and critique the work of these aspiring educators. Learning how to become a great teacher or principal is not part- time work. It should be highly focused--the only thing these aspiring educators are doing-- while they learn their craft. Then, in their first year on the job with our children, they need to be mentored and coached the way we hope that our beginning heart surgeon or master plumber was trained and coached!

Yes, this will be expensive and a much bigger commitment than it ever has been before. Aspiring teachers and principals will need to commit to longer and more costly training. School districts will need to create mentoring and coaching positions to help nurture and perfect their new hires. Universities will have to put the same type of commitment and resources into training our teachers and principals as their importance requires. Most of their training programs will need to be totally redesigned. Our states will have to figure out ways to compensate teachers and principals at levels that reflect the length and cost of their preparation.

LM- Wayne, thanks so much for your thoughts here. To conclude, will you comment on where you do and/or don’t see value added in some of the things we’re doing in leadership development, especially in the principal training and licensure arena, and district-based programs?

WO- Over the last few years, most State Departments of Education have tried to make it easier for people to become teachers and principals. Provisional educator licenses have been granted to adults who have a college degree and can pass some kind of state exam. These teacher and principal licensure requirements generally include no serious apprenticeship or formal training in teaching or in leading. That is simply nuts! At the same time, in an effort to meet the needs for training more new teachers and administrators, to replace the retiring baby boomers, State DOE’s have approved what I call “licensure light” programs that thrive on quick and simple training regimens that meet the barest minimum of the skills and knowledge necessary to be competent teachers and principals.

In-district leadership programs continue to appear and disappear as they have for three decades.  They use a master set of recognizable readings and exercises but, as you and I have repeatedly seen in our work together, without excellent facilitation, outside provocation and critical friendship, and a willingness to challenge in-district orthodoxy, norms, and “coziness”, these programs routinely fall short of generating outstanding leaders.

Thanks for inviting me to comment and I look forward to being part of ERC initiative to restore faith in front-line educators to lead the change in our schools!