California Middle Grades Study Will Examine College Preparation And Much More

Last blog featured an ACT study that found a profound impact of middle grade education upon college preparation and student success in college. Edsource in California is conducting a large study of middle grades schools as described below. I am part of the research team along with study director Trish Williams of Edsource and Stanford Professor Ed Haertel
Research Questions and Purpose
• What school-wide middle grades practices and policies exist in California schools?
• How do these practices and policies differ between K–8, 7–8, and 6–8 school configurations?
• How do they differ between schools serving low income versus middle income students?
• What district/school-wide and subject area practices and policies appear to differentiate higher from lower performing middle grades serving similar student populations?

One strength of this study is that it will survey schools with differing grade configurations at the middle grades level, as well as schools serving primarily low income families as well as those serving primarily middle income families. The study will be large scale, gathering survey responses from hundreds of principals and thousands of teachers in hundreds of different school districts across California. In addition to large total numbers, we will have survey responses from the district to the principal to the classroom teacher (ELA and math). Survey responses from teachers within a school, between teachers and the principal, and between the school and the district can be examined. Another strength is that where possible the survey items have response scales that will measure intensity. There are, however, many limitations to this kind of study. Although the analysis may find strong correlations between some practices and student outcomes on California’s Standards Tests, it is not a best practices or case study. Also, because the survey will cover a broad range of middle grades approaches and practices, it won’t delve deeply and narrowly into any one particular practice area.

We hope the study will generate interest in improving middle grades instruction and will help inform discussion among educators and policymakers who are working to improve the educational attainment of middle grades students. We also hope the study reveals areas worthy of more in-depth examination.

Research Sample and Methodology
The research team will survey K-8, 6-8, and 7-8 schools in two different bands of the School’s Characteristics Index: the 20th-35th percentile band (schools serving predominantly students from low income families) and the 70-85th percentile band (schools serving predominantly students from middle income families). The two samples combined include 525 schools housed in over 300 different districts across California. Over two dozen of these schools are charters. We are soliciting the participation of all the schools within these two samples with a goal of getting 50% or more of schools in both samples to agree to complete and return the principal, teacher, and district surveys.

We will ask every participating school to return a completed principal survey. We also ask that a minimum of 60% of each school’s English Language Arts and Math teachers return completed surveys. A small number of questions about English learner instruction will be folded into all surveys. Later in the spring, district superintendents (or CEOs of charter management organizations) of participating schools will also be asked to complete a short survey.

EdSource will utilize a survey administration process that ensures that schools can keep survey responses confidential from other staff at the school or district. In addition, EdSource will ensure that at no time will the names of participating individuals or the names of participating districts and schools be released publicly.

This study will focus on concrete practices, policies, and actions at the school level, but it will also gather information about district and classroom variables. The research team has conducted a review of available literature on middle grades issues, as well as of reports issued by various California and national organizations, and model programs, articulating various effective middle grades approaches. The survey questions are framed by this literature, but have been further operationalized to be relevant to the current standards-based K–12 education policy context and expectations in California.

The middle grades conceptual domains framing the surveys, and the survey questions themselves, have been reviewed by consultants and volunteer advisors: middle grades knowledgeable researchers (both national and in California), K–12 educators, and state policymakers.

In addition to surveying superintendents, principals, and ELA and math teachers at the middle grades level, the research team may follow-up the surveys with a more limited number of interviews for the primary purpose of adding clarity, rich context, or deeper understanding to the survey results.

Plan of Analysis
By conducting a large-scale survey of schools across California serving 7th and 8th graders, the research team seeks to document the variety of middle grades approaches, practices, and policies, and the status and intensity of their implementation, currently in place in the state.
Further, we will use regression and other analyses, and other statistical methods, to identify middle grades practices and policies that are most strongly correlated with stronger student performance (overall, and for student subgroups) on the California Standards Tests after controlling for demographic and other student differences. We may also look at other middle grades student outcomes depending upon the availability of the data. The surveys and sampe have been designed to enable various sub-analyses if they appear fruitful. If useful longitudinal student data is available we will develop a plan for its use.

Survey data responses will be keyed in by hand by research and evaluation staff at WestEd (under contract with EdSource), who will create a data file for the research team. The data file of the survey responses will be merged with student outcome, school performance and other data from the California Department of Education.

The research team will construct composite school-level variables to capture critical aspects of schools' policies and practices; and the team will examine correlations of both individual survey items and composites with a school's student achievement on the various middle grades California Standards Tests (mean scale scores) and other measures. Next steps will involve multivariate modeling to identify broad factors predicting school success, for middle grades schools in general and also for important subsets of schools (e.g., schools with different grade configurations, charter schools, schools serving students from differing socio-economic backgrounds, and others).
The full analysis plan will be developed, with input from a variety of technical and other research advisors, in the spring of 2009 as the completed surveys are being retrieved.
Financial Support for the Study
Reed Hastings, CEO of Netflix, is funding this large-scale survey of California middle grades schools.

Preliminary Study Timeline 2008–10
September–October 2008–09
Organize core research team, recruit consultants and advisors, review middle grades research and reform reports, develop survey framework.
November – January 2008-09
Send Survey Framework to consultants and advisors for review and input; develop survey items and get practitioner input on questions for ELA, Math, and ELD instruction; finalize and format survey; contact districts and schools to request participation in study.
January–March 2009
Confirm participating schools and districts; print, mail, and retrieve school-level teacher and principal surveys; input responses into data file. Develop more detailed analysis plan based upon survey composites and items and student outcome data available from the state. Start survey for superintendents.
April–June 2009
Finalize superintendent survey, mail and retrieve.
Begin analysis of data with input from technical advisors.
July–October 2009
Continue analysis. Convene stakeholder and consultants advisory group for feedback on initial findings. Begin to draft sections of research report and appendices.
Early November 2009
Report of study’s initial general findings released to the media, policymakers, and general public; also mailed to all participating schools and districts.
January–May 2010