BILL ANALYSIS
AB 429
Page 1
ASSEMBLY THIRD READING
AB 429 (Brownley)
As Amended April 29, 2009
Majority vote
EDUCATION 8-0 APPROPRIATIONS 17-0
------------------------------------------------------------------
|Ayes:|Brownley, Nestande, |Ayes:|De Leon, Nielsen, Ammiano, |
| |Ammiano, Arambula, | | |
| |Carter, Eng, Miller, | |Charles Calderon, Davis, |
| |Solorio | |Duvall, Fuentes, Hall, |
| | | |Harkey, Miller, |
| | | |John A. Perez, Price, |
| | | |Skinner, Solorio, Audra |
| | | |Strickland, Torlakson, |
| | | |Krekorian |
|-----+--------------------------+-----+---------------------------|
| | | | |
------------------------------------------------------------------
SUMMARY : Requires examination of methods for making and
reporting valid comparisons of individual academic performance
over time and for making potential improvements in the Academic
Performance Index (API), so as to be able to measure and report
both a student's and a school's academic growth over time.
Specifically, this bill :
1)Requires the advisory committee advising the Superintendent of
Public Instruction (SPI) on matters related to the API, to
make recommendations to the SPI by July 1, 2011, concerning
the establishment of a methodology for making the state's
assessment system longitudinally valid, and for measuring
academic growth more accurately and validly over time for
individual students and for schools; also requires the
advisory committee to use the pilot study conducted pursuant
to provision 10 of Item 6110-113-0890 of Section 2.00 of the
Budget Act of 2007 in making its recommendations.
2)Requires the SPI to forward the committee's recommendations,
along with cost estimates and a timeline for implementation of
each recommendation, to the State Board of Education (SBE),
the appropriate policy and fiscal committees of the
Legislature, and the Department of Finance by October 1, 2011.
AB 429
Page 2
3)Prohibits these recommendations or any other proposal to
develop longitudinally valid measures from being implemented
until funds are appropriated by the Legislature specifically
for that purpose.
EXISTING LAW :
1)Requires the SPI, with the approval of the SBE, to develop and
implement the API to measure the performance of schools, and
to include a variety of indicators, including achievement test
results, attendance rates, and graduation rates in that
measure, and requires the SPI to establish an advisory
committee to provide advice on all appropriate matters
relative to the creation of the API.
2)Directs the advisory committee by July 1, 2005, to make
recommendations to the SPI on the appropriateness and
feasibility of a methodology for generating a measurement of
academic performance by using unique pupil identifiers and
annual academic achievement growth to provide a more accurate
measure of a school's growth over time.
FISCAL EFFECT : According to the Assembly Appropriations
Committee, General Fund Proposition 98 cost pressure, likely in
the tens to hundreds of millions of dollars, to the California
Department of Education (CDE) to modify the state's assessment
system and the API to measure student growth over time. Actual
costs will be determined by the specific recommendations made by
the advisory committee. For example, if it chooses to
statistically modify the state's existing assessment system,
costs will be less. However, if the advisory committee chooses
to design new assessments, the costs will be significantly more.
This measure prohibits any proposal to develop a longitudinally
valid assessment system from being implemented unless funds are
appropriated for this purpose.
COMMENTS : The SPI established, pursuant to SB 1 X1 (Alpert),
Chapter 3, Statutes of 1999-2000 First Extraordinary Session, an
advisory committee to advise the SPI and the SBE on all
appropriate matters relative to the creation of the API. SB 1
X1 also requires the SPI, with the approval of the SBE, to
develop the API to measure the performance of schools and
districts. Currently only achievement test results are
incorporated into the API, and the API is configured to produce
AB 429
Page 3
scores measuring a school's performance at each grade level and
content area at one point in time. In addition the SPI also
produces a "Growth API" that compares this static performance
from one year to the next. This growth API, however, does not
measure growth for a specific group of students and is not based
on information for individual pupils; in other words that
measure may only be reflecting the differences in two cohorts of
pupils who were in one grade level in two different years,
rather than actual growth for a fixed set of students over time.
There is a broad spectrum of methodologies that could be
employed to either eliminate or work around this problem. On
one end of that spectrum might be a full vertical scaling
effort, which would allow a student's growth to be tracked as
the student moves up the score scale that runs from the lowest
grade level up through the highest scores at the highest grade
level and which would reflect a progression through the content.
Since the API is an aggregation of STAR test scores, vertical
scaling of the test scores would eliminate most of the problems
associated with using the API to compare school and district
performance across time. At the other end of the spectrum might
be approaches that rely on statistical procedures to estimate or
project what score, on the average, should be achieved in a
given year based on the previous year's score or other
information. In this way a student's or school's actual score
can be compared to the projected score, and a judgment could be
made about whether the student or school grew at a greater or
lesser rate than the average. There are many other approaches
and methodologies that could be employed to allow comparisons
over time. The trade-off among these procedures is generally
between the increased validity and accuracy of the results, and
the cost and time involved in implementing that approach. At
the two ends of the spectrum, a vertical scaling process would
be the most involved of the approaches, while direct statistical
mediations would be less costly and faster; on the other hand
statistical mediation does not solve the underlying problems and
would suffer from greater issues with validity.
This bill does not presume that any of these approaches is best
in terms of either maximizing the validity and accuracy of the
comparisons of individual scores or aggregate API measure that
will eventually be compared over time or in terms of minimizing
the costs of producing these comparable measures. Instead this
bill directs the advisory committee, with the expertise to
AB 429
Page 4
balance these goals, to make recommendations on the best course
for the state to proceed; the bill does, however, constrain the
advisory committee by requiring it to solve this lack of
longitudinal comparability for both individual assessment
results and for the state's aggregate accountability measure.
In other words, this bill leads the advisory committee to those
many possible approaches where individual test scores that can
validly be compared over time are developed and used to build up
to an API that is also longitudinally valid. What this approach
rules out is an approach that mediates the aggregate API measure
without allowing the underlying individual test scores to be
compared over time.
This bill also requires the SPI to forward the advisory
committee's recommendations, along with cost estimates and a
timeline for implementation, to the SBE, the appropriate policy
and fiscal committees of the Legislature, and the Department of
Finance; in addition, the bill prohibits these recommendations
or any other proposal to develop longitudinally valid measures
from being implemented until funds are appropriated by the
Legislature specifically for that purpose. Making a change in
how we measure progress of both students and schools potentially
has significant impacts on individual students, schools and
school districts in terms both the state and the federal
accountability system, as well as in overall school reform; a
change of this significance should have the involvement of the
Legislature and the Governor.
Provision 10 of Item 6110-113-0890 of section 2.00 of the Budget
Act of 2007 required a study of academic growth measures to
evaluate multiple approaches for measuring individual pupil
annual growth on the state standards. The study examined five
approaches to measuring growth, including vertical scaling and
different statistical mediations. The study recommended that
the state proceed with a regression based approach, consider the
development of vertical scales, and not pursue certain specific
statistical approaches; the study also provided caveats about
the problems involved in these approaches, the possibility of
misunderstanding or misinterpretation of the resulting
comparisons, and the unintended consequences that could occur
with the release of growth information to students and parents.
Problems with misuse and misinterpretation, as well as
unintended consequences, present serious threats to the validity
of any approach used to produce measures of student or aggregate
AB 429
Page 5
achievement.
Related and previous legislation: This bill is one of four
bills that propose changes to the state's accountability system,
specifically to the API measure, and that have been introduced
in the Assembly in 2009. Those four bills are AB 173 (Price),
AB 429 (Brownley), AB 1130 (Solorio), and AB 1435 (V. Manuel
Perez). AB 2776 (Mullin), held in the Senate Appropriations
Committee in 2008, would have required examination of the
collection of individual student data, the state's emerging data
systems, the possibility of making real comparisons of student
performance over time, and the long-term availability of
assessment data related to the acquisition of English language
by English learners with respect to making potential
improvements in the API. SB 219 (Steinberg), Chapter 731,
Statutes of 2007, made changes in the calculation of and in the
process for revising the API. SB 257 (Alpert), Chapter 782,
Statutes of 2003, required the advisory committee established to
advise the SPI on the API to make recommendations to the SPI on
a methodology for generating a "gain" score measurement to
provide more accurate measure of a school's growth over time.
SB 1 X1 (Alpert), Chapter 3, Statutes of 1999-2000 First
Extraordinary Session, known as the Public Schools
Accountability Act (PSAA), authorized the state's current
accountability program, including establishment of the PSAA
Advisory Committee and development of the API.
Analysis Prepared by : Gerald Shelton / ED. / (916) 319-2087
FN: 0001145