BILL ANALYSIS
AB 429
Page 1
CONCURRENCE IN SENATE AMENDMENTS
AB 429 (Brownley)
As Amended September 4, 2009
Majority vote
-----------------------------------------------------------------
|ASSEMBLY: |79-0 |(June 2, 2009) |SENATE: |35-0 |(September 10, |
| | | | | |2009) |
-----------------------------------------------------------------
Original Committee Reference: ED.
SUMMARY : Requires examination of methods for making and
reporting valid comparisons of individual academic performance
over time, and for making potential improvements in the Academic
Performance Index (API) and state assessment system, so as to be
able to measure and report both a student's and a school's
academic growth over time. Specifically, this bill :
1)Requires the advisory committee advising the Superintendent of
Public Instruction (SPI) on matters related to the API, to
make recommendations to the SPI by January 1, 2011, concerning
the establishment of a methodology for making the state's
assessment system longitudinally valid, and for measuring
academic growth more accurately and validly over time for
individual students and for schools.
2)Requires the advisory committee to consider the pilot study
made pursuant to provision 10 of Item 6110-113-0890 of Section
2.00 of the Budget Act of 2007 in making recommendations.
3)Requires the SPI to forward the committee's recommendations,
immediately upon receipt, to the State Board of Education
(SBE), the appropriate policy and fiscal committees of the
Legislature and the Department of Finance (DOF), and requires
cost estimates and a timeline for the implementation of each
recommendation to be submitted as well.
4)Requires the committee's recommendations be consistent with
any federal guidance under the Elementary and Secondary
Education Act of 1965 (ESEA) and the American Recovery and
Reinvestment Act of 2009 (ARRA), including any guidance issued
for the Race to the Top (RTTT) grant programs.
5)Prohibits these recommendations or any other proposal to
AB 429
Page 2
develop longitudinally valid measures from being implemented
until funds are appropriated by the Legislature specifically
for that purpose.
The Senate amendments :
1)Make technical changes and add coauthors.
2)Change the date by which the advisory committee is required to
make recommendations to the SPI from July 1, 2011, to January
1, 2011.
3)Change the date by which the SPI is required to forward the
committee's recommendations to the SBE, Legislature and DOF
from October 1, 2011, to "immediately upon receipt."
4)Require the recommendations be consistent with any federal
guidance under ESEA and the ARRA, including any guidance
issued for the RTTT grant programs.
EXISTING LAW :
1)Requires the SPI, with the approval of the SBE, to develop and
implement the API to measure the performance of schools, and
to include a variety of indicators, including achievement test
results, attendance rates, and graduation rates in that
measure, and requires the SPI to establish an advisory
committee to provide advice on all appropriate matters
relative to the creation of the API.
2)Directs the advisory committee by July 1, 2005, to make
recommendations to the SPI on the appropriateness and
feasibility of a methodology for generating a measurement of
academic performance by using unique pupil identifiers and
annual academic achievement growth to provide a more accurate
measure of a school's growth over time.
AS PASSED BY THE ASSEMBLY , this bill was substantially similar
to the version passed by the Senate.
FISCAL EFFECT : According to the Senate Appropriations
Committee, $280,000 to $300,000 one-time General Fund (GF) costs
to fully analyze the issues and make recommendations, and
potential GF cost pressures in the millions to implement the
committee's recommendations.
AB 429
Page 3
COMMENTS : The SPI established, pursuant to SB 1 X1 (Alpert),
Chapter 3, Statutes of 1999-2000 First Extraordinary Session, an
advisory committee to advise the SPI and the SBE on all
appropriate matters relative to the creation of the API,
developed the API to measure the performance of schools and
districts. Currently only achievement test results are
incorporated into the API, and the API is configured to produce
scores measuring a school's performance at each grade level and
content area at one point in time. The API neither measures
growth for a specific group of students nor is based on
information for individual pupils; in other words the API only
reflects the differences in two cohorts of pupils who were in
one grade level in two different years or who were in successive
grades, rather than actual growth for a fixed set of students or
individual students over time. At the same time, the state's
testing system has not been designed to produce test scores
(which provide the foundation for the API) that provide a
measure of real academic growth for individual pupils or
schools.
There is a broad spectrum of methodologies that could be
employed to either eliminate or work around this inability to
make comparisons over time. On one end of that spectrum might
be a full vertical scaling effort, which would allow a student's
growth to be tracked as the student moves up the score scale
that runs from the lowest grade level up through the highest
scores at the highest grade level and which would reflect a
progression through the content. Since the API is an
aggregation of STAR test scores, vertical scaling of the test
scores would eliminate most of the problems associated with
using the API to compare school and district performance across
time. At the other end of the spectrum might be approaches that
rely on statistical procedures to estimate or project what
score, on the average, should be achieved in a given year based
on the previous year's score or other information. In this way
a student's or school's actual score can be compared to the
projected score, and a judgment could be made about whether the
student or school grew at a greater or lesser rate than the
average. There are many other approaches and methodologies that
could be employed to allow comparisons over time. The trade-off
among these procedures is generally between the increased
validity and accuracy of the results, and the cost and time
involved in implementing that approach. At the two ends of the
spectrum, a vertical scaling process would be the most involved
AB 429
Page 4
of the approaches, while direct statistical mediations would be
less costly and faster; on the other hand statistical mediation
does not solve the underlying problems and would suffer from
greater validity problems.
This bill does not presume that any of these approaches is best
in terms of either maximizing the validity and accuracy of the
comparisons of individual scores or aggregate API measure that
will eventually be compared over time, or in terms of minimizing
the costs of producing these comparable measures. Instead this
bill directs the advisory committee, with the expertise to
balance these goals, to make recommendations on the best course
for the state to proceed; the bill does, however, constrain the
advisory committee by requiring it to solve this lack of
longitudinal comparability for both individual assessment
results and for the state's aggregate accountability measure.
In other words, this bill leads the advisory committee to those
many possible approaches where individual test scores that can
validly be compared over time are developed and used to build up
to an API that is also longitudinally valid. What this approach
rules out is an approach that mediates the aggregate API measure
without allowing the underlying individual test scores to be
compared over time.
This bill also requires the SPI to forward the advisory
committee's recommendations, immediately upon receipt, along
with cost estimates and a timeline for implementation, to the
SBE, the appropriate policy and fiscal committees of the
Legislature, and the Department of Finance; in addition, the
bill prohibits these recommendations or any other proposal to
develop longitudinally valid measures from being implemented
until funds are appropriated by the Legislature specifically for
that purpose. Making a change in how we measure progress of
both students and schools potentially has significant impacts on
individual students, schools and school districts in terms both
the state and the federal accountability system, as well as in
overall school reform; a change of this significance should have
the involvement of the Legislature and the Governor. This is
especially true given the increased importance that these
changes have taken on with respect to guidance issued by the
U.S. Department of Education regarding the Race to the Top grant
programs under ARRA. For example, the application criteria
under RTTT that calls for states to use pupil performance data
in the evaluation of teachers only makes sense if a state's
testing system is designed to produce scores that clearly
AB 429
Page 5
measure growth in individual pupil performance from year to
year. As long as comparisons of a pupil scores over time are
invalid, any conclusion about whether specific factors (e.g.,
programs or teachers) contributed to growth in a student's
performance will be equally invalid.
Provision 10 of Item 6110-113-0890 of section 2.00 of the Budget
Act of 2007 required a study of academic growth measures to
evaluate multiple approaches for measuring individual pupil
annual growth on the state standards. The study examined five
approaches to measuring growth, including vertical scaling and
different statistical mediations. The study recommended that
the state proceed with a regression based approach, consider the
development of vertical scales, and not pursue certain specific
statistical approaches; the study also provided caveats about
the validity problems involved in these approaches, the
possibility of misunderstanding or misinterpretation of the
resulting comparisons, and the unintended consequences that
could occur with the release of growth information to students
and parents.
Analysis Prepared by : Gerald Shelton / ED. / (916) 319-2087
FN: 0003065