BILL ANALYSIS
AB 1130
Page 1
CONCURRENCE IN SENATE AMENDMENTS
AB 1130 (Solorio)
As Amended September 8, 2009
Majority vote
-----------------------------------------------------------------
|ASSEMBLY: |74-0 |(May 14, 2009) |SENATE: | |(September 11, |
| | | | | |2009) |
-----------------------------------------------------------------
(vote not available)
Original Committee Reference: ED.
SUMMARY : States legislative intent regarding the examination of
methods for making and reporting comparisons of school and
district academic achievement over time based on a cohort growth
measure. Specifically, this bill :
1)Makes findings and declarations regarding California's
accountability system and the benefits of incorporating a
cohort growth measure into that system.
2)States legislative intent that the advisory committee advising
the Superintendent of Public Instruction (SPI) on matters
related to the academic performance index (API):
a) Make recommendations to the SPI and the State Board of
Education (SBE) concerning establishment of a methodology
for measuring academic achievement by cohort to more
accurately measure academic growth for schools and
districts by providing the ability to determine both
achievement and growth toward proficiency; and,
b) Take into consideration the pilot study conducted
pursuant to provision 10 of Item 6110-113-0890 of Section
2.00 of the Budget Act of 2007, federal statute and
regulation associated with the Elementary and Secondary
Education Act (the current reauthorization of which is
known as NCLB) accountability waivers granted by the United
States Secretary of Education, and measures in use in other
states that reflect student, subgroup, school and district
growth.
3)Requires that any measure of academic growth must be in the
public domain and meet specified statistical standards, if it
is implemented by the SPI after being approved by the SBE,
AB 1130
Page 2
adopted by the state educational agency (i.e., the SBE) for
the purposes of federal education programs as part of any plan
or waiver request submitted to the federal government under
the Elementary and Secondary Education Act (currently NCLB),
or adopted by the state as part of any other plan required for
receipt or allocation of federal funds.
4)Requests that the advisory committee also notify the
chairpersons of the Committees on Education and on
Appropriations in both houses, when making any notification
required under the Bagley-Keene Open Meeting Act.
The Senate amendments add a legislative request that the
advisory committee also notify the chairpersons of the
Committees on Education and on Appropriations in both houses,
when making any notifications required under the Bagley-Keene
Open Meeting Act.
EXISTING LAW :
1)Requires the SPI, with the approval of the SBE, to develop and
implement the API to measure the performance of schools, and
to include a variety of indicators, including achievement test
results, attendance rates, and graduation rates in that
measure, and requires the SPI to establish an advisory
committee to provide advice on all appropriate matters
relative to the creation of the API.
2)Directs the advisory committee by July 1, 2005, to make
recommendations to the SPI on the appropriateness and
feasibility of a methodology for generating a measurement of
academic performance by using unique pupil identifiers and
annual academic achievement growth to provide a more accurate
measure of a school's growth over time.
AS PASSED BY THE ASSEMBLY , this bill was substantially similar
to the version passed by the Senate.
FISCAL EFFECT : This bill is keyed non-fiscal.
COMMENTS : The SPI established, pursuant to SB 1 X1 (Alpert),
Chapter 3, Statutes of 1999-2000 First Extraordinary Session, an
AB 1130
Page 3
advisory committee to advise the SPI and the SBE on all
appropriate matters relative to the creation of the API. SB 1
X1 also requires the SPI, with the approval of the SBE, to
develop the API to measure the performance of schools and
districts. Currently only achievement test results are
incorporated into the API, and the API is configured to produce
scores measuring a school's performance at each grade level and
content area at one point in time. In addition, the SPI also
produces a "Growth API" that compares this static performance
from one year to the next. This growth API, however, does not
measure growth for a specific group of students and is not based
on information for individual pupils; in other words that
measure may only be reflecting the differences in two cohorts of
pupils who were in one grade level in two different years,
rather than actual growth for a fixed set of students over time.
There is a broad spectrum of methodologies that could be
employed to either eliminate or work around this problem. On
one end of that spectrum might be a full vertical scaling
effort, which would allow a student's growth to be tracked as
the student moves up the score scale that runs from the lowest
grade level up through the highest scores at the highest grade
level and which would reflect a progression through the content.
Since the API is an aggregation of STAR test scores, vertical
scaling of the test scores would eliminate most of the problems
associated with using the API to compare school and district
performance across time. At the other end of the spectrum might
be approaches that rely on statistical procedures to estimate or
project what score, on the average, should be achieved in a
given year based on the previous year's score or other
information. In this way a student's or school's actual score
can be compared to the projected score, and a judgment could be
made about whether the student or school grew at a greater or
lesser rate than the average. There are many other approaches
and methodologies that could be employed to allow comparisons
over time. The trade-off among these procedures is generally
between the increased validity and accuracy of the results, and
the cost and time involved in implementing that approach. At
the two ends of the spectrum, a vertical scaling process would
be the most involved of the approaches, while direct statistical
mediations would be less costly and faster; on the other hand
statistical mediation does not solve the underlying problems and
would suffer from greater issues with validity.
This bill proposes to state legislative intent to focus the
AB 1130
Page 4
advisory committee on cohort growth, or the growth of aggregate
scores for a group of pupils across grade levels. According to
the bill's sponsor, this approach would produce estimates or
projected aggregate scores that would be used to determine
whether actual aggregate growth was occurring at, above, or
below some desired trajectory. In other words this bill appears
to propose a direct statistical mediation of the aggregate
accountability measure based on a cohort analysis. It is not
clear whether this approach would be used to generate individual
pupil scores that would be comparable over time.
Provision 10 of Item 6110-113-0890 of section 2.00 of the Budget
Act of 2007 required a study of academic growth measures to
evaluate multiple approaches for measuring individual pupil
annual growth on the state standards. The study examined five
approaches to measuring growth, including vertical scaling and
different statistical mediations. The study recommended that
the state proceed with a regression based approach, consider the
development of vertical scales, and not pursue certain specific
statistical approaches; the study also provided caveats about
the problems involved in these approaches, the possibility of
misunderstanding or misinterpretation of the resulting
comparisons, and the unintended consequences that could occur
with the release of growth information to students and parents.
Problems with misuse and misinterpretation, as well as
unintended consequences, present serious threats to the validity
of any approach used to produce measures of student or aggregate
achievement. This bill requires that the results of any adopted
or implemented growth model be in the public domain, be
replicable, and meet specified statistical standards related to
the accuracy (i.e., reliability) of the measure; the bill does
not have a similar standard with respect to the validity of the
measure or its uses.
Related legislation: AB 429 (Brownley), pending in the Senate,
can be viewed as a companion bill that requires examination of
methods for making and reporting valid comparisons of individual
academic performance over time and for making potential
improvements in the Academic Performance Index, so as to be able
to measure and report both a student's and a school's academic
growth.
Analysis Prepared by : Gerald Shelton / ED. / (916) 319-2087
AB 1130
Page 5
FN: 0003123