BILL ANALYSIS
AB 1130
Page 1
ASSEMBLY THIRD READING
AB 1130 (Solorio)
As Amended April 22, 2009
Majority vote
EDUCATION 11-0
-----------------------------------------------------------------
|Ayes:|Brownley, Nestande, | | |
| |Ammiano, Arambula, | | |
| |Buchanan, Carter, Eng, | | |
| |Garrick, Miller, Solorio, | | |
| |Torlakson | | |
| | | | |
-----------------------------------------------------------------
SUMMARY : States legislative intent regarding the examination of
methods for making and reporting comparisons of school and
district academic achievement over time based on a cohort growth
measure. Specifically, this bill :
1)Makes findings and declarations regarding California's
accountability system and the benefits of incorporating a
cohort growth measure into that system.
2)States legislative intent that the advisory committee advising
the Superintendent of Public Instruction (SPI) on matters
related to the academic performance index (API), to make
recommendations to the SPI and the State Board of Education
(SBE) concerning establishment of a methodology for measuring
academic achievement by cohort to more accurately measure
academic growth for schools and districts by providing the
ability to determine both achievement and growth toward
proficiency.
3)States the intent of the Legislature that the advisory
committee take into consideration the pilot study conducted
pursuant to provision 10 of Item 6110-113-0890 of Section 2.00
of the Budget Act of 2007, federal statute and regulation
associated with the Elementary and Secondary Education Act
(the current reauthorization of which is known as NCLB)
accountability waivers granted by the United States Secretary
of Education, and measures in use in other states that reflect
student, subgroup, school and district growth.
AB 1130
Page 2
4)Requires that any measure of academic growth must be in the
public domain and meet specified statistical standards, if it
is implemented by the SPI after being approved by the SBE,
adopted by the state educational agency for the purposes of
federal education programs (i.e., the SBE) as part of any plan
or waiver request submitted to the federal government under
the Elementary and Secondary Education Act (currently NCLB),
or adopted by the state as part of any other plan required for
receipt or allocation of federal funds.
EXISTING LAW :
1)Requires the SPI, with the approval of the SBE, to develop and
implement the API to measure the performance of schools, and
to include a variety of indicators, including achievement test
results, attendance rates, and graduation rates in that
measure, and requires the SPI to establish an advisory
committee to provide advice on all appropriate matters
relative to the creation of the API.
2)Directs the advisory committee by July 1, 2005, to make
recommendations to the SPI on the appropriateness and
feasibility of a methodology for generating a measurement of
academic performance by using unique pupil identifiers and
annual academic achievement growth to provide a more accurate
measure of a school's growth over time.
FISCAL EFFECT : Unknown
COMMENTS : The SPI established, pursuant to SB 1 X1 (Alpert),
Chapter 3, Statutes of 1999-2000 First Extraordinary Session, an
advisory committee to advise the SPI and the SBE on all
appropriate matters relative to the creation of the API. SB 1
X1 also requires the SPI, with the approval of the SBE, to
develop the API to measure the performance of schools and
districts. Currently only achievement test results are
incorporated into the API, and the API is configured to produce
scores measuring a school's performance at each grade level and
content area at one point in time. In addition, the SPI also
produces a "Growth API" that compares this static performance
from one year to the next. This growth API, however, does not
measure growth for a specific group of students and is not based
on information for individual pupils; in other words that
measure may only be reflecting the differences in two cohorts of
AB 1130
Page 3
pupils who were in one grade level in two different years,
rather than actual growth for a fixed set of students over time.
There is a broad spectrum of methodologies that could be
employed to either eliminate or work around this problem. On
one end of that spectrum might be a full vertical scaling
effort, which would allow a student's growth to be tracked as
the student moves up the score scale that runs from the lowest
grade level up through the highest scores at the highest grade
level and which would reflect a progression through the content.
Since the API is an aggregation of STAR test scores, vertical
scaling of the test scores would eliminate most of the problems
associated with using the API to compare school and district
performance across time. At the other end of the spectrum might
be approaches that rely on statistical procedures to estimate or
project what score, on the average, should be achieved in a
given year based on the previous year's score or other
information. In this way a student's or school's actual score
can be compared to the projected score, and a judgment could be
made about whether the student or school grew at a greater or
lesser rate than the average. There are many other approaches
and methodologies that could be employed to allow comparisons
over time. The trade-off among these procedures is generally
between the increased validity and accuracy of the results, and
the cost and time involved in implementing that approach. At
the two ends of the spectrum, a vertical scaling process would
be the most involved of the approaches, while direct statistical
mediations would be less costly and faster; on the other hand
statistical mediation does not solve the underlying problems and
would suffer from greater issues with validity.
This bill proposes to state legislative intent to focus the
advisory committee on cohort growth, or the growth of aggregate
scores for a group of pupils across grade levels. According to
the bill's sponsor, this approach would produce estimates or
projected aggregate scores that would be used to determine
whether actual aggregate growth was occurring at, above, or
below some desired trajectory. In other words this bill appears
to propose a direct statistical mediation of the aggregate
accountability (API) measure based on a cohort analysis. It is
not clear whether this approach would be used to generate
individual pupil scores that would be comparable over time.
There are two concerns with this bill. The first concern stems
AB 1130
Page 4
from the lack of any specified timeline. Since there are no
deadlines, time references or time-related conditions that bear
on the delivery of the recommendations from the advisory
committee nor on the implementation of those recommendations by
the SPI, the SBE, or the "state" - all of which are parties that
are seen by this bill as being authorized to trigger
implementation of the recommendations - there is no guarantee
that the recommendations are made by the advisory committee or
that those recommendations, if made, are implemented. The
second concern is also associated with implementation of the
recommendations. The bill does not require the advisory
committee recommendations to be implemented, but does refer to
the recommendations being implemented in any one of three ways:
a) by the SPI after being approved by the SBE (current law); b)
by the state educational agency designated for the purposes of
federal education programs (i.e., the SBE) as part of any plan
or waiver request submitted to the federal government under NCLB
or subsequent legislation replacing it; or, c) by the state as
part of any other plan required for receipt or allocation of
federal funds. This ambiguity in specifying how and by whom the
changes envisioned in this bill would be implemented, does not
match the importance or the magnitude of these changes. Making a
change in how we measure progress of both students and schools
potentially has significant impacts on individual students,
schools and school districts in terms both the state and the
federal accountability system, as well as in overall school
reform; a change of this significance should have the
involvement of the Legislature and the Governor.
Provision 10 of Item 6110-113-0890 of section 2.00 of the Budget
Act of 2007 required a study of academic growth measures to
evaluate multiple approaches for measuring individual pupil
annual growth on the state standards. The study examined five
approaches to measuring growth, including vertical scaling and
different statistical mediations. The study recommended that
the state proceed with a regression based approach, consider the
development of vertical scales, and not pursue certain specific
statistical approaches; the study also provided caveats about
the problems involved in these approaches, the possibility of
misunderstanding or misinterpretation of the resulting
comparisons, and the unintended consequences that could occur
with the release of growth information to students and parents.
Problems with misuse and misinterpretation, as well as
unintended consequences, present serious threats to the validity
AB 1130
Page 5
of any approach used to produce measures of student or aggregate
achievement. This bill requires that the results of any adopted
or implemented growth model be in the public domain, be
replicable, and meet specified statistical standards related to
the accuracy (i.e., reliability) of the measure; the bill does
not have a similar standard with respect to the validity of the
measure or its uses.
Related and previous legislation: This bill is one of four bills
that propose changes to the state's accountability system,
specifically to the API measure, and are pending in the
Assembly. Those four bills are AB 173 (Price), AB 429
(Brownley), AB 1130 (Solorio), and AB 1435 (V. Manuel Perez).
AB 2776 (Mullin), held in the Senate Appropriations Committee in
2008, would have required examination of the collection of
individual student data, the state's emerging data systems, the
possibility of making real comparisons of student performance
over time, and the long-term availability of assessment data
related to the acquisition of English language by English
learners with respect to making potential improvements in the
API. SB 219 (Steinberg), Chapter 731, Statutes of 2007, makes
changes in the calculation of and in the process for revising
the API. SB 257 (Alpert), Chapter 782, Statutes of 2003,
requires the advisory committee established to advise the SPI on
the API to make recommendations to the SPI on a methodology for
generating a "gain" score measurement to provide more accurate
measure of a school's growth over time. SB 1 X1 (Alpert),
Chapter 3, Statutes of 1999-2000 First Extraordinary Session,
known as the Public Schools Accountability Act (PSAA),
authorizes the state's current accountability program, including
establishment of the PSAA Advisory Committee and development of
the API.
Analysis Prepared by : Gerald Shelton / ED. / (916)
319-2087FN: 0000530