BILL ANALYSIS
SENATE COMMITTEE ON EDUCATION
Gloria Romero, Chair
2009-2010 Regular Session
BILL NO: AB 1130
AUTHOR: Solorio
AMENDED: June 18, 2009
FISCAL COMM: No HEARING DATE: June 25, 2009
URGENCY: No CONSULTANT: James Wilson
SUBJECT : Academic Performance Index (API)
SUMMARY
This bill requires that if an existing advisory committee
to the Superintendent of Public Instruction (SPI) considers
a measure of annual academic achievement growth, then that
measure must not be proprietary, must be replicable and its
results, as well as its statistical features, must be able
to be fully and accurately explained to the public.
BACKGROUND
Current law requires the Superintendent of Public
Instruction (SPI) to establish a broadly representative
committee to advise the SPI and the State Board of
Education (SBE) on the creation of the Academic Performance
Index (API) and make recommendations on the feasibility of
measuring academic performance utilizing unique pupil
identifiers.
The Academic Performance Index (API) was established
pursuant SB 1 X (Alpert, Ch. 3 of 1999). The index was
proposed as a means of combining multiple indicators of
school performance into one easy-to-compare index. A
school's API score is based primarily on the test scores of
pupils on the California Standards Tests of pupils in
grades 2 through 11. The California Standards Tests measure
pupil performance against the statewide standards adopted
by the State Board for each grade level. Although the
standards are considered among the best in the nation, they
were not designed to track pupil performance in specific
skills from one grade to the next. Each grade has a unique
set of standards and the tests for that grade are designed
to measure pupil performance against those standards, but
AB 1130
Page 2
aligned as they are to the standards at each grade level,
the tests are not "vertically aligned" so that a pupil's
performance on any particular set of skills may be tracked
over time. As a result, this system does not allow an
accurate comparison of a pupil's growth or decline in
performance between grade levels.
.
The Department of Education also calculates a "Growth API"
that compares grade level performance from one year to the
next. The Growth API does not measure growth for a
specific group of students and is not based on information
for individual pupils; in other words that measure may only
be reflecting the differences in two cohorts of pupils, for
example last year's third grade class vs. this year's third
grade class. The current state testing system does not
measure the actual growth for the same students over time.
ANALYSIS
This bill:
1) Requires that if the advisory committee on API matters
considers a measure of annual academic achievement
growth or growth by cohort, and adopts such a measure
for any of several specified reasons, it must:
a) Utilize a growth model in the public domain
that is not proprietary.
b) Be able to be replicated by an independent
statistician.
c) Be able to be fully and accurately
explained, including the
generation of all results, the specification of
the standard error, and the stringency of the
confidence interval used to determine whether the
annual change in test scores is statistically
significant, in a document available to the
public.
2) Declares legislative findings that, among other
things:
a) California's school accountability system
compares snapshots of
AB 1130
Page 3
individual school and school district performance
by grade level thereby comparing the difference
in achievement of different cohorts of pupils
from one year to the next.
b) This accountability system fails to adjust
for beginning levels of
achievement, and schools and districts are often
unfairly held accountable for the low performance
of the school the pupils previously attended.
c) Tracking each age group's (cohort) academic
growth over time will
provide better information to identify pupils who
need additional assistance and target resources
to close achievement gaps.
3) Declares legislative intent that, when conducting
their responsibilities, the advisory committee on API
matters consider recommendations of a 2007 Budget Act
required study and federal requirements and guidance
from the federal Department of Education, and waivers
for cohort growth measures approved for other states.
4) Declares legislative intent that the advisory
committee on API matters consider measures already in
use by other states to determine grade level
performance benchmarks that indicate the following
with reasonable statistical confidence:
a) High achievement with a growth rate
indicating ability to remain at
proficiency or to move into the highest range of
achievement.
b) High achievement with a growth rate
indicating ability to remain at
least at proficiency.
c) Low achievement with a growth rate
indicating ability to reach
proficiency within a specified timeframe.
d) Low achievement with a growth rate
indicating significant inability to
reach proficiency within a specified timeframe.
AB 1130
Page 4
5) Requests the advisory committee on API matters, when
making any notice pursuant to the Bagley-Keene Open
Meeting Act, to also notify the chairpersons of the
Committees on Education and Appropriations of any
activities that may be conducted relative to measures
of annual academic achievement growth or growth by
cohort.
STAFF COMMENTS
1) Measuring academic growth . There are several methods
that could be employed to measure achievement growth.
On one end of that spectrum might be a full vertical
scaling effort, which would allow a student's growth
to be tracked as the student moves up the score scale
that runs from the lowest grade level up through the
highest scores at the highest grade level and which
would reflect a progression through the content. Since
the API is an aggregation of STAR test scores,
vertical scaling of the test scores would eliminate
most of the problems associated with using the API to
compare school and district performance across time.
At the other end of the spectrum are approaches that
rely on statistical procedures to estimate what score,
on the average, should be achieved in a given year
based on the previous year's score or other
information. In this way a student's or school's
actual score can be compared to the projected score,
and a judgment could be made about whether the student
or school grew at a greater or lesser rate than the
average. There are many other methodologies that
could be employed to allow comparisons over time. The
trade-off among these procedures is generally between
the increased validity and accuracy of the results,
and the cost and time involved in implementing that
approach. At the two ends of the spectrum, a vertical
scaling process would be the most involved of the
approaches, while direct statistical mediations would
be less costly and faster; on the other hand
statistical mediation does not solve the underlying
problems and could suffer from greater issues with
validity.
2) Intent to use statistical methods . This bill declares
legislative intent to focus the Superintendent's
AB 1130
Page 5
advisory committee on cohort growth, or the growth of
aggregate scores for a group of pupils as they move
through grade levels. This approach would use
estimates or benchmarks for aggregate scores that
would be compared against actual aggregate scores to
determine if actual aggregate growth was occurring at,
above or below an established objective. In other
words this bill appears to propose a direct
statistical mediation of the aggregate accountability
(API) measure based on a cohort analysis. It is not
clear whether this approach could be used to generate
individual pupil scores that would be comparable over
time.
3) Related legislation . AB 429 (Brownley), which is also
being heard today, would require the Superintendent's
advisory committee to make recommendations by July 1,
2011 for development of a longitudinally valid
assessment system in which annual academic growth can
be measured for both schools and pupils over time. The
two bills address the same issues and direct the same
advisory committee, although AB 429 calls for
recommendations while this measure specifies
parameters that the recommendations must meet, without
actually calling for a growth measurement proposal.
SUPPORT
Association of California School Administrators
California School Boards Association
California Teachers Association
EdVoice (Sponsor)
OPPOSITION
None received.