BILL ANALYSIS
SENATE COMMITTEE ON EDUCATION
Gloria Romero, Chair
2009-2010 Regular Session
BILL NO: AB 391
AUTHOR: Torlakson
AMENDED: January 25, 2010
FISCAL COMM: Yes HEARING DATE: June 23, 2010
URGENCY: No CONSULTANT:Lynn Lorber
SUBJECT : Evaluation of the Standardized Testing and
Reporting Program
KEY POLICY ISSUES
Should the Superintendent of Public Instruction contract for
an independent evaluation of the Standardized Testing and
Reporting (STAR) program?
Do the reports submitted annually by the STAR test vendor
provide the type of information the Legislature needs to
deliberate the upcoming reauthorization of the STAR program?
Would an evaluation provide more valuable information?
Should the State Board of Education have a voice in awarding
a contract for this evaluation?
SUMMARY
This bill requires the Superintendent of Public Instruction
to contract for an independent evaluation of the Standardized
Testing and Reporting (STAR) program.
BACKGROUND
The STAR program requires testing of pupils in grades 2
through 11, including English language arts and mathematics
in most grades, and science and history-social science at
specified grade levels. In 2003, the California Standards
Tests (CSTs) replaced a nationally published "off the shelf"
test as the primary battery of STAR tests. The CSTs are
written specifically to test California's content standards.
In February 2009, the federal U.S. Department of Education
issued an invitation to the States to compete for
AB 391
Page 2
approximately $4.4 billion of American Recovery and
Reinvestment Act one-time funding known as Race to the Top
(RTTT) grants. The RTTT grants are to be issued in two
competitive rounds. California was not successful in its
first attempt. As of June 1, California submitted an
application for the second round of funding. In late August
/ early September Phase 2 grant recipients will be announced.
RTTT is a competitive grant program designed to encourage and
reward States that are creating the conditions for education
innovation and reform; achieving significant improvement in
student outcomes, including making substantial gains in
student achievement; closing achievement gaps; improving high
school graduation rates; and ensuring student preparation for
success in college and career; and implementing ambitious
plans in four core education reform areas:
Adopting high quality standards and
assessments to prepare students for higher
education or work.
Recruiting, developing, retaining and
rewarding effective teachers and principals.
Creating data systems to measure student
success and support instruction.
Turning around the lowest performing schools.
SB 1 of the Fifth Extraordinary Session (Steinberg, Ch. 2,
2010), among other things, extended the sunset on the STAR
program from July 1, 2011, to July 1, 2013. SB 1 also stated
legislative intent that the reauthorization of the statewide
pupil assessment program include all of the following:
1) A plan for transitioning to a system of high-quality
assessments, as defined in the federal Race to the Top
guidance and regulations.
2) Alignment with the standards developed by Academic
Content Standards Commission (established in SB 1) to
develop academic content standards in
language arts and math, 85% of which are to be the
common core standards developed by the National
Governors Association and the Council of Chief State
School Officers.
AB 391
Page 3
3) Any common assessments aligned with these standards.
4) Conforms to the assessment requirements of any
reauthorization of the federal Elementary and Secondary
Education Act or any other federal law that effectively
replaces that act. (Education Code 60640 and 60604.5)
Results for STAR tests are reported for the individual pupil,
but no accountability attaches to these individual results;
the state and federal accountability systems are primarily
based on the aggregated STAR test scores from all pupils in a
school or school district. Many elements of the STAR program
are used by California to meet the assessment and
accountability requirements of the federal No Child Left
Behind (NCLB) Act, which requires standards-aligned
achievement testing in reading and mathematics to all pupils
in grades 3-8 and grade 10; and also requires science testing
in grades 5, 8, and 10.
ANALYSIS
This bill requires the Superintendent of Public Instruction
(SPI) to contract for an independent evaluation of the STAR
program. Specifically, this bill:
1) Requires the SPI, by April 1, 2011, to contract with an
independent evaluator, who is to report to the SPI, for
evaluation of the STAR program. The evaluation must be
a meta-analysis of existing information and data from
the STAR program based upon all of the following:
a) Information gathered in field testing and
annual administrations of the STAR assessments.
b) Existing technical reports, peer reviews, and
other studies, reports and evaluations of the STAR
program conducted by or at the request of the
California Department of Education, the Legislature
or the State Board of Education.
c) State and federal requirements.
d) A review of research-based alternative
assessment models.
e) A review of existing and emerging practices in
AB 391
Page 4
large-scale assessment from across the nation.
2) Requires the evaluation to include, but not be limited
to, all of the following:
a) A report on the results of prior analyses
regarding the alignment between the STAR
assessments and the full range of the content
standards, and a determination of whether the STAR
program assesses pupil knowledge in the same manner
and at the same level of complexity as expected in
those content standards.
b) An independent analysis of the grade level
continuity and vertical articulation of the content
standards.
c) An independent analysis of the ability of the
tests to produce scores for an individual pupil
that can be validly compared from year to year for
both groups and individuals.
d) An independent analysis of the use of content
standards in other core curriculum areas for
testing items, as applicable.
e) A report on the results of prior analyses
regarding pupil performance, broken down by
assessment, grade level, race or ethnicity, and
end-of-course assessments, including any trends
that become apparent over time.
f) An independent analysis of the degree to which
the STAR program complies with professional testing
standards and satisfies or exceeds state and
federal requirements for assessments for each grade
level.
g) An independent analysis of the usefulness of
the STAR program in terms of state and local
program evaluations.
h) An independent analysis of the usefulness of
the STAR program in providing individual results,
providing a diagnostic assessment for classroom
use, and providing formative and interim
AB 391
Page 5
assessments in order to better inform instruction
and improve learning.
i) An independent analysis of the feasibility and
cost of the development and administration of a
diagnostic alternative test in grade levels and
content standard areas that are not required to
have an assessment under federal law.
3) Requires the evaluator to submit to the SPI a report
containing the findings of the evaluation and include
recommendations on specified factors, such as improving
the usefulness of the test, integrating content
standards in other curriculum areas, developing and
implementing alternatives to the current test format,
and generating multiple measures of pupil achievement.
4) Requires the Public School Accountability Act Advisory
(PSAA) Committee to advise the SPI on the independent
evaluation by providing all of the following:
a) Recommendations regarding the parameters of
the evaluation.
b) Recommendations regarding any request for
proposals or request for applications used to
solicit contract proposals.
c) Recommendations regarding the selection of the
contractor.
d) A review of any reports submitted by the
independent evaluator, including any midterm
reports as well as the final evaluation.
5) Requires the SPI to appoint four additional members to
the PSAA Committee, who shall be educators or
individuals having expertise in large-scale assessment
and who shall serve only for the purposes of this bill.
6) Requires the SPI to provide the evaluation to the
Legislature, Governor and State Board of Education by
November 1, 2011.
7) Requires the California Department of Education to use
federal funds to contract for the evaluation, and makes
the operation of this bill contingent upon an
AB 391
Page 6
appropriation for this purpose in the annual Budget Act
or another statute.
8) Defines "formative assessment, "high-quality
assessment," and "interim assessment." The definition
of "high-quality assessment" in this bill is identical
to how it is defined in the federal Race to the Top
guidance and regulations.
STAFF COMMENTS
1) Need for the bill : According to the author, "the
overall STAR model has not been reconsidered since 1997,
and there are numerous models of testing, content and
reporting that should be examined to ensure that the
STAR program is most effective for teaching, learning
and providing performance accountability. With
discussions on a federal level regarding national
assessments, and increasing pressure in the classroom
because of focus on STAR test scores, California must
examine this model and its results to maximize our
investment of teacher and student time and tax dollars.
The STAR system is up for reauthorization in 2013, and
the Legislature must be prepared for discussion of
alternative models, particularly considering the
national assessment possibility either through Race to
the Top or the Elementary and Secondary Education Act
reauthorization projected to happen in 2011."
2) What do we need to know about STAR prior to
reauthorization ? The STAR program sunsets on July 1,
2013. This bill suggests that the state needs specific
information, beyond what has been included in the
technical reports provided annually by the test
administrator. These technical reports include
information such as the number and percentage of pupils
tested, the mean scale score, and the number and
percentage of pupils scoring at each performance level.
Do these technical reports provide the type of
information the Legislature needs when deliberating the
reauthorization of the STAR program? Would an
evaluation provide more valuable information?
3) Nature of independence . This bill calls for an
independent evaluation. Staff understands that it is
the author's intent to have an evaluation completed by
an entity that is independent of the STAR test vendor.
AB 391
Page 7
Some have questioned the nature of this independence if
the authority to enter into a contract for the
evaluation only rests with the Superintendent of Public
Instruction, particularly considering the California
Department of Education is the administrator of the
testing contract. Should the State Board of Education
also have a voice in the awarding of a contract for an
evaluation?
4) Fiscal impact . According to the Assembly Appropriations
Committee, this bill would create cost pressure (General
Fund/Proposition 98 or federal funds), likely between
$250,000 and $500,000 to contract for an evaluation of
the STAR program. This bill requires CDE to use federal
funds to contract for the evaluation. The 2009 Budget
Act allocates a total of $24 million Title VI No Child
Left Behind funds for state assessment programs,
including $4 million for the STAR program, and an
additional $5.4 million in federal assessment funding.
5) Related budget action . The Governor proposed budget
language to require the Legislative Analyst's Office and
the Department of Finance to jointly review the STAR
contract and report on its components, current costs,
and program improvement recommendations. This item is
being considered by the Budget Conference Committee, as
the Assembly approved alternative language proposed by
the Legislative Analyst to include a process to review
options for developing and implementing pupil growth
measures in the proposed review of the STAR program,
while the Senate did not approve either the Governor's
nor the Legislative Analyst's language.
6) Technical amendment . This bill does not clearly state
that the evaluator's report is to include the findings
of the evaluation and recommendations on specified
factors (see # 3 in the Analysis section).
7) Prior and related legislation .
AB 476 (Torlakson, 2009) was nearly
identical to this bill. The Governor vetoed AB 476
with the following veto message:
The objectives of this bill are duplicative of
work already being done by a variety of
sources. Not only have there been reviews of
AB 391
Page 8
California's standards and assessment system
by the United States Department of Education's
peer review process, the California Department
of Education has a process which has included
an independent alignment study and review of
test items by
various content and test development experts.
Finally, this bill circumvents the State Board
of Education in the selection of the
independent evaluator and approving the
evaluation and its recommendations.
SB 80 (Committee on Budget and Fiscal
Review, Ch, 174, 2007), among other things,
reinstated second grade testing and reset the
sunset on the STAR Program to July 1, 2011.
SB 1448 (Alpert, Ch. 233, 2004) eliminated
second grade testing on July 1, 2007, and extended
sunset on the STAR Program for grades 3-11 to
January 1, 2011.
SUPPORT
Association of California School Administrators
Business for Science, Math and Related Technologies Education
California Council for the Social Studies
California Federation of Teachers
California School Boards Association
California State PTA
San Francisco Unified School District
Small School Districts' Association
OPPOSITION
None received.