BILL ANALYSIS Ó
SB 1002
Page 1
Date of Hearing: June 27, 2012
ASSEMBLY COMMITTEE ON LOCAL GOVERNMENT
Cameron Smyth, Chair
SB 1002 (Yee) - As Amended: June 21, 2012
SENATE VOTE : 35-0
SUBJECT : Public records: electronic format.
SUMMARY : Requires public agencies to provide or post to the
Internet required public documents in an open format, as
defined, and authorizes state or local public agencies to
provide a copy of an electronic record in a format in which
electronic record text is searchable. Specifically, this bill :
1)Authorizes a public agency to provide a copy of a requested
electronic record in a format in which the text in the
electronic record is searchable in an open format, as
specified, if the agency does not already have the electronic
record in an open format.
2)Requires the requester to bear the cost of converting the
electronic record into an open format, including the cost of
programming and computer services necessary to produce the
electronic record.
3)Provides that whenever a state or local agency is required by
law to make electronic data or an electronic document
available to the public, and the agency has the software,
hardware, or services that allow it to retain the data or
document in an open format, it shall provide the data or
document in an open format.
4)Provides that whenever a state or local agency is required to
make data or a document available on the Internet, and the
agency has the software, hardware, or services that allow it
to retain the data or document in an open format, it shall
post in an open format.
5)Provides that the requirements of this bill shall not apply to
data or documents posted on the Internet before January 1,
2013.
6)Provides that the requirements of this bill shall not be
SB 1002
Page 2
construed to require an agency to re-post existing data or
documents currently posted and available on the Internet.
7)Defines "open format" to mean all of the following:
a) The data or document can be located, downloaded and read
by a requester or member of the public using
publicly-accessible software or public Internet
applications without any purchase price or paid
subscription fees;
b) The data or text in the document is machine readable and
can be searched, indexed, organized, categorized, and is
otherwise automatically processable; and,
c) The data or document provides data granularity,
definitions, and structured formats in the original quality
available to the state or local agency.
8)Provides that this bill shall not be construed to require a
state or local agency to convert data or a document into an
open format or to update its software or hardware.
9)Provides that the provisions of this bill shall not be
construed to require a public agency to release data or a
document in an open format if its release would jeopardize or
compromise the security or integrity of the original record or
any proprietary software in which it is maintained.
10) Provides that nothing in this bill shall be construed to
permit public access to records held by a public agency to
which access is otherwise restricted.
EXISTING LAW :
1)Requires public agencies, via the California Public Records
Act (CPRA), to make their records available for public
inspection and, upon request of a person, to provide a copy of
a public record unless the record is exempt from disclosure.
2)Requires a public agency to make non-exempt electronic public
records available in any electronic format in which it holds
the information or, if requested, in an electronic format used
by the agency to create copies for its own or other agency's
use.
SB 1002
Page 3
3)Authorizes a public agency to charge to the requester the
direct cost of producing the electronic public record.
4)Requires the requester of the electronic record to bear the
cost of producing a copy of the record, including the cost to
construct a record, and the cost of programming and computer
services necessary to produce a copy of the record if the
public agency produces the electronic record only at regularly
scheduled intervals or the request requires data compilation,
extraction, or programming to produce the record.
5)Provides that a public agency is not required to release an
electronic record in the electronic form in which it is held
by the agency if its release would jeopardize or compromise
the security or integrity of the original record or of any
proprietary software in which it is maintained.
FISCAL EFFECT : According to the Senate Appropriations
Committee, this bill contains:
1)Minor ongoing costs to state and local agencies to provide and
post electronic data and documents in open format on a
prospective basis, if the data and documents are maintained in
that format by the agency and release of the data or documents
would not compromise the security or integrity of the
information, as specified.
2)Potential significant future cost pressure on state and local
agencies to move to systems/software enabling
creation/conversion of data and documents to open format.
COMMENTS :
1)This bill aims to encourage state and local public agencies to
maintain and provide electronic data or documents in an 'open
format", as defined, under the CPRA in order to facilitate the
electronic access, searching, indexing and extraction of data
from government records. The measure is author-sponsored.
2)According to the author, "Ýt]he main deficiency in current law
is that there is not a set standard for what format electronic
data Ýis] to be provided in. Oftentimes agencies will post
online or turn over via ÝCPRA] requests Ýfor] electronic data
that is nothing more than an image file. Meaning, the data or
document cannot even utilize basic functions such as searching
SB 1002
Page 4
or indexing. This bill would create a set standard and ensure
that when data goes online it is useable."
According to Californians Aware, "Ýt]he CPRA was amended in the
1990s to bring its disclosure requirements up to date to
prevent new technology from becoming a barrier to government
transparency rather than a supportive factor. Another
updating is overdue to accomplish the same objective, by
encouraging public agencies to make information not only
available in static electronic format - humanly readable - but
in a machine readable form useable by the public for purposes
of the kind of sophisticated analysis that was beyond the
reach or even imagination of most people a decade ago or so."
3)Under current law, any member of the public may request,
pursuant to the CPRA, an identifiable public record, not
otherwise exempt, that an agency provide a particular record
in the electronic format in which that agency keeps the
document. Requesters generally bear the cost of any extra
duplication, construction or programming required. These
provisions were put in place by AB 2799 (Shelley), Chapter
982, Statutes of 2000.
The author notes that a problem arises when the electronic
format of the document is an image file that cannot be
immediately searched or indexed by a program. An example
might be a .tif file, or 'tagged image file format', commonly
used in the past for scanned documents that is not always
'searchable' by software or a search engine like Google,
although a .tif file can be made readable with reasonable
accuracy through 'optical character recognition' software if
the original image is of sufficient quality.
One practical result of driving government data into an 'open
format' would be to make that data easily accessible to data
mining and software applications. For example, making fare
and schedule data for a local transportation agency would
enable private sector programmers to develop mobile computer
applications for sale to commuters - a potential boon to
consumers that would increase the value of the transportation
system without direct costs to the agency.
Finally, the author points to the US Director of the Office of
Management and Budget's December 2009 Open Government
Directive (M10-06), which was intended to assist executive
SB 1002
Page 5
departments and agencies in implementing the principles of
transparency, participation, and collaboration set forth by
President Obama.
According to the author, the Directive instructs federal
government agencies to "publish information online in an open
format that can be retrieved, downloaded, indexed, and
searched by commonly used web search applications." The
Directive also defines "open format" to mean "one that is
platform independent, machine readable, and made available to
the public without restrictions that would impede the re-use
of that information."
4)The bill would amend the CPRA to authorize state and local
public agencies, upon request, to provide electronic data or
documents in an 'open format' even if the record does not
maintain the document in an open format, as long as the
requester covers the cost of conversion. Public agencies would
also be required to provide upon request, and post where
required by law, any electronic data or documents in an open
format when the agency has the software, hardware, or services
to maintain the data or document in an open format.
The measure exempts from the posting requirement any record
posted before January 1, 2013. It also explicitly exempts
public agencies from having to re-post data currently posted
and available on the Internet. This bill also states that it
should not be construed to require a public agency to convert
data or documents into an open format or update its hardware
or software.
The bill also provides a definition of 'open format', which
refers to data that can be located, downloaded and read by the
public using free, publicly accessible software. It also
requires that data or text be machine-readable and
automatically processable, and can be searched, indexed,
organized, and categorized. The definition also requires the
open format record to provide 'data granularity',
'definitions', and 'structured formats' in the document's
original quality.
5)Opponents of the bill and others raise the following concerns:
a) The California Association of Clerks and Elections
Officials (CACEO) contends that "portions of Section 2 seem
SB 1002
Page 6
to be internally conflicting. On the one hand,
subdivisions (a) and (b) clearly would require an agency to
provide, upon request, electronic data and electronic
documents in an open format, whether the requested records
are available on-line or are maintained by the agency on
its various electronic systems. Subdivisions (e) and (f)
clearly state that agencies would not be required to
convert data or a document into an open format and that
agencies would not be required to update their software or
hardware in order to comply with the section. It is
unclear, then, what agencies would be required to do in
order to be in compliance with Section 6278."
b) The League of California Cities (League) argues that
"the CPRA does not require local agencies to create records
in response to records requests. It only requires agencies
to disclose existing, non-exempt records in the agency's
possession. This is in keeping with the CPRA's fundamental
purpose of letting the public know how its business is
being conducted. The other, related considerations that
are carefully balanced in the CPRA and case law are
protection of individual privacy rights and furthering
effective government operations. Imposing an expensive and
difficult new requirement on local agencies to reformat
electronic government information, including information
received from third parties, for purposes of revising and
manipulating data that is already available for disclosure
does not advance Ýthe] goals of the CPRA."
c) The California State Association of Counties (CSAC)
points out that the permissive language of GC 6253.9(e)
Ý"Upon request, an agency may provide a copy of an
electronic record in a format in which the text in the
electronic record is searchable in an open format?if the
agency does not already have the electronic record in an
open format. The requester shall bear the cost of
converting the electronic record?"] raises the question of
whether or not the conversion is at the discretion of the
county clerk, or if the mandate of GC 6253.91(a)-(b) to
provide a document in the open format if technically
feasible trumps that discretionary language.
d) CSAC further states that "Section 6253.91(c)(2) requires
that a document be machine-readable as well as searched,
indexed, organized and categorized. There are many
SB 1002
Page 7
documents that are reproduced as a PDF so that a local
agency may safely share the information with the public
without fear of the document being manipulated by outside
parties. Local agencies should not be required to release
documents such as meeting minutes, which include voting
results, resolutions or ordinances in a format that would
allow for easy manipulation by outside interests. These
documents should be and are available to the public, but
for liability reasons are scanned and not released in
formats that downstream can be easily manipulated or
inappropriately amended."
e) The California Special Districts Association (CSDA)
argues that "staff tasked with fulfilling ÝCPRA] requests
will be required to determine how electronic data and
documents meet the newly created standard of 'open format'
in the bill. This will require time and resources to train
staff on this new standard."
6)A coalition of CSAC, Urban Counties Caucus (UCC), the Regional
Council of Rural Counties (RCRC), and the League has taken an
'oppose unless amended' position, requesting that the bill be
amended to require that only non-sensitive documents be
required to be provided in any open and searchable format, and
that the 'open format' provisions be deleted in their
entirety.
7)As a technical matter, in the definition provided for "open
format" by this bill ÝSection 6253.91(c)(3)], it requires the
data or document to provide "data granularity", "definitions",
and "structured formats". As these words or phrases do not
have existing definitions as technical terms in the Government
Code and are not otherwise defined by this bill, it may be
necessary to add definitions in order to provide public
agencies with clearer guidance on how to achieve compliance.
8)The Committee may also wish to consider the larger question
surrounding the imposition of a specific data standard on all
state and local public documents - is the proposed standard
the right standard?
This bill aims to impose a single data standard for all state
and local government that would make its data interoperable.
The 'open format' standard presented here is loose, reflecting
more of a set of principles than an exacting technology
SB 1002
Page 8
standard per se. However, that same flexibility may not
provide sufficient specificity to public agencies looking for
guidance on whether or not specific programs or file formats
would be in compliance with the 'open format'.
The Committee may wish to ask the author to describe the
process behind the development of the proposed 'open format';
the individuals, groups, companies and public agencies
consulted; the different standards and elements considered;
and the shortcomings of current law that necessitate the
proposed standard.
The Committee may also wish to discuss whether or not the
issue should be studied further by the Legislative Analyst's
Office or other research entity.
9)This bill was double-referred to the Assembly Committee on
Governmental Organization, where it was heard on June 20,
2012, and approved on an 11-3 vote.
10)Support arguments : According to Common Cause, this bill
"will bring accessibility to documents and information not
easily available in their current state. SB 1002 will make
gathering information easier, in a user-friendly manner. The
current inaccessibility to documents, due to a lack of easily
searchable text and format, makes the process of gathering
information difficult. Accessibility to these public records
would encourage voters to seek out information that would help
them make informed decisions."
Opposition arguments : According to a coalition led by CSAC,
"SB 1002, as currently drafted, leaves requirements regarding
open format definitions somewhat vague, creates uncertainty
and confusion on the part of the public as well as state and
local agencies, and could make compliance with this measure
difficult."
REGISTERED SUPPORT / OPPOSITION :
Support
American Federation of State, County and Municipal Employees
(AFSCME)
California Council of the Blind
California Faculty Association
SB 1002
Page 9
California Teachers Association
Californians Aware
Common Cause
San Francisco Public Utilities Commission
United Auto Workers Local 5810 (Berkeley)
United Reporting Crime Beat News
Opposition
California Association of Clerks and Elections Officials
(concerns) (5/23)
California Peace Officers Association (4/26)
California Special Districts Association (6/19)
California State Association of Counties (unless amended) (6/25)
California State Sheriff's Association (4/11)
Chief Probation Officers of California (4/11)
City of Cathedral City (4/13)
Community College League of California (unless amended) (6/22)
League of California Cities (unless amended) (6/19)
Regional Council of Rural Counties (unless amended) (6/25)
Urban Counties Caucus (unless amended) (6/25)
Analysis Prepared by : Hank Dempsey / L. GOV. / (916) 319-3958