i.
TECHNICAL PROPOSAL COVER PAGE
TITLE: EASE Project: Research and Development of Sustainable, Open Source, Multi-
Platform Applications for use by election officials to improve outreach, services,
communication, and ballot delivery to UOCAVA voters
BAA NUMBER:
BAA H98210-BAA-11-0001
APPLICANT CAGE CODE:
APPLICANT DUNS:
APPLICANT:
WENDY S. NOREN
BOONE COUNTY CLERK
COUNTY OF BOONE, MISSOURI
KNOWN CONTRACTORS/SUB-RECIPIENTS
UNIVERSITY OF MISSOURI - COLUMBIA
OFFICE OF SECRETARY OF STATE - STATE OF MISSOURI
TECHNICAL CONTACT:
Wendy S. Noren
Boone County Clerk
801 E. Walnut Rm 236
Columbia, Mo 65201
Phone: 573-886-4295
Fax: 573-886-4300
EMAIL: wsnoren@gmail.com and wnoren@boonecountymo.org
ADMINISTRATIVE/BUSINESS CONTACT
Wendy S. Noren
Boone County Clerk
801 E. Walnut Rm 236
Columbia, Mo 65201
Phone: 573-886-4295
Fax: 573-886-4300
EMAIL: wsnoren@gmail.com and wnoren@boonecountymo.org
PROPOSED PERIOD OF PERFORMANCE
8/01/2011 through 12/31/2012 for (system maintenance continued through November 2018)
2
ii.
TABLE OF CONTENTS
Technical Approach and Justification:
Executive Summary ________________________________________________
3
Goals and Objectives________________________________________________
4
Develop tools for online and electronic communication tools _________
4
How the Goals meet the Evaluation Criteria_____________________________
5
Significance__________________________________________________
5
Sustainability of Products______________________________________
5
Sustainability of Research_____________________________________
6
Impact______________________________________________________
6
Strategic Approach____________________________________________
7
Innovation ___________________________________________________
8
Scalability __________________________________________________
8
Collaborative Qualities________________________________________
9
Cost Benefit Analysis_________________________________________
9
Security __________________________________________________
9
Schedule and Milestones___________________________________________
10
Reports _________________________________________________________
12
Management Approach ___________________________________________
14
Current and Pending Project Submissions ___________________________
19
Qualification of Key Personnel _____________________________________
20
3
iii. TECHNICAL APPROACH AND JUSTIFICATION
EXECUTIVE SUMMARY
Boone County Clerk Wendy Noren will lead a team of state, local and university experts who
will:
- Research and develop UOCAVA outreach and education programs through non-traditional
sources, emerging technology and modern data mining techniques to connect UOCAVA voters
to their Local Election Authority (LEA);
- design online FPCA system designed to serve UOCAVA voters, reduce local election authority
workload, increase data integrity and improve UOCAVA voter knowledge of and access to
alternate ballot delivery availability;
- Research, design and develop online or other ballot delivery methods for all Missouri
jurisdictions that will increase the probability that a UOCAVA voter will be able to receive, vote,
cast and have counted their ballot;
-Research and potential development of online ballot delivery system that would be open to
states and jurisdictions outside Missouri – subject to University capacity limits;
- Research and develop alternative processes for low cost processing of paper ballots returned by
mail to improve accuracy and reduce LEA workload;
-Design systems with end goal of providing a low maintenance system for perpetual use by states
and local governments;
- Provide research and produce mobile apps that can be used to help UOCAVA voters connect
the LEA;
These systems serve to extend and expand the initial research and development done by Boone
County to communicate to UOCAVA voters new opportunities for ballot deliver, provide web
based access to request electronically delivered ballots and participation in 2010 Electronic
Voting Support Wizard through the FVAP.
This grant will incorporate lessons learned from the design/development program to provide
expanded ballot delivery systems and communication tools for both UOCAVA voters and LEA’s
in Missouri.
The primary deliverable under this grant will be an online application and ballot delivery system
that is designed to provide synergy with current LEA process and improve data accuracy.
The University of Missouri and the Boone County Clerk will also research and develop ways to
utilize technology concepts standard in commercial operations but unknown and currently
unavailable election official to indentify and link UOCAVA voters with their LEA. The research
will look into how to leverage available “big data” to develop tools to assist UOCAV voters.
The open source applications developed by the University will provided to all Missouri
jurisdictions through 2018. The systems will be designed with the end goal of providing either
perpetual use through migration to state or local hosting sites or development of a cost fee for
service system affordable by even the smallest jurisdiction in the state. In addition, applications
will be made available at no cost to state and local jurisdictions outside Missouri that have the
ability to implement independent of the University of Missouri.
4
iii. TECHNICAL APPROACH AND JUSTIFICATION
Goals and Objectives:
A. Develop deliverables for online and electronic communication tools:
The primary deliverable under this grant will be the development of online and other electronic
applications that can be used by UOCAVA voters and LEAs to process registration and absentee
applications; provide delivery of electronic format ballots, keep the UOCAVA voter informed on
ballot status and improve tracking and reporting capabilities. The Boone County Clerk will
coordinate the research and design of these applications with University of Missouri College of
Engineering Computer Science Department, the Secretary of State through their state voter
registration database vendor and local election authorities in Missouri. The design and
development will build on previously designed applications used in Boone County for the
implementation of MOVE to incorporate them into the state election management system as well
as enhance and expand them to local jurisdictions statewide.
Missouri jurisdictions other than Boone County are reliant on the state voter registration database
for absentee and election management processing. These functionality of this system can be
enhanced to improve services available to voters under MOVE.
The Missouri rate for utilization of electronic delivery in the 2010 election was dismally low
(approximately 1%) except in Boone County where more than 90% of the UOCAVA voters
eligible for electronic delivery requested and received their ballot through automated ballot
delivery systems. This utilization rate was so low because of the following identifiable barriers:
- UOCAVA voters were not aware of the availability of this service
- Election Authorities had no incentive to encourage the use of electronic ballot delivery
because the process available was significantly more burdensome than mailing a ballot.
Through the expansion of programs developed in Boone County statewide these barriers can be
significantly reduce through systems that:
- communicate availability of electronic delivery of ballots to UOCAVA voters early
- develop online registration and absentee application systems that allow UOCAVA voters
easy access to entering and where necessary printing appropriate applications
- to reduce LEA data entry workload and error rate provide for collection of the application
data entered by the UOCAVA voter for delivery to LEA in a format that can be merged
electronically to the voter registration database after the application is received and
approved by the LEA
- eliminate duplication of effort currently needed to manage voter database, ballot counting
systems and a new layer of electronic ballot delivery through the development of a
consolidated ballot layout and coding system utilized for all systems
- maintain an automated communication flow to the voter of the status of the application so
the voter does not have to go to a website to determine status;
- research the effectiveness of various alternative methods to capture voted ballot data
submitted by mail and incorporate it electronically to ballot counting systems;
5
- develop systems to assist in the reporting requirements for LEA’s
- research and develop tools that assist in the prevention of fraudulent use of technology
used to provide services to UOCAVA voters
- develop all applications with the knowledge that over the next few years more people will
be communicating and processing transactions through mobile smartphone technology
rather than traditional computers;
- identify and develop mobile applications that can enhance the democracy experience of
UOCAVA voters
- identify and develop sources for partnerships in getting these applications to UOCAVA
voters (i.e. National Guard agencies for inclusion in materials provided to members being
sent overseas, college registrars for students going abroad etc.)
B. How the Goals meet evaluation criteria:
Significance – The significant variation between the rates of electronic delivery in Boone
County versus the balance of the state demonstrates that these tools have potential to
dramatically increase utilization of electronic ballot delivery. The Boone County experiments
indicate that comprehensive programs that increase UOCAVA voter access and reduce LE costs
can have a significant impact on usage rates of ballot delivery systems. By developing an open
source system designed to be transferable to many jurisdictions across the country these grant
funds can dramatically enhance the ability of any UOCAVA voter to have their ballot cast and
counted in a timely fashion.
By researching how modern technology tools are used commercially to identify and provide
services a specified customer base we can expand the knowledge base of potential services
available to voters. Utilizing modern data mining techniques (with privacy by design concepts)
could change the tools available to identify, track and serve UOCAVA voters. The technology
exists today but no one has researched and developed tools to bring those concepts and
technology to the voter service area. As an example, when I book a flight to XYZ I get email
from rental car companies asking if I want to rent are car. This grant will research how that
technology might be translated to our business function of serving voters. Other research area
can be how we might utilize smart phone technology to identify and connect UOCAVA voters
with the correct LEA.
The capacity to do this type of research identify cost effectiveness of application development is
not available to the current vendor base serving the election community. Research on this scale
can best be filtered and tested through a major research institution, such as the University of
Missouri, that has the mission and technical skill set to develop a multi-disciplinary approach to
the problems faced by UOCAVA voters and election officials.
Sustainability of product deliverables
The design and implementation of this system will be focused on the primary areas critical to
sustainability of the product:
- The system must reduce increase the probability that a UOCAVA voter has the correct
ballot in time to have it cast and counted;
- The system needs to decrease the workload andcosts for local election officials
- There is a long term commitment on the part of the University of Missouri to insure the
end product will be available after 2018 for a low fee for service rate that is affordable to
6
any Missouri jurisdiction. We believe the model by then could be in the range of $100
per election set up and 2.00 to 3.00 per ballot transmitted. This price range will keep
LEA’s utilizing a system because the cost of transmitting ballots through this system will
be less than processing them by mail;
- There is a commitment on the part of the University of Missouri to allow migration of all
applications at no charge to a state or local host site for ongoing use;
- The system must duplicate and enhance due diligence efforts currently used by LEAs to
prevent fraudulent use of the UOCAVA process;
- The system must provide UOCAVA voters with easy access to the processes necessary
for casting a ballot and utilize technology to enhance the experience for disabled voters;
- The system must provide confidence to UOCAVA voters that their voter data is used
appropriately and only for processing ballots;
- The system must provide the UOCAVA voters the ability to use the technology available
to them i.e. mobile phones, to easily process their requests, and communicate with their
LEA through the development of downloadable applications from app stores;
These concepts will guide the development of all applications identified as deliverables to extend
the useful life of any product developed.
Sustainability of Research Components:
The research on voter outreach through use of technology will identify costs associated with and
how we leverage scarce funds to maximize our access to tools that connect us with UOCAVA
voters.
It is highly unlikely that any local jurisdiction would be able to subscribed to and maintain the
kind of data mining structures that will be explored but properly through research and building
partnerships there may be components of this grant that will identify systems that could be
appropriate at the state level.
Voter outreach applications will, like the other applications developed under this grant, continue
to be available to Missouri local and state election officials.
Impact
Missouri currently processes approximately 16,000 UOCAVA ballots in a Presidential Election
and up to 10,000 in an off year General Election. There is minor participation in primary
elections. The Boone County numbers indicate that given easy access to choosing electronic
ballot delivery, 90% of the UOCAVA voters will choose that option. Current usage in other
Missouri jurisdictions was approximately 1%. Implementation of these proposals statewide can
result in dramatic increase in services envisioned under MOVE.
In the 2008 Presidential Election 17% of UOCAVA voters whose applications were processed
did not have a ballot counted. Primary reasons include the ballot was received too late, it was
returned undeliverable, the status was unknown (indicating it was received by the voter too late
or lost in mail). An additional 5% of the UOCAVA voters had a ballot counted from the FWAB
submitted which indicates the regular ballot was not received in time to be returned and counted.
7
Approximately 100 ballots that were received timely were rejected because of failure sign or
complete the appropriate affidavits. Primary elections present an additional set of errors when
voters do not request a party ballot (required by state law)
Implementation of these proposals will reduce the error rates of ballot delivery, return and
affidavit processing through the use of online application, ballot delivery and affidavit generation
procedures that validate information, reduce data entry errors, and are evaluated for usability to
reduce voter confusion.
Design of this system will also be evaluated to possibly allow other states and local jurisdictions
to utilize the tools developed as capacity permits. Due to the open source nature of the products
developed potential impact is nationwide adoption of common software.
Strategic Approach
The primary deliverables under this will be based on prototype systems that were in place in
Boone County and utilized in 2010. Boone County had versions of web based application
processing and automated notifications of applications/ballot in place for several election cycles.
Electronic ballot delivery through automated email system was developed and implemented in
2010. In addition an online ballot marking was designed to specifications developed in Boone
County and provided to Boone County voters through the 2010 FVAP Electronic Voting Support
Wizard (EVSW).
The Boone County experience demonstrates that the deliverables for electronic application,
ballot delivery and ballot status notification increases UOCAVA service levels and reduced
workload in processing applications, ballot delivery and responding to voter queries. In addition
the Boone County Clerk will research and design best method for processing returned paper
ballots and incorporating them into the ballot counting systems currently in use in Missouri.
This grant will be utilized to analyze how best to incorporate these successful design elements
into the software used by the other 115 local jurisdictions in Missouri as well as develop new
applications that will set up the online ballot delivery system. The online ballot delivery system
will contain many of the elements that were designed by Boone County for the EASE project but
will be modified to address issues not adequately covered during the short time frame of that
grant:
- Usability testing of design elements as well as interface with assistive technology
devices;
- Security issues related to downloadable ballots
- Conversion of voted paper ballots returned to LEA to format that allows producing
machine counted results
- Utilization of ballots by UOCAVA voters who have disabilities
The University of Missouri will provide expert cyber security support throughout the design and
maintenance to insure ballot integrity and will provide facilities and personnel for usability
testing throughout the project. Additional consultant work is included in the budget to have
usability consultations from expert(s) who have specific research and testing experience with
ballots and where necessary “privacy by design” expert review.
8
The State of Missouri will provide under contract the modifications to the state voter database
that will be required to import and export data elements necessary to accomplish full electronic
process of application and online ballot delivery.
The University of Missouri will develop mobile applications that enhance the UOCAVA voter’s
ability to access the tools developed under this grant and process their applications and ballots.
This will be enhanced by the military initiative to place a Smartphone device in the hands of all
personnel. As more people move to these systems and away from traditional web/internet pages
this will be critical need;
The Boone County Clerk, in conjunction with the University, will identify and recruit key
government, industry, political, not-for-profits, commercial, university and journalists to research
and identify best methods for:
- identifying the customer base and strategies to provide content to them through
innovative search engine, social media, data mining that successfully used in other
industries but are not available to election officials;
- development of partnerships for putting the tools in the hands of the UOCAVA voters
Innovation:
- No current open source applications exist for use by election official in the areas we have
identified as deliverables;
- No current affordable fee for service ballot delivery system exists that would provide for
perpetual sustainability of the concepts outlined in the grant notice;
- The grant will fund research into alternative ways to process returned paper ballots to
determine the most cost effective and accurate method for incorporating these votes
- The mobile applications to be researched and developed by the University will be unique
and designed to be available to all election officials;
- New research will be been done to identify sources and develop cost modeling for
alternative service delivery methods;
- New research to identify appropriate partners to enhance the availability of the tools
developed by having them incorporate the tools into their service structure;
- All systems will be designed to meet Section 508 requirements and this study will further
work with consultants who have specialized skills in identifying assistive technology
components and usability requirements in the election process and incorporate those
concepts into all design development processes;
Scalability:
The design of this system is geared toward maximum scalability of software components and
ongoing service through 2018. The end cost model is designed for ongoing usability. The
product will also incorporate design elements that would allow other states or local
jurisdictions (assuming same ballot design criteria as Missouri) to utilize the system up to
current capacity limits. These limits will be identified and costs developed to expand the
process to any jurisdictions.
9
The project will utilize, if available in a timely fashion, common data format criteria to be
established by NIST. The project is not designed to incorporate state specific ballot design
issues that do not exist in Missouri (i.e. straight party voting, ballot rotation).
Collaborative Qualities of the Project:
The project will be collaboration between State, local election officials (116 jurisdictions)
and University of Missouri -Columbia.
The University of Missouri Columbia is one of the nation’s 76 land grant Universities whose
mission is to expand the research conducted at the University beyond the campus and into
communities. As such, the University works within a consortium of land grant universities to
expand the research not only to Missouri communities but also to other states. This unique
system may assist in expanding the work product developed under the grant to other states
and local jurisdictions. In addition the University has developed prior working relationships
with military base personnel, other colleges and universities and state agencies that will be
involved in this project.
Cost Benefit Analysis:
Although this is a statewide project covering all election jurisdictions we propose to focus
cost benefit and return on investment analysis to the jurisdictions that process more than 200
ballots. These represent 75% of the UOCAVA voters and reduce the study pool from 116
jurisdictions to less than 20. These entities cover a range of jurisdictions with varying
degrees of available staffing. They also contain jurisdictions that have military bases,
colleges and universities and National Guard sites that are focal point for most UOCAVA
applications. This way we can focus on more in depth analysis.
The system will incorporate tracking processes to analyze site visits, application access,
online ballot delivery, error tracking. Cost benefits of various designs will be analyzed prior
to implementation to determine most economical and broadest use design for long term
implementation. Research programs and communication/marketing application will identify
usage rates. If possible, traffic flow will be analyzed to determine ways to improve usage by
UOCAVA voters. Utilizing partnership development will increase usage by UOCAVA
voters thereby lowering cost per vote and LEA overhead.
Critical to this will be developing a cost effective method to translate voted paper ballots to
readable formats in the ballot counting process. Allowing online delivery of ballots without
this will cause an increase in LEA workload through having to re-mark ballots. Current
estimates of vendor provided systems are not feasible on statewide basis so alternative
processes need to be researched, designed and developed.
Security Measures
Throughout the process we will have the University Cyber Security program assist with
development of state of the art security procedures and controls. In addition, research will be
conducted on the specialized nature of ballot security in the online environment and tools to
protect against fraudulent use of services provided to UOCAVA voters.
10
Schedule and milestones:
FPCA(Registration and Absentee Application) Web Tool
August 2011 – contract awards and determine data elements for state database; determine
functional elements work plan for state vendor to develop quotes
August 2011 – Meet with targeted LEA’s for design meeting and have Cyber Security
research and reviews completed
September – initial design & design phase usability testing and define automated email
system requirements
October – develop and test security/privacy plan for FPCA data transmission
October 1 – finalize design elements for University program development
November 1 – final design plan to State database vendor for contracting April 1 deliverable
November – develop voter notification protocols to connect voter to the LEA and inform on
the status of application
October 1 to December 1 University coding and testing of FPCA functions
December 1-7 – usability testing of final product
MILESTONE Key Assessment point: December 15 – FPCA testing and dual processing in
Boone County.
Development of LEA training materials
January – April – live processing in Boone County and development of LEA training
materials
April – final testing of State Database merge programs.
Late April/ May – LEA training of MCVR merged application processing
MILESTONE May 15 2011 – live implementation statewide
Key Assessment Point
Simultaneous – research and development of mobile apps for this function
Online Ballot Delivery Tool:
2011
August 2011 – define data elements for state database. Review design options. Meet with
LEAs for design meeting and present design options
August 2011 – meet will ballot counting system subcontractors (Premier and Sequoia) to
conduct feasibility study on creating exports of ballot definitions and review returned ballot
processing options
September 1-15 – Cyber Security research and reviews and develops security plan and design
phase usability testing and review IEEE and NIST common data format requirements
September 15-30th – review system plan to LEAs at state conference and define automated
email system requirements
October – develop design elements for State Database – submit to vendor for pricing
Milestone October 15 – final design plan with University
October 15 through December 15 – design and development of online system; research on
ballot printing system
November 15 – Boone County sample presidential primary file extracts available to
University
November 15 – finalize design contract with State database vendor
December 1 – usability testing of sample online ballot data
December 15 – Boone County notifies UOCAVA voters of new system for online signup test
voter notification email system.
11
December 20 – data extracts of precinct district data for presidential primary from Boone
County
Milestone December 27 – online ballot definition system live for Boone County
2012
Milestone week of January 15, 2012 – online ballot delivery system live for Boone County
UOCAVA presidential primary voters (legal deadline January 20)
January 15 through March 15 – ongoing processing of Boone County presidential primary
voters, statistical reporting and voter usage; Develop training materials
April 10-15 Usability testing of state database modifications
April 15 – complete final testing of State Database extract programs
April-May – LEA training
Milestone May 29, 2012 online ballot definition system goes live for 20 largest UOCAVA
jurisdictions (and others who choose to participate)
Milestone Week of June 17 Online ballot marking system goes live for participating
jurisdictions and voters
June 17-August 15 – ongoing statistical reporting and monitor modifications needed
modifications needed
August 15 – Lessons learned review and modifications due
August 15 – August 25 additional training programs
August 25th final modifications due and online ballot definition system goes live for all
jurisdictions
Week of September 17th – online ballot marking system available statewide (legal deadline
September 21
Mobil Apps – this process will run in conjunction with the above systems and will include
identifying and meeting with Missouri military base and National Guard personnel early in
the process to insure timely delivery of services that service group;
Research –
September 2011 – November identify and coordinate technology based voter outreach
summits to develop technology distribution partners and pathways to connect UOCAV voter
with LEA; survey UOCAVA voters and third party application processors (i.e. Overseas
Vote Foundation)
November 2011 to January 2012 – define research models and additional mobile application
concepts, finalize partnership contacts;
January to May develop partnership applications identified in research and conduct Cyber
security research on electronic return of ballots;
June – deploy partnership applications in conjunction with state FPCA rollout
June – November – monitor partnership participation for connecting UOCAVA voter to LEA
November – December 2012 – review and report data evaluate lesions learned and determine
programs for future implementation; report on cyber security findings for potential for future
models for online delivery of voted ballots;
Post 2012 (through 2018) – maintain online systems developed under this grant at no cost
and provide ongoing assistance to LEA’s; provide software to other state and local entities at
no cost; develop low cost fee for service pricing model for post 2018 that any jurisdiction
will be able to use. Continue research and development of applications for partnership
development;
12
REPORTS
Administration, status and financial;
Boone County will provide status reports on application development at least quarterly;
Boone County will provide implementation reports within one week of each milestone that
identifies problems and successes;
Research reports and recommendations for cyber security controls will be provided upon receipt
by the Boone County Clerk;
Contracts will be available upon approval by Boone County Commission;
Status updates with state database vendor will be provided quarterly;
Boone County will arrange for FVAP personnel to participate in application testing and
consolidate reports and recommendations from that testing;
Boone County will provide review and report on consultant work at least quarterly;
Boone County will file report of progress in conjunction with any payment in excess of 10,000 to
single source;
Boone County will separately track and report all financial transactions and maintain records
appropriate with state and federal requirements provide all required financial reporting;
Boone County will comply with all audit requirements arising from this grant but is requesting
funding for additional costs Boone County will incur as a result of this grant (i.e. major program
audit – see budget request)
Boone County will separately track all equipment and software purchase made from grant funds;
All audit reports for Boone County will be available to the FVAP and the public at
www.showmeboone.com/auditor
Boone County will development contracts and schedules to insure appropriate reporting
compliance with federal grant requirements from sub-recipients;
Boone County will develop, in conjunction with the state and LEA’s a method for defining any
cost savings, error reduction, and possible cost increases; this data will be used in conjunction
with application, ballot delivery and ballot processing statistics to develop a final return on
investment report in January 2013;
Boone County will use statistical reports generated from current and developed systems to
identify and report improvement in successful completion of ballots from application to counting
for UOCAVA voters in Missouri;
Statistical Reports:
Online application and ballot delivery systems will be designed to track and report statistics of
accessed, submitted, processed, received, approved and notified;
Email notification systems will provide statistics on quantity and content coding (i.e. application
accepted emails sent; application rejected emails sent: etc)
Online systems will be designed to collect detailed transaction data and produce reports statistics
for usage, error, voter status, voter activity and other including but not limited to:
Date time source (ip address);
13
Prior to implementation, research will be conducted to identify a comprehensive set of
transaction tracking recommendations that will be incorporated into the system;
Transaction statistics will be reportable by state or jurisdiction on demand and downloadable in
common format (i.e. csv) for additional desired analysis and report generation;
Helpdesk transactions and reporting will be provided at state and local level;
Immutable audit logs will be developed and report generation of those logs will be available;
Mobile apps will be developed with tracking systems for downloads;
Mobile apps will be evaluated for inclusion in the Reynolds Journalism Institute annual survey of
IPod apps
Research reporting
How to utilizing successful business models for data mining, extracting social network data, web
transactions to connect UOCAVA voters with tools developed and their LEA;
Cyber security report on potential avenues for the future of returning ballots electronically;
Survey data from UOCAVA voters before and after the development of these products;
Survey data from states on availability of web based absentee/registration application and ballot
delivery processing;
Reports and recommendations from each of the four proposed summits (see budget narrative)
designed to assess best uses of available data mining options to identify and reach our customer
base; build partnerships with social media, web search, not for profit and political who provide
third party applications to create convergent technology; and engage media leaders to develop
public education campaigns on availability of tools;
Usability testing reports and documentation on how to incorporate applications with assistive
technology to maximize usage by disabled voters;
Quarterly updates of research status, findings and recommendations;
Special reports and papers that may derive from any of the above research;
How the technologies and systems developed can be expanded to the disability community and
general voting population to increase return on investment;
These reports will be completed and filed by end of January 2013 along with a final report -
contents will be developed in consultation with the FVAP staff.
14
iv.
MANAGEMENT APPROACH
Personnel:
PROJECT DIRECTOR:
Wendy Noren Boone County Clerk
Wendy Noren will manage the grant on behalf of the County of Boone and coordinate with
subcontractors in the design, development, testing and implementation of all products under this
grant. She will also advise on research topics and coordinate consultants and identify potential
participants in research studies on technical, security, and outreach efforts. She will be
responsible for all administrative aspects of the grant and coordinating compliance by the sub-
contractors.
Ms Noren is serving as Project Director and is the applicant for this grant because many of the
products to be designed and delivered have already been in production in Boone County.
She has previously designed, programmed and implemented the automated web voter/absentee
application sites, automated ballot email notification system and secure data transfer systems to
an from the Secretary of State database. She also designed specifications and programmed the
data extracts used in the pilot online ballot delivery system used in 2010 through EVSW.
Ms. Noren will be responsible for working with University of Missouri, Secretary of State and
LEA’s to create functional design specifications, assist with the development of specifications
needed state database modifications; work with University staff on design of online applications;
develop test programs for data extraction and transfer; conduct pilot program in Presidential
Primary using live system; assist with development of training programs and provide financial
management of the grant. Ms Noren will investigate methods of electronically counting ballots
returned by mail (i.e. through barcode of votes cast). She will also assist with the development
of survey applications and results, coordinate seminar development and review research
progress.
It should be noted that most of the design elements already exist for these programs. Ms Noren
will need to review them with the state and University, develop test scripts for any modifications
monitor usability testing results and manage the dual processing. Programs for use in Boone
County were developed in 2010 and will require only minor modification.
This project will be accomplished through establishment of inter-governmental contracts
between the County of Boone (Boone County Clerk), the Office of the Secretary of State and the
University of Missouri-Columbia.
Secretary of State Role:
The contracted responsibility for the Secretary of State is to coordinate services that will be
provided by vendor who has the contract to maintain and develop enhancements to the state
voter registration database (MCVR). These services will include the development of extracts
15
from and imports to MCVR to accomplish the necessary updates from web based transactions
and creation of ballot data for online ballot delivery.
The goals for this part of the project are to design the imports, interfaces and exports of data to
reduce redundancies in election management processing between the MCVR system, the ballot
counting system and the new system for online ballot delivery two options will be studied:
Option 1
- create exports of precinct, precinct split, district, and jurisdictions files for export to a
web interfaced developed by the University
- entry of candidate, proposition, races files on the web portal designed by the University
by LEA’s
- export of the necessary date from the University back to the MCVR and for use with the
ballot counting system used by the county
Option 2
- create the all of the files on the MCVR system and export all data to the University
system
Both systems will need to be reviewed to determine the best, most economical, usable and
expandable process. Advantages and disadvantages have been identified for both options.
The Secretary of State and the Boone County Clerk will work with LEA’s in determining the
preferred option based on current and future needs.
The State of Missouri contract for maintenance of the MCVR system is currently in an RFP
process. At this point we cannot say who that vendor will be or what the terms of the contract
will provide.
We know certain portions of these extracts will fall under the general terms of the vendor
maintenance contract (creation of extracts for currently stored data) and other components will
probably require additional payment to the vendor as special enhancements.
The State and the Ms. Noren have already started preliminary review of current and future fields
needed to accomplish either option so they will be prepared to quickly develop criteria for the
vendor to review and develop cost estimates.
University of Missouri – Columbia role
The major portion of this products, research and services arising from this grant will be provided
through the inter-governmental contract with the University of Missouri-Columbia. The
University of Missouri is one of 76 land grant institutions in the US and the state’s only higher
education research institution. By law and longstanding practice, the mission of the Missouri’s
flagship campus is to extend the benefits of its research capacity to all Missouri citizens and
communities.
16
The responsibilities and designated divisions are broken down as follow:
College of Engineering Computer Science Department
Reynolds Journalism Institute (RJI)
-Design, develop and implement online application and ballot delivery systems; develop
electronic communication systems for LEA’s and UOCAVA voters and maintain these systems
through 2018;
- Provide open source applications that could be available at no cost for migration and
implementation by any state or local government who can support its functions.
- Provide a low cost perpetual solution affordable to all Missouri jurisdictions after the
maintenance period is over;
- Research and develop mobile apps for extending the use of these systems to the broadest
number of UOCAVA voters;
- Research modern business technologies for data mining to for identifying and serve a specific
customer base and make recommendation for possible development of such technologies;
- Research, identify and develop partnerships with targeted groups to maximize usage of
products developed;
- Provide cyber security research, recommendations and implementation for all products
developed;
- Incorporate design elements that maximize usability and interface with assistive technology to
insure a positive voting experience for persons with disabilities;
- Research and develop lowest cost and most accurate method for processing electronically
delivered ballots that are returned by mail (i.e. votes embedded in barcode to scan);
- Research the potential future needs and security issues relating to electronic return of voted
ballots and make recommendations for potential areas of development;
- Other research topics that arise through summits organized to maximize usage and outreach;
- Research and recommend ways the products developed under this grant can be expanded to
disability groups or the general public to increase the return on investment;
- Assist in the development of return on investment criteria;
- Prepare necessary files and systems to meet the statistical reporting requirements of this grant;
- Work with designated personnel to develop and provide facilities for training programs for
LEA’s
These two divisions will provide a strong pool of experts to perform the above services and are
committed to providing all resources necessary to meet the outlined tasks.
RJI was established in 2004 with a $31 million grant and its mission is to use develop technology
for media and advertising use in promoting democracy. It has a one-of-a-kind technology
futures lab that is at the forefront in the development of new ways to keep citizens informed
through media and advertising.
The College of Engineering has state of the art facilities, technology experts and cyber security
personnel that can develop a safe, secure, usable solution to the problems faced by UOCAVA
voters.
17
Truman School of Public Administration
- develop and provide training for LEA’s on the systems developed
- work with College of Engineering to develop training materials and programs
- develop a lessons learned component for post 2012 election review
Boone County will contract separately under this grant for the training services provided by the
Truman School. Missouri LEAs and the Secretary of State have a longstanding relationship with
the Truman School personnel who have previously developed and provided both training
programs and “lessons learned” sessions. They have also managed one of the most successful
College Pollworker Programs in the country through a grant from the EAC.
They have previous knowledge of the kinds of training materials needed for the wide range of
people who conduct elections. They will review the products during design and testing phase
and work with the Boone County Clerk on the initial live processing phase to develop necessary
materials and arrange the training schedules. Approximately 150 LEA personnel will need
training on the new systems.
Usability testing:
The County will contract separately under this grant for use of the University’s usability testing
labs and personnel. The testing will be based on outside consultant recommendations for
usability testing of ballots.
Strategic Goals:
- To improve the ability of UOCAVA voters to apply, vote and have their vote counted;
-To provide low cost, usable, permanent process for Missouri LEA’s to provide electronic
application and ballot delivery tools to UOCAVA voters;
- Develop systems that will reduce the workload for LEA’s and improve the service to
UOCAVA voters;
- To incorporate sound usability testing and components that enhance the experience of disabled
UOCAVA voters;
- Develop tools to increase the probability the systems developed are used by UOCAVA voters;
- Develop partnerships to make the systems developed available to UOCAVA voters
- Research methods to identify and serve UOCAVA voters;
- Provide the strategies and research previously detailed in the University responsibilities
Current Analysis:
No Missouri county other than Boone County has any automated, web-based registration and
application system. All other counties must enter data from forms submitted.
The only electronic delivery method available to all counties but Boone is to individually address
and write an email, attach a pdf of the correct ballot and affidavits and send.
Justification for modification
These procedures are time consuming and have a higher rate of error than automated systems
18
These procedures discourage use of electronic ballot delivery
Process:
Each of the defined responsibilities contain the elements of processes to be used
Potential risks and mitigation factors:
Unless system is designed correctly it could cause another layer of work for LEA;
Unless designed correctly the user interface could cause a higher error rates by voters;
Collected and transmitting sensitive data across multiple platforms increase possibility of privacy
breach;
Mitigation factors:
Strong design elements that identify areas to reduce LEA workload
Strong usability testing to insure user interfaces do not create ballot errors
Create strong privacy by design and security controls into the system
Performance indicators:
Evaluation of statistics on ballots applied, sent received and comparison to prior data.
Voter satisfaction surveys
LEA satisfaction surveys
Projections for effectiveness of modifications:
Boone County had 90% participation rate when voters given were the option for electronic
delivery of ballots;
Boone County had reduced data entry errors when voters were allowed to enter their own data;
Boone County saw a reduction in staff time need to process applications and ballot delivery
packets;
Development of ballot tabulation method for returned ballot will reduce labor cost to transfer
voted paper ballot to scan ballots.
19
1. Current and Pending Project Proposal Submissions
Boone County Clerk currently has the following grant agreement:
Title of Proposal and Summary:
EAC Logic and Accuracy Grant – develop a toolkit for automating the pre-election logic and
accuracy test process
Source and amount of funding: US EAC $25,000 through 12/31/2012
Percentage of effort to funding: < 10%
Prime Applicant: Wendy S. Noren Boone County Clerk
Technical Contact:
Wendy S. Noren Boone County Clerk 801 E Walnut Rm. 236 Columbia Mo.
573-883-4295 573-886-4296
Period of performance June 1, 2011 through Dec 31, 2012
Award period – same as above
Labor hours devoted to project – 280
How projects are related: Data elements extractions pulled for Logic and Accuracy Grant will be
same as data elements extractions pulled for this grant
Data extractions are used to create random ballot selections for logic and accuracy testing
20
Qualifications: Project Director
Ms. Noren has been managing elections in Boone County Missouri since 1978 and designed,
developed and programmed all of the voter registration software utilized in Boone County and
all programs referenced in this proposal including: programs incorporating voter web
transactions for registration and absentee application to the registration database, automated
email notifications of application and ballot status and secure transfer voter data for to and from
the state voter database. In addition she had designed as worked with county web programmers
who coded all of the front end voter entry interfaces for voter registration and absentee
application.
In 2010, Ms. Noren designed the specifications for Missouri’s participation in the FVAP EVSW
pilot program was responsible for programming all of the data extractions and FTP secure to the
FVAP vendor to use in providing the online ballot marking system. Approximately 70% of
Boone County UOCAVA voters utilized the system even though the final version from the
vendor was not available until a week after they received the PDF version of their ballot. Based
on the comments of users and Ms. Noren’s own experience with the system, she believes this is
the best method for providing electronic delivery of ballots to UOCAVA voters and it is
important to expand its use to all Missouri jurisdictions.
Ms. Noren has also had extensive experience on a state and national level in researching and
recommending policy and procedures for elections officials. Specific to this project is work on
the Board of Advisors to the US EAC and its subcommittee reviewing voting system standards
and the National Academy of Science State Database Interoperability study. These projects have
given her extensive contact with leaders in all fields relating to integration of new technology to
the election process.
Biography attached
Also attached:
Sub-contractor CV:
Dale Musser – Project Director for University of Missouri – Columbia
BUDGET PROPOSAL
COSTS ASSOCIATED WITH DELIVERABLES
1. Direct Labor
Project Director – 200 hours 39.45 per hour 82,061 annual
7,800
Hours:
Design Development – 40 hrs
Coding and Testing - 60 hrs
Research Review - 40 hrs
Administration and Reporting – 60
Legal Counselor
819.00
40.95 per hour 20 hours $85,176 annual
Contract development and review – 20 hours
Administrative and Clerical
100 hours at 12.00
Account payment processing and general clerical (copies, correspondence)
Total 1200.00
Fringe Benefits – county does not have negotiated F&A
1,137 Project Director
112 Legal Counsel
1,249 Total
Calculated by:
11,966 annual for benefits total
Project director @ 9.5% of total annual
Legal counselor @ .96% of total annual
Travel 16,500
Training on new system requirements in Columbia Mo. – participants will be election
authorities from all over state.
150 at 110 per person
Based on CONUS lodging, breakfast and lunch MIE
Training sessions for election officials on new programs developed under grant
Subcontractor
University of Missouri – Columbia 500,948
Proposal and justification attached
Additional:
University of Missouri
Training: 30,000
To be negotiated after training modules defined. Pricing based on prior services cost
Training sessions over varying period for 150 people in computer labs
Design development of training materials
University of Missouri - Usability lab and testing program – 30,000
To be determined after usability testing program finalized
Estimate provided based on prior usage and projected test modules
State of Missouri Secretary of State
75,000
Amount will reimburse the state selected vendor who will provide maintenance and
modifications to state voter registration software.
Amount is estimate until:
1. State completes its RFP process and finalizes contract
2. We finalize the design specifications for new procedures
3. Modifications are submitted to vendor to negotiate items covered under general maintenance
(no charge) or if they are modifications not covered under general maintenance. Modification
costs will be billed to state and reimbursed by grant funds.
Consultants:
Consultants can not be contracted for until grant funds approved and purchasing policy followed.
We anticipate these will be sole source service contracts but the scope of work will have to be
reviewed under purchasing policies:
Consultants will be used for the following purposes:
#1 - $12,000
Usability test design consulting – need to have someone who has specific research, design and
testing with ballots and election related materials.
Cost based on prior estimate maximum 1500 per day.
#2 – $10,000
Develop reporting system, assist with compiling and review research reports, and assist with
survey review and consolidation
Current election schedule (4 elections in 8 months) will make it impossible to compile and
submit necessary reports from project director. Need someone with experience in monitoring
and reporting election costs and translating that into public policy goals.
Based on cost estimate – rate and terms will be determined in contracting phase
#3 – $5,000
Need consultation on assistive technology issues to advise system developers
Contract to be negotiated if appropriately funded
Materials and Supplies:
Training manuals – 150 at 40.00 per unit. Based on prior pricing of training materials.
Other Direct Costs:
$35,000
Ballot Converter system (1) – will be used to test cost effectiveness of one option for tabulating
returned voted ballots. Includes software and hardware for ballot conversion to optical scan
ballot. Quote from vendor (Advance Ballot Solutions) will be subject to County RFP
requirements unless sole source.
$5,000 Reimburse County of Boone for additional audit costs for under major program audit
requirements. County would not have these costs without the grant. Request is based on cost of
last major program audit.
ADDITIONAL FUNDING REQUEST FOR HOSTING SEMINARS
The research and outreach components of this grant would be greatly enhanced through the
convening of 3 small group (no more than 8) seminars with experts in the certain areas of
concentration. They would actually be work sessions that would allow researchers to have
access to expertise to focus research and outreach concepts. We believe these are key to the
success of the overall program.
Group 1 – expertise in data mining and commercial data collection – guide research on ways to
identify and serve UOCAVA voters. What tools are businesses using to identify and reach their
customer base through data mining. When I book a flight to XYZ city how am I getting emails
from rental car companies with deals for that city. Can we mine that same kind of data, what
data should we target and how much does it cost
4-5 people
Possible cost:
25,000 – We may have to pay this group to participate
Group 2 – representative of groups that are 3rd party application providers (i.e. Overseas Vote
Foundation, League of Women Voters)
It does us no good to have forms that collect and merge data with our database if people go to
these sites – how do we get them to help us get the data we need.
4-5 people
Possible cost:
7,500 travel etc.
Group 3 - Google Facebook Linkedin Twitter etc (maybe PEW with this group)
Can we utilize their resources to target the groups we need to service?
Could we use Google as the host site for 3rd party users to post the application data they collect
to merge with our data? What technology do they have that we can use.
Example – can we get to a Facebook person who was born in Columbia Mo and is now in
Prague?
4-5 people
Cost could be 25,000
All of these we would like to get for free. We would like to negotiate this with FVAP and get
the best brains for the lowest price.
BUDGET PROPOSAL
COSTS ASSOCIATED WITH DELIVERABLES
1. Direct Labor
Project Director – 200 hours 39.45 per hour 82,061 annual
7,800
Hours:
Design Development – 40 hrs
Coding and Testing - 60 hrs
Research Review - 40 hrs
Administration and Reporting – 60
Legal Counselor
819.00
40.95 per hour 20 hours $85,176 annual
Contract development and review – 20 hours
Administrative and Clerical
100 hours at 12.00
Account payment processing and general clerical (copies, correspondence)
Total 1200.00
Fringe Benefits – county does not have negotiated F&A
1,137 Project Director
112 Legal Counsel
1,249 Total
Calculated by:
11,966 annual for benefits total
Project director @ 9.5% of total annual
Legal counselor @ .96% of total annual
Travel 16,500
Training on new system requirements in Columbia Mo. – participants will be election
authorities from all over state.
150 at 110 per person
Based on CONUS lodging, breakfast and lunch MIE
Training sessions for election officials on new programs developed under grant
Subcontractor
University of Missouri – Columbia 500,948
Proposal and justification attached
Additional:
University of Missouri
Training: 30,000
To be negotiated after training modules defined. Pricing based on prior services cost
Training sessions over varying period for 150 people in computer labs
Design development of training materials
University of Missouri - Usability lab and testing program – 30,000
To be determined after usability testing program finalized
Estimate provided based on prior usage and projected test modules
State of Missouri Secretary of State
75,000
Amount will reimburse the state selected vendor who will provide maintenance and
modifications to state voter registration software.
Amount is estimate until:
1. State completes its RFP process and finalizes contract
2. We finalize the design specifications for new procedures
3. Modifications are submitted to vendor to negotiate items covered under general maintenance
(no charge) or if they are modifications not covered under general maintenance. Modification
costs will be billed to state and reimbursed by grant funds.
Consultants:
Consultants can not be contracted for until grant funds approved and purchasing policy followed.
We anticipate these will be sole source service contracts but the scope of work will have to be
reviewed under purchasing policies:
Consultants will be used for the following purposes:
#1 - $12,000
Usability test design consulting – need to have someone who has specific research, design and
testing with ballots and election related materials.
Cost based on prior estimate maximum 1500 per day.
#2 – $10,000
Develop reporting system, assist with compiling and review research reports, and assist with
survey review and consolidation
Current election schedule (4 elections in 8 months) will make it impossible to compile and
submit necessary reports from project director. Need someone with experience in monitoring
and reporting election costs and translating that into public policy goals.
Based on cost estimate – rate and terms will be determined in contracting phase
#3 – $5,000
Need consultation on assistive technology issues to advise system developers
Contract to be negotiated if appropriately funded
Materials and Supplies:
Training manuals – 150 at 40.00 per unit. Based on prior pricing of training materials.
Other Direct Costs:
$35,000
Ballot Converter system (1) – will be used to test cost effectiveness of one option for tabulating
returned voted ballots. Includes software and hardware for ballot conversion to optical scan
ballot. Quote from vendor (Advance Ballot Solutions) will be subject to County RFP
requirements unless sole source.
$5,000 Reimburse County of Boone for additional audit costs for under major program audit
requirements. County would not have these costs without the grant. Request is based on cost of
last major program audit.
ADDITIONAL FUNDING REQUEST FOR HOSTING SEMINARS
The research and outreach components of this grant would be greatly enhanced through the
convening of 3 small group (no more than 8) seminars with experts in the certain areas of
concentration. They would actually be work sessions that would allow researchers to have
access to expertise to focus research and outreach concepts. We believe these are key to the
success of the overall program.
Group 1 – expertise in data mining and commercial data collection – guide research on ways to
identify and serve UOCAVA voters. What tools are businesses using to identify and reach their
customer base through data mining. When I book a flight to XYZ city how am I getting emails
from rental car companies with deals for that city. Can we mine that same kind of data, what
data should we target and how much does it cost
4-5 people
Possible cost:
25,000 – We may have to pay this group to participate
Group 2 – representative of groups that are 3rd party application providers (i.e. Overseas Vote
Foundation, League of Women Voters)
It does us no good to have forms that collect and merge data with our database if people go to
these sites – how do we get them to help us get the data we need.
4-5 people
Possible cost:
7,500 travel etc.
Group 3 - Google Facebook Linkedin Twitter etc (maybe PEW with this group)
Can we utilize their resources to target the groups we need to service?
Could we use Google as the host site for 3rd party users to post the application data they collect
to merge with our data? What technology do they have that we can use.
Example – can we get to a Facebook person who was born in Columbia Mo and is now in
Prague?
4-5 people
Cost could be 25,000
All of these we would like to get for free. We would like to negotiate this with FVAP and get
the best brains for the lowest price.
