1. education
2. school

New Approaches for Evaluating and Improving the
Efficacy of Digital Courseware: Request for Proposal
Application Deadline: March 20, 2015 – 5:00 pm (PST)
Overview
Introduction
In 2013, The Bill & Melinda Gates Foundation invested $2 million in new approaches to facilitating short-cycle feedback
so that teachers, school decision-makers, and parents have better information about the effectiveness of digital
courseware, and developers of such products could receive rapid input to improve their offerings. That investment
supported the launch two types of new initiatives:


Test bed networks, organizations that support schools in piloting innovative courseware products and help
schools and entrepreneurs to collaboratively define a structured process to evaluate and improve the efficacy of
digital courseware; and
Efficacy platforms, technology platforms that use validated diagnostic and formative assessments to facilitate
rapid iteration and improvement of digital content and tools.
To date, these initiatives have supported more than 50,000 students and 2,000 teachers in piloting new courseware.
Based on that success, we are requesting proposals for a second cohort of test bed networks and efficacy platforms that
can help improve digital courseware’s efficacy in:
1. Student Learning: effective digital courseware should not only enable students to achieve explicit learning objectives
but also accelerate the pace at which they do so.
2. Teacher Support: effective digital courseware should be easy to implement and integrate into instructional plans,
saving teachers time and helping them support their students’ learning needs.
3. Engagement: effective digital courseware should provide for a rich, compelling, and interactive experience that will
encourage repeated, prolonged, and self-motivated use.
4. Satisfaction: effective digital courseware should sufficiently meet student and teacher needs and provide for a
consistent, inviting, and intuitive experience.
This document is an invitation for proposals for rapid digital courseware evaluation and improvement approaches that:




Focus the development and adoption of personalized learning products on helping students achieve desired
learning outcomes.
Put teachers and school decision-makers at the center of the shift toward personalized learning.
Lower risks and barriers to all parties of adopting new, potentially transformative products.
Encourage the rapid development of a healthy, transparent market for highly effective learning technologies.
Page 1 of 15
© 2013 Bill & Melinda Gates Foundation
Request for Proposal
Vision for Personalized Learning
Our vision for education is that all students have learning experiences that are tailored to their individual needs, skill
levels, and interests. For some, this will mean accelerating instruction beyond the fixed pace of many of today’s
classrooms. For others it will mean being given opportunities for extended practice of skills or content they have yet to
master. For all students, it will mean learning experiences that are tailored specifically to their progress against a clear
set of academic standards; receiving constant, actionable information on their progress; experiencing more one-on-one
engagement with their teacher and developing deep connections with other students.
Implementing this vision, which we and many others call personalized learning, at scale will require new tools to help
teachers, parents, and students diagnose gaps in knowledge and skills and adapt learning experiences based on
student progress. Technology can help support effective teachers, by giving them new ways to design and provide
personalized instruction to their students. Comprehensive courseware products such as Carnegie Learning’s Cognitive
Tutor, Reasoning Mind, ST Math, and Dreambox are facilitating significant learning gains for students in even the
highest-need schools, while apps like Refractions, DragonBox, and Motion Math are demonstrating that even rigorous
academic content can be highly engaging. Our goal is for all students to have ready access to these types of tools,
enabling them to learn what they need, when they need it by personalizing content, learning experiences, and support in
real time.
Market Context
Fueled by the growth of personalized learning schools, technological advances, and an influx of new talent, “start-up
companies and individuals are developing digital learning resources at a rapid pace.” (US Department of Education,
2013). At the same time, tech-savvy teachers are increasingly seeking products to support their efforts to personalize
learning and improve student outcomes. More than 90% of all teachers in a recent Harris Interactive survey indicated
that “they would like to use more education technology in the classroom than they do now” (Harris Interactive, 2013);
however, the market is not yet rich enough with choices for pioneers implementing personalized learning, nor is
information on the impact of digital courseware on student learning readily available enough to meaningfully drive
purchasing decisions or to support the transition to a performance-driven market. As a result, many incumbent providers
are able to compete based on the strength of their sales and marketing efforts alone, while innovative players with
promising products struggle to achieve sustainability and scale.
We envision a healthy, rational US K-12 digital courseware market characterized by effectiveness, affordability, and
transparency. In such a market, vendors would have ready access to short-cycle efficacy data and information that
would be used to drive continuous product improvement. In turn, teachers and school decision-makers would use
standardized, easily comparable efficacy data in deciding whether to continue using existing products or to adopt new
ones. Making this data and information readily available would help evolve the market to one in which school decisionmakers are well supported in identifying the most effective tools for student learning, vendors develop and compete
primarily based on performance, and all students have ready access to highly effective personalized learning tools.
One way in which we support these types of market shifts is by providing resources that cast a light on gaps and
opportunities that are critical to realizing the vision of personalized learning for all students. Over the past year, our
analysis of the market – informed by input from leading teachers and school decision-makers, innovative vendors, and a
range of experts – identified three major shifts required to move towards a transparent, performance-based, and
responsive K-12 digital courseware market:
1. A dramatic increase in the number of effective personalized learning products, particularly in high-need areas such
as literacy. While investment in education has increased dramatically over the past several years, critical areas of
the market remain unserved. For example, of the $427 million in venture funding provided for K-12 education
entrepreneurs in 2012, only 6% of it was invested in digital instructional content (NewSchools Venture Fund, 2012).
2. Providing readily available, independent information on learning technologies that allow teachers and school
decision-makers to discover new products, quickly identify product alignment to standards, and share feedback on
learning technologies based on personal experience. Nearly 80% of teachers agree that “it’s at least somewhat
difficult to find out about high quality educational technologies,” and nearly a third of all teachers spend an hour or
more each week researching which tools to use (Harris Interactive, 2013).
3. The emergence of lightweight, rapid feedback loops between teachers, school decision-makers and entrepreneurs
that use student learning and engagement data to drive decision-making and continuous improvement. The federal
Page 2 of 15
© 2013 Bill & Melinda Gates Foundation
Request for Proposal
What Works Clearinghouse was designed to show what programs and products are most effective. However, due
to its methodological requirements, it does not sufficiently address the needs of schools or vendors. Increasing the
speed and transparency with which valid data is produced and made available should increase incentives for
vendors to improve the performance of their products.
Our Literacy Courseware Challenge invested $6 million to help facilitate the first shift. Our support of organizations that
provide ready information on digital learning products, such as Graphite, EdReports, and EdSurge, seeks to facilitate the
second shift. This proposal will seek to facilitate the third shift.
Thus, we are requesting proposals for new approaches to facilitating short-cycle feedback on product efficacy to both
developers of innovative, digital courseware and the students, teachers, and school decision-makers they serve.
Detailed information on the problem we seek to solve, our definition of product efficacy, grant awards, and eligibility is
contained within this request for proposal.
Scope and Approach
This request for proposal will support a three-year grant program targeting product entrepreneurs, market catalysts, and
school networks, or any partnerships or combinations of these parties. The Bill & Melinda Gates Foundation does not
envision being a sole funder of these initiatives during the two-phase process or thereafter. Applicants should have the
ability and inclination to generate additional funds and to support on-going operations.
Solution Requirements
This RFP was developed by the Bill & Melinda Gates Foundation with the input and guidance of an advisory group
composed of eight innovative school and non-profit leaders, and entrepreneurs. While the content and viewpoints
expressed herein are those of the foundation, the vision and solution requirements for the RFP were shaped through
extensive research on existing solutions and in-depth interviews with the advisory group.
Throughout 12 interviews, advisory group members consistently described product development and piloting as an
ambiguous, time-consuming, and often isolated process that poses large risks to schools and creates a very high barrier
to entry for entrepreneurs with potential solutions. Faced with an increasing number of product options, school decisionmakers reported both a lack of data on what products work for which students under which circumstances and a lack of
means to quickly evaluate whether a given tool is the right fit for them – a particular concern at a time when lean school
budgets have only heightened the need to justify and assess the return on any investment in technology.
Entrepreneurs, in turn, cited barriers including limited feedback channels and an insufficient number of teachers and
students to building effective products that satisfy school needs.
Both groups agreed that currently available data sources were insufficient for evaluating product efficacy, in particular a
product’s impact on student learning. Among the major limitations they cited are:




The rigorous effectiveness studies that have traditionally produced this data are time-consuming, expensive, and
unable to keep pace with the abundance of emerging tools. For example, the federal What Works Clearinghouse
lists fewer than 50 K-12 literacy interventions that show positive or potentially positive impact on student learning.
Data produced by existing methods is often based on infrequent interim or end-of-year tests that assess student
performance at a single point in time, forcing school decision-makers to rely on data that is inconsistently available
and lags far behind ongoing product improvements.
Product efficacy data from credible sources typically provides only a single dimension of product quality and
statements about large, general populations. This type of data is useful in assessing a product independent of
context, but is not helpful to school decision-makers who need to understand how a particular product is affecting
an individual student’s learning and take immediate action based on this information.
Self-reported vendor claims of efficacy cannot be easily evaluated and corroborated. This leads to even validly
produced studies being viewed with distrust, and schools in some cases unnecessarily duplicating efforts by
contracting with independent evaluation firms to conduct their own efficacy studies.
Page 3 of 15
© 2013 Bill & Melinda Gates Foundation
Request for Proposal

Vendor-produced efficacy data is often presented in unwieldy, hard-to-interpret, and inflexible formats, eroding its
usefulness for already-oversubscribed school staffs.
School Needs
In a recent survey of more than 1,000 teachers, the most cited reason for frustration with technology in the classroom
was that “they don’t do exactly what I want” (EdShelf, 2013). Still, nearly all teachers “agree they would like to use more
education technology than they currently do,” citing its potential positive impact on student outcomes and personalizing
learning (Common Sense Media, 2013). In order to achieve this, schools need technologies that are both efficient and
effective at delivering personalized learning experiences, and that make the best use of school and teacher time, effort,
and expertise. More than ever, teachers and school decision-makers need practical, credible evidence on the
effectiveness of new tools – and that evidence’s applicability to their particular needs before adopting the tools. As a
part of that evidence, they need to understand how to implement new tools based on prior success, their unique student
needs, and fit between test conditions and real classroom environments. Post-implementation, teachers need regular
data showing whether the tools they are using are having a positive impact on student learning and engagement. If
something is not working optimally, they need to have at their disposal built-in feedback channels with entrepreneurs to
help diagnose and address those deficiencies through product refinements in a timely manner.
Vendor Needs
Entrepreneurs tell us that these shortcomings pose a Catch-22: their products are often not trusted by schools due to a
lack of independent, long-term evidence of efficacy – but they also cannot get their products used by a critical mass of
people to generate the evidence that they need. Furthermore, while entrepreneurs are increasingly adopting an agile
product development process that creates opportunities to refine and strengthen products in response to feedback in a
matter of weeks, the lack of real-time, product efficacy data makes this process at best one guided primarily on
engagement alone rather than actual student learning. As a result, some entrepreneurs build products that are
engaging enough for individual teachers to employ in classrooms as an instructional change of pace, but that are not
optimized for student learning. When successful, this approach generates an initial surge in use, but often fails to build
an evidence base that justifies wider deployment. Other entrepreneurs focus immediately on demonstrating impact on
student learning, but struggle either to test their products with enough students to generate statistically significant data
or to show clear, positive impact without incurring major expenses from hiring independent evaluators. In both cases,
innovative entrepreneurs struggle to establish themselves in the market and to compete against incumbents.
Entrepreneurs, therefore, need a critical mass of pilot teachers and students to help inform and validate their product
efforts. They also need to be able to lower the risk of trying new, untested products by quickly proving to schools that
they have the processes in place to gather and provide credible, actionable efficacy data at intervals short enough to
support short-cycle adjustments to student learning experiences without compromising the entrepreneur’s own ability to
test and refine products. As adoption increases, entrepreneurs need robust mechanisms within their product
development processes to continuously measure efficacy, to learn about the effects of different implementations on
student learning and engagement, and to rapidly iterate and optimize their products in response to both qualitative and
quantitative feedback. Rapid cycles of modification, analysis of results, and redesign are “key to producing dramatic
change while reducing risk” (Tony Bryk, 2011).
Solutions Requested
To address these needs, we are requesting proposals for test bed networks, platforms that utilize A/B testing with
validated assessments, or solutions combining the two. Successful proposals will address at least one of the solution
descriptions below and will rate highly against the evaluation criteria below.
Test Bed Networks
Forums that bring teachers and entrepreneurs together in the hopes of facilitating collaborative product development are
increasingly being convened across the country by organizations such as the Education Technology Start-up
Collaborative. These events typically feature teachers “test-driving” personalized learning products across various
stages of development for free or at a reduced price and shaping the product development process by offering valuable
feedback to entrepreneurs. Events such as these are a step in the right direction, but they are limited in continuity and
scope of impact. A sustained test bed network with consistent and frequent interactions between teachers, their
Page 4 of 15
© 2013 Bill & Melinda Gates Foundation
Request for Proposal
students, and entrepreneurs on a larger scale – in either an online or offline medium – can facilitate a meaningful
feedback loop about the qualitative and quantitative dimensions of product efficacy. For entrepreneurs, these networks
would provide a range of benefits that would help them compete more effectively in the market and increase their
chances of success, including: access to a critical mass of pilot teachers and students; the feedback necessary to build
more effective products; the ability to generate early evidence of their products’ impact on student learning and
engagement, and greater teacher satisfaction. For schools, these networks would create a way to quickly determine
which products merit further exploration, greater influence on product design, and better, more regular data about the
effectiveness of the learning technologies used in their classrooms.
As a part of these test bed networks, schools and entrepreneurs would collaboratively define a structured process to
evaluate and improve the efficacy of digital courseware, including: (1) the implementation model for each product tested,
(2) the format and frequency of any requisite professional development, (3) the specific measures of product efficacy to
be evaluated and by whom, (4) the diagnostic and assessment to be administered, (5) the nature of data reporting and
data-sharing agreements between school and vendor, and (6) the format and structure of the feedback loop.
Platforms (Utilizing A/B Testing with Validated Assessments)
While software programs such as Google Analytics and Mixpanel have provided an effective means by which to digitally
measure engagement and satisfaction in web-based products, the largest gap when it comes to evaluating product
efficacy remains in measuring a product’s impact on student learning.
The randomized controlled trial (RCT) has long been recognized in many fields, including education, as the gold
standard for evaluating the efficacy of an intervention. While this experimental model is viewed by researchers as the
most conclusive means by which to establish a causal relationship between a given practice and a learning outcome, in
its traditional form it poses a number of often-insurmountable challenges in schools, including in a digital learning
context. Namely, RCTs can be cost-prohibitive for resource-starved schools and entrepreneurs, costing hundreds of
thousands of dollars (Coalition for Evidence-Based Policy, 2012), and can often take years to fully execute, a timeline
which is incompatible with the velocity of change and decision-making in technology and education. Random
assignment of students to treatment and control groups is also very difficult to accomplish, given the way schools
typically assign students to teachers and programs. For these reasons, fully leveraging the transformative potential of
innovative, personalized learning products must first involve “accepting that the strongest level of causal evidence is not
necessary for every learning resource decision" (US Department of Education, 2013).
A/B testing, a widely-practiced product development heuristic employed by many education entrepreneurs, captures the
essence of random assignment in RCTs and allows for systematic comparison of different product features – without the
prohibitive cost or timeline required. Under this method, two randomly-assigned groups of students are given versions
of the product that vary in a single, deliberate manner – thus, allowing comparisons to be drawn between the
experiences and outcomes of the two groups, and influential product changes identified. For early-stage entrepreneurs
with new products and teacher and student bases insufficient for large-scale, multi-site RCTs and schools with short
decision-making timelines and low tolerance for risk, a collaborative form of A/B testing utilizing validated diagnostic and
formative assessments represents the ideal vehicle with which to carry out small-scale efficacy studies. The information
gleaned from these studies can serve as crucial inputs into the product development and purchasing processes in the
near-term by proving that impacts on student learning are maintained even outside of the specific product environment.
In the long-term, it will also form the foundation upon which larger-scale experiments can be performed to demonstrate
fidelity of results across a broader diversity of teachers, students, and contexts.
Page 5 of 15
© 2013 Bill & Melinda Gates Foundation
Request for Proposal
Exhibit 1: Evolution of Efficacy Studies
Use Cases
The scenarios depicted below are representative of our overall vision (Phase I + Phase II). They are presented here as
potential examples of proposals to help guide potential respondents only and should not be perceived as prescriptive or
limiting in any fashion.
Test Bed Networks
Better Schools for All (BSA), a non-profit organization that supports the expansion of high-performing charter schools,
submitted an application for Phase I of the New Approaches to Evaluating and Improving the Efficacy of Digital
Courseware RFP. BSA, whose network of twenty schools nationally serves a population of over 8,000 students, had
been thinking about a systematic means of evaluating the efficacy of a seemingly ever-increasing pipeline of new digital
learning products up for consideration for use in classrooms. Intrigued by the vision of a test bed network that would
facilitate the structured piloting of digital courseware in classrooms, connect teachers and students with entrepreneurs in
a mutually-beneficial way, and ultimately enable rapid decision-making based on product efficacy data, BSA formed a
working team composed of a select group of entrepreneurs, school decision-makers, and teachers, to begin exploring a
potential solution. Over a series of planning meetings, the team agreed upon the following parameters of the test bed
network:






Implementation format for each product (station rotation blended model, 30 minutes a session, two times per week)
Professional development format and frequency (half-day, in-person, vendor-facilitated overview combined with justin-time online PD as needed)
Efficacy metrics to be tracked and evaluated (impact on student learning, teacher support, student engagement, and
student/teacher satisfaction)
The external measure of impact on student learning (a standards-aligned, initial diagnostic and formative
assessment already in use presently)
The content, level of detail, and visual format of data outputs for all stakeholders
The structure of the feedback loop (weekly reports on student engagement, monthly reports on student/teacher
satisfaction as measured by surveys, monthly reports on impact to student learning, real-time IT/trouble-shooting
support)
The team applied for and won a Phase 1 grant for a six-month pilot of the solution for three new literacy products, across
ten sixth-grade classrooms within the BSA network.
Page 6 of 15
© 2013 Bill & Melinda Gates Foundation
Request for Proposal
For entrepreneurs, the pilot generated a continuous stream of valuable feedback from real teachers and students that
could be used to improve the product experience. It also exposed inconsistencies between how the products were
designed to be implemented and how they were actually being utilized in the classroom. For teachers, the pilot provided
a way to quickly assess how effective new products actually are in the classroom, and an opportunity to help shape the
product development process as well. For school decision-makers, the pilot revealed significant and persistent relative
differences in student learning gains and engagement across the three products, allowing them to make faster decisions
about which products to explore further or discontinue. Based on the success of this initial pilot, BSA is currently
seeking Phase II funding to deploy the solution across its entire portfolio of schools.
Platforms
In hopes of building a decision-making tool for teachers and school decision-makers to quickly evaluate the efficacy of
digital courseware products in a systematic fashion, DecisionEngine, an enterprise software company specializing in
data analytics, submitted an application for Phase 1 of the New Approaches to Evaluating and Improving the Efficacy of
Digital Courseware RFP. The proposal was for ProductAssess, a web-based evaluation system that would:





Integrate with web-based products so as to be able to measure student learning and engagement resulting from use
of those products
Create a student portal within which students would be able to securely log-in, access, and use the digital products
under evaluation, and subsequently complete standards-aligned diagnostics and formative assessments
Create teacher/administrator/parent portals within which stakeholders would be able to access actionable data on
student engagement and learning gains in short cycles, and in both granular and aggregate levels of detail as
needed
Provide continuous feedback on product efficacy and fidelity of implementation to personalized learning vendors,
who can then quickly translate the feedback into product refinements/enhancements
Have the capacity to perform automated randomized assignment tests on a large scale
After winning the Phase I grant and building out a beta version of ProductAssess, DecisionEngine was able to secure a
six-month pilot with a large, urban public school district and a subset of its portfolio of product vendors. Though this
initial pilot was insufficient in scope to generate any statistically-definitive conclusions, the data suggested that there
were indeed significant relative differences in student engagement and learning gains across the various digital learning
products currently employed by the district. It also established for school decision-makers a standardized framework
with which to measure product efficacy, provided useful feedback for ProductAssess improvements back to
DecisionEngine, and provided information regarding fidelity of implementation back to product vendors.
DecisionEngine is currently seeking Phase II funding for a national launch. With the efficacy data gleaned from a
broader cross-section of schools and vendors, DecisionEngine hopes to utilize the full capability of ProductAssess to
conduct automated randomized assignment tests in order to:



Quickly assess the efficacy of newly-introduced personalized learning tools
A/B test new product features/enhancements
Recommend different implementations of products based on data from other schools
Page 7 of 15
© 2013 Bill & Melinda Gates Foundation
Request for Proposal
Rules & Guidelines
Eligibility
Funding Criteria
For this RFP (Phase I), we are targeting product entrepreneurs, market catalysts, and school networks/districts - or
partnerships or combinations of these parties – with proposals for the following:

Test Bed Networks: collaborative partnerships between schools and entrepreneurs that facilitate broad usage of
digital courseware by teachers and their students in order to build a continuous feedback loop on product efficacy for
school decision-makers and entrepreneurs.

Platforms: technology products that will support data collection on digital courseware in order to assess product
efficacy and that can support large-scale, short-cycle A/B testing.

Combination: solutions that combine Test Bed Networks with Platforms. While we anticipate that most respondents
will seek to address either a platform or test bed network in their proposals, special consideration will be given to
organizations that – whether on their own or through proposed or already established partnerships – can deliver on
a comprehensive solution that combines both.
We will consider only those proposals that adequately address all four components of product efficacy as defined in this
document, with the heaviest emphasis on impact on student learning.
We encourage for- and non-profit organizations with demonstrated capacity for or promise of undertaking the work
described herein, to respond. Teams proposing partnerships are required to submit a single application and identify the
financial recipient of the grant funds.
Given the nascent role of product efficacy efforts in the K-12 education ecosystem currently, we will evaluate and grant
awards to ideas both in the Concept Development and Testing/Deployment stages.
Type of Award
Stage of Development
Concept Development
New concepts pre-beta testing
Definition
In use by <10 teachers and 300 students
Size of Award
Number of Awards
Grant Period
Up to $100K
Up to 6
12 months
Purpose of Grant
To facilitate concept development and
testing in preparation for pilot
Testing/Deployment
School-ready products
In use by >10 teachers and 300 students and
poised to grow
Up to $350K
Up to 4
12 months
To facilitate solution refinement and pilot in
preparation for broad implementation and
independent, third-party evaluation
Do I retain intellectual property ownership?
Yes, all entrants, regardless of winner status, will retain intellectual property ownership of their software applications.
However, winners must agree to support global access objectives by conducting and managing the project and
information in a manner that enables (a) the knowledge gained during the project to be promptly and broadly
disseminated and (b) the intended products to be made available and accessible for free or at reasonable cost to the
people most in need. For more details, please see the Representative Terms and Conditions that will be included in the
grant agreement.
Page 8 of 15
© 2013 Bill & Melinda Gates Foundation
Request for Proposal
Exclusion Criteria
We will NOT consider funding for:







Respondents that do not propose either a platform and/or test bed network solution
Proposals that fail to adequately address all four dimensions of product efficacy (as defined in this document)
Single LLCs, or individuals not affiliated with a for- or non-profit company
o If you are an LLC, please provide evidence that you're taxed as a partnership, or that you've made an
election to be taxed as a corporation
Respondents without an existing, minimally-viable solution
Individual schools
Isolated, one-off events bringing together entrepreneurs and school leaders (e.g. “School Start-Up Days”)
Proposals with budgets >$350,000 or >12 months in duration
Evaluation Criteria
Submissions received by the deadline will be evaluated on the following criteria and rubric:
Scoring Criteria
Alignment with Objectives
Potential for Impact
Capacity to Deliver
TOTAL
Points
25
50
25
100

Alignment with Objectives (25%)
o Fully addresses the problem statement and vision raised herein
o Proposes to adequately and comprehensively measure all dimensions of product efficacy (as per the
definition described herein)
o Will serve a diverse cross-section of schools, teachers, and students
o Is affordable and accessible to all schools

Potential for Impact (50%)
o Establishes robust, continuous, and scalable feedback channels and processes between stakeholders
o Integrates diagnostics and formative assessments that are broadly-applicable and have been validated by
instructional leaders and the research community
o Includes short-cycle reporting interfaces and outputs that are user-friendly, flexible, intuitive, and actionable
o Proposes a realistic implementation plan that requires minimal training and IT support by vendors and/or
pilot schools
o Utilizes technology that is interoperable, secure, scalable, and has the analytics capability to perform largescale A/B testing (if applicable)

Capacity to Deliver (25%)
o Led by a team with the skills and experience necessary for the proposed work
o Proposes a work plan that can be realistically executed on time and within budget
o Attests to the long-term financial health, sustainability, and self-sufficiency of the organization
o Engenders confidence that the team has the human and financial capital to viably scale the solution if it
succeeds in pilot testing
o Demonstrates buy-in from proposed partners (if applicable)
Page 9 of 15
© 2013 Bill & Melinda Gates Foundation
Request for Proposal
Activities & Timeline
February 18, 2015: Request for Proposal (RFP) Open.
February 24, 2015 – 1:00 PM (PST): Q&A call/webinar.
March 20, 2015 – 5:00 PM (PST): Completed applications due.
April 2015: Finalists notified.
June 2015: Countersigned Grant Agreements received.
Note: Grant awards will be contingent on the successful completion of due diligence processes.
How to Apply
Response Requirements
To apply, please submit a proposal using the ‘Proposal Narrative’ template that includes the following:

Summary (1/4 page or less):
o Award category: Concept Development or Testing/Deployment
o Amount requested ($)
o Brief description: 1-2 sentence summary of your proposal

Organization (3 pages or less): Please tell us about the people and organization(s) that will develop the solution,
and provide:
o Background about how long the organization has been in operation, its mission, and any products that are
already available
o A list and description of key existing or pipeline users of the solution
o Historical information on number of existing users currently supported and anticipated growth trajectory
o A list of all relevant team members complete with biographical information. Describe how your team
members know one another and specify the activities each person will be involved in
o An explanation for why this team has the unique capability to develop and implement this solution

Solution Description (5 pages or less): Please provide a detailed description of the platform and/or process you
aim to develop, and address the following:
o What is a use case that accurately depicts how you envision your solution to work?
o What short- and long-term goals do you propose for measuring and evaluating the success of your solution?
o What specific metrics of product efficacy do you seek to measure, facilitate a feedback loop around, and
optimize for?
o What technologies will the solution utilize, and how will such technologies support facilitating a short-cycle
feedback loop around product efficacy?
o What assessments and/or feedback channels will be employed in the solution? To what extent have the
assessments been validated by instructional leaders and the research community? What quantitative and
qualitative feedback will teachers, school decision-makers, and entrepreneurs receive based on these
assessments and channels, how, in what form, and how often?
o What stakeholders and/or partners need to be involved and what is the desired nature, number, frequency,
and structure of intercepts linking them together?
o How do you specifically plan to deliver feedback on product efficacy to drive continuous improvement efforts
for vendors?
o If a platform, service or product, how will you price it? If a process, what rubric will you employ to determine
who gets to participate and when?
o What training, professional development, and ongoing support will be required to enable a pilot?
o What is your cost to develop the platform?
o How does your pricing compare to similar existing products?
Additionally, respondents may provide up to five additional pages of appendices that specifically relate to solution
description, such as screenshots, sample assessments, etc.
Page 10 of 15
© 2013 Bill & Melinda Gates Foundation
Request for Proposal

Work Plan (1 page or less): Please outline the major activities associated with the development of the proposed
solution, when they will be completed, and by what resources.

Recruitment Plan (1 page or less): Please tell us how you plan to attract customers of the solution to demonstrate
proof of concept. Include specific information on partnerships, distribution channels and/or marketing strategies.
Submission Instructions
Start by clicking the application link below. The first time you visit the portal you will need to create an account. After you
enter your information, a temporary password will be emailed to you. Return to the portal to personalize your password
and complete your account profile.
1. Click on the Application Portal link.
2. Create an Account
3. Once you are logged in, select “If you have received an invitation… you may respond here.” on the home page
and enter the following information:
Request ID: SOL1128135
Access Code: educ@tion2015
Please Note:
 You can save the application and return to it later by logging in to the Application Portal at any time before the
submission deadline with your email address and password. The Request ID and Access Code are only
necessary the first time you access the application.
 You can only hit “submit” once using the application tool, so we suggest iterating on documents offline and
submitting your final versions.
 You will receive a confirmation email once you have successfully submitted your full application materials. If
you do not receive the email (check your spam folder), please return to the portal and try again.
 To view frequently asked questions about submitting your application, click Help on the Application Portal.
 We use the application portal mainly to create an entry in our internal database. After you use it once to submit
your documents, you’ll send all subsequent documents as attachments via email.
 The Application Portal will be updated periodically with information regarding the Q&A Call/ Webinar currently
scheduled for February 24, 2015 – 1:00 PM (PST)
Filling out the Online Application
Follow the directions on the application portal.
1. Getting Started tab - Please review the all Guideline and Reference documents noted below.
Guidelines and Reference
Document
New Approaches for Evaluating and
A Improving the Efficacy of Digital
Courseware - RFP
B
BMGF Representative Terms and
Conditions
C
BMGF College Ready Strategy
D
BMGF Indirect Cost Policy
E
ER Guidelines
F
Data Stewardship Principles
Page 11 of 15
Purpose
Invitation to participate and guidelines for the RFP
Provides the Representative Terms and Conditions
for the grant. If you’re proposal is selected, you
should be prepared to agree to the terms as stated
in this document. Please review it carefully.
Provides information about the Gates foundation
USP College Ready Strategy
Provides guidance about the foundation’s Indirect
Cost policy
Provides guidance on private foundation
expenditure responsibility requirements
Provides guidance on private foundation
stewardship principles to protect student data and
privacy
© 2013 Bill & Melinda Gates Foundation
Request for Proposal
2. Describe Project tab - Please fill this out to the best of your ability, using the following guidelines:




Project Title – a short title, including the name of the solution.
Project Description – a short description about the solution and any unique features.
Project Duration – 12 months.
Project Amount – enter the amount you are applying for. Please consult the RFP Information above for
guidance on the maximum amount for each award.
Referred By – please note where you heard about this RFP.

3. Upload Files tab - Please complete and upload the documents noted below using the templates provided.
Templates and Organizational Documents
Document
Purpose
Organization’s Current Year Budget
Narrative Guidelines and template for narrative
responses
Captures broad budget and spending categories
for the work
Identify how the proposed work under the Project
will fulfill the foundation’s Global Access
objectives
A collection of information about your organization
(Note: Please gather the documents below into
PDF form, and then submit as one document)
A copy of your IRS tax determination letter or
certification of formation (depending on what type
of entity you are), as different grant terms and
conditions may apply
Applicable only to LLCs. If you are an LLC,
please provide evidence that you are taxed as a
partnership, or that you have made an election to
be taxed as a corporation
Audited financial statements for the past two
years, or a proxy if you have not had an audit
Your organization’s current-year budget
List of Board of Directors
List of your board of directors, if applicable
1
Proposal Narrative
2
Budget
3
IP Due Diligence Module
4
Organizational Documents
IRS Tax Determination Letter or Certificate
of Formation
Evidence of Partnership
Last 2 Years of Audited Financial
Statements (or proxy)
4. Add Contacts tab - Please make sure you add a contact for the person who will sign the grant agreement if you are
awarded a grant in order to facilitate the award process.
Help Contact(s)
Please contact college.ready@gatesfoundation.org for questions.
Selection and Award Process
Initial Application Review
After all applications are submitted, Foundation staff will screen submissions to address whether the proposal addresses
the key needs described in the topic and adheres to the application requirements. Applications excluded during the
screening process will be notified that their proposals were declined.
Page 12 of 15
© 2013 Bill & Melinda Gates Foundation
Request for Proposal
Application Review, Scoring, and Finalist Notification
Next, a panel made up of Foundation and external reviewers will review submissions and issue a preliminary scoring
based on the scoring rubric. At this point, a select number of organizations will be notified that they have been
designated as finalists.
Final Application Review and Winner Selection
Organizations selected for investment will be notified in November. We will ask organizations selected for investment to
answer a final set of questions that enable us to perform a more detailed due diligence review. For-profit grantees or
other entities subject to expenditure responsibility will have to answer a specific set of questions to enable the
Foundation to comply with its IRS obligations as a private foundation. For more on expenditure responsibility, please
review the ER Guidelines in the application portal. Upon successful completion of this review, finalists will be asked to
sign grant agreements with the Foundation.
Execution of Grant Terms and Conditions
All grant awards will be contingent on execution of a definitive grant agreement. The Representative Terms and
Conditions can be found on the Application Portal.
These Terms and Conditions have been developed specifically for this RFP and are not negotiable. You are advised to
ensure that your institution can accept these Terms and Conditions at the time of proposal submission. Grantees
subject to expenditure responsibility may be subject to additional terms and conditions
If your proposal is selected, you will have a very limited amount of time after the notification of award to accept the grant
and return the award letter with an appropriate institutional signature. You must return a fully executed Grant
Agreement to the foundation post-marked no later than the date indicated by the Grants Management team to receive a
grant award.
Intent and Disclaimer
This RFP is made with the intent to identify organizations to build solutions as described in this RFP. The Foundation will
rely on an organization’s representations and consider them to be truthful as described. The Foundation assumes it can
be confident in an applicant’s ability to deliver the activities described in this RFP. The responses will be incorporated
into a future grant agreement should the Foundation wish to support the proposal submitted by the applicant.
This RFP is not an offer to enter into a funding agreement. The Foundation assumes no responsibility for your cost to
respond to this RFP. Until a written funding agreement is fully executed, the Foundation will have no obligations to any
applicant.
The Foundation has put in place policies and procedures to restrict public dissemination of grant application materials
including, when possible, having external reviewers sign confidentiality agreements and requiring that reviewers destroy
or return to the foundation all copies of information acquired or created during the course of performing a review. In
some instances, we are unable to put in place confidentiality agreements or to police the use of grant application
materials.
As a general policy, the Foundation does not publicly disseminate or "publish" proposals or supporting information
related to grant applications. For IRS compliance reasons, we are required to publish a list of grants that we have made.
We also provide a general description of the grant on our web sites including www.gatesfoundation.org. These brief
descriptions are also made available in press releases and other marketing materials.
To identify and avert conflicts of interest among reviewers, reviewers will not be permitted to submit proposals, or to
review proposals from organizations in which they have self-identified conflicts of interest.
Page 13 of 15
© 2013 Bill & Melinda Gates Foundation
Request for Proposal
More Information
Reference
Blank, Steve. The Startup Owner’s Manual: The Step-by-Step Guide for Building a Great Company. Pescadero: K&S
Ranch, 2012.
Blank, Steve. “Why the Lean Startup Changes Everything.” Harvard Business Review. May 2013.
Chatterji, Aaron and Benjamin Jones. 2012. “Harnessing Technology to Improve K-12 Education.”
Coalition for Evidence-Based Policy. 2012. “Rigorous Program Evaluations on a Budget.”
Costa, Kristina. “Finding ‘What Works’ in Education.” Center for American Progress. 9 Feb. 2012. 15 July 2013.
http://www.americanprogress.org/issues/open-government/news/2012/02/09/11053/finding-what-works-ineducation/.
Education Market Research. 2012. The Complete K-12 Report®: Market Facts & Segment Analyses .
Fullan, Michael and Katelyn Donnelly. 2013. “Alive in the Swamp: Assessing Digital Innovations in Education.”
Gage, Deborah. “The Venture Capital Secret: 3 out of 4 Startups Fail.” The Wall Street Journal Online. 19 Sep. 2012.
16 July 2013. http://online.wsj.com/article/SB10000872396390443720204578004980476429190.html
Global Silicon Valley Advisors. 2012. “Fall of the Wall: Capital Flows to Education Innovation.”
Gray, L., Thomas, N., & Lewis, L. 2009. Teachers' Use of Educational Technology in U.S. Public Schools. Washington,
D.C.: U.S. Department of Education.
Havens, David. 2012. “A Closer Look at K12 EdTech Venture Funding.” Innovation and Entrepreneurship in Education
– NewSchools Venture Fund Blog. 20 Dec. 2012. 15 July 2013. http://www.newschools.org/blog/closer-look
IDEO & The Bill and Melinda Gates Foundation. 2013. “Catalyzing Ed Tech: User Journeys.”
Lee, Mike. “The Top EdTech Frustrations of Educators.” EdShelf Blog. 9 Apr. 2013. 15 July 2013.
http://edshelf.tumblr.com/post/47541656401/the-top-edtech-frustrations-of-educators
Maxwell, Lesli A. “Study: Hybrid Algebra Program ‘Nearly Doubled’ Math Learning. Education Week. 20 June 2013.
15, July 2013. http://blogs.edweek.org/edweek/inside-schoolresearch/2013/06/government_study_finds_gains_f.html?cmp=ENL-EU-MOSTPOP
mind the gap & The Bill and Melinda Gates Foundation. 2013. “Teachers and Technology.”
Purcell, Kristen and Alan Heaps. 2013. “How Teachers are Using Technology at Home and in their Classrooms.”
Ries, Eric. “Minimum Viable Product: A Guide.” Startup Lessons Learned. 3 Aug. 2009. 15 July 2013.
http://www.startuplessonslearned.com/2009/08/minimum-viable-product-guide.html.
Rogers, Everett M. Diffusion of Innovations. 5th ed. New York: Free Press, 2003.
The Bill and Melinda Gates Foundation. 2012. “Innovation in Education: Technology and Effective Teaching.”
US Department of Education Office of Educational Technology. 2013. “Expanding Evidence Approaches for Learning
in a Digital World.”
About the Bill & Melinda Gates Foundation
Page 14 of 15
© 2013 Bill & Melinda Gates Foundation
Request for Proposal
Guided by the belief that every life has equal value, the Bill & Melinda Gates Foundation works to help all people lead
healthy, productive lives. We work with partner organizations worldwide to tackle critical problems in four program areas.
Our Global Development Division works to help the world’s poorest people lift themselves out of hunger and poverty.
Our Global Health Division aims to harness advances in science and technology to save lives in developing countries.
Our United States Division works to improve U.S. high school and postsecondary education and support vulnerable
children and families in Washington State. And our Global Policy & Advocacy Division seeks to build strategic
relationships and promote policies that will help advance our work. Our approach to grant making emphasizes
collaboration, innovation, risk-taking, and, most importantly, results.
To learn more about the foundation's work, visit www.gatesfoundation.org.
Page 15 of 15
© 2013 Bill & Melinda Gates Foundation
Request for Proposal