What 18F will look for in RFQ Responses
-- Greg Elin - 06.17.2015
I wanted to share the below thoughts on the Agile BPA based on the rules of evaluating solicitation responses and what I know about 18F.
1) Quoters will scored against the checklists and questions in the US Digital Services Playbook.
My bet is the Selection Committee is going to score the Technical Responses as "Exceptional", "Acceptable", "Non-Acceptable" against each and every checklist item and question in the US Digital Services Playbook.
The Selection Committee will score each Quoter (vendor) against each checklist item and question in the Playbook and then give an overall rating for each "Play" to each Quoter. Then they will add up all the points for the Technical Approach an overall rating and a quantitative score. The awardees with be those with the highest score (and some consideration of price).
It is procurement law to have an objective process, using objective criteria that is communicated to public. There is surprisingly little detail about the criteria in the proposal except, "the Quoter must demonstrate that they followed the U.S. Digital Services Playbook by providing evidence in the repository."
2) Quoters demonstrating a user-driven build will score higher than any features or execution.
The biggest critique of government IT is that it builds the wrong things, that are not useful to real users.
The RFQ describes three pools of services, each building on the other. In other words, user-centric design are the plays that come first and must be done before the others.
To be blunt, the Team should demonstrate User Focus from day one. That focus should be represented in the Git Repo. Personally, I think it makes sense for the Team to spend at least three days focused on Pool1 (design) before progressing to Pool 2 (Development) or Pool 3 (Full Stack). This would demonstrate a real investment in establishing an agile process that connected with real users. (Of course, there some tech exploration spikes and creating the CI can be done in parallel to end-user driven research.) Also, user interactions would continue throughout the process for continuous iteration. But at big investment with end-users to define what is being built (not just the features) is important.
It is the classic government IT fail to say, "We don't have time to spend a lot of time with users, we have to meet this deadline." GSA 18F will absolutely be looking for groups that demonstrate they follow the Agile Manifesto and put people and collaboration over process, tools, and contracts.
Selection Committee will be looking to see in the repo if and how end-users drove the MVP core feature(s) and there was actual testing and iteration.
3) The Git Repo needs to have as much detail as a traditional proposal
The RFQ emphasizes continually how the repo should demonstrate how the organization follows the playbook.
The actual MVP code should only be one part of the repo. Much of the repo should reflect the agile process and assets demonstrating how the MVP grew out of a collaboration and iteration with target users.
4) Assign a Budget and demonstrate how we stuck to it
CivicActions should have a budget for this project and stick to it.
Play #5 is "Structure budgets and contracts to support delivery". Most Quoters will give only lip service to this play while throwing lots of resources at the deadline, (e.g. budget creep).
Quoters providing evidence of assigning a budget and managing the project to constraints will score "Exceptional" for this play.
There’s clearly an implied budget of 10 days in the project. I think a firm can demonstrate within the repo structuring budgets and contracts to support delivery by showing how we selected a team, a leader, stuck with that team, had structured hours when work was done, made choices, etc. Even better if we show time boxing for different activities and our internal interactions with our own executives to meet the “budget.”
5) Working software is a must.
It doesn’t have to set the world on file, but Dave Zvenyach has tweeted as saying working software is needed. Agile Methodologies promote the idea of Always Being Releasable. You should document when you got the MVP done and how you iterated on it.