New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open source by default #73

Closed
konklone opened this Issue Mar 22, 2016 · 8 comments

Comments

Projects
None yet
9 participants
@konklone
Contributor

konklone commented Mar 22, 2016

(I’m Eric, an engineer at 18F, an office in the U.S. General Services Administration (GSA) that provides in-house digital services consulting for the federal government. I’m commenting on behalf of 18F; we’re an open source team and happy to share our thoughts and experiences. This comment represents only the views of 18F, not necessarily those of the GSA or its Chief Information Officer.)

The draft policy’s introduction asks:

Would an “open source by default” approach that required all new Federal custom code to be released as OSS, subject to exceptions for things like national security, be more or less effective in [fueling innovation, lowering costs, benefitting the public, and meeting the operational and mission needs of covered agencies]?

Yes, an “open source by default” approach would more effectively meet the White House’s goals with this policy.

18F’s policy is that all software it produces and procures is open source from the first line of code, published publicly and dedicated to the international public domain. In our two years of operation, 18F has created around 400 original public repositories containing software, documentation, and other operational materials. These repositories are where 18F collaborates on projects with our agency partners, and within our team. It’s our direct experience that the most efficient, cost-effective, and secure way to develop software is in the open.

We have consistently seen that the most effective way to share information, software, and experience among agencies is the ongoing public release of data, code, and documentation. Managing and guarding access to “private” software and information consistently entails significant operational overhead when compared to sharing public information. The bureaucratic overhead of secrecy can sometimes be extreme, depending on the scale and temperament of the collaborators. However, this overhead is frequently discounted or unobserved by teams that default to working in private.

The cost of secrecy is one of the factors that leads 18F to be very clear with potential partners that when we build things with them, we work in the open. But another stated reason is that it leads to higher quality software and brings the best out of our engineers:

Working in the open helps to encourage good documentation and coding practices. Everyone is aware and following processes for open information from day one. There is no just-before-launch, last minute review of everything. All code and documentation is reviewed as it’s developed.

Good software practices, such as writing appropriate documentation and separating passwords from the software which uses them, are good for all software and not unique to open source software. However, working in the open makes following these practices more consistent and more likely, even for the most competent staff — and when mistakes are made, working in public makes it more likely that they will be spotted and corrected early.

Releasing more procured software

The draft policy requires 100 percent of custom software written by government employees to be open source by default, which is a strong and welcome requirement.

However, the policy requires agencies to release only 20 percent of code developed on their behalf by vendors or third parties, as a “pilot.” Today, the majority of code written for the government is written by vendors and third parties, and so this policy will not directly affect the majority of software produced for the U.S. government in its current form.

(Throughout section 5.1, the policy refers to the 20 percent pilot as governing the release of all custom government code, when it actually governs the release of externally developed custom government code. These references should be updated to be more precise.)

Section 5 of the draft policy says:

As outlined in the OMB Open Government Directive, the three principles of transparency, participation, and collaboration form the cornerstone of an open government. Federally released OSS embodies these principles.

Leveraging the skills and knowledge of individuals across the Federal Government and beyond can result in, among other things, enhancements to code quality and security as a result of public scrutiny of open source code.

Federal OSS can also contribute to economic growth and innovation as state and local governments, private sector companies, taxpayers, and others can reuse that code to develop products and services for the public.

These statements all apply equally well to procured software as they do to software built in-house. The policy describes various benefits of open source software in section 5.1 (the pilot program) and section 5.2 (membership in the open source community). However, the policy doesn’t specify why it applies a different standard to procured software than it does to in-house software.

Rather than a 20 percent threshold for externally developed custom software, agencies should be required to release 100 percent of externally developed custom software as open source. This would eliminate entire classes of metrics that the White House would otherwise need to measure, and greatly reduce the overhead of implementing and overseeing this policy for OMB and agencies alike.

To the extent that procuring custom software development from third parties has unique and demonstrable challenges that make the policy reluctant to impose a blanket open source requirement, we believe those challenges would be best met by offering agencies the option of providing a written justification for not releasing the results of taxpayer-funded procurements to the public.

For example, if OMB is concerned that an open source requirement for contracting officials might lead to reduced competition or higher prices for contracts, and an agency shares this concern, the agency could say this in a written justification to OMB. OMB could then allow the agency to experiment with continued procurement of closed source software — for specific procurements — for the purposes of testing and comparison with the agency’s general procurement of open source software.

This would allow agencies with clear and articulable concerns about their marketplace to explore those concerns while simplifying the policy’s effect and implementation, and would allow OMB to more clearly meet the White House’s stated goal in the second Open Government National Action Plan to “adopt an open source software policy”.

To summarize, our recommendations are:

  • Update section 5.1 so that references to the 20 percent pilot unambiguously refer only to externally developed custom code.
  • Streamline and strengthen the policy by eliminating the 20 percent threshold, and require all custom software developed for the U.S. Government to be open source by default. OMB could allow agencies with procurement-specific concerns to evaluate those concerns through narrow written justifications.
@freephile

This comment has been minimized.

Show comment
Hide comment
@freephile

freephile Mar 28, 2016

The MIT Media Lab has adopted an "open by default" policy (See https://medium.com/mit-media-lab/mit-media-lab-changes-software-default-to-floss-4305e478e40#.oaesmqjx1). I'm suggesting an even better approach: "Free by default" #78

freephile commented Mar 28, 2016

The MIT Media Lab has adopted an "open by default" policy (See https://medium.com/mit-media-lab/mit-media-lab-changes-software-default-to-floss-4305e478e40#.oaesmqjx1). I'm suggesting an even better approach: "Free by default" #78

@GSA-CIO-das

This comment has been minimized.

Show comment
Hide comment
@GSA-CIO-das

GSA-CIO-das Mar 30, 2016

100% agree, Eric. Regarding the views of the GSA CIO (me), we moved to an open source-first strategy for the enterprise in August of 2014, confirmed it again in October of 2015, operate against it at the strategic enterprise level, project level and at the data layer. Open source-first is baked into the GSA IT DNA and is one of our core operating principles. Just in case anyone reading your blog post was curious. DAS

GSA-CIO-das commented Mar 30, 2016

100% agree, Eric. Regarding the views of the GSA CIO (me), we moved to an open source-first strategy for the enterprise in August of 2014, confirmed it again in October of 2015, operate against it at the strategic enterprise level, project level and at the data layer. Open source-first is baked into the GSA IT DNA and is one of our core operating principles. Just in case anyone reading your blog post was curious. DAS

@brainwane

This comment has been minimized.

Show comment
Hide comment
@brainwane

brainwane Apr 8, 2016

I'm a manager whose consultancy focuses on free and open source software. In my experience, organizations that use an open-source-by-default approach have a much easier time exchanging working code and ideas with other organizations, and even with different departments of their own organizations. This fuels innovation. And since an open-source-by-default strategy would lead to less friction around release and more source code being shared, state, municipal, and other government agencies would have a much easier time benefiting from that work -- meaning that they, and the public, would benefit more.

http://archive.civiccommons.org/2011/01/be-open-from-day-one/ by open source processes expert Karl Fogel, who has worked with government on open source workflow issues, goes into several good reasons to prefer the "open from day one" approach.

brainwane commented Apr 8, 2016

I'm a manager whose consultancy focuses on free and open source software. In my experience, organizations that use an open-source-by-default approach have a much easier time exchanging working code and ideas with other organizations, and even with different departments of their own organizations. This fuels innovation. And since an open-source-by-default strategy would lead to less friction around release and more source code being shared, state, municipal, and other government agencies would have a much easier time benefiting from that work -- meaning that they, and the public, would benefit more.

http://archive.civiccommons.org/2011/01/be-open-from-day-one/ by open source processes expert Karl Fogel, who has worked with government on open source workflow issues, goes into several good reasons to prefer the "open from day one" approach.

@benbalter

This comment has been minimized.

Show comment
Hide comment
@benbalter

benbalter Apr 10, 2016

I agree. I further expand upon why government software should be open by default, rather than the government only making a small subset of taxpayer-funded and taxpayer-owned code available to the taxpayers who made its development possible and for whom the software is intended to benefit over in #90 (comment).

benbalter commented Apr 10, 2016

I agree. I further expand upon why government software should be open by default, rather than the government only making a small subset of taxpayer-funded and taxpayer-owned code available to the taxpayers who made its development possible and for whom the software is intended to benefit over in #90 (comment).

@peterbenmeyer

This comment has been minimized.

Show comment
Hide comment
@peterbenmeyer

peterbenmeyer Apr 12, 2016

The sources cited are all right on, but they don't go so far as to support 100% open-sourcing of every line of code written by a government staffer. The Media Lab says yes immediately to REQUESTS to open source. Fogel refers to projects that will eventually be open-source, so open-source them now; leaving the possibility that some aren't even considered for open-source. 18F and vendors are satisfying some kind of CONTRACT with the code, a formal handoff of some kind, leaving open the possibility that they wrote lots of experimental code which never got into the final product. What about cases where one just throws a program together to see what it does? or a program where the author is its only user? or a one-time calculation? a program which never had any requirements specification? It is transparency extremism to include these in the 100% -- it adds more costs than it's worth to impose such requirements on every government-employed human interacting with a computer. I advocate at the very least a privacy/experimentation exception in #171

peterbenmeyer commented Apr 12, 2016

The sources cited are all right on, but they don't go so far as to support 100% open-sourcing of every line of code written by a government staffer. The Media Lab says yes immediately to REQUESTS to open source. Fogel refers to projects that will eventually be open-source, so open-source them now; leaving the possibility that some aren't even considered for open-source. 18F and vendors are satisfying some kind of CONTRACT with the code, a formal handoff of some kind, leaving open the possibility that they wrote lots of experimental code which never got into the final product. What about cases where one just throws a program together to see what it does? or a program where the author is its only user? or a one-time calculation? a program which never had any requirements specification? It is transparency extremism to include these in the 100% -- it adds more costs than it's worth to impose such requirements on every government-employed human interacting with a computer. I advocate at the very least a privacy/experimentation exception in #171

@konklone

This comment has been minimized.

Show comment
Hide comment
@konklone

konklone Apr 12, 2016

Contributor

18F and vendors are satisfying some kind of CONTRACT with the code, a formal handoff of some kind, leaving open the possibility that they wrote lots of experimental code which never got into the final product.

Just to clarify for 18F - while staff might write super-trivial scripts to briefly toy around with things that never make it into any repo, any code that has enough utility to end up in source control, even if it's experimental or prototypical, should end up in a public repo. We try to work in public as our normal course of business, rather than delivering software at the conclusion of an engagement.

Contributor

konklone commented Apr 12, 2016

18F and vendors are satisfying some kind of CONTRACT with the code, a formal handoff of some kind, leaving open the possibility that they wrote lots of experimental code which never got into the final product.

Just to clarify for 18F - while staff might write super-trivial scripts to briefly toy around with things that never make it into any repo, any code that has enough utility to end up in source control, even if it's experimental or prototypical, should end up in a public repo. We try to work in public as our normal course of business, rather than delivering software at the conclusion of an engagement.

@kfogel

This comment has been minimized.

Show comment
Hide comment
@kfogel

kfogel Apr 12, 2016

Just to clarify, regarding @peterbenmeyer's comment referring to my post "Be Open Source From Day One" from five years ago:

I completely agree with @benbalter and others that "open by default" is the best policy for publicly-funded code, and that a goal of merely 20% is far too low. My earlier post was not meant to imply that some government projects shouldn't be open source (conceivably there could be exceptions, for the usual national security reasons, but those should be quite rare). The post was simply about a different topic: how a government agency should implement a successful open source project, assuming the agency has already decided it wants an open source result. Independently of that implementation advice, I also think the policy should default to wanting an open source result :-).

(Hat-tip to @brainwane for letting me know that what I'd written back then was open to misinterpretation now. Not @peterbenmeyer's fault; I should have been clearer in the original post.)

kfogel commented Apr 12, 2016

Just to clarify, regarding @peterbenmeyer's comment referring to my post "Be Open Source From Day One" from five years ago:

I completely agree with @benbalter and others that "open by default" is the best policy for publicly-funded code, and that a goal of merely 20% is far too low. My earlier post was not meant to imply that some government projects shouldn't be open source (conceivably there could be exceptions, for the usual national security reasons, but those should be quite rare). The post was simply about a different topic: how a government agency should implement a successful open source project, assuming the agency has already decided it wants an open source result. Independently of that implementation advice, I also think the policy should default to wanting an open source result :-).

(Hat-tip to @brainwane for letting me know that what I'd written back then was open to misinterpretation now. Not @peterbenmeyer's fault; I should have been clearer in the original post.)

@david-a-wheeler

This comment has been minimized.

Show comment
Hide comment
@david-a-wheeler

david-a-wheeler Apr 15, 2016

I agree that software developed with government funding should be released by default as OSS. I think i would also include the 20% minimum, simply to get people started... because once they learn how to release software as OSS, and develop it publicly, it's easy to get near 100%.

david-a-wheeler commented Apr 15, 2016

I agree that software developed with government funding should be released by default as OSS. I think i would also include the 20% minimum, simply to get people started... because once they learn how to release software as OSS, and develop it publicly, it's easy to get near 100%.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment