Make it easier for companies to fund open source

[chadwhitacre]

So the basic idea evolving on this PR is to walk the entire dependency graph of a company's source code, and provide a way to give money to the entire bundle at once. We need the projects to opt in first, of course; our first attempt at streamlining that is in #4148. The current plan once #4148 is in is to use the Libraries.io API to get the dependency graph based on a company's uploaded package.json file.

[chadwhitacre]

Closes #2695.

Actually, nevermind that. I was gonna say that we should also have a nice onboarding page for companies ("Step 1, Step 2"), but since we only have one company giving right now, I think we should just make the homepage the onboarding page. 😶

[chadwhitacre]

5d9e275

[chadwhitacre]

92d411f

[mattbk]

Blog post about this when it goes live? Part of Step 6 (Tell the world!) should be "tell the projects you use to get on Gratipay."

[chadwhitacre]

Step 4 should be "link your GitHub and we will crawl your private repos to find out what packages you depend on."

Then the company sets a dollar amount and we split it equally between all of the packages in the dependency tree that have been claimed.

We should crawl all the packages repos for metadata and make it available at /on/{pypi,npm,cpan,cocoa-pods,etc}/package.

[chadwhitacre]

Actually, that should be step 1. That's the interesting thing, the hook. I'm a CTO at a small to midling software company, it's interesting to me to know what my open source inventory is and especially what licenses are used. On the result page we prompt them to take the next step and fund the projects listed in the report.

We call it a "bundle" or something: the set of open source packages you depend on it. Each week we recrawl to generate a fresh list, and we add a step to payday to distribute an amount equally across teams (projects) in a bundle.

Then we need some way for package owners to claim their package on Gratipay, i.e., make it a team. I guess for demo purposes we can do that manually in the review ticket. The "Claim" button on the /on/pypi/aspen page can simply POST to /new with the form partially filled out, and they can finish applying there. They're into the existing team flow at that point.

[chadwhitacre]

Restarting this PR with a different tack. Previous head was 92d411f.

[chadwhitacre]

[chadwhitacre]

[chadwhitacre]

I constrained the "1.0 payouts" notice to just pages under ~username (there's no other way to access the page, so we need to keep something). The rest of the time there is now a "Gratipay is a great way to fund open source" message that points to the dedicated page.

[chadwhitacre]

https://martin-thoma.com/analyzing-pypi-metadata/
https://martin-thoma.com/analyzing-pypi-metadata-2/

[chadwhitacre]

Version 0 should probably just take a requirements.txt in a textarea.

[chadwhitacre]

I really want to demo this in my lightning talk at ATO. I need to get slides tomorrow! 😮

[chadwhitacre]

[chadwhitacre]

We could probably replace the welcome notice about the number of teams with a link to Fund Open Source.

[chadwhitacre]

I'm looking at the next part, where we take in requirements.txt and we want to spit out a list of all of the dependencies (not just the first-level ones, all of them), as well as their PyPI names and licenses. I had thought maybe we could just use pip programmatically to parse the requirements, but we need to actually install everything in order to see what's installed. That means we need some sort of sandboxing, because installing packages involves executing arbitrary code. Hmm ...

[chadwhitacre]

I'm looking at building a micro-service like we have for processing images. I'm looking at using Docker for the sandboxing, hosted on a Digital Ocean droplet.

[chadwhitacre]

Blam!

root@gdr:~# docker run gdr ./licenselife.py
Greetings, program!
root@gdr:~#

Docker is nice. :-)

[techtonik]

Wow! Time to get back to Gratipay now that gratipay/inside.gratipay.com#317 became actionable. =)

- "Gratipay is a great way to fund open source"
+ "Funding your invisible infrastructure"

[chadwhitacre]

Perhaps we should take the angle that this is a matter of corporate social responsibility.

[chadwhitacre]

I mean, right?

Corporate social responsibility (CSR, also called corporate conscience, corporate citizenship or responsible business) is a form of corporate self-regulation integrated into a business model. CSR policy functions as a self-regulatory mechanism whereby a business monitors and ensures its active compliance with the spirit of the law, ethical standards and national or international norms. With some models, a firm's implementation of CSR goes beyond compliance and engages in "actions that appear to further some social good, beyond the interests of the firm and that which is required by law." The aim is to increase long-term profits and shareholder trust through positive public relations and high ethical standards to reduce business and legal risk by taking responsibility for corporate actions. CSR strategies encourage the company to make a positive impact on the environment and stakeholders including consumers, employees, investors, communities, and others.

https://en.wikipedia.org/wiki/Corporate_social_responsibility

[chadwhitacre]

We've received some awesome feedback from @arasmussen ("this guy" ;) in private email. Sharing with permission ...

Hey Chad,

I'd gladly contribute to a project on Gratipay, but I'd like to contribute to a project that we actually use and benefit from at Loopify.

Specifically, here's a list of npm modules I'm grateful for:

• async
• babel-* (also css-loader, file-loader, saas-loader, style-loader, url-loader, etc)
• iframe-resizer
• mongoose
• nodemon
• react
• react-helmet
• react-modal
• react-redux
• react-router
• react-router-redux
• redux
• redux-thunk
• webpack

I'd suggest you hit up all these project maintainers and say "I've got a startup founder who wants to pay for your project if you sign up for Gratipay". Then let me know! :)

---

Gosh—thank you! 🙇

This actually might fit in super well with the next feature we're turning our attention to, which is integration with npm. The plan is to let you post a package.json, pay one amount, and have it distributed among all of the packages you depend on. Does that sound like a good way to address the "actually use and benefit" criterion?

---

Like ... how did you come up with that list? Is it the same as or different than your package.json? Is it enough to start with package.json, or do you want to hand-curate the list to give to?

---

Kinda. It'd be nice if I could remove some projects from the list though. Some projects are extremely simple and don't really get maintained, whereas other projects are a full time effort from teams of people. The latter is the type I'd rather support.

There are also npm modules in my package.json that I've simply forgotten to "npm uninstall". So I think being able to upload the package.json is cool, but then let me select which projects from that list I'd like to support.

---

I started with my package.json and trimmed a significant number of projects that are either very simple, unmaintained, or we don't actually use actively. See my other email :P

---

Awesome. Our targets are:

-1. load up npm in the Gratipay db—done!
0. mirror all npm packages on Gratipay

1. enable npm packages to opt-in to receive funding
2. enable pledging to projects that haven't opted in yet
3. aggregate opt-in so maintainers can opt-in many at once
4. maybe naive package.json parsing with no deep traversal and equal split
5. possibly deep traversal (deps of deps)
6. eventually maybe fine-tuning of "bundles" like you're talking about

Honestly, you should be able to manually set up payments as you want after (1) or maybe (2).

May I post your comments to public GitHub for the rest of the team to see? Feedback is oxygen!

---

Sure!

[kaguillera]

This is great. Lets do it.

[nobodxbodon]

• if package owner doesn't come to get their piece of donation, will it expire?
• will the total fund be distributed equally among all the projects in the whole dependency tree?
• any reason to paste package.json instead of uploading the file?

[chadwhitacre]

• if package owner doesn't come to get their piece of donation, will it expire?

Hmm ... that's a new idea, I think. We did have "pledging" in the past: a promise to give, but we don't actually move money until the receiver opts in. Pledges didn't expire, but receivers could "lock" their /on/foo/bar/ page to opt-out even from pledges.

• will the total fund be distributed equally among all the projects in the whole dependency tree?

For the manual setup we're talking about at #4135 (comment), amounts would be manually set by the giver. More complicated distributions fall under #1493.

• any reason to paste package.json instead of uploading the file?

Easier to build. :-)

[bbangert]

I understand the appeal of this concept to companies, but I'm not sure I understand how it's a great thing for developers that want real project sustenance vs. some beer money.

An apt example might be musicians.... who get peanuts from the divided lump-sum a listener throws at the streaming service. The musicians always prefer direct money to them, with concert tickets and album sales second, and streaming music a distant reluctant thing. Given the size of these dependency chains for some projects, it could also be as useless to many projects as most class action settlements are to the actual plaintiffs. A good/huge chunk of money split so many ways its not enough to make any real difference to the people that actually get it.

Ignoring that, the next biggest issue I'd imagine is what @nobodxbodon points out, how are the funds to be split amongst the dependency tree? By LoC? How does one blindly say what is the most significant, took the most time, or requires the most additional resources (actual new funding)?

Is the person at the company responsible for allocating money going to be uploading requirements files for all their project code-bases? How exactly does that work? Can companies instead choose existing 'stacks' they know they use, and trust that someone managing the funds for that 'stack' is overseeing fair allotment of funds?

Is this feature going to be appealing enough to OSS developers that they encourage companies to donate via this route instead of direct donations just to their project? (Where they'd get the entirety of it without having to worry about how its divided amongst a dependency chain)

I'm quite curious about what this feature entails since gratipay/inside.gratipay.com#637 is requesting a specific amount of money to implement what seems like a very vaguely defined feature with unknown desirability amongst the OSS developers it's supposed to benefit.

[kaguillera]

@bbangert IMHO.

I understand the appeal of this concept to companies, but I'm not sure I understand how it's a great thing for developers that want real project sustenance vs. some beer money.

The distant/difference between zero and one is infinite (at least in mathematics). In some case those developer are not getting any contributions for companies that want to contribute to them so something is better than nothing. That in itself is a step in the right direction.

Is this feature going to be appealing enough to OSS developers that they encourage companies to donate via this route instead of direct donations just to their project? (Where they'd get the entirety of it without having to worry about how its divided amongst a dependency chain)

The problem with the OSS developers trying to encourage companies to donate is if the company actually does pay attention to the developers and want to donate how easy/difficult is it in the current infrastructure. The financial reporting for just that one project would be a major turn off for the company . This is what we are trying to do, make the who process easier for both OSS developer and company willing to donate.

That being said the rest of you questions are what we are trying to work out.

[chadwhitacre]

@bbangert It's interesting that your feedback from a developer's perspective actually aligns with @arasmussen's feedback from the company giver point of view:

I started with my package.json and trimmed a significant number of projects that are either very simple, unmaintained, or we don't actually use actively.

Some projects are extremely simple and don't really get maintained, whereas other projects are a full time effort from teams of people. The latter is the type I'd rather support.

There are also npm modules in my package.json that I've simply forgotten to "npm uninstall". So I think being able to upload the package.json is cool, but then let me select which projects from that list I'd like to support.

In other words, "enlightened" companies want to direct their payments to the right place. They want their money to have the most impact. I think there's still a role for surfacing requirements via the package repos, because recognition is easier than recall. If all you have to go on is the packages you remember that you use, then your giving will be (even more) biased towards the projects with the best marketing.

Is this feature going to be appealing enough to OSS developers that they encourage companies to donate via this route instead of direct donations just to their project? (Where they'd get the entirety of it without having to worry about how its divided amongst a dependency chain)

I think, done right, this can benefit smaller projects, where there is no time or budget to do the sales necessary to get companies to give just to them. An alternative would be to band together under a shared entity (PSF?), run marketing and sales through there, and divvy up the money however is agreed upon by that group.

[chadwhitacre]

I just got off the horn with @jdorfman, who pitched me an idea about providing assurance to companies that the long-tail of open source is sustainable for the long term, and I was able to share with him what we're working on here in terms of building on package managers towards that very end. :-)

Helpful in the back-and-forth was putting a point on the fact that what we need to balance here is:

1. making it easy for companies to get money into the ecosystem without the "chore" (@jdorfman's word) of sifting through the long tail to figure out where the money should go, and
2. allocating money fairly, to balance core tech with hot new projects, as @bbangert especially has been advocating here and even moreso at seek funding from Mozilla inside.gratipay.com#637 (comment).

[jdorfman]

No one likes chores.

[mattbk]

No one likes chores.

Part of avoiding chores seems like setting up company annual or monthly invoicing. That seems important for the corporate side of things.

[mattbk]

+1 from https://twitter.com/bbatsov/status/819392359087517696.

[chadwhitacre]

This is proving to be a larger task than it makes sense to have a single PR for. Closing in favor of our first epic: gratipay/inside.gratipay.com#987.