This repository has been archived by the owner. It is now read-only.

implement funds #449

Closed
chadwhitacre opened this Issue Jan 9, 2013 · 88 comments

Comments

Projects
None yet
6 participants
@chadwhitacre
Contributor

chadwhitacre commented Jan 9, 2013

The idea here it to provide a way to lump individuals together into groups and projects for funding purposes. A "fund" here is a set of weighted percentages for individuals and other funds (funds can be nested). Then we use these funds as levels of indirection when shuffling money around each week on payday.

Was: group tips together for simultaneous adjustment

I believe this is what @joelmccracken is asking for on #434. The coarse-grained case would be adjusting all tips at once (#448). This ticket is for grouping tips together so that you could adjust 10 tips at once without touching the other 20. The language that Joel used was "a single tip covering multiple people."

@chadwhitacre

This comment has been minimized.

Show comment
Hide comment
@chadwhitacre

chadwhitacre Jan 16, 2013

Contributor

After conversations with @MikeFair in email and IRC, I'm now looking at this as the answer to the question of aggregating individuals into groups on Gittip.

!m @joelmccracken

Contributor

chadwhitacre commented Jan 16, 2013

After conversations with @MikeFair in email and IRC, I'm now looking at this as the answer to the question of aggregating individuals into groups on Gittip.

!m @joelmccracken

@chadwhitacre

This comment has been minimized.

Show comment
Hide comment
@chadwhitacre

chadwhitacre Jan 16, 2013

Contributor

Gittip users can create one or more "funds" (as I'm calling them). A fund is a mapping of participant_id to integer weight:

{ foo: 1000
, bar: 400
, baz: 200
, buz: 10
 }

The part of the Gittip profile page where you see everyone you're giving to ... that becomes a UI over this data structure. Then in the API (#462) we have this as an endpoint that can be posted to by the Gittip UI as well as by third-party applications. Then, we allow for funds to be the keys as well:

{ foo: 1000
, "bar/baz": 400
, baz: 200
, buz: 10
 }

One thing to tease out is the interplay between absolute dollar amounts and percentages. Another is how to keep the damper on volatility.

Next step for me here is some paper prototypes ...

Contributor

chadwhitacre commented Jan 16, 2013

Gittip users can create one or more "funds" (as I'm calling them). A fund is a mapping of participant_id to integer weight:

{ foo: 1000
, bar: 400
, baz: 200
, buz: 10
 }

The part of the Gittip profile page where you see everyone you're giving to ... that becomes a UI over this data structure. Then in the API (#462) we have this as an endpoint that can be posted to by the Gittip UI as well as by third-party applications. Then, we allow for funds to be the keys as well:

{ foo: 1000
, "bar/baz": 400
, baz: 200
, buz: 10
 }

One thing to tease out is the interplay between absolute dollar amounts and percentages. Another is how to keep the damper on volatility.

Next step for me here is some paper prototypes ...

@chadwhitacre

This comment has been minimized.

Show comment
Hide comment
@chadwhitacre

chadwhitacre Jan 17, 2013

Contributor

Just got off the horn with @MikeFair. He had the interesting idea of implementing this using CouchDB. We would store funds as full json docs in there, and use Couch's map-reduce implementation to compute dollar amounts from there. He's offered to spike that out for us. Reticketing as #488.

Contributor

chadwhitacre commented Jan 17, 2013

Just got off the horn with @MikeFair. He had the interesting idea of implementing this using CouchDB. We would store funds as full json docs in there, and use Couch's map-reduce implementation to compute dollar amounts from there. He's offered to spike that out for us. Reticketing as #488.

@chadwhitacre

This comment has been minimized.

Show comment
Hide comment
@chadwhitacre

chadwhitacre Jan 18, 2013

Contributor

Users have zero, one, or more funds. We create a "default" fund for them when they register (cf. #489). In the UI we want there to be a default fund, to streamline the simple case where a user has a single fund. (Whether we enforce a one-fund minimum under the hood or allow the user to drop to zero funds is an implementation detail.)

All funds are public. When I go to a profile page I see a list of funds that the user has created, as well as the weekly dollar amount being given to each fund. These are also shown on community pages (#496). Funds are anonymous in that the allocation of each fund is not public. Users can't be part of their own funds. Non-self users and other funds can be part of funds.

Users allocate dollar amounts to funds. They no longer allocate dollar amounts directly to other Gittip accounts.

Each payday, the fund graph is reduced to dollar amounts from one account to another.

Issues subsumed by this one: #27, #55, #94, #216, #316, #327, #372, #434, #448.

Contributor

chadwhitacre commented Jan 18, 2013

Users have zero, one, or more funds. We create a "default" fund for them when they register (cf. #489). In the UI we want there to be a default fund, to streamline the simple case where a user has a single fund. (Whether we enforce a one-fund minimum under the hood or allow the user to drop to zero funds is an implementation detail.)

All funds are public. When I go to a profile page I see a list of funds that the user has created, as well as the weekly dollar amount being given to each fund. These are also shown on community pages (#496). Funds are anonymous in that the allocation of each fund is not public. Users can't be part of their own funds. Non-self users and other funds can be part of funds.

Users allocate dollar amounts to funds. They no longer allocate dollar amounts directly to other Gittip accounts.

Each payday, the fund graph is reduced to dollar amounts from one account to another.

Issues subsumed by this one: #27, #55, #94, #216, #316, #327, #372, #434, #448.

@justinabrahms

This comment has been minimized.

Show comment
Hide comment
@justinabrahms

justinabrahms Jan 18, 2013

Contributor

This sounds legit to me. I did a double take on "fund can't contain creator". I understand why, but I would have initially suspected that JKM would create a Django fund and put himself in it (though he likely wouldn't, it serves as a good example). Skipping it seems to remove some of the possible abuse potential of having someone tip a fund, then dropping everyone out but you, etc.

I think this is a clear move in a solid direction. 👍

Contributor

justinabrahms commented Jan 18, 2013

This sounds legit to me. I did a double take on "fund can't contain creator". I understand why, but I would have initially suspected that JKM would create a Django fund and put himself in it (though he likely wouldn't, it serves as a good example). Skipping it seems to remove some of the possible abuse potential of having someone tip a fund, then dropping everyone out but you, etc.

I think this is a clear move in a solid direction. 👍

@chadwhitacre

This comment has been minimized.

Show comment
Hide comment
@chadwhitacre

chadwhitacre Jan 18, 2013

Contributor

Exactly, @justinabrahms. Presumably Jacob's Django fund includes your Django fund, which includes Jacob, etc.

Contributor

chadwhitacre commented Jan 18, 2013

Exactly, @justinabrahms. Presumably Jacob's Django fund includes your Django fund, which includes Jacob, etc.

@justinabrahms

This comment has been minimized.

Show comment
Hide comment
@justinabrahms

justinabrahms Jan 18, 2013

Contributor

Oh, interesting. You're suggesting that these can be nested. Are the weights accumulative? Something to watch out for is that you limit the depth of these transitive tips, else you'll need to ensure its not a cyclical graph. :)

Contributor

justinabrahms commented Jan 18, 2013

Oh, interesting. You're suggesting that these can be nested. Are the weights accumulative? Something to watch out for is that you limit the depth of these transitive tips, else you'll need to ensure its not a cyclical graph. :)

@joelmccracken

This comment has been minimized.

Show comment
Hide comment
@joelmccracken

joelmccracken Jan 18, 2013

Oh wow, this sounds confusing. So a group fund can pay to other group
funds, not only other individuals?

On Fri, Jan 18, 2013 at 1:07 PM, Justin Abrahms notifications@github.comwrote:

Oh, interesting. You're suggesting that these can be nested. Are the
weights accumulative? Something to watch out for is that you limit the
depth of these transitive tips, else you'll need to ensure its not a
cyclical graph. :)


Reply to this email directly or view it on GitHubhttps://github.com/zetaweb/www.gittip.com/issues/449#issuecomment-12434005.

Oh wow, this sounds confusing. So a group fund can pay to other group
funds, not only other individuals?

On Fri, Jan 18, 2013 at 1:07 PM, Justin Abrahms notifications@github.comwrote:

Oh, interesting. You're suggesting that these can be nested. Are the
weights accumulative? Something to watch out for is that you limit the
depth of these transitive tips, else you'll need to ensure its not a
cyclical graph. :)


Reply to this email directly or view it on GitHubhttps://github.com/zetaweb/www.gittip.com/issues/449#issuecomment-12434005.

@chadwhitacre

This comment has been minimized.

Show comment
Hide comment
@chadwhitacre

chadwhitacre Jan 18, 2013

Contributor

@justinabrahms Yeah, designing the algorithm for computing concrete gift amounts each payday from the graph of funds is going to be an interesting challenge (cf. https://github.com/zetaweb/www.gittip.com/issues/129#issuecomment-12341021). :-)

@joelmccracken My thought was a fund is a set of percentages:

40% Adrian Holovaty -> Django
20% Guido van Rossum -> Python
10% Linus Torvalds -> Linux
10% Guido van Rossum
10% Adrian Holovaty
10% ACDC

That means: "Give 10% directly to the Gittip accounts of Guido, Adrian, and ACDC, then distribute 40% amongst whatever Gittip accounts are represented by Adrian's 'Django' fund, and 20% amongst whatever Gittip accounts are underneath Guido's 'Python' fund, and the remaining 10% amongst accounts under Linus' 'Linux' fund."

The simple case is that you'll have a single fund—you won't even have to know the word "fund" to use it. It'll just look like a set of percentages as above, and then a big fat "$100.00" at the top, which is the total amount you're giving each week; it will be split up according to the percentages. On payday we'd do a lot of number crunching to determine exactly where your money goes.

Does that make any sense?

Contributor

chadwhitacre commented Jan 18, 2013

@justinabrahms Yeah, designing the algorithm for computing concrete gift amounts each payday from the graph of funds is going to be an interesting challenge (cf. https://github.com/zetaweb/www.gittip.com/issues/129#issuecomment-12341021). :-)

@joelmccracken My thought was a fund is a set of percentages:

40% Adrian Holovaty -> Django
20% Guido van Rossum -> Python
10% Linus Torvalds -> Linux
10% Guido van Rossum
10% Adrian Holovaty
10% ACDC

That means: "Give 10% directly to the Gittip accounts of Guido, Adrian, and ACDC, then distribute 40% amongst whatever Gittip accounts are represented by Adrian's 'Django' fund, and 20% amongst whatever Gittip accounts are underneath Guido's 'Python' fund, and the remaining 10% amongst accounts under Linus' 'Linux' fund."

The simple case is that you'll have a single fund—you won't even have to know the word "fund" to use it. It'll just look like a set of percentages as above, and then a big fat "$100.00" at the top, which is the total amount you're giving each week; it will be split up according to the percentages. On payday we'd do a lot of number crunching to determine exactly where your money goes.

Does that make any sense?

@joelmccracken

This comment has been minimized.

Show comment
Hide comment
@joelmccracken

joelmccracken Jan 18, 2013

it does! Very good =D

On Fri, Jan 18, 2013 at 3:57 PM, Chad Whitacre notifications@github.comwrote:

@justinabrahms https://github.com/justinabrahms Yeah, designing the
algorithm for computing concrete gift amounts each payday from the graph of
funds is going to be an interesting challenge (cf. #129https://github.com/zetaweb/www.gittip.com/issues/129#issuecomment-12341021).
:-)

@joelmccracken https://github.com/joelmccracken My thought was a fund
is a set of percentages:

40% Adrian Holovaty/Django
20% Guido van Rossum/Python
10% Linus Torvalds/Linux
10% Guido van Rossum
10% Adrian Holovaty
10% ACDC

That means: "Give 10% directly to the Gittip accounts of Guido, Adrian,
and ACDC, then distribute 40% amongst whatever Gittip accounts are
represented by Adrian's 'Django' fund, and 20% amongst whatever Gittip
accounts are underneath Guido's 'Python' fund, and the remaining 10%
amongst accounts under Linus' 'Linux' fund."

The simple case is that you'll have a single fund—you won't even have to
know the word "fund" to use it. It'll just look like a set of percentages
as above, and then a big fat "$100.00" at the top, which is the total
amount you're giving each week; it will be split up according to the
percentages. On payday we'd do a lot of number crunching to determine
exactly where your money goes.

Does that make any sense?


Reply to this email directly or view it on GitHubhttps://github.com/zetaweb/www.gittip.com/issues/449#issuecomment-12440694.

it does! Very good =D

On Fri, Jan 18, 2013 at 3:57 PM, Chad Whitacre notifications@github.comwrote:

@justinabrahms https://github.com/justinabrahms Yeah, designing the
algorithm for computing concrete gift amounts each payday from the graph of
funds is going to be an interesting challenge (cf. #129https://github.com/zetaweb/www.gittip.com/issues/129#issuecomment-12341021).
:-)

@joelmccracken https://github.com/joelmccracken My thought was a fund
is a set of percentages:

40% Adrian Holovaty/Django
20% Guido van Rossum/Python
10% Linus Torvalds/Linux
10% Guido van Rossum
10% Adrian Holovaty
10% ACDC

That means: "Give 10% directly to the Gittip accounts of Guido, Adrian,
and ACDC, then distribute 40% amongst whatever Gittip accounts are
represented by Adrian's 'Django' fund, and 20% amongst whatever Gittip
accounts are underneath Guido's 'Python' fund, and the remaining 10%
amongst accounts under Linus' 'Linux' fund."

The simple case is that you'll have a single fund—you won't even have to
know the word "fund" to use it. It'll just look like a set of percentages
as above, and then a big fat "$100.00" at the top, which is the total
amount you're giving each week; it will be split up according to the
percentages. On payday we'd do a lot of number crunching to determine
exactly where your money goes.

Does that make any sense?


Reply to this email directly or view it on GitHubhttps://github.com/zetaweb/www.gittip.com/issues/449#issuecomment-12440694.

@chadwhitacre

This comment has been minimized.

Show comment
Hide comment
Contributor

chadwhitacre commented Jan 18, 2013

A heart coin for you!

@MikeFair

This comment has been minimized.

Show comment
Hide comment
@MikeFair

MikeFair Jan 19, 2013

I'd like to allow creators to be a member of their own funds for a number of reasons including A) the ease with which a creator could work around such a restriction, B) the difficulty of implementing such a restriction at this time, and C) This kind of thing is more 'one man's opinion' versus functional requirement and therefore it's better to give the gifters/upstream funds a tool/setting to express such an opinion. For instance, a checkbox that says "This fund does not allocate to the creator's of the funds it contains" or a user that is able to mark "Reallocate tips for the creator's of funds" in their profile.

  1. One of the things I would fully expect to see would be Chad's name in the GitTip fund. Something would seem off if it wasn't there.... (though if I am reading the idea right, the contents of these funds are a big black box atm.)

  2. Another tool that comes into effect to help with the actual dollar amount allocation of these funds is the user's "target tip amount" (assuming they've set one).

The formula:
[User's Target Tip Amount] / [Total Amount of All Tips Given] = [User's Percentage of Doneness]

Aside from the weight an item in a fund could have other settings (and this likely should go in as a separate GitHub ticket to be worked on after funds themselves are initially working).
Two of these setting I expect to be a hard lower and upper bound for how much to give.

For instance,
"Lower Bound = If this minimum amount cannot be allocated then allocate nothing and redistribute the weight to the rest of the fund"
and
"Upper Bound = Any amount allocated over this amount should be redistributed to the other items in this fund"

With an upper bound capability, a creator could allocate to themselves but set the upper bound to be "when user's doneness reaches 100%" which is a way of saying "Do not allocate any more than their target amount".

These bounds are cumulative to the total amount the user is to be tipped that payday.
This means that if a single user receives tips through many projects or sources, (and assuming the upper bound setting of 100% doneness is in effect) then on payday the user will receive up to their target amount and no more. Any excess amount will get redistributed to the other items.

I'd like to allow creators to be a member of their own funds for a number of reasons including A) the ease with which a creator could work around such a restriction, B) the difficulty of implementing such a restriction at this time, and C) This kind of thing is more 'one man's opinion' versus functional requirement and therefore it's better to give the gifters/upstream funds a tool/setting to express such an opinion. For instance, a checkbox that says "This fund does not allocate to the creator's of the funds it contains" or a user that is able to mark "Reallocate tips for the creator's of funds" in their profile.

  1. One of the things I would fully expect to see would be Chad's name in the GitTip fund. Something would seem off if it wasn't there.... (though if I am reading the idea right, the contents of these funds are a big black box atm.)

  2. Another tool that comes into effect to help with the actual dollar amount allocation of these funds is the user's "target tip amount" (assuming they've set one).

The formula:
[User's Target Tip Amount] / [Total Amount of All Tips Given] = [User's Percentage of Doneness]

Aside from the weight an item in a fund could have other settings (and this likely should go in as a separate GitHub ticket to be worked on after funds themselves are initially working).
Two of these setting I expect to be a hard lower and upper bound for how much to give.

For instance,
"Lower Bound = If this minimum amount cannot be allocated then allocate nothing and redistribute the weight to the rest of the fund"
and
"Upper Bound = Any amount allocated over this amount should be redistributed to the other items in this fund"

With an upper bound capability, a creator could allocate to themselves but set the upper bound to be "when user's doneness reaches 100%" which is a way of saying "Do not allocate any more than their target amount".

These bounds are cumulative to the total amount the user is to be tipped that payday.
This means that if a single user receives tips through many projects or sources, (and assuming the upper bound setting of 100% doneness is in effect) then on payday the user will receive up to their target amount and no more. Any excess amount will get redistributed to the other items.

@MikeFair

This comment has been minimized.

Show comment
Hide comment
@MikeFair

MikeFair Jan 19, 2013

I'd like see the fund's contents to be public rather than private.

On larger projects I fully expect there to be multiple versions of a fund for the same project because people have different opinions about who the important people are and how important they are relative to the other items in the fund.

Being able to see the contents of the fund and hearing their creator's describe how they are constructed is an important aspect of the ongoing nature of selecting which funds to give to.

A user's specific allocations would be private, other people can't see who specifically is giving to what fund as that's more about personal choices, but what would remain visible is the total aggregate amount that each fund is receiving.

I'd like see the fund's contents to be public rather than private.

On larger projects I fully expect there to be multiple versions of a fund for the same project because people have different opinions about who the important people are and how important they are relative to the other items in the fund.

Being able to see the contents of the fund and hearing their creator's describe how they are constructed is an important aspect of the ongoing nature of selecting which funds to give to.

A user's specific allocations would be private, other people can't see who specifically is giving to what fund as that's more about personal choices, but what would remain visible is the total aggregate amount that each fund is receiving.

@chadwhitacre

This comment has been minimized.

Show comment
Hide comment
@chadwhitacre

chadwhitacre Jan 19, 2013

Contributor

@MikeFair

I'd like to allow creators to be a member of their own funds.
I'd like see the fund's contents to be public rather than private.
(though if I am reading the idea right [...])

You're right that these go together: creators aren't allowed to be members of their own funds precisely because funds are private.

the ease with which a creator could work around such a restriction

Sure. If I'm Linus, I can create a shadow account and include that in my "Linux" fund, and then when people give to my Linux fund I'm able to recover some or all of that for myself. The restriction on creators in their own funds is a soft constraint intended to steer the growth of the community, not a hard line. We wouldn't even make this a terms of service violation; it'd be an acknowledged workaround: "If you want to receive money from your own fund, create a second account." We want to discourage people from being greedy on Gittip, but the point is that we trust Linus. The "no creators" constraint removes 99% of the burden of proof from Linus; his reputation for trustworthiness easily makes up the other 1%. And if he really, really wants to include himself in his own fund, he can.

the difficulty of implementing such a restriction at this time

Not sure I see this one. Does this refer to some constraint with the CouchDB option? If so, the answer is that, if at all possible, user experience drives technology, not vice versa.

C) This kind of thing is more 'one man's opinion' versus functional requirement

It's part of the defining character of Gittip that gifts are anonymous in the particulars (we had a ticket for making tips non-anonymous, #326; I just closed it). Private funds fit that character better than public funds. If we try to support both public and private funds we muddy the product.

and therefore it's better to give the gifters/upstream funds a tool/setting to express such an opinion. For instance, a checkbox that says "This fund does not allocate to the creator's of the funds it contains" or a user that is able to mark "Reallocate tips for the creator's of funds" in their profile.

I'd like to see this work the other way around: Just as Linus is free to create a second account, he is free to disclose what his funds contain. Disclosure is not a violation of Gittip's terms of service, but Gittip is not going to do it for you.

Being able to see the contents of the fund and hearing their creator's describe how they are constructed is an important aspect of the ongoing nature of selecting which funds to give to.

If that's true then it seems like when we roll this out we'll expect to see a) demand for public funds, b) funds that disclose elsewhere do better than funds that don't, c) third party tools to implement public funds on top of Gittip. For now, I'd like to move forward with private funds, depending primarily on the reputation of the fund managers to drive fund selection. We can revisit as needed once we have this out there for a while.

Contributor

chadwhitacre commented Jan 19, 2013

@MikeFair

I'd like to allow creators to be a member of their own funds.
I'd like see the fund's contents to be public rather than private.
(though if I am reading the idea right [...])

You're right that these go together: creators aren't allowed to be members of their own funds precisely because funds are private.

the ease with which a creator could work around such a restriction

Sure. If I'm Linus, I can create a shadow account and include that in my "Linux" fund, and then when people give to my Linux fund I'm able to recover some or all of that for myself. The restriction on creators in their own funds is a soft constraint intended to steer the growth of the community, not a hard line. We wouldn't even make this a terms of service violation; it'd be an acknowledged workaround: "If you want to receive money from your own fund, create a second account." We want to discourage people from being greedy on Gittip, but the point is that we trust Linus. The "no creators" constraint removes 99% of the burden of proof from Linus; his reputation for trustworthiness easily makes up the other 1%. And if he really, really wants to include himself in his own fund, he can.

the difficulty of implementing such a restriction at this time

Not sure I see this one. Does this refer to some constraint with the CouchDB option? If so, the answer is that, if at all possible, user experience drives technology, not vice versa.

C) This kind of thing is more 'one man's opinion' versus functional requirement

It's part of the defining character of Gittip that gifts are anonymous in the particulars (we had a ticket for making tips non-anonymous, #326; I just closed it). Private funds fit that character better than public funds. If we try to support both public and private funds we muddy the product.

and therefore it's better to give the gifters/upstream funds a tool/setting to express such an opinion. For instance, a checkbox that says "This fund does not allocate to the creator's of the funds it contains" or a user that is able to mark "Reallocate tips for the creator's of funds" in their profile.

I'd like to see this work the other way around: Just as Linus is free to create a second account, he is free to disclose what his funds contain. Disclosure is not a violation of Gittip's terms of service, but Gittip is not going to do it for you.

Being able to see the contents of the fund and hearing their creator's describe how they are constructed is an important aspect of the ongoing nature of selecting which funds to give to.

If that's true then it seems like when we roll this out we'll expect to see a) demand for public funds, b) funds that disclose elsewhere do better than funds that don't, c) third party tools to implement public funds on top of Gittip. For now, I'd like to move forward with private funds, depending primarily on the reputation of the fund managers to drive fund selection. We can revisit as needed once we have this out there for a while.

@chadwhitacre

This comment has been minimized.

Show comment
Hide comment
@chadwhitacre

chadwhitacre Jan 19, 2013

Contributor

@MikeFair

  1. Another tool that comes into effect [...]

I'm reading this as unrelated to the matter of funds being private (correct me if I'm wrong). This gets into the details of computing amounts from funds on payday, right?

Contributor

chadwhitacre commented Jan 19, 2013

@MikeFair

  1. Another tool that comes into effect [...]

I'm reading this as unrelated to the matter of funds being private (correct me if I'm wrong). This gets into the details of computing amounts from funds on payday, right?

@MikeFair

This comment has been minimized.

Show comment
Hide comment
@MikeFair

MikeFair Jan 19, 2013

Four things:

  1. There's a distinction between a user's "Basket" (where they personally are giving) and a "Fund" (some opinion/allocation function other people use to select where to give).

A user's Basket is clearly a private affair.
A Fund/Allocation Algorithm I consider a public affair.

  1. The model I'm working within is that we are always doing direct person to person gifts.
    We do not ever give to funds, we are always giving directly to other people.

So in this model, what's a fund then?

A Fund (and the nesting of funds) is an organizational tool to aid us in the process of selecting the people we're going to give to and how much we're going to give to each person.

A Fund is a mechanism to, in essence, create a function call that expands one label in our own basket into a list of people and amounts.

In this model none of us give anything to Linus' fund.
We use the allocation function Linus published (called a Fund) in the execution of our own allocation function.
This is conceptually no different than executing other's published algorithms in our own computer systems.

Since the model is that it's actually the user's basket that is executing some allocation function to figure out who to give to, this makes these allocation functions more akin to source code in that respect.

This then creates a very consistent story of what's happening on payday that exactly matches what's already happening today.

Since I've subscribed some fraction of my giving to Linus' fund, then what happens on payday is my basket executes the 'Linus' Linux Kernel Fund' allocation function which expands to a list of people that I'm going to give to and how much I'm going to give them this payday.

I'm still the one ultimately allocating those direct gifts, I'm using Linus' opinion as part of my selection process.

The anonymity of where the money came from to the recipient in this point of view remains intact.
This remains consistent with exactly what's already happening right now, people choosing people to tip.

  1. Relating to creator's not being able to put themselves into the fund.
    A fund can come from anything and be about anything.
    It's not only the simple case that a single individual creates the Fund.

It could be created by anything, a person, a committee, the outcome of some election, a computer algorithm; anything that is capable of creating a list of names and weights. This makes it really hard to determine who the creator is in some of these cases (and what's driving the difficulty of enforcement).

For example, if Debian chooses to hold a semi-annual election to reconstruct the contents of the Debian Allocation Fund each period, we're obviously not going to say that all the people who voted can't be in it (and how would we know who they were anyway), it makes sense the sitting president of Debian would likely be in the Fund despite that's also the most likely person to take responsibility for publishing the fund...

What gives a fund power is people subscribing to it and pulling the function/algorithm that it represents into their own basket and using it to make their allocations with.

If people think the creators, however the chooser defines creator, shouldn't put themselves into their own funds, they won't subscribe to or use that person's allocation algorithm thereby removing power from that Fund.

As the Funds contents are public in this model, if someone didn't want to give to a Fund's creator, the person would be able to create their own new Fund and construct it like this:
"Start with the Published fund I don't like because the creator is in there and apply this transform to remove the creator."
Once published, other people could subscribe to that version of the Fund if that was their wish.
(In fact I've always envisioned that such a transform (think of it like an XSLT), in and of itself, would be a popular published code snippet to be applied by people in many cases.)

Now instead of subscribing directly to the other fund, they subscribe to their own version of it.

I haven't discussed the pipelining of Fund transformations yet as I felt it was way too early in the implementation to bring it up.

Like with our source code, I envisioned there would be a whole suite of tools to aid us in putting together these Funds. Allowing people to start with someone else's list of names (source code) and then apply a series of transforms (like a series of patches/commits/XSLTs) as a pipeline or build process an integral part of the equation to make it so that we can easily take other people's ideas for where to give and transform it to make it our own.

It's the same thing we do with our code, applied in a new context.

  1. All that said, it seems the main concern here is having people be able to figure out how much they are getting through a particular fund somehow reducing the anonymity of the gifts to the recipient.

I've somewhat addressed that by showing how the Funds are functions people begin with, not direct allocators.

Instead of keeping the Fund's contents private keep the amounts that Fund is responsible for allocating private.
This way everyone can see what these Funds contain, but not how much is going where as a result of that.

So instead of a big $100 at the top of the Fund (because we aren't actually tipping a fund, the fund is a function people are using to figure out who and how much to tip), perhaps we assign some impact scores that say something about the fund's popularity, influence, and impact that it's having within the social framework.

Perhaps a "download count" or "number of subscribers". This keeps private how much came from where and to where and is simply saying how popular this Fund is for people to use in their allocation algorithms.

Four things:

  1. There's a distinction between a user's "Basket" (where they personally are giving) and a "Fund" (some opinion/allocation function other people use to select where to give).

A user's Basket is clearly a private affair.
A Fund/Allocation Algorithm I consider a public affair.

  1. The model I'm working within is that we are always doing direct person to person gifts.
    We do not ever give to funds, we are always giving directly to other people.

So in this model, what's a fund then?

A Fund (and the nesting of funds) is an organizational tool to aid us in the process of selecting the people we're going to give to and how much we're going to give to each person.

A Fund is a mechanism to, in essence, create a function call that expands one label in our own basket into a list of people and amounts.

In this model none of us give anything to Linus' fund.
We use the allocation function Linus published (called a Fund) in the execution of our own allocation function.
This is conceptually no different than executing other's published algorithms in our own computer systems.

Since the model is that it's actually the user's basket that is executing some allocation function to figure out who to give to, this makes these allocation functions more akin to source code in that respect.

This then creates a very consistent story of what's happening on payday that exactly matches what's already happening today.

Since I've subscribed some fraction of my giving to Linus' fund, then what happens on payday is my basket executes the 'Linus' Linux Kernel Fund' allocation function which expands to a list of people that I'm going to give to and how much I'm going to give them this payday.

I'm still the one ultimately allocating those direct gifts, I'm using Linus' opinion as part of my selection process.

The anonymity of where the money came from to the recipient in this point of view remains intact.
This remains consistent with exactly what's already happening right now, people choosing people to tip.

  1. Relating to creator's not being able to put themselves into the fund.
    A fund can come from anything and be about anything.
    It's not only the simple case that a single individual creates the Fund.

It could be created by anything, a person, a committee, the outcome of some election, a computer algorithm; anything that is capable of creating a list of names and weights. This makes it really hard to determine who the creator is in some of these cases (and what's driving the difficulty of enforcement).

For example, if Debian chooses to hold a semi-annual election to reconstruct the contents of the Debian Allocation Fund each period, we're obviously not going to say that all the people who voted can't be in it (and how would we know who they were anyway), it makes sense the sitting president of Debian would likely be in the Fund despite that's also the most likely person to take responsibility for publishing the fund...

What gives a fund power is people subscribing to it and pulling the function/algorithm that it represents into their own basket and using it to make their allocations with.

If people think the creators, however the chooser defines creator, shouldn't put themselves into their own funds, they won't subscribe to or use that person's allocation algorithm thereby removing power from that Fund.

As the Funds contents are public in this model, if someone didn't want to give to a Fund's creator, the person would be able to create their own new Fund and construct it like this:
"Start with the Published fund I don't like because the creator is in there and apply this transform to remove the creator."
Once published, other people could subscribe to that version of the Fund if that was their wish.
(In fact I've always envisioned that such a transform (think of it like an XSLT), in and of itself, would be a popular published code snippet to be applied by people in many cases.)

Now instead of subscribing directly to the other fund, they subscribe to their own version of it.

I haven't discussed the pipelining of Fund transformations yet as I felt it was way too early in the implementation to bring it up.

Like with our source code, I envisioned there would be a whole suite of tools to aid us in putting together these Funds. Allowing people to start with someone else's list of names (source code) and then apply a series of transforms (like a series of patches/commits/XSLTs) as a pipeline or build process an integral part of the equation to make it so that we can easily take other people's ideas for where to give and transform it to make it our own.

It's the same thing we do with our code, applied in a new context.

  1. All that said, it seems the main concern here is having people be able to figure out how much they are getting through a particular fund somehow reducing the anonymity of the gifts to the recipient.

I've somewhat addressed that by showing how the Funds are functions people begin with, not direct allocators.

Instead of keeping the Fund's contents private keep the amounts that Fund is responsible for allocating private.
This way everyone can see what these Funds contain, but not how much is going where as a result of that.

So instead of a big $100 at the top of the Fund (because we aren't actually tipping a fund, the fund is a function people are using to figure out who and how much to tip), perhaps we assign some impact scores that say something about the fund's popularity, influence, and impact that it's having within the social framework.

Perhaps a "download count" or "number of subscribers". This keeps private how much came from where and to where and is simply saying how popular this Fund is for people to use in their allocation algorithms.

@MikeFair

This comment has been minimized.

Show comment
Hide comment
@MikeFair

MikeFair Jan 19, 2013

@whit537

  1. Another tool that comes into effect [...]

I'm reading this as unrelated to the matter of funds being private (correct me if I'm wrong).
This gets into the details of computing amounts from funds on payday, right?

It's separate, and not so much about privacy as it is creator's being members of their own Funds.
The idea addresses the concern of people being greedy and using funds to try and suck money from the system.

Ironically, the most powerful tool we would have to audit these Funds to ensure they represent what they claim to represent, that is making their contents public, is exactly what you're suggesting we don't do.

Making funds public is how we create oversight for these otherwise black boxes. It's how we as a community go about forming our opinions about and distinguishing between the funds we want to give to, just like what we do to choose what people we want to give to.

I can publish a GitTip fund just as easily as Chad can and so there'd be two GitTip funds in that case and people could choose between them. Without making them public how could anyone make a sane choice between them?

We also might just as easily say, the contents of the project's official fund is some formula that combines the ideas from the Fund's published by these people (a form of board of directors).

Seeing the results of that formula is an easy way to audit that the board is doing a good job. Over time, a trust develops where people aren't so sensitive, but the ability to trust but verify what's in a fund is always there.

  1. Making the contents public won't change what it takes for a Fund to get subscribers which is how a Fund gets power to be abused in the first place (who cares if you get 100% of nothing).

If anything, the openness helps prove that it's doing exactly what it's supposed to be doing.

If it remained closed and private, and even if the contents were supposedly published elsewhere, how would anyone know that's actually what was happening. That's exactly how Bernie was able to do what he did. He told everyone one thing, but in practice something else was actually happening (and those that were supposed to validating what he was claiming didn't do a good job at that).

  1. In the end a project is the people that make it up. It's not separate from them, the body of people that take up implementing the project's mission/goals are what make the project go.
    You make the project go by making the people go.

We want people to get done.
That's how we're going to empower them to be part of the open solution.

If you as a donor can see that people are going to cap out when their targets reach 100% doneness then you've eliminated the greed factor as a concern.

They could set their fund to be 95% themselves and 5% everyone else and they won't receive a dime more than their target. Once they reach 100% doneness (regardless of how that happens), the fund becomes 100% everyone else for all remaining gifts.

The really popular people with no cap are the people introducing the most greed risk.

So my thoughts on counteracting that (and again this will need to go in as a separate issue) are that if you haven't set a target, then your target gets implicitly set according to some kind of normalization function.

The idea is that if you haven't set a target, then your target gets automatically set to somewhere to the middle of the pack preventing you from being able to abuse the Fund system for your own personal gain.

If you have set a target, then you're going to cap out at 100% doneness (a public number) and therefore again, you aren't going to be able to abuse the system for your own personal gain.

If you have said, I don't want tips, well that's the same as a target of 0 for this calculation and you pose no risk.

The most powerful tools we have to discourage greed is capping people out and making the whole thing as transparent as possible.

Now that said, exactly who you are giving to and how you are choosing to allocate (aka your personal basket) is your own private and personal information.

Thereby, recipients are not allowed to see where the money came from because it would violate the privacy policy of where someone gives is personal and private information to the giver. This is separate from whether or not the Funds, the formulas the community is using to execute their private baskets, are private.

This model makes both intuitive and practical sense to me:

  1. All donations are gifts from the giver to the receiver and always people (or at least legalized entities)

  2. All donations are the private and personal information of the giver and not shared

3a) Givers use Funds and other tools to aid in the recipient selection/allocation process
3b) Like all other tools/source code in our ecosystem, the Funds and tools used are open and shareable

@whit537

  1. Another tool that comes into effect [...]

I'm reading this as unrelated to the matter of funds being private (correct me if I'm wrong).
This gets into the details of computing amounts from funds on payday, right?

It's separate, and not so much about privacy as it is creator's being members of their own Funds.
The idea addresses the concern of people being greedy and using funds to try and suck money from the system.

Ironically, the most powerful tool we would have to audit these Funds to ensure they represent what they claim to represent, that is making their contents public, is exactly what you're suggesting we don't do.

Making funds public is how we create oversight for these otherwise black boxes. It's how we as a community go about forming our opinions about and distinguishing between the funds we want to give to, just like what we do to choose what people we want to give to.

I can publish a GitTip fund just as easily as Chad can and so there'd be two GitTip funds in that case and people could choose between them. Without making them public how could anyone make a sane choice between them?

We also might just as easily say, the contents of the project's official fund is some formula that combines the ideas from the Fund's published by these people (a form of board of directors).

Seeing the results of that formula is an easy way to audit that the board is doing a good job. Over time, a trust develops where people aren't so sensitive, but the ability to trust but verify what's in a fund is always there.

  1. Making the contents public won't change what it takes for a Fund to get subscribers which is how a Fund gets power to be abused in the first place (who cares if you get 100% of nothing).

If anything, the openness helps prove that it's doing exactly what it's supposed to be doing.

If it remained closed and private, and even if the contents were supposedly published elsewhere, how would anyone know that's actually what was happening. That's exactly how Bernie was able to do what he did. He told everyone one thing, but in practice something else was actually happening (and those that were supposed to validating what he was claiming didn't do a good job at that).

  1. In the end a project is the people that make it up. It's not separate from them, the body of people that take up implementing the project's mission/goals are what make the project go.
    You make the project go by making the people go.

We want people to get done.
That's how we're going to empower them to be part of the open solution.

If you as a donor can see that people are going to cap out when their targets reach 100% doneness then you've eliminated the greed factor as a concern.

They could set their fund to be 95% themselves and 5% everyone else and they won't receive a dime more than their target. Once they reach 100% doneness (regardless of how that happens), the fund becomes 100% everyone else for all remaining gifts.

The really popular people with no cap are the people introducing the most greed risk.

So my thoughts on counteracting that (and again this will need to go in as a separate issue) are that if you haven't set a target, then your target gets implicitly set according to some kind of normalization function.

The idea is that if you haven't set a target, then your target gets automatically set to somewhere to the middle of the pack preventing you from being able to abuse the Fund system for your own personal gain.

If you have set a target, then you're going to cap out at 100% doneness (a public number) and therefore again, you aren't going to be able to abuse the system for your own personal gain.

If you have said, I don't want tips, well that's the same as a target of 0 for this calculation and you pose no risk.

The most powerful tools we have to discourage greed is capping people out and making the whole thing as transparent as possible.

Now that said, exactly who you are giving to and how you are choosing to allocate (aka your personal basket) is your own private and personal information.

Thereby, recipients are not allowed to see where the money came from because it would violate the privacy policy of where someone gives is personal and private information to the giver. This is separate from whether or not the Funds, the formulas the community is using to execute their private baskets, are private.

This model makes both intuitive and practical sense to me:

  1. All donations are gifts from the giver to the receiver and always people (or at least legalized entities)

  2. All donations are the private and personal information of the giver and not shared

3a) Givers use Funds and other tools to aid in the recipient selection/allocation process
3b) Like all other tools/source code in our ecosystem, the Funds and tools used are open and shareable

@chadwhitacre

This comment has been minimized.

Show comment
Hide comment
@chadwhitacre

chadwhitacre Jan 21, 2013

Contributor

!m @MikeFair. Some important considerations here regarding oversight and trust.

It's not only the simple case that a single individual creates the Fund.

Likewise, it's not only the case that Gittip accounts represent single individuals. We already have accounts that represent groups, e.g.:

https://www.gittip.com/readthedocs/
https://www.gittip.com/workforpie/

For example, if Debian [...]

There'd be a "Debian" Gittip account that would be separate from the sitting president's Gittip account. I can see a need in this use case for transparency: if I'm in the Debian fund but I'm not directly controlling it, I want to be able to see that the fund is being allocated in the way we all voted for. To support this case we could allow funds to opt-in to disclosing their contents to individuals who are directly named in the fund. Perhaps #502 expands to encompass this: when you register an account on Gittip and promote it to a "group" account, then the fund disclosure rule kicks in automatically as part of that. This isn't needed for v1 of the fund feature, though.

I can publish a GitTip fund just as easily as Chad can and so there'd be two GitTip funds in that case and people could choose between them. Without making them public how could anyone make a sane choice between them?

Trust. If a person allocates 67% to MikeFair/Gittip and 33% to whit537/Gittip that means they trust you twice as much as me.

Making funds public is how we create oversight for these otherwise black boxes.

Oversight is needed when there are strings attached to the money. With government, the expectation of return is defense, infrastructure, social programs, etc. With Madoff, the expectation was a financial return on investment. With Gittip, there are no strings attached to the money. These are free gifts. What would oversight even mean here, when there are no direct expectations?

[I]t seems the main concern here is having people be able to figure out how much they are getting through a particular fund somehow reducing the anonymity of the gifts to the recipient.

The root principle is inspiring generosity. Both anonymous particulars and private funds are derived from that. When I give money to Debian on Gittip, I'm fully entrusting the money to them. I'm disclaiming any right to oversee or influence what they do with the money.

Contributor

chadwhitacre commented Jan 21, 2013

!m @MikeFair. Some important considerations here regarding oversight and trust.

It's not only the simple case that a single individual creates the Fund.

Likewise, it's not only the case that Gittip accounts represent single individuals. We already have accounts that represent groups, e.g.:

https://www.gittip.com/readthedocs/
https://www.gittip.com/workforpie/

For example, if Debian [...]

There'd be a "Debian" Gittip account that would be separate from the sitting president's Gittip account. I can see a need in this use case for transparency: if I'm in the Debian fund but I'm not directly controlling it, I want to be able to see that the fund is being allocated in the way we all voted for. To support this case we could allow funds to opt-in to disclosing their contents to individuals who are directly named in the fund. Perhaps #502 expands to encompass this: when you register an account on Gittip and promote it to a "group" account, then the fund disclosure rule kicks in automatically as part of that. This isn't needed for v1 of the fund feature, though.

I can publish a GitTip fund just as easily as Chad can and so there'd be two GitTip funds in that case and people could choose between them. Without making them public how could anyone make a sane choice between them?

Trust. If a person allocates 67% to MikeFair/Gittip and 33% to whit537/Gittip that means they trust you twice as much as me.

Making funds public is how we create oversight for these otherwise black boxes.

Oversight is needed when there are strings attached to the money. With government, the expectation of return is defense, infrastructure, social programs, etc. With Madoff, the expectation was a financial return on investment. With Gittip, there are no strings attached to the money. These are free gifts. What would oversight even mean here, when there are no direct expectations?

[I]t seems the main concern here is having people be able to figure out how much they are getting through a particular fund somehow reducing the anonymity of the gifts to the recipient.

The root principle is inspiring generosity. Both anonymous particulars and private funds are derived from that. When I give money to Debian on Gittip, I'm fully entrusting the money to them. I'm disclaiming any right to oversee or influence what they do with the money.

@carsomyr

This comment has been minimized.

Show comment
Hide comment
@carsomyr

carsomyr Jan 23, 2013

Contributor

@whit537 I believe that this is the equation we're after.

![equation](http://latex.codecogs.com/gif.latex?%28I - %28P - diag%28P%29%29%29^{-1} * diag%28P%29 * D)

Here, P is the payout matrix: Each row represents an entity i (fund or person), and each column represents the fraction of that entity i's purse paid out to entity j. Note that the entries (i, i) represent the amount of the entity's purse retained.

D is the column vector of initial purse allocations. Note that many entities will start with a purse of 0. Any nonzero purses necessarily correspond to people who are the ones giving money, and all funds start with a purse of 0 (people are kicking off the giving, after all).

The computed result is the payout column vector. I'll write this up, show my derivations, and provide examples in the near future.

Contributor

carsomyr commented Jan 23, 2013

@whit537 I believe that this is the equation we're after.

![equation](http://latex.codecogs.com/gif.latex?%28I - %28P - diag%28P%29%29%29^{-1} * diag%28P%29 * D)

Here, P is the payout matrix: Each row represents an entity i (fund or person), and each column represents the fraction of that entity i's purse paid out to entity j. Note that the entries (i, i) represent the amount of the entity's purse retained.

D is the column vector of initial purse allocations. Note that many entities will start with a purse of 0. Any nonzero purses necessarily correspond to people who are the ones giving money, and all funds start with a purse of 0 (people are kicking off the giving, after all).

The computed result is the payout column vector. I'll write this up, show my derivations, and provide examples in the near future.

@MikeFair

This comment has been minimized.

Show comment
Hide comment
@MikeFair

MikeFair Jan 24, 2013

Hey Roy,

I've been putting together a recursive SQL function that does much the same
as you've been describing though slightly different from the phases you
described, all the same math though I believe.

Step one: sum tip amounts grouped by participant (this get the totals that
represent the denominators for each fund/participant) call that "totals".

Step two:
select tipper, tippee, amount/(total for this tipper)
This creates a weight contribution that goes into each tipper from this
tipper.

Iteratively do a UNION ALL
Select tipper, tipper,
(amount / (total for this tipper)) * (the percentage from the prior
iteration)
Where this tipper is the tippee from the prior iteration and the tippee in
this iteration is not a tipper from prior a iteration on this path

This iterative multiplication puts everything in terms of a percentage of
the original total amount and avoids giving cycles. Technically a cycle
would eventually wind down to something so small we could ignore it, but
for now I'm just outright excluding them.

Step three: sum up all the weights grouped by tippee. Because a tippee can
appear in multiple paths, we want to sum up the total weight/percentage
gong to the same tippee.

Step four:
Multiply all percentages by the total for the tipper to get the actual
amounts for each tippee.

I need to test the code a bit more but expect to post it here shortly (
next day or two).
On Jan 23, 2013 1:02 PM, "Roy Liu" notifications@github.com wrote:

@whit537 https://github.com/whit537 I believe that this is the equation
we're after.

[image: equation]https://a248.e.akamai.net/camo.github.com/dc0992bd548d918ea5ccffe8a9f24630af428286/687474703a2f2f6c617465782e636f6465636f67732e636f6d2f6769662e6c617465783f28492532302d25323028502532302d2532306469616728502929292535452537422d312537442532302a253230646961672850292532302a25323044

Here, P is the payout matrix: Each row represents an entity (fund or
person), and each column represents the fraction of that entity's purse
paid out. Note that the entries (i, i) represent the amount of the
entity's purse retained.

D is the column vector of initial purse allocations. Note that many
entities will start with a purse of 0. Any nonzero purses necessarily
correspond to people who are the ones giving money, and all funds start
with a purse of 0 (people are kicking off the giving, after all).

The computed result is the payout column vector. I'll write this up, show
my derivations, and provide further examples.


Reply to this email directly or view it on GitHubhttps://github.com/zetaweb/www.gittip.com/issues/449#issuecomment-12622083.

Hey Roy,

I've been putting together a recursive SQL function that does much the same
as you've been describing though slightly different from the phases you
described, all the same math though I believe.

Step one: sum tip amounts grouped by participant (this get the totals that
represent the denominators for each fund/participant) call that "totals".

Step two:
select tipper, tippee, amount/(total for this tipper)
This creates a weight contribution that goes into each tipper from this
tipper.

Iteratively do a UNION ALL
Select tipper, tipper,
(amount / (total for this tipper)) * (the percentage from the prior
iteration)
Where this tipper is the tippee from the prior iteration and the tippee in
this iteration is not a tipper from prior a iteration on this path

This iterative multiplication puts everything in terms of a percentage of
the original total amount and avoids giving cycles. Technically a cycle
would eventually wind down to something so small we could ignore it, but
for now I'm just outright excluding them.

Step three: sum up all the weights grouped by tippee. Because a tippee can
appear in multiple paths, we want to sum up the total weight/percentage
gong to the same tippee.

Step four:
Multiply all percentages by the total for the tipper to get the actual
amounts for each tippee.

I need to test the code a bit more but expect to post it here shortly (
next day or two).
On Jan 23, 2013 1:02 PM, "Roy Liu" notifications@github.com wrote:

@whit537 https://github.com/whit537 I believe that this is the equation
we're after.

[image: equation]https://a248.e.akamai.net/camo.github.com/dc0992bd548d918ea5ccffe8a9f24630af428286/687474703a2f2f6c617465782e636f6465636f67732e636f6d2f6769662e6c617465783f28492532302d25323028502532302d2532306469616728502929292535452537422d312537442532302a253230646961672850292532302a25323044

Here, P is the payout matrix: Each row represents an entity (fund or
person), and each column represents the fraction of that entity's purse
paid out. Note that the entries (i, i) represent the amount of the
entity's purse retained.

D is the column vector of initial purse allocations. Note that many
entities will start with a purse of 0. Any nonzero purses necessarily
correspond to people who are the ones giving money, and all funds start
with a purse of 0 (people are kicking off the giving, after all).

The computed result is the payout column vector. I'll write this up, show
my derivations, and provide further examples.


Reply to this email directly or view it on GitHubhttps://github.com/zetaweb/www.gittip.com/issues/449#issuecomment-12622083.

@carsomyr

This comment has been minimized.

Show comment
Hide comment
@carsomyr

carsomyr Jan 24, 2013

Contributor

@MikeFair Looks about right, although we should account for cycles in a formal way. If not, that would be like a bank programmer using floats to represent monetary sums, and everyone cites that as a canonical example of why representation is important. Note that the matrix inversion in my formula is actually encoding repeated tipping iterations in a geometric sum of matrices. If matrix inversion is too intensive, we can use a sparse representation, compute successive terms of the geometric sum, and stop whenever the total change in all tips for everyone is less than a penny.

Also, what do you mean by "the percentage from the prior iteration"? Are you able to write down your calculations as matrix multiplications? Verbiage without code has the potential for multiple interpretations. Looking forward to seeing the SQL! A useful tool for testing all of this is GNU Octave; you can input payout matrices and see what happens under successive iterations.

Contributor

carsomyr commented Jan 24, 2013

@MikeFair Looks about right, although we should account for cycles in a formal way. If not, that would be like a bank programmer using floats to represent monetary sums, and everyone cites that as a canonical example of why representation is important. Note that the matrix inversion in my formula is actually encoding repeated tipping iterations in a geometric sum of matrices. If matrix inversion is too intensive, we can use a sparse representation, compute successive terms of the geometric sum, and stop whenever the total change in all tips for everyone is less than a penny.

Also, what do you mean by "the percentage from the prior iteration"? Are you able to write down your calculations as matrix multiplications? Verbiage without code has the potential for multiple interpretations. Looking forward to seeing the SQL! A useful tool for testing all of this is GNU Octave; you can input payout matrices and see what happens under successive iterations.

@MikeFair

This comment has been minimized.

Show comment
Hide comment
@MikeFair

MikeFair Jan 24, 2013

If you're familiar with CTE (Common Table Expressions) they can be used to
write "recursive" queries. Technically they are iterative queries, they
arr able to use the resultset from a query as part of the FROM clause for
another query. They continue to execute until the "recursive part" of the
query returns an empty result set.

So a really simple incomplete example would be:
WITH RECURSIVE totals(tipper, total) (
/* query to select totals grouped by tipper */
),
payday(funder, total, tipper, tippee, percentage, depth) (
Select t.tipper funder, tot.total total, t.tipper, t.tippee,
(t.amount/tot.total) percentage
From tips t inner join totals tot on t.tipper=tot.tipper
UNION ALL
select p.funder, p.total, t.tipper, t.tippee, ((t.amount/tot.total) *
p.percentage) percentage
From tips t
inner join totals tot on t.tipper=tot.tipper
Inner join payday p on p.tippee=t.tipper
)
Select * from payday

The totals CTE is not recursive, it creates a subquery named "totals" that
acts like a view within the scope of the query.

The payday query also creates the equivalent of a view called payday for
the scope of the query, but this one is recursive. It executes the first
part, and then iteratively executes the second part (the part after the
"union all"). In the second part, the reference to payday is a reference
to the results of the prior iteration and therefore useable for this
iteration. When the second part returns the empty set (aka there are no
more results to be added), it stops.

Using this, the query is able to continuously multiply the weight from the
row on which the new tipper was a tippee to the current weight of this row.
This normalizes all the weightings to the total amount provided by the
original top level funder.

The steps I didn't do here were take the results of payday and sum
percentage grouping by funder and tippee; and then take those results and
multiply them again by the funder's total to get back to a dollar figure.

The real trick however is going to be that after we've got this basic part
working there's going to be a lot of sub-pennies running around. My plan,
though I'm sure it will need to be discussed, is to track/accumulate these
sub-pennies until they accrue to something contributable.

In the same vein, it's totally possible that a tippee receives less than
the minimum allocation amount from each tipper, but that the aggregate of
all the tippers together is above the minimum allocation amount.

My plan here is to credit the tippee that total anyway, drawing the funds
from one (or maybe a couple) of the tippers, but earmark the debit from the
other tippers whose accounts didn't contribute. Then next time this kind
of allocation is needed, it'll come from those who didn't do it last time.
I'm looking at this as a different kind of tip that is more amount focused
than refurring focused. To track the debit of these other tippers the
system would automatically generates a kind of IOU tip from those that
didn't pitch in to those that did. The system then uses these imbalances to
select which tippers to draw from first next time keeping everything in
balance overall.

This might be overcomplicating the problem a bit but I don't think so.
Once you start doing percentages of percentages the numbers get small
quickly and it's not like the tippers are starting off with large amounts
and that combination tells me these sub-penny amounts are going to be a
real phenomena that we'll want a good way to address.

As for cycles, a naive, ignore them and let the system crank on the math
until the percentages get too small will do the right thing (although
expensively).

I think we can do better though.
I say we start a new thread to deal with handling cycles once we have some
kind of fund implementation working in which cycles can occur. There's
going to be a way to precompute the weight of each cycle and simply
redistribute that weight to the other non-cyclic nodes as we iterate
through but that's not now. At least not for me. :)

What do you think/see?
Make sense?
On Jan 23, 2013 5:29 PM, "Roy Liu" notifications@github.com wrote:

@MikeFair https://github.com/MikeFair Looks about right, although we
should account for cycles in a formal way. If not, that would be like a
bank programmer using floats to represent monetary sums, and everyone uses
that example as a canonical example of why representation is important.
Note that the matrix inversion in my formula is actually encoding repeated
tipping iterations in a geometric sum of matriceshttp://math.stackexchange.com/questions/21055/a-geometric-infinite-sum-of-matrices.
If matrix inversion is too intensive, we can use a sparse representation,
compute successive terms of the geometric sum, and stop whenever the total
change in all tips for everyone is less than a penny.

Also, what do you mean by "the percentage from the prior iteration"?


Reply to this email directly or view it on GitHubhttps://github.com/zetaweb/www.gittip.com/issues/449#issuecomment-12632755.

If you're familiar with CTE (Common Table Expressions) they can be used to
write "recursive" queries. Technically they are iterative queries, they
arr able to use the resultset from a query as part of the FROM clause for
another query. They continue to execute until the "recursive part" of the
query returns an empty result set.

So a really simple incomplete example would be:
WITH RECURSIVE totals(tipper, total) (
/* query to select totals grouped by tipper */
),
payday(funder, total, tipper, tippee, percentage, depth) (
Select t.tipper funder, tot.total total, t.tipper, t.tippee,
(t.amount/tot.total) percentage
From tips t inner join totals tot on t.tipper=tot.tipper
UNION ALL
select p.funder, p.total, t.tipper, t.tippee, ((t.amount/tot.total) *
p.percentage) percentage
From tips t
inner join totals tot on t.tipper=tot.tipper
Inner join payday p on p.tippee=t.tipper
)
Select * from payday

The totals CTE is not recursive, it creates a subquery named "totals" that
acts like a view within the scope of the query.

The payday query also creates the equivalent of a view called payday for
the scope of the query, but this one is recursive. It executes the first
part, and then iteratively executes the second part (the part after the
"union all"). In the second part, the reference to payday is a reference
to the results of the prior iteration and therefore useable for this
iteration. When the second part returns the empty set (aka there are no
more results to be added), it stops.

Using this, the query is able to continuously multiply the weight from the
row on which the new tipper was a tippee to the current weight of this row.
This normalizes all the weightings to the total amount provided by the
original top level funder.

The steps I didn't do here were take the results of payday and sum
percentage grouping by funder and tippee; and then take those results and
multiply them again by the funder's total to get back to a dollar figure.

The real trick however is going to be that after we've got this basic part
working there's going to be a lot of sub-pennies running around. My plan,
though I'm sure it will need to be discussed, is to track/accumulate these
sub-pennies until they accrue to something contributable.

In the same vein, it's totally possible that a tippee receives less than
the minimum allocation amount from each tipper, but that the aggregate of
all the tippers together is above the minimum allocation amount.

My plan here is to credit the tippee that total anyway, drawing the funds
from one (or maybe a couple) of the tippers, but earmark the debit from the
other tippers whose accounts didn't contribute. Then next time this kind
of allocation is needed, it'll come from those who didn't do it last time.
I'm looking at this as a different kind of tip that is more amount focused
than refurring focused. To track the debit of these other tippers the
system would automatically generates a kind of IOU tip from those that
didn't pitch in to those that did. The system then uses these imbalances to
select which tippers to draw from first next time keeping everything in
balance overall.

This might be overcomplicating the problem a bit but I don't think so.
Once you start doing percentages of percentages the numbers get small
quickly and it's not like the tippers are starting off with large amounts
and that combination tells me these sub-penny amounts are going to be a
real phenomena that we'll want a good way to address.

As for cycles, a naive, ignore them and let the system crank on the math
until the percentages get too small will do the right thing (although
expensively).

I think we can do better though.
I say we start a new thread to deal with handling cycles once we have some
kind of fund implementation working in which cycles can occur. There's
going to be a way to precompute the weight of each cycle and simply
redistribute that weight to the other non-cyclic nodes as we iterate
through but that's not now. At least not for me. :)

What do you think/see?
Make sense?
On Jan 23, 2013 5:29 PM, "Roy Liu" notifications@github.com wrote:

@MikeFair https://github.com/MikeFair Looks about right, although we
should account for cycles in a formal way. If not, that would be like a
bank programmer using floats to represent monetary sums, and everyone uses
that example as a canonical example of why representation is important.
Note that the matrix inversion in my formula is actually encoding repeated
tipping iterations in a geometric sum of matriceshttp://math.stackexchange.com/questions/21055/a-geometric-infinite-sum-of-matrices.
If matrix inversion is too intensive, we can use a sparse representation,
compute successive terms of the geometric sum, and stop whenever the total
change in all tips for everyone is less than a penny.

Also, what do you mean by "the percentage from the prior iteration"?


Reply to this email directly or view it on GitHubhttps://github.com/zetaweb/www.gittip.com/issues/449#issuecomment-12632755.

@carsomyr

This comment has been minimized.

Show comment
Hide comment
@carsomyr

carsomyr Jan 24, 2013

Contributor

@MikeFair I'm confused: In your calculation, percentages seem to be a fluctuating amount. I thought funds paid each other at fixed percentages? Tell you what, let's run through an example. How about the following matrix?

0.8 0.2
0.1 0.9

In other words, person 1 tips person 2 20%. Person 2 tips person 1 10%. Let's say each person starts with $1. According to my formula, the payouts come out to (0.89796, 1.10204). What would the SQL arrive at?

FYI, this is the Octave/Matlab code.

> a = [0.8 0.2; 0.1 0.9]
> diag_a = [0.8 0; 0 0.9]
> [1 1] * inverse(eye(2) - (a - diag_a)) * diag_a

The last line is the computation for the payout vector.

Contributor

carsomyr commented Jan 24, 2013

@MikeFair I'm confused: In your calculation, percentages seem to be a fluctuating amount. I thought funds paid each other at fixed percentages? Tell you what, let's run through an example. How about the following matrix?

0.8 0.2
0.1 0.9

In other words, person 1 tips person 2 20%. Person 2 tips person 1 10%. Let's say each person starts with $1. According to my formula, the payouts come out to (0.89796, 1.10204). What would the SQL arrive at?

FYI, this is the Octave/Matlab code.

> a = [0.8 0.2; 0.1 0.9]
> diag_a = [0.8 0; 0 0.9]
> [1 1] * inverse(eye(2) - (a - diag_a)) * diag_a

The last line is the computation for the payout vector.

@carsomyr

This comment has been minimized.

Show comment
Hide comment
@carsomyr

carsomyr Jan 24, 2013

Contributor

@MikeFair Also, if you're going to read the entire tipping table multiple times anyways, why not just implement the formula and use something like NumPy's sparse array package to represent whole iterations as matrix multiplication? Scientific computing is what Python excels at.

Contributor

carsomyr commented Jan 24, 2013

@MikeFair Also, if you're going to read the entire tipping table multiple times anyways, why not just implement the formula and use something like NumPy's sparse array package to represent whole iterations as matrix multiplication? Scientific computing is what Python excels at.

@MikeFair

This comment has been minimized.

Show comment
Hide comment
@MikeFair

MikeFair Jan 25, 2013

@carsomyr
Because it's what I know.
I don't know about NumPy (I can't even say I know Python really), I don't think in matrix terms the way you can, and I was writing code that I was thinking others would really be able to understand (and remake to be better). ;)

I'm excited to learn that you think in matrices and use MatLab/Octave et al.
A lot of what I'd like to do in the future relies on stuff like Octave and can take advantage of CUDA, et al. to generate aggregated information on the various graphs and what not we can produce to push up to websites.

That all said, SQL excels at set operations, and I saw this as a set operations challenge. :)
(I'll make another post just after this explaining the math part a bit better)

To your point about reading the table many times, that's why SQL Server has creatable indeces.

In the end, all the same math operations need to be performed, the question is more a matter of what's the process used to do all the math.

The query would be executed entirely on the SQL server CPU and then be handed back to the web server.

The database could/would be optimized with a tippee/tipper index on the tips table (and if we can precompute using triggers and/or cron jobs it will be even better). This means we won't ever be doing full table scans, we're doing fast B-Tree lookups where all the valid tippee/tipper links have already been prestored.

The data itself is also slowly changing (relatively speaking).
That means that more often than not, it will be the exact same numbers from prior weeks being recrunched over and over again each week. If we use triggers/cron jobs to deal with computing the results as the inserts/updates/deletes are happening in an incremental way, then the query to calculate the final answers are a really trivial task.

It was with that in mind that really cemented my thoughts about doing the whole thing on the database side. It idn't hurt that I'd already written some SQL code like this before a couple times using CTEs in SQL, so I knew that the CTE approach would work, I knew we already had Postrges installed, up, and running as a core part of the environment, and so the number of patches to the code to get this functional would be minimal. The tables and data seemed to already exist and all we really have to do is just start using it that way.

@carsomyr
Because it's what I know.
I don't know about NumPy (I can't even say I know Python really), I don't think in matrix terms the way you can, and I was writing code that I was thinking others would really be able to understand (and remake to be better). ;)

I'm excited to learn that you think in matrices and use MatLab/Octave et al.
A lot of what I'd like to do in the future relies on stuff like Octave and can take advantage of CUDA, et al. to generate aggregated information on the various graphs and what not we can produce to push up to websites.

That all said, SQL excels at set operations, and I saw this as a set operations challenge. :)
(I'll make another post just after this explaining the math part a bit better)

To your point about reading the table many times, that's why SQL Server has creatable indeces.

In the end, all the same math operations need to be performed, the question is more a matter of what's the process used to do all the math.

The query would be executed entirely on the SQL server CPU and then be handed back to the web server.

The database could/would be optimized with a tippee/tipper index on the tips table (and if we can precompute using triggers and/or cron jobs it will be even better). This means we won't ever be doing full table scans, we're doing fast B-Tree lookups where all the valid tippee/tipper links have already been prestored.

The data itself is also slowly changing (relatively speaking).
That means that more often than not, it will be the exact same numbers from prior weeks being recrunched over and over again each week. If we use triggers/cron jobs to deal with computing the results as the inserts/updates/deletes are happening in an incremental way, then the query to calculate the final answers are a really trivial task.

It was with that in mind that really cemented my thoughts about doing the whole thing on the database side. It idn't hurt that I'd already written some SQL code like this before a couple times using CTEs in SQL, so I knew that the CTE approach would work, I knew we already had Postrges installed, up, and running as a core part of the environment, and so the number of patches to the code to get this functional would be minimal. The tables and data seemed to already exist and all we really have to do is just start using it that way.

@carsomyr

This comment has been minimized.

Show comment
Hide comment
@carsomyr

carsomyr Jan 25, 2013

Contributor

@MikeFair Understood. I agree that the implementation can be kept separate from the ideas. I don't really insist on matrices either. It's just that they are:

  1. Close to the math. Expressing a problem in matrix terms is a compact way of getting your point across.
  2. Close to the implementation, too. Once the tipper/tippee information is read from the database, that's it. The computation goes down in one involved algebraic step.

Have you tried running the SQL on the two-entity example I mentioned above? I'm not trying to snow you in with math; rather, I think it's important to have a sanity check for what a procedure is doing. After all, we are dealing with people's money here. If they ask why they got paid as much as they did, we had better have a concise explanation. It's not even a big deal if our answers don't match. Much more important is a diagnosis of why they don't match and which policy most naturally models the tipping process.

Tell you what: If matrices don't suit you, can you write down the SQL in pseudocode? It would be something like "At every iteration, tippers compute the sum of their incoming tips and ...".

Alternatively, we can split up the work. If you give me two tables consisting of (tipper, tippee, percentage) rows and (initial allocation) rows, I can spit out a table of (payout) rows.

Contributor

carsomyr commented Jan 25, 2013

@MikeFair Understood. I agree that the implementation can be kept separate from the ideas. I don't really insist on matrices either. It's just that they are:

  1. Close to the math. Expressing a problem in matrix terms is a compact way of getting your point across.
  2. Close to the implementation, too. Once the tipper/tippee information is read from the database, that's it. The computation goes down in one involved algebraic step.

Have you tried running the SQL on the two-entity example I mentioned above? I'm not trying to snow you in with math; rather, I think it's important to have a sanity check for what a procedure is doing. After all, we are dealing with people's money here. If they ask why they got paid as much as they did, we had better have a concise explanation. It's not even a big deal if our answers don't match. Much more important is a diagnosis of why they don't match and which policy most naturally models the tipping process.

Tell you what: If matrices don't suit you, can you write down the SQL in pseudocode? It would be something like "At every iteration, tippers compute the sum of their incoming tips and ...".

Alternatively, we can split up the work. If you give me two tables consisting of (tipper, tippee, percentage) rows and (initial allocation) rows, I can spit out a table of (payout) rows.

@chadwhitacre

This comment has been minimized.

Show comment
Hide comment
Contributor

chadwhitacre commented Apr 12, 2013

chadwhitacre added a commit that referenced this issue Apr 13, 2013

Store companies in `participants`; #449
Rather than having a separate `companies` table, we can just the
participants table for both companies and people. That's why it's named
participants in the first place, instead of users or humans or some
such.

chadwhitacre added a commit that referenced this issue Apr 13, 2013

Back out ill-advised changes to Participant; #449
We're going to greatly simplify the schema changes for this feature. I
had started modifying set_tip_to for the more complicated schema, and it
made the test suite reeeeeaaallly borked. Backing this out fixes the
test suite and I'll commit the new schema changes next.

chadwhitacre added a commit that referenced this issue Apr 13, 2013

Greatly simplify schema for #449
I had originally thought we would have new tables for `companies` and
`brands` but we're going to use the existing `participants` table.
Instead, this adds a `type` column to `participants` which is an ENUM
with three values: individual, group, and open group. Individuals will
behave as participants currently do. Groups will say "We are" instead of
"I am" but will otherwise be the same. Open groups will be where we go
into all of the voting to decide how to split up the money.

chadwhitacre added a commit that referenced this issue Apr 13, 2013

Back out changes to js side of tipping; #449
I backed out the server-side changes to tipping in 579933e.

chadwhitacre added a commit that referenced this issue Apr 13, 2013

Gotta represent. #449, #657
I'm changing the 'open group' to 'open company'.

chadwhitacre added a commit that referenced this issue Apr 14, 2013

UI for open groups is MVP; #449
Trusted users can answer the question "Who is Gittip?" and we compute a
percentage split for the pot based on those answers. The rest is
details. Or faucets and fixtures. ;-)

chadwhitacre added a commit that referenced this issue Apr 14, 2013

chadwhitacre added a commit that referenced this issue Apr 14, 2013

Append branch.sql to schema.sql; #449
This has been applied to production.
@chadwhitacre

This comment has been minimized.

Show comment
Hide comment
@chadwhitacre

chadwhitacre Apr 14, 2013

Contributor

This shit is released, people! Will reticket ... everything else I didn't do yet. :)

Contributor

chadwhitacre commented Apr 14, 2013

This shit is released, people! Will reticket ... everything else I didn't do yet. :)

@mvdkleijn

This comment has been minimized.

Show comment
Hide comment
@mvdkleijn

mvdkleijn Sep 17, 2013

Contributor

Sorry for commenting on a closed issue... I'm confused. How are funds implemented? Or are Teams supposed to be funds? I don't see options to donate to communities and I don't see funds in the UI anywhere. The closest thing I can find is Teams?

Contributor

mvdkleijn commented Sep 17, 2013

Sorry for commenting on a closed issue... I'm confused. How are funds implemented? Or are Teams supposed to be funds? I don't see options to donate to communities and I don't see funds in the UI anywhere. The closest thing I can find is Teams?

@chadwhitacre

This comment has been minimized.

Show comment
Hide comment
@chadwhitacre

chadwhitacre Sep 17, 2013

Contributor

@mvdkleijn Right, funds ended up morphing into Teams (via "open groups"). Down the line we can revisit the idea of funds as originally described on this ticket, but I don't want to get into that right now. Let's stay focused on cleaning up what already exists.

Contributor

chadwhitacre commented Sep 17, 2013

@mvdkleijn Right, funds ended up morphing into Teams (via "open groups"). Down the line we can revisit the idea of funds as originally described on this ticket, but I don't want to get into that right now. Let's stay focused on cleaning up what already exists.

@chadwhitacre

This comment has been minimized.

Show comment
Hide comment
@chadwhitacre

chadwhitacre Sep 23, 2013

Contributor

This took a left turn into Teams. Reviving the original proposal as #1493.

Contributor

chadwhitacre commented Sep 23, 2013

This took a left turn into Teams. Reviving the original proposal as #1493.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.