run Gratipay 195

[clone1018]

#508

---

Docs

http://inside.gratipay.com/howto/run-payday

Rotation

Month	Pilot	Copilot
February	@clone1018   ✈️	@rohitpaulk
March	@whit537	@clone1018
April	@rohitpaulk	@whit537
May	@clone1018	@rohitpaulk
June	@whit537	@clone1018
July	@rohitpaulk	@whit537
August	@clone1018	@rohitpaulk
September	@whit537	@clone1018
October	@rohitpaulk	@whit537
November	@clone1018	@rohitpaulk
December	@whit537	@clone1018
January	@rohitpaulk	@whit537

[clone1018]

Gotta run to the doctor this morning. I should be back around my usual time + 1 hour.

[clone1018]

Let's gooo!

[rohitpaulk]

🚀

[clone1018]

Approved 2 1.0 Payout Applications

[clone1018]

$ cat migrate-tips.log 
Done.

[clone1018]

Backup completed and verified.

[clone1018]

Got a whole crap ton of InsecurePlatformWarning's from payday, but everything appeared to work

[clone1018]

Script ran for 59 seconds (0:00:59.372523).

[clone1018]

Updated three users to a fee cap of 20.00 per the masspay None bug #485 (comment)

[rohitpaulk]

@clone1018 is this on a droplet or your local? It usually takes around 3 minutes on your system, I see that it is < 1 minute today..

[clone1018]

That was on a droplet, normally on my laptop it takes 3 minutes.

[clone1018]

1 GB Memory / 30 GB Disk / NYC3

[rohitpaulk]

👍

[clone1018]

Of course running into a couple of masspay wonky fees, working on that.

[clone1018]

@rohitpaulk so we had 3 None's in masspay, I manually uploaded each and set an adjusted fee for two of them, but I'm still not getting the correct masspay numbers, is my next step splitting this file up?

[rohitpaulk]

I manually uploaded each and set an adjusted fee for two of them

Are you saying that you tried thrice, by setting one of the three to $1, and the other two to $20? Isn't it possible that two of them are supposed to be $1? (or that all of them are supposed to be $1)

[rohitpaulk]

@clone1018 if you upload only 3 of them, and you're getting a fee mismatch error - then there definitely is an error with one (or more) of those 3 records.

[clone1018]

My normal process has been to split out the paypay output csv into 3 new files with each person I manually added a fee_cap for, and check that their Gratipay estimated fee matches paypal, if it doesn't I update their fee_cap to be whatever paypal says it is. The past 3 paydays that worked fine and the numbers ended up matching up

[clone1018]

One file per person

[rohitpaulk]

Alright.. So after your adjustments - the three files (with one record each) individually pass the fee checks is what you're saying, correct?

[clone1018]

Yeah

[rohitpaulk]

Ah, then no other option 😞 Binary search it is.

If the amounts don't match then we probably have a bad paypal_fee_cap for one of the users. Manually bisect the input file from step 1, running through steps 2 through 4 with half the file at a time until you discover the culprit. Manually update paypal_fee_cap for the culprit in the participants table, then start all over from step 1.

[clone1018]

😭

[clone1018]

Sorry for the delay, had a bunch of stuff come up, bisecting these files now!

[clone1018]

Just gonna vomit everything I'm going through here so I dont mess anything up.

First Chunk: 1-33
Second Chunk: 34-67

Split the file in two, both chunks have incorrect fees.

First Chunk Off: 5.55
Second Chunk Off: 0.02

[clone1018]

44-67 are good.
39-43 paypal says 0.12 I say 0.10
34-38 are good

[clone1018]

Looks like the off person is someone I modified a couple of paydays back?

[clone1018]

I have to be doing something wrong, the other incorrect fees are my previous payday modifications? @rohitpaulk

[clone1018]

@whit537 just in case

[clone1018]

Went through previous paydays to find out what we normally do, normally @whit537 updates all null fee_caps to be 20.00 before running masspay.

[chadwhitacre]

...

[clone1018]

So really I'm trying to figure out: If a user has an incorrect fee should I be setting their exchange_routes fee_cap to the Paypal fee cap or something else?

If so wouldn't this change week by week? How do we correctly calculate their fee?

[clone1018]

From what I see so far, some of my previously updated exchange_routes (updating by setting fee_cap to whatever paypal says) are now incorrect because their receiving amount went up?

[chadwhitacre]

@clone1018 PayPal has two possible fee caps:

• $20 for folks outside the U.S. of A.
• $1 for folks inside the U.S. of A.

There's no way (that I found when I looked) of determining which category PayPal puts a user in except actually uploading a MassPay input CSV and checking the computed fees (kinda annoying).

[clone1018]

Okay, didn't know that

=> select * from exchange_routes where fee_cap <> 20.00 and fee_cap <> 1.00;
(1 row)

[chadwhitacre]

some of my previously updated exchange_routes

On the face of it this doesn't sound totally implausible, since it can depend on the amount we're paying out. I.e., if we are paying out $100 to someone inside the U.S. then there fee will be $1 (2% = $2, but it's capped). If we're paying them $40 then the fee will be $0.80 and we won't notice the cap.

[chadwhitacre]

(1 row)

Interesting.

My habit has been to set all that are null to $20 (per #516 (comment)), and then revise downwards to $1 as need arises (fees are off on CSV upload during MassPay).

[clone1018]

Yeah, I've been setting a users fee cap to the fee reported by paypal

[chadwhitacre]

Ah ... yeah, it should be either $20 or $1. The (1 row) is you, then?

/me updates docs ...

[clone1018]

Okay, awesome, so I need chunk out this csv, check for anyone with a fee_cap of 1.00 and then verify they're not supposed to be 20.00, or vise versa?

[clone1018]

Paypal fee: 23.42
Our fee: 28.97

Since the numbers are in that direction does it mean a couple of users have an incorrectly set 20.00 fee_cap when it should be $1?

[chadwhitacre]

(Docs updated in c8f5275.)

[chadwhitacre]

@clone1018 We're out by +$5.55. In the simplest case that means that we think someone has a $20 fee cap when they really have a $1 fee cap. We think they owe $6.55 (below the $20 fee cap), when they really owe $1 (fee capped at $1). However, the fee delta is a sum, it could include more than one fee cap discrepancy. Unfortunately our current best practice is to find out via binary search.

[clone1018]

Alright I've got the tools and the knowledge then to complete this, thanks for popping in @whit537

[chadwhitacre]

@clone1018 I'm done with COS for the day and just hanging out in my hotel room. I can copilot since hopefully @rohitpaulk is asleep. :-) ✈️

!m @clone1018

[clone1018]

Invalid fee caps:
11625: 20 => 1

D'oh! Should have been able to guess that one!

[clone1018]

POSTed MassPay back to Gratipay for 67 users.

[clone1018]

Masspay: https://github.com/gratipay/logs/commit/24c79af488cea328a60eb3f4c5978ac61b408d6d
Payday: https://github.com/gratipay/logs/commit/b288652455e195be87309ad4dfee719ef89dbfff

[clone1018]

[clone1018]

Decline rate = 11.49

[clone1018]

1285 * 5 = 6425, paypal says we have 4,272.47. @whit537 I noticed you top up to 10,000. Should I do that vs what the docs recommend?

[clone1018]

Adding $3,000 for now.

[clone1018]

It's done!

[clone1018]

I'm gonna go have a 🍻

[chadwhitacre]

Should I do that vs what the docs recommend?

Not a biggie. I can drop back to what the docs recommend. Mostly I think $10,000 was a round number, and maybe also was influenced by uncertainty around 1.0 payouts (I believe we spiked PayPal upwards just in case, and have gradually dwindled it back down).

It's done!

You rock! !m @clone1018!

I'm gonna go have a 🍻

I shall virtually join you! :-)

🍻

[rohitpaulk]

👏