Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot import project from SplitWise #159

Closed
iluvatyr opened this issue Jul 11, 2022 · 8 comments
Closed

Cannot import project from SplitWise #159

iluvatyr opened this issue Jul 11, 2022 · 8 comments
Labels
bug Something isn't working testing

Comments

@iluvatyr
Copy link

Hello, I wanted to go and import from Splitwise as advertised. Went to Splitwise, pressed export, copied the export.csv to nextcloud and from there went to the cospend app to import it into a new project. Unfortunately, it says the csv is malformed and cannot be imported.
Has something changed or why isnt this possible?

@julien-nc
Copy link
Owner

Hi, Thanks for the bug report. Maybe Splitwise changed their export format. Maybe there is a bug in Cospend 😁. If you provide an example file I can debug that and fix Cospend import process.

@iluvatyr
Copy link
Author

iluvatyr commented Aug 1, 2022

Hi, Thanks for the bug report. Maybe Splitwise changed their export format. Maybe there is a bug in Cospend grin. If you provide an example file I can debug that and fix Cospend import process.

I just saw right now you replied.

Exported csv is attached
export.csv

julien-nc pushed a commit that referenced this issue Aug 1, 2022
Signed-off-by: Julien Veyssier <eneiluj@posteo.net>
@julien-nc julien-nc added bug Something isn't working testing labels Aug 1, 2022
@julien-nc
Copy link
Owner

Thanks for the file. Splitwise had the strange idea to translate the csv column names... So they can't be predicted unless we know their sources... Even then, translations are not permanent, they can change.

There is no obvious good solution to fix the import process in Cospend... If the column order is always the same we can use that to identify them instead of their names.

Immediate solution for you: rename the columns in your file:

Datum,Beschreibung,Kategorie,Kosten,Währung,Person3,Person1,Person2

becomes

Date,Description,Category,Cost,Currency,Person3,Person1,Person2

A small fix to the import process was still needed. If you can try v1.4.9-4-nightly, great. If not, next release is coming soon and will include the fix. You will still need to bring the column names back to English 😁.

@iluvatyr
Copy link
Author

iluvatyr commented Aug 2, 2022

It worked to import the Splitwise file after upgrading to nightly and changing to english names like you said. I don't even understand why the language there is in German since my device language aswell as my app language is english. Pretty weird.

Thank you for solving this issue and thank you for doing this awesome project.

Edit: Switching the language within the account settings on Splitwise.com and then exporting makes them export in english again. So maybe everyone who wants to export should switch to english before exporting. Or like you said, you can just delete the first 5 columns of the first row and put the english versions there within the import script if the order is always the same which it seems it is.

Edit2: I double checked if it imported correctly and it seems the numbers for how much debt each person has is wrong. Here is the data for settlements.
I confirmed wrong debt data with another import of just me and another person, where the debt also differs from the correct one (from splitwise).
The issue seems to be that in the export there is all data, from the first entry, But it since there is nothing indicating when it was settled up, Cospend just adds up all entries (so also settled up ones already). Anyways, still, when I deleted the entries that are already settled up and imported the edited csv then, it was still not correct (although more correct).
With settlement on 02.April, everything before "Other pizza stuff" on 02. April is already settled up and can be deleted from the csv. The settlement payments splitwise calculates with the entries left (not settled up).csv is:

Person 2 --> Person1 : 42,43
Person 3 --> Person1: 296.73
So Person1 should receive 339,16

Whereas in cospend it will calculate to following settlement payments:
Person2 --> Person1 213.24
Person3 --> Person1 140.96
So Person1 should receive 354.19

So as visible, the data is wrong. Maybe you can troubleshoot the problem with the information provided now.

By the way, exporting non group sheets from splitwise apparently adds a line so that it looks like this, I guess you have to also create a fix for that:

Note: does not include group expenses

Date,Description,Category,Cost,Currency,Person1,Person2

2020-10-25,Rewe,Groceries,31.37,EUR,15.69,-15.69
2020-10-25,Person1 paid Person2,Payment,15.69,EUR,-15.69,15.69

julien-nc pushed a commit that referenced this issue Aug 2, 2022
…ll splits

Signed-off-by: Julien Veyssier <eneiluj@posteo.net>
julien-nc pushed a commit that referenced this issue Aug 2, 2022
…ne is not the first one

Signed-off-by: Julien Veyssier <eneiluj@posteo.net>
@julien-nc
Copy link
Owner

julien-nc commented Aug 2, 2022

Ouch, I had a look at the implementation. It was only working with equally spit bills which involve all members. I didn't test this feature enough when reviewing the related PR...

It is fixed now, both bill creation and handling when there is some content before the "column names" line. v1.4.9-5-nightly is available.

It was a bit tricky to find a way to import data correctly. Splitwise does not leave any trace of who pays and owes. It just mentions the effect of each bill on the balances. The only way I found to generate valid Cospend bills with this information is:

  • We ignore the "Cost" column.
  • For each line:
    • Each positive member value => we create a bill with this value as the paid amount
    • The owers of each of these bills are the members with a negative value

So in the end the Cospend bill list does not look like the Splitwise one because we don't have enough information in the exported files.
But the essential information of who owes what to the group is preserved. Fine by me. What do you think?

I hope your file covers all the split scenarios and the Cospend import process is now generic enough to work all time.

The balances for your example project now fit the Splitwise ones. There still is a small difference due to an unavoidable divergence in rounding choices 😁 :

  • Splitwise rounds the owed amounts for each bills
  • Cospend does not round anything but the balances, this way it keeps the maximum precision and sticks more to the truth ©️

Thank you very much for helping there. I would have never noticed those bugs.

@iluvatyr
Copy link
Author

iluvatyr commented Aug 5, 2022

Hello, I'm sorry I answer just now,

EDIT: Works flawlessly after correct update!

(Old answer)
So thank you for your time on fixing this. From what you are saying, it sounds like everything is fixed now.
Did you implement this also in the latest nightly version?
I ran "occ app:update --allow-unstable cospend", restarted nextcloud, and imported the export.csv, but the values are just like in my previous comment, where everything was still unfixed.

@julien-nc
Copy link
Owner

Did you implement this also in the latest nightly version?

Yes, v1.4.9-5-nightly

Are you sure it upgraded to v1.4.9-5-nightly?

I just tried again with your original file (after fixing the column names) and it works fine.

@iluvatyr
Copy link
Author

iluvatyr commented Aug 17, 2022

I'm sorry I could not try since my internet providers cable was destroyed by some construction works and my server is just back online now - horrible days.

Anyways, I uninstalled cospend and reinstalled and then updated it with nightly and now it works. Awesome! I tested with a few untouched splitwise exports, where even in between it was settled up and also that worked perfectly. So now it's really possible for everyone to export and import without problems. Flawless fix!

So again, thank you very much for this awesome support! Can't be seen every day that a dev fixes the problems so fast. I'm impressed!

@eneiluj you can mark this as resolved

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working testing
Projects
None yet
Development

No branches or pull requests

2 participants