Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.Sign up
Improve budgeting interface/features #391
I know it's not your preferred method, but I had a play around with the "You Need A Budget" app and it seemed to make a lot of sense for anything beyond regular monthly expenses.
I find the current budget pretty hard to use, and not very useful when looking forward.
I think at the very least it should be simpler to edit the budget (in the table itself), and it'd be really useful to have something like a monthly income/expense view for the budget that's visible while editing it (or to be able to edit the budget column in the monthly profit/loss sheet). I know it's not that straightforward, and that's actually be a more feature limited budget, but right now it just seems too hard to quickly update, and then not really worth the effort of updating each month (since it only works for half my expenses)
Not quite sure what the answer is, but hopefully that gives some perspective if nothing else...
I'm not happy with how budgets ended up either, but I have to admit that I'm not too sure where I can go from there.
I don't do much budgeting myself, so I'm not well placed for designing such a feature. Moreover, doing something that remotely approaches what other apps are already doing is significant work.
Making sure that the plugin API is good enough to develop a budget plugin is an interesting possibility, but we lack a developer for such plugin :)
@brownnrl @tuxlifan how about we brainstorm on envelope budgeting here? Since the next release is a major one (v3.0), we can do a clean break with budgets. I don't like how budgets turned out, I don't use the feature, and worse: it's a rather complicated feature, code-wise.
How would you see things work, ideally?
My early take:
I haven't thought much about this yet though, but it's just my first intuition.
I've just quickly uploaded the HACKING.nvlp (nvlp being my abbreviation for envelope - in case you hadn't guessed it already) I had created for myself to
Word of warning: it's unedited, raw and contains some general thoughts and ideas, explanations, as well as some ramblings, wishes and thoughts that might or might not make sense to you ;)
I didn't have a clear vision for the UI back then, just the feeling to not outright copy YNAB, focus on the technical side and see where it goes.
In addition to plain envelopes I think "Goals" would be a nice thing (basically assigning some amount to an envelope each "nvlp-period" (c.f. HACKING.nvlp ;)) to achieve a certain amount at a certain date) to have. They would be like recurring transactions but for envelope money distribution, with automatically calculated actual amounts (based off your goal and time frame) and shown in their own list to not confuse the user.
I quickly read over @tuxlifan write up, and later I'll try to see any delta with what I have worked on.
It might be a lot of work. But I think it would be possible to create a separate set of budget records that create a view of the transactions, and then makes a comparison within each period.
A few quick assumptions, first is to fix the budget periods to start to a window on the order of two weeks. This way when edits are applied to budget records they don't have to be spread or consolidated arbitrarily as I'm not sure how that might work.
I created a LibreOffice spreadsheet, attached as a PDF (unfortunately, they don't let me attach the ODS file.)
The first page shows a budget view as you might see it in MG. The top gives a summary of the starting cash balance (as calculated from asset accounts marked as contributory), the flow of money into these accounts, and outflows from these accounts.
Important to note that the Bold and underlined entries of this view are the only ones that a user can actually edit. The rest of the values come from transactions on the ledger or methods to create new goals. There may also be some hot links to go direct to the account maybe on a view over the same budget period in which they can enter transactions.
To begin with, the person making these edits doesn't know and has not yet entered any transactions to actual accounts other than the starting balance and their paycheck ($120 and $20, respectively). They also know their current credit card balance and their minimum payment, consisting of $1 in interest and $4 to principle.
This person also has some goals, like saving a little for future bills and creating an emergency fund. These goals are listed at the bottom of the sheet. Goals are a second record type associated with budgets aside from the budget values themselves.
This person doesn't do much editing beyond that, and doesn't pay too much attention to the "Available Cash" and the "Meet Budget" at the top of the budget, and waits until actual transactions from this period are available.
Next Page (2) the person actually has some transactions entered for this period. They enter those transactions and sees that there is a difference between what they spent and what they have budgeted. They modify their budget to drive these differences down to zero. This causes the "Meet Budget" goal to change. They modify the Meet budget up to the amount that is indicated, and notice that they still have some available cash. Seeing that their future bills and emergency fund goals are already 100%, they increase their credit card payment until there is no available cash left.
NOTE: Available cash can go negative, it is really just an indication of whether they have money left to spend outside of goals and cash already spent
The person notices that new credit created goes positive, an indication that they are paying down their debt.
Next Page (3) Two weeks go by, and they are now into budget period 2.
Next Page (4) The third budget period comes around, and the expense look very much like the first budget period (rent, electric, heat all due spending has changed slightly).
Here the fact that we only make $20 every two weeks and are spending nearly double that starts to come into play. We choose to start to dip into our "Future Bills" goal and the "To Future Credit" to make our ends meet. We also start to reduce our budget items and increase the difference carry to hopefully influence the way we spend on some of these items in the coming budget periods.
But unlike other zero sum budgeting applications, we can let ourselves rely on credit if that is the choice we want to make. It makes it plain and apparent where our money is flowing and how it is changing over time. We can see in the total balance where we stand and how much new credit we are creating each period. We can see in the New Carry how much we are on or off track compared with where we want to spend to meet our budget goals. We can see how far off we are from our other goals.
These goals can also be used to prepare to pay for large one time expenses that occur in future budget periods.
@tuxlifan I find your approach a bit restrictive. One has to adhere to these budgeting practices to use the system. I had something a bit more analytical in mind, that is, giving you the information you need to adjust spending behavior to maximize your interest rather than being the red light telling you "you're doing it wrong!".
But thanks for your insight, this gives an idea of the kind of tools that moneyGuru should provide to facilitate this kind of budgeting.
I'm, in fact, much more partial to something like what @brownnrl suggests. But isn't such a system, designed around the idea of managing debt, be more useful if we had compound interest calculation implemented?
One thing is certain, both approaches imply a significant development effort. It might have to be implemented in multiple milestones.
Or what about... exports to ODS?
Budgets is about rules that we give ourselves and those rules can change a lot depending on the person. Spreadsheet is a very good tool to express those rules in a flexible manner.
What about exporting to ODS, with support for updating an existing file? The flow would be:
Export would mainly be about exporting account balances (or flow, in case of income and expense) at certain dates. moneyGuru could be configured to export a table of accounts (rows) and periods (columns) with cells being the balance/flow. Maybe other nice things could be part of the export?
Example budgets ODS could be made available for download to inspire users.
Do you think that budgets like the one you made @brownnrl could be plugged into such an export?
I think a method to export the underlying information to an ODS file would be a great feature, and it would be worthwhile at the very least in researching the best presentation and development of any integrated budgeting in MG.
I hand wrote several small cases on paper, watched a few YNAB videos on their approach, and read a few ways GnuCash handled zero sum budgets, and then re-did the cases on paper. It took a few attempts to arrive at the information I had on the top of the attachment, and I’m not sure if that is the most optimal way of presenting that information. Providing a spreadsheet export would provide the most flexible approach toward calculating budgets.
The one issue I would have with doing a spreadsheet export-only approach is that users increase the amount of time they may spend fiddling with the budgets between the two applications. As it is now, I spend probably one to two hours importing and updating all of my transactions into MG, reconciling, and then figuring out the next month’s transfers.
It would be nice if budgeting took less time, even if it was a bit more opinionated and didn’t match my exact interpretation of a budget. If MG provided a budget view with hot links to asset and expense accounts, it reduces the time spent jumping between the transaction and budget views.
But again, it would be nice if there was a full set of exported data broken out by budget period and selectable budget-contributing accounts. I can see if I can come up with an example using MG’s existing export features and the attached spreadsheet. Though the idea might be worth a separate ticket?
A minimal milestone might be to implement only the expense / budget / difference / carry view. You can also not implement the Cash/Credit breakdown, and only count up the total outflow to the expense. You would ignore the available cash / credit allocations, and phantom goals (maybe to implement in a later milestone if it is worth it). But they aren't the critical focus.
This would be a replacement for the current budget system, although it's possible to implement in tandem with the old system. The biggest problem there is the arbitrary start and end date of budgets of the current system, and how that might not align to a budget period. It's possible it could work out some way, I haven't worked out any examples of that. I don't know if it would be worth it. It probably would be worthwhile to figure out how to convert existing budgets as to not alienate any users that currently rely on the feature. I have a set of monthly and annual budgets based on my spending habits of the previous year, which is really just to compare how I spent within those two years.
Difference and carry would only really be for reporting purposes, and not impact the value that would be used for future forecasting. This would work the same as it does now if realized transactions are more than the budget then it there is no "final amount" subtracted at the end of a budget period. If there is an amount left over then it is used like it is used now, as a non-editable transaction that appears in future forecasts. The amount is applied as a transaction which is coincident with the end of the budget period (so all budgets would be based on the budget period).
So when you establish a budget, you state when your budget should begin and when it should "end" (i.e., reset the diff and carry). By default this could be the month you start your budget and the end of the calendar year.
Then the budgets are adjusted manually within each budget period. The difference and carry are part of the budget tab.
Everything else I have written is a crutch for moving and categorizing existing cash as a cost/benefit comparison of direct payments to expenses through an asset account transfers or indirectly offsetting it into a liability. Everyone currently just does this anyway. The biggest problem with the current budget system that users ask about (I think?) is they can't tell how much they go under or over their desired budget over time. The under is currently possible but the over just gets lost completely.
It's the foremost thing I would like to see, and everything else just helps with the adjustments.
I would volunteer to help with this. I feel guilty I left development of the import feature, so you wouldn't be wrong to take the offer with a handful of salt. Maybe a collaborative effort with @tuxlifan. I do have some experience in C with embedded programming, and a few large team projects at work that used C++. I can definitely try to commit a few hours a week at least in a few PRs on a development branch. I'd like to see it in MG.
Your first milestone is very close to what I've been forming in my head. Easy to achieve, commits the "break" it needs with the old system (which is why we have a major version release), and useful.
I'm in big part simply repeating what you wrote, but the goal here is to outline the steps ahead explicitly.
As for the C rewrite, yes, I'm aiming for a rewrite of budgets in C, but I think it's a better idea to do the rewrite after the budget changes: with a single period and no target accounts, the mechanics will be simpler, so the rewrite in C will be easier.
Alright, let's do this. The strategic way to proceed here would be to strip stuff, convert to C, add stuff. That would mean steps 1, 2 (partial) and 3, convert, then 2 (rest), 4 and 5.
I prefer to take care of the stripping part myself because an "outsider" might be too cautious ("am I removing something I shouldn't") and make the process harder than it should. Or, inversely, an "outsider" might strip too much (remove a test when what needed to be done was behavior adjustment) and I might not notice the loss of test coverage during review. Also, to be frank, I really like to strip stuff...
Conversion to C is also a bit tricky. If you're interested in trying out @brownnrl, I wouldn't mind assigning the task to you, but it's very challenging and require a deep knowledge of the code, of Python's C API, and good experience in refactoring code in general.
Then, I'll be delighted to help you do the rest (if you want to do it). I suggest this plan because you mention that you know C, but if you prefer, we can do everything in python before conversion to C. The difference in overall effort isn't very high I think.
I'd be down once I get my environment set up and aligned with your workflow (see my current hiccup in #503).
I can't promise I'll live up to it, but I'll certainly try. I wrote a few python C modules working on a few embedded projects, but that was a few years ago now.
So there is some dust to blow off there. I can commit 5-6 hours a week plus a little more in research time, and I can update you where I stand or if I hit a snag. So with those caveats, whatever you think is the best way to proceed is fine with me. We can do the C API in ccore, and I'll take tasks as you want to break them off. Or take the pure python route and port changes to C API afterwards. Either way.
I'm keeping up (or trying, at least) with your commits as you make them. Please let me know when you want me to assist. No rush of course, just letting you know I'm looking for an sync point.
I did like the budget target feature when it was around. I missed it a little when I switched to master and looked at my asset accounts. It was also minor, so I see the benefit in reducing code complexity. But without it, it would be nice after this first phase to try to implement the available cash and goal accounts. They'll essentially do the same thing.
The "stripping" part is taking longer than I expected, but I should be finished rather soon. I'm half way I guess...
The "converting to C first" plan won't work, it will be too long before we reach a "sync point". Let's implement this in python first. Overall effort is similar.
@brownnrl I'd be tempted to say that the "stripping" part is over. It's a bit "raw" (another way to say it would be "barely functional") as it is, but that's where you come in :)
So, if you want, I'd leave it to you to implement the rest of the plan. Don't hesitate to refactor as needed: The
You'll notice that I haven't touched the file format: you'll have a better idea of what the format will look like.
Don't hesitate to ping me if you need guidance. My nick is
While you work on this, I'll continue preparation for v3, but I'll try to stay clear of anything related to schedules and budgets.
I was doing some review and planning this morning over how to execute on this feature. I'd like to get some clarity about the user interface flow around steps "2 (rest) and 4" that you have listed above.
I've come up with primarily two options in terms of direction, my initial view and one that I've come to in reviewing the code and practical matters of implementation. The trade off being some usability. So I'll state the options, and let you decide. It'll be good to level-set on the expectations for this feature.
Option 1. As I originally had pictured it, the budget view for any one period would look like the profit/loss report in the expense section, with some additional columns and an editable "budgeted amount".
This option requires is that you have to design a second date spinner into the budget view to select the period. Then there is some minor complexity in knowing which budget periods fall under the current date view. Next, we'd have to add editable cells into a report view, which would require changes to the core
I thought those changes might be too far reaching for a first attempt. Most of these report-style views are not currently editable and almost all edition is performed through discrete transaction tables or individual modal, panel interfaces.
Option 2. Re-use the current edition mechanisms of transaction elements. Instead of making a report style view, we re-create the budget similar to a transaction view. Here, date ranges would not be editable (selected as the last date in the budget period) but selected based on the budget periods overlapping with the currently selected date range, one line per expense grouped into individual budget periods. Maybe the date range instead of the last date is listed in the date cell.
There would be less general work to do. A downside to this is that you would lose the nice hierarchical view of expenses and their ordering available in the report (though maybe we could re-use that information to decrease the pain slightly).
My thought is go with option 2, and if it works out all right maybe integrate the budgets into the profit/loss reports directly at a later point. Ultimately, that's where that kind of interaction belongs, especially if we're ever going to do the goal accounts mentioned before. But I can't see the viability in re-creating a profit/loss report with editable items in a budget view now.
I was figuring that to have something that works, we'll have to ignore the date range in the budget view. What I had in mind was a read only period indicator (from 24/01/2019 to 31/01/2019 for example, for a weekly budget) with and right arrows to navigate the budget periods (with leftward bounds being the budget start date). That could be a lot of clicking around in some situations, but I expect them to be rare. Who would want to budget in a distant future?
But yes, maybe we could consider the current date range and make budget bounds and "initial date" be conditioned by it. It would be a nice feature and allow elegant far-reaching date navigation. I hadn't though of it.
I agree that re-using
I'm not sure why you talk about transaction editing though. You mean the "inline" editing mechanism (which we also see in the split table)? Then yes, it's rather easy to transpose to any kind of underlying data. That's also what I had in mind.
With regards to re-creating the ordering (you mean the ordering brought through grouping I guess?), it will be easy to re-create that in a table's flattened hierarchy. Much simpler than trying to re-implement a new
With regards to integrating it directly to the P&L view, I had the same thought too. Ultimately, we'll probably want this. The challenge will be to make budget widgets available in a nonintrusive way and elegant way, making regular date range and budget date range work well together. But this will be more work. Let's think about this later...
So yeah, option 2, but what do you think of navigating budget ranges with arrows instead of a second date widget?
So for a first round in v3, something that looks roughly like this for the "transaction view" for budgets, where budget periods are trimmed down to the smallest set of periods that fits in the currently selected date range. Difference is per period and track is per "cycle". I don't know if you want to adopt that term, but essentially when you want to reset the carry over. We should probably normalize on terminology, I see you use "range" for what I may be referring to as a period.
At a later point, I can see people wanting to have more granular control on when a carry over resets.
Here, let's say a custom range of 13 months starting 1/1/2019 was selected. A budget period of a month with a cycle of 12 months is made. The only editable column is the "Budget" column. Expenses come from account creation and are ordered according to the layout in P&L.
So I got a little bit further. Though before I go too much further, I wanted to get consensus on the data model and user story around starting out budgeting.
We have an idea of a start date, cycle duration when carry reports resets, and periods which are divisions that the cycle is broken up into where differences with actual transactions are reported.
You had originally placed three fields on the budget view, but I removed it as changing those dates isn't a view action but a destructive one. It changes each underlying budget object with a new start and end date, which also means that the amounts may also be invalid (would need to be spread or consolidated).
I thought that idea would be better to make that a modal change (like the rest of the app). So I moved the start date field into the budget panel removing the other fields (for now), so when making a "New Budget" the dialog would be to create a new budget plan.
I created an object
The idea here being that a user creates a plan (which is singleton to the model and either exists or doesn't), and then doesn't make specific one-off additions or deletions of specific budget items. The application will initially create the current
However, the action of creating a new budget plan feels a little clunky.
Another way to do this, that might make more sense and be a lot easier in terms of the code, is to assume a set period. My recommendation would be for Monthly budgeting periods. We'd hit probably 90% of users out there. I only really budget one month at a time, and that's how other applications like YNAB present it.
This way instead of creating many different budget records explicitly, you could simply have a view of budget lines one per expense but only create the associated budget entry if there is some non-null element in that line that the user actually created.
Difference carry could start at the first period with a non-null budget item giving us an effective start date, and we could record a reset carry flag with each budget record. Either as a global "Reset Carry" button, or users could define individual budget line items on which they want to reset the carry.
The down side to this is that you'd lose the variable budget period selection when the budget is first defined. But as I see it, most users budget on the month anyway and making that adjustable is going to add a fair bit of complexity.
I wanted to clear some of this with you before I fixed all the tests and serialization stuff since that's going to be a fair bit of work.
I agree that "Add budget" feels clunky (because, hum, we agree here that we don't want to get in supporting multiple budget plans (whatever that would mean), right?). Those 3 buttons at the bottom of the window, I think, shouldn't be used for editing the budget plan singleton, it will confuse users.
You are right to note that the change to those widgets are destructive. Their location was temporary. This zone should indeed be read-only. What about simply adding a "Change Budget Plan" button in that zone? We could warn that a change in period is destructive and simply reset budgets on plan change.
As for assuming a set period, I don't mind limiting ourselves to this for v3, but we'll surely want to make this customizable in the near future. I find the idea of syncing budgeting periods with pay checks interesting and I guess that many users will want this. It fits well with the idea of envelope budgeting.
@tuxlifan was making a good point when he talked about the awkwardness of fitting a "pay check" period into a fixed monthly period. If we have an easy way to avoid that pain to users, we should take that extra step.
Also, for "TODO: Change to generalize". look at
I won't leave the periods static currently if there is use in it, as that will probably work best as an up front change. I'll also come up with a separate
I'll revert the
So on changing a budget plan, the application will reset all budgets items for all periods by removing them. Per your note in the review, we will lazily generate budget items. The
I do think it would be nice (useful to me at least) to leave budget records that existed prior or up to the start date of the new budget. This would be to answer questions like, "how good did we do with our budget up until we created a new budget plan?" This would be to done by only removing
I think that's enough to make a more solid plan and modify the tests and loader. I may incrementally ping you if I think there is something worth review again. I will definitely use the
I was suggesting lazy budget generation, but that's in fact already what happens:
I don't know about that. It seems that you started working on a system where we had one
I have a feeling that it could be a nest of complexity lurking there, waiting for the right moment to jump on us and devour us alive. A nice to have indeed, but I have to say that I'm happy to see your will to stay within scope :)
Don't hesitate to ping me (IRC might work better for quick questions), especially on the
It's probably good to clear this up, it is either my misconception either about how it works or how we want budgets to work. As I had envisioned it originally, the amount for the next budget period wouldn't necessarily be the same amount in the subsequent period. It would always be zero by default. The user would then enter a different manual-input of an amount so that the spawns are only generated for their pro-rated nature within the same period. That is to say, they would reflect a transaction pro-rated to the end of the period in the net worth and P&L and not utilize the recurrence outside of that.
Difference reporting would be based on the budget amount as defined, rather than as pro-rated used for future forecasting in profit/loss and net worth.
What I think you are saying is that the system should work as it does in MG now, and all amounts entered into a single budget item are edited/allocated for the whole budget cycle and then divided equally into each budget period? If that is correct, I worry it might be too complicated to edit and update? For instance, if my carry is $10 low, in the next period I may allocate more. And if $10 high, I may allocate less. This would lend itself later to the idea of applying available cash to a budget in the next phase of this feature.
One middle ground (resulting in less manual input) would be if the last budget period that was actually defined by the user have a recurrence date equal to the period but no stop date. Then the views could use the spawns to fill in the last amount defined per period.
I apologize if I'm not understanding or belaboring. Just trying to get the intention clear.
We get that clear. It's what I think too.
No, that's not what I think.
So, I think we're in sync. Because I can't be behind your shoulder as you implement, I'm just trying to avoid wasted effort. In my head, the implementation of this new budget system would look like the "exceptions" and "global changes" system that we already have in
But maybe you have a better idea than that, I don't know. I'm simply advising a gathering a good knowledge of the existing system.
I do see what you mean.
I'm going to try to revert the commit you made decoupling
I was going to store the cycle reset values in these spawned instances, or keep a reference to the budget. But that's too coupled with the C code and I don't want to subclass transaction as I see you are moving away from that model. I'll store the cycle reset dates with the
I'm trying to get a complete understanding of the
I've tried searching for instances in the tests, and debug runs of MG where I modify some attribute and select "Just this one". I can't seem to locate instances where some attributes are recorded within an exception. The loader suggests that it can happen. However, in practice it seems to be that the transaction is materialized and the exceptions are treated as a set for not spawning specific dates. But the
A serialized exception within a recurrence is provided below. Is there any point where I would store something within an
Ultimately, I'm looking to try to re-use this mechanism for budgets to record the allocated budget_amounts and notes for a budget period that the user may want to enter. Or at least understand before I re-invent the wheel or break it. :)
You are correct when you say that a "just this one" scenario will result in a normal "deletion + materialize". So no, it will not generate a "valued" exception (it used to, however, which is why the system might be a little bit over-engineered right now).
The only case where a "valued" exception occurs is when two things happen:
At the very core, I guess you've noticed that it happens at https://github.com/hsoft/moneyguru/blob/master/core/model/recurrence.py#L222 . If you grep the code for
Hum, now that I look at the code, it looks like my second condition listed above isn't correct. A "valued" exception will be created for every global change. I think that at some point I wanted to optimize global changes out, but it looks like I ended up never doing it.
Does it clear things up for you?
Oh, there we go, I'm stupid, I didn't remember the two separate dicts! It seems that the
Actually, I think that will work for budgets (at least how I was thinking of using it). So my intention is to use the
I was just trying to get the lay of the land. That does help. Thank you.
Do you have a time table for the next release? I just finished modifying the core model, implementing budget amounts, differences, carry and carry resets in the model to the point that I'm confident it's correct. I used direct APIs to create some test cases, and everything is working as expected. I plan on using the TestApp API while I integrate the model into the view / document / app. It was just a bit too much of a push to do the whole thing at once.
I store the budget amount and carry reset flags in the spawns themselves. I'm using the budget date (last date of the period) to reference the exception. I use a date range lookup for budget amounts in between or outside of the exception ranges, using 0 amounts for dates before the first exception and the last defined exception from a given budget date for amounts with no defined exception.
I cache the result of all budget spawns (past and future) and a different property to access the complete set which I'll use for editing the budget. What gets returned to the oven is as it works today so the transaction table will still have only have budget spawns which occur in the future and have a greater than zero amount.
Now that the model is complete, I'll start integrating into the view / app. I just wanted to make sure I wasn't slowing you down.
I am waiting after you to continue with the release (I'll leave loader/savers alone for this release after all. too much work for now, so there's nothing left to do than budget + the subsequent C-ification of the Recurrence and budgets), but I wouldn't say your slowing me down. There's no rush. I have plenty of other things to do elsewhere meanwhile :)
Again, let me know if you need my involvement.
As you told me privately @brownnrl, you lack availability to complete the budget feature in a reasonable time frame.
I was waiting after this for the v3 release and I'm not willing to do it myself. Because I wanted to use v3 as an opportunity to break with the old badly implemented budget system, I will strip budgets altogether for v3. We'll re-add it later, without the time pressure.
I'm adding a