Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve budgeting interface/features #391

Open
andrew-hill opened this Issue May 5, 2014 · 37 comments

Comments

Projects
None yet
4 participants
@andrew-hill
Copy link

andrew-hill commented May 5, 2014

I know it's not your preferred method, but I had a play around with the "You Need A Budget" app and it seemed to make a lot of sense for anything beyond regular monthly expenses.

I find the current budget pretty hard to use, and not very useful when looking forward.

  • the current budget entry is cumbersome, this is something where YNAB really excels (since it's the primary feature of YNAB), it's not a chore to regularly (monthly) adjust budgets to match the current state of affairs (it even has simple ways of updating the budget for a category based on recent data, eg last 3 months, which is useful when just learning what you spend, or when you can't control the spending)
  • the existing moneyguru budget predictions work well for "everyday" expenses like groceries, but get worse the less frequent the payments are. As time passes, the budgeted remainder for an annual bill reduces pro rata, even though the bill payment may occur in full at some future date.
  • further, there's no way to "save up" for an item without having a "sinking fund" for it. I would like to put aside some funds for various categories (saving for holidays, a new car, whatever) each week/month but there's no real way to earmark such savings when all my money is in a single account (eg mortgage or offset). This is another place where YNAB excels, by having envelope budgeting.

I think at the very least it should be simpler to edit the budget (in the table itself), and it'd be really useful to have something like a monthly income/expense view for the budget that's visible while editing it (or to be able to edit the budget column in the monthly profit/loss sheet). I know it's not that straightforward, and that's actually be a more feature limited budget, but right now it just seems too hard to quickly update, and then not really worth the effort of updating each month (since it only works for half my expenses)

Not quite sure what the answer is, but hopefully that gives some perspective if nothing else...

@andrew-hill

This comment has been minimized.

Copy link
Author

andrew-hill commented May 5, 2014

Maybe this is a good candidate for a plug in? Either a simpler monthly + one-entry-per-category budget-editor, or actually implementing envelope budgeting as well... I don't know if either are really possible though...

@hsoft hsoft added feature labels May 5, 2014

@hsoft

This comment has been minimized.

Copy link
Owner

hsoft commented May 5, 2014

I'm not happy with how budgets ended up either, but I have to admit that I'm not too sure where I can go from there.

I don't do much budgeting myself, so I'm not well placed for designing such a feature. Moreover, doing something that remotely approaches what other apps are already doing is significant work.

Making sure that the plugin API is good enough to develop a budget plugin is an interesting possibility, but we lack a developer for such plugin :)

@hsoft hsoft removed the thinking label Jul 2, 2015

@hsoft

This comment has been minimized.

Copy link
Owner

hsoft commented Jan 10, 2019

@brownnrl @tuxlifan how about we brainstorm on envelope budgeting here? Since the next release is a major one (v3.0), we can do a clean break with budgets. I don't like how budgets turned out, I don't use the feature, and worse: it's a rather complicated feature, code-wise.

How would you see things work, ideally?

@hsoft

This comment has been minimized.

Copy link
Owner

hsoft commented Jan 10, 2019

My early take:

  1. Easier interface: the budget view has a list of all expense accounts where we can easily change budgeted amounts in a tabular way
  2. Drop the pro-rata thing: budgets are like "ghost" scheduled entries that materialize in a "ghost" account, parallel to the expense.
  3. Budgeting then becomes a difference between the real expense account and the "ghost" expense account: you have a surplus or a deficit with regards to when the budget started (every budget has a start date).

I haven't thought much about this yet though, but it's just my first intuition.

@tuxlifan

This comment has been minimized.

Copy link
Contributor

tuxlifan commented Jan 10, 2019

@hsoft, @brownnrl I'm certainly in for brainstorming, might just be a bit slow to respond from time to time.

I've just quickly uploaded the HACKING.nvlp (nvlp being my abbreviation for envelope - in case you hadn't guessed it already) I had created for myself to
https://gitlab.com/snippets/1796820
(not github since they don't seem to care about not-supported-browsers and can't be bothered to make a browser-agnostic UI anymore... :o rollseyes)
so you can have a look. (code still needs vetting & cleanup :))

Word of warning: it's unedited, raw and contains some general thoughts and ideas, explanations, as well as some ramblings, wishes and thoughts that might or might not make sense to you ;)

I didn't have a clear vision for the UI back then, just the feeling to not outright copy YNAB, focus on the technical side and see where it goes.
At the moment, however, I think that the financier.io / YNAB UI works well if taking the best parts that work from them (to be expanded...) and there would be a big potential user base that could feel right at home with it.
(on some points though, their functionalities differ and mG will certainly differ too, so the hard part will be to make it different enough at the same time to not trip users up with their engrained expectations/habits)

In addition to plain envelopes I think "Goals" would be a nice thing (basically assigning some amount to an envelope each "nvlp-period" (c.f. HACKING.nvlp ;)) to achieve a certain amount at a certain date) to have. They would be like recurring transactions but for envelope money distribution, with automatically calculated actual amounts (based off your goal and time frame) and shown in their own list to not confuse the user.

@brownnrl

This comment has been minimized.

Copy link
Contributor

brownnrl commented Jan 14, 2019

I quickly read over @tuxlifan write up, and later I'll try to see any delta with what I have worked on.

It might be a lot of work. But I think it would be possible to create a separate set of budget records that create a view of the transactions, and then makes a comparison within each period.

A few quick assumptions, first is to fix the budget periods to start to a window on the order of two weeks. This way when edits are applied to budget records they don't have to be spread or consolidated arbitrarily as I'm not sure how that might work.

I created a LibreOffice spreadsheet, attached as a PDF (unfortunately, they don't let me attach the ODS file.)

budgets_proposal.pdf

The first page shows a budget view as you might see it in MG. The top gives a summary of the starting cash balance (as calculated from asset accounts marked as contributory), the flow of money into these accounts, and outflows from these accounts.

Important to note that the Bold and underlined entries of this view are the only ones that a user can actually edit. The rest of the values come from transactions on the ledger or methods to create new goals. There may also be some hot links to go direct to the account maybe on a view over the same budget period in which they can enter transactions.

To begin with, the person making these edits doesn't know and has not yet entered any transactions to actual accounts other than the starting balance and their paycheck ($120 and $20, respectively). They also know their current credit card balance and their minimum payment, consisting of $1 in interest and $4 to principle.

This person also has some goals, like saving a little for future bills and creating an emergency fund. These goals are listed at the bottom of the sheet. Goals are a second record type associated with budgets aside from the budget values themselves.

This person doesn't do much editing beyond that, and doesn't pay too much attention to the "Available Cash" and the "Meet Budget" at the top of the budget, and waits until actual transactions from this period are available.

Next Page (2) the person actually has some transactions entered for this period. They enter those transactions and sees that there is a difference between what they spent and what they have budgeted. They modify their budget to drive these differences down to zero. This causes the "Meet Budget" goal to change. They modify the Meet budget up to the amount that is indicated, and notice that they still have some available cash. Seeing that their future bills and emergency fund goals are already 100%, they increase their credit card payment until there is no available cash left.

NOTE: Available cash can go negative, it is really just an indication of whether they have money left to spend outside of goals and cash already spent

The person notices that new credit created goes positive, an indication that they are paying down their debt.

Next Page (3) Two weeks go by, and they are now into budget period 2.
The rent, electric, and credit card payments are not due, but they are still spending on A/B/C and maybe by a little too much. They note that in this period, they really only wanted to spend the amounts indicated by the budget. But they do have the money, so they will let that difference carry into the next budget period. They also put away little money into their "To Future Credit" goal so they can use that money to pay the current balance.

Next Page (4) The third budget period comes around, and the expense look very much like the first budget period (rent, electric, heat all due spending has changed slightly).

Here the fact that we only make $20 every two weeks and are spending nearly double that starts to come into play. We choose to start to dip into our "Future Bills" goal and the "To Future Credit" to make our ends meet. We also start to reduce our budget items and increase the difference carry to hopefully influence the way we spend on some of these items in the coming budget periods.

But unlike other zero sum budgeting applications, we can let ourselves rely on credit if that is the choice we want to make. It makes it plain and apparent where our money is flowing and how it is changing over time. We can see in the total balance where we stand and how much new credit we are creating each period. We can see in the New Carry how much we are on or off track compared with where we want to spend to meet our budget goals. We can see how far off we are from our other goals.

These goals can also be used to prepare to pay for large one time expenses that occur in future budget periods.

@hsoft

This comment has been minimized.

Copy link
Owner

hsoft commented Jan 14, 2019

Thanks @tuxlifan and @brownnrl, very interesting!

@tuxlifan I find your approach a bit restrictive. One has to adhere to these budgeting practices to use the system. I had something a bit more analytical in mind, that is, giving you the information you need to adjust spending behavior to maximize your interest rather than being the red light telling you "you're doing it wrong!".

But thanks for your insight, this gives an idea of the kind of tools that moneyGuru should provide to facilitate this kind of budgeting.

I'm, in fact, much more partial to something like what @brownnrl suggests. But isn't such a system, designed around the idea of managing debt, be more useful if we had compound interest calculation implemented?

One thing is certain, both approaches imply a significant development effort. It might have to be implemented in multiple milestones.

@hsoft

This comment has been minimized.

Copy link
Owner

hsoft commented Jan 15, 2019

Or what about... exports to ODS?

Budgets is about rules that we give ourselves and those rules can change a lot depending on the person. Spreadsheet is a very good tool to express those rules in a flexible manner.

What about exporting to ODS, with support for updating an existing file? The flow would be:

  1. Start with an initial export after having configured the export periodicity (weekly, monthly, start date, end date (include future with schedules?), etc.)
  2. Create yourself adjacent sheets where you do your budgeting rules
  3. Periodically update your ODS to update your budgets. moneyGuru would only update its own sheets, leaving the rest of the file intact.

Export would mainly be about exporting account balances (or flow, in case of income and expense) at certain dates. moneyGuru could be configured to export a table of accounts (rows) and periods (columns) with cells being the balance/flow. Maybe other nice things could be part of the export?

Example budgets ODS could be made available for download to inspire users.

Do you think that budgets like the one you made @brownnrl could be plugged into such an export?

@brownnrl

This comment has been minimized.

Copy link
Contributor

brownnrl commented Jan 15, 2019

I think a method to export the underlying information to an ODS file would be a great feature, and it would be worthwhile at the very least in researching the best presentation and development of any integrated budgeting in MG.

I hand wrote several small cases on paper, watched a few YNAB videos on their approach, and read a few ways GnuCash handled zero sum budgets, and then re-did the cases on paper. It took a few attempts to arrive at the information I had on the top of the attachment, and I’m not sure if that is the most optimal way of presenting that information. Providing a spreadsheet export would provide the most flexible approach toward calculating budgets.

The one issue I would have with doing a spreadsheet export-only approach is that users increase the amount of time they may spend fiddling with the budgets between the two applications. As it is now, I spend probably one to two hours importing and updating all of my transactions into MG, reconciling, and then figuring out the next month’s transfers.

It would be nice if budgeting took less time, even if it was a bit more opinionated and didn’t match my exact interpretation of a budget. If MG provided a budget view with hot links to asset and expense accounts, it reduces the time spent jumping between the transaction and budget views.

But again, it would be nice if there was a full set of exported data broken out by budget period and selectable budget-contributing accounts. I can see if I can come up with an example using MG’s existing export features and the attached spreadsheet. Though the idea might be worth a separate ticket?

@hsoft

This comment has been minimized.

Copy link
Owner

hsoft commented Jan 15, 2019

I don't know if it's worth creating a ticket. I don't think I'll do it if there's a functional budgeting system in place. I don't have an unlimited will to implement stuff. It was just a random idea. If it doesn't stick, no biggie.

@brownnrl

This comment has been minimized.

Copy link
Contributor

brownnrl commented Jan 16, 2019

A minimal milestone might be to implement only the expense / budget / difference / carry view. You can also not implement the Cash/Credit breakdown, and only count up the total outflow to the expense. You would ignore the available cash / credit allocations, and phantom goals (maybe to implement in a later milestone if it is worth it). But they aren't the critical focus.

This would be a replacement for the current budget system, although it's possible to implement in tandem with the old system. The biggest problem there is the arbitrary start and end date of budgets of the current system, and how that might not align to a budget period. It's possible it could work out some way, I haven't worked out any examples of that. I don't know if it would be worth it. It probably would be worthwhile to figure out how to convert existing budgets as to not alienate any users that currently rely on the feature. I have a set of monthly and annual budgets based on my spending habits of the previous year, which is really just to compare how I spent within those two years.

Difference and carry would only really be for reporting purposes, and not impact the value that would be used for future forecasting. This would work the same as it does now if realized transactions are more than the budget then it there is no "final amount" subtracted at the end of a budget period. If there is an amount left over then it is used like it is used now, as a non-editable transaction that appears in future forecasts. The amount is applied as a transaction which is coincident with the end of the budget period (so all budgets would be based on the budget period).

So when you establish a budget, you state when your budget should begin and when it should "end" (i.e., reset the diff and carry). By default this could be the month you start your budget and the end of the calendar year.

Then the budgets are adjusted manually within each budget period. The difference and carry are part of the budget tab.

Everything else I have written is a crutch for moving and categorizing existing cash as a cost/benefit comparison of direct payments to expenses through an asset account transfers or indirectly offsetting it into a liability. Everyone currently just does this anyway. The biggest problem with the current budget system that users ask about (I think?) is they can't tell how much they go under or over their desired budget over time. The under is currently possible but the over just gets lost completely.

It's the foremost thing I would like to see, and everything else just helps with the adjustments.

@brownnrl

This comment has been minimized.

Copy link
Contributor

brownnrl commented Jan 16, 2019

I would volunteer to help with this. I feel guilty I left development of the import feature, so you wouldn't be wrong to take the offer with a handful of salt. Maybe a collaborative effort with @tuxlifan. I do have some experience in C with embedded programming, and a few large team projects at work that used C++. I can definitely try to commit a few hours a week at least in a few PRs on a development branch. I'd like to see it in MG.

@hsoft

This comment has been minimized.

Copy link
Owner

hsoft commented Jan 16, 2019

Your first milestone is very close to what I've been forming in my head. Easy to achieve, commits the "break" it needs with the old system (which is why we have a major version release), and useful.

  1. Single period for all budget items
  2. A summary view over the table with a period navigator, inflows, outflows, other relevant totals.
  3. Remove "target account": only global net worth and profits are affected. The goal of the "target account" for budgets was to somehow manage the balance of a checking account w.r.t future schedules and expected expense, but it never worked for me, I doubt it worked for many users. Also, it brings a big complexity penalty to the code.
  4. All expense accounts pre-listed in the budget tables, with budgeted amounts editable inline
  5. Budget figures can be changed per-period.

I'm in big part simply repeating what you wrote, but the goal here is to outline the steps ahead explicitly.

As for the C rewrite, yes, I'm aiming for a rewrite of budgets in C, but I think it's a better idea to do the rewrite after the budget changes: with a single period and no target accounts, the mechanics will be simpler, so the rewrite in C will be easier.

@hsoft

This comment has been minimized.

Copy link
Owner

hsoft commented Jan 16, 2019

Alright, let's do this. The strategic way to proceed here would be to strip stuff, convert to C, add stuff. That would mean steps 1, 2 (partial) and 3, convert, then 2 (rest), 4 and 5.

I prefer to take care of the stripping part myself because an "outsider" might be too cautious ("am I removing something I shouldn't") and make the process harder than it should. Or, inversely, an "outsider" might strip too much (remove a test when what needed to be done was behavior adjustment) and I might not notice the loss of test coverage during review. Also, to be frank, I really like to strip stuff...

Conversion to C is also a bit tricky. If you're interested in trying out @brownnrl, I wouldn't mind assigning the task to you, but it's very challenging and require a deep knowledge of the code, of Python's C API, and good experience in refactoring code in general.

Then, I'll be delighted to help you do the rest (if you want to do it). I suggest this plan because you mention that you know C, but if you prefer, we can do everything in python before conversion to C. The difference in overall effort isn't very high I think.

@brownnrl

This comment has been minimized.

Copy link
Contributor

brownnrl commented Jan 17, 2019

I'd be down once I get my environment set up and aligned with your workflow (see my current hiccup in #503).

I can't promise I'll live up to it, but I'll certainly try. I wrote a few python C modules working on a few embedded projects, but that was a few years ago now.

So there is some dust to blow off there. I can commit 5-6 hours a week plus a little more in research time, and I can update you where I stand or if I hit a snag. So with those caveats, whatever you think is the best way to proceed is fine with me. We can do the C API in ccore, and I'll take tasks as you want to break them off. Or take the pure python route and port changes to C API afterwards. Either way.

hsoft added a commit that referenced this issue Jan 17, 2019

hsoft added a commit that referenced this issue Jan 17, 2019

hsoft added a commit that referenced this issue Jan 17, 2019

hsoft added a commit that referenced this issue Jan 17, 2019

hsoft added a commit that referenced this issue Jan 22, 2019

Decouple repeat_type descs from Schedule
Will be useful for the new budgeting.

ref #391
@brownnrl

This comment has been minimized.

Copy link
Contributor

brownnrl commented Jan 22, 2019

I'm keeping up (or trying, at least) with your commits as you make them. Please let me know when you want me to assist. No rush of course, just letting you know I'm looking for an sync point.

I did like the budget target feature when it was around. I missed it a little when I switched to master and looked at my asset accounts. It was also minor, so I see the benefit in reducing code complexity. But without it, it would be nice after this first phase to try to implement the available cash and goal accounts. They'll essentially do the same thing.

@hsoft

This comment has been minimized.

Copy link
Owner

hsoft commented Jan 22, 2019

The "stripping" part is taking longer than I expected, but I should be finished rather soon. I'm half way I guess...

The "converting to C first" plan won't work, it will be too long before we reach a "sync point". Let's implement this in python first. Overall effort is similar.

hsoft added a commit that referenced this issue Jan 22, 2019

hsoft added a commit that referenced this issue Jan 22, 2019

Make Budget not subclass Recurrence any more
This gives us more leeway for the refactoring to come.

ref #391

hsoft added a commit that referenced this issue Jan 22, 2019

Use only the first budget's schedule attrs
Transitional code to get to schedule attrs outside `Budget`.

ref #391

hsoft added a commit that referenced this issue Jan 22, 2019

hsoft added a commit that referenced this issue Jan 23, 2019

hsoft added a commit that referenced this issue Jan 23, 2019

@hsoft

This comment has been minimized.

Copy link
Owner

hsoft commented Jan 23, 2019

@brownnrl I'd be tempted to say that the "stripping" part is over. It's a bit "raw" (another way to say it would be "barely functional") as it is, but that's where you come in :)

So, if you want, I'd leave it to you to implement the rest of the plan. Don't hesitate to refactor as needed: The Budget and BudgetList structure as it is is very probably not optimal for what we want to do.

You'll notice that I haven't touched the file format: you'll have a better idea of what the format will look like.

Don't hesitate to ping me if you need guidance. My nick is vdupras on freenode and my IRC client is permanently logged.

While you work on this, I'll continue preparation for v3, but I'll try to stay clear of anything related to schedules and budgets.

@brownnrl

This comment has been minimized.

Copy link
Contributor

brownnrl commented Jan 23, 2019

Copy, I'll start working on this tomorrow morning.

@brownnrl

This comment has been minimized.

Copy link
Contributor

brownnrl commented Jan 24, 2019

I was doing some review and planning this morning over how to execute on this feature. I'd like to get some clarity about the user interface flow around steps "2 (rest) and 4" that you have listed above.

I've come up with primarily two options in terms of direction, my initial view and one that I've come to in reviewing the code and practical matters of implementation. The trade off being some usability. So I'll state the options, and let you decide. It'll be good to level-set on the expectations for this feature.

Option 1. As I originally had pictured it, the budget view for any one period would look like the profit/loss report in the expense section, with some additional columns and an editable "budgeted amount".

This option requires is that you have to design a second date spinner into the budget view to select the period. Then there is some minor complexity in knowing which budget periods fall under the current date view. Next, we'd have to add editable cells into a report view, which would require changes to the core Report, and/or duplication of the profit/loss view.

I thought those changes might be too far reaching for a first attempt. Most of these report-style views are not currently editable and almost all edition is performed through discrete transaction tables or individual modal, panel interfaces.

Option 2. Re-use the current edition mechanisms of transaction elements. Instead of making a report style view, we re-create the budget similar to a transaction view. Here, date ranges would not be editable (selected as the last date in the budget period) but selected based on the budget periods overlapping with the currently selected date range, one line per expense grouped into individual budget periods. Maybe the date range instead of the last date is listed in the date cell.

There would be less general work to do. A downside to this is that you would lose the nice hierarchical view of expenses and their ordering available in the report (though maybe we could re-use that information to decrease the pain slightly).

My thought is go with option 2, and if it works out all right maybe integrate the budgets into the profit/loss reports directly at a later point. Ultimately, that's where that kind of interaction belongs, especially if we're ever going to do the goal accounts mentioned before. But I can't see the viability in re-creating a profit/loss report with editable items in a budget view now.

@hsoft

This comment has been minimized.

Copy link
Owner

hsoft commented Jan 24, 2019

I was figuring that to have something that works, we'll have to ignore the date range in the budget view. What I had in mind was a read only period indicator (from 24/01/2019 to 31/01/2019 for example, for a weekly budget) with and right arrows to navigate the budget periods (with leftward bounds being the budget start date). That could be a lot of clicking around in some situations, but I expect them to be rare. Who would want to budget in a distant future?

But yes, maybe we could consider the current date range and make budget bounds and "initial date" be conditioned by it. It would be a nice feature and allow elegant far-reaching date navigation. I hadn't though of it.

I agree that re-using Report might make the attempt for v3 a bit much, but we'll want to get some something that look like this (we'll also want to add graphs and stuff). I was thinking of a simple table view with all expenses listed, much simpler to implement.

I'm not sure why you talk about transaction editing though. You mean the "inline" editing mechanism (which we also see in the split table)? Then yes, it's rather easy to transpose to any kind of underlying data. That's also what I had in mind.

With regards to re-creating the ordering (you mean the ordering brought through grouping I guess?), it will be easy to re-create that in a table's flattened hierarchy. Much simpler than trying to re-implement a new Report-like view.

With regards to integrating it directly to the P&L view, I had the same thought too. Ultimately, we'll probably want this. The challenge will be to make budget widgets available in a nonintrusive way and elegant way, making regular date range and budget date range work well together. But this will be more work. Let's think about this later...

So yeah, option 2, but what do you think of navigating budget ranges with arrows instead of a second date widget?

@brownnrl

This comment has been minimized.

Copy link
Contributor

brownnrl commented Jan 24, 2019

So for a first round in v3, something that looks roughly like this for the "transaction view" for budgets, where budget periods are trimmed down to the smallest set of periods that fits in the currently selected date range. Difference is per period and track is per "cycle". I don't know if you want to adopt that term, but essentially when you want to reset the carry over. We should probably normalize on terminology, I see you use "range" for what I may be referring to as a period.

At a later point, I can see people wanting to have more granular control on when a carry over resets.

Here, let's say a custom range of 13 months starting 1/1/2019 was selected. A budget period of a month with a cycle of 12 months is made. The only editable column is the "Budget" column. Expenses come from account creation and are ordered according to the layout in P&L.

budget_pic

@hsoft

This comment has been minimized.

Copy link
Owner

hsoft commented Jan 24, 2019

Oh, I see. Interesting... More usable and elegant than single period selection. Looks worthy of trying out!

brownnrl added a commit to brownnrl/moneyguru that referenced this issue Jan 28, 2019

(hsoft#391) Draft at a "budget plan"
Budget plans as modified through the document. Very rough, very
unstable first cut.
@brownnrl

This comment has been minimized.

Copy link
Contributor

brownnrl commented Jan 28, 2019

So I got a little bit further. Though before I go too much further, I wanted to get consensus on the data model and user story around starting out budgeting.

We have an idea of a start date, cycle duration when carry reports resets, and periods which are divisions that the cycle is broken up into where differences with actual transactions are reported.

You had originally placed three fields on the budget view, but I removed it as changing those dates isn't a view action but a destructive one. It changes each underlying budget object with a new start and end date, which also means that the amounts may also be invalid (would need to be spread or consolidated).

I thought that idea would be better to make that a modal change (like the rest of the app). So I moved the start date field into the budget panel removing the other fields (for now), so when making a "New Budget" the dialog would be to create a new budget plan.

I created an object Budget Plan and a function change_budget_plan and got it minimally working through the interface. These are probably the only two items in that change set worth looking at, the rest is just me trying to knock the rust off and remember the information flow between the gui and core layers.

The idea here being that a user creates a plan (which is singleton to the model and either exists or doesn't), and then doesn't make specific one-off additions or deletions of specific budget items. The application will initially create the current Budget objects with a start and end date equal to the divided periods of the cycle up, one for each expense account that currently exists.

However, the action of creating a new budget plan feels a little clunky.

Another way to do this, that might make more sense and be a lot easier in terms of the code, is to assume a set period. My recommendation would be for Monthly budgeting periods. We'd hit probably 90% of users out there. I only really budget one month at a time, and that's how other applications like YNAB present it.

This way instead of creating many different budget records explicitly, you could simply have a view of budget lines one per expense but only create the associated budget entry if there is some non-null element in that line that the user actually created.

Difference carry could start at the first period with a non-null budget item giving us an effective start date, and we could record a reset carry flag with each budget record. Either as a global "Reset Carry" button, or users could define individual budget line items on which they want to reset the carry.

The down side to this is that you'd lose the variable budget period selection when the budget is first defined. But as I see it, most users budget on the month anyway and making that adjustable is going to add a fair bit of complexity.

I wanted to clear some of this with you before I fixed all the tests and serialization stuff since that's going to be a fair bit of work.

@hsoft

This comment has been minimized.

Copy link
Owner

hsoft commented Jan 28, 2019

I agree that "Add budget" feels clunky (because, hum, we agree here that we don't want to get in supporting multiple budget plans (whatever that would mean), right?). Those 3 buttons at the bottom of the window, I think, shouldn't be used for editing the budget plan singleton, it will confuse users.

You are right to note that the change to those widgets are destructive. Their location was temporary. This zone should indeed be read-only. What about simply adding a "Change Budget Plan" button in that zone? We could warn that a change in period is destructive and simply reset budgets on plan change.

As for assuming a set period, I don't mind limiting ourselves to this for v3, but we'll surely want to make this customizable in the near future. I find the idea of syncing budgeting periods with pay checks interesting and I guess that many users will want this. It fits well with the idea of envelope budgeting.

@tuxlifan was making a good point when he talked about the awkwardness of fitting a "pay check" period into a fixed monthly period. If we have an easy way to avoid that pain to users, we should take that extra step.

Also, for "TODO: Change to generalize". look at DateCounter and inc_date. They exist precisely for this.

@brownnrl

This comment has been minimized.

Copy link
Contributor

brownnrl commented Jan 28, 2019

I won't leave the periods static currently if there is use in it, as that will probably work best as an up front change. I'll also come up with a separate BudgetPlanPanel launched from a button on the view, with the start date and budget period with a new big green (+) icon to attract a users attention.

I'll revert the BudgetPanel so that users can see details and if we want to add additional data removing their ability to modify the budget periods (so only amounts and notes will remain). I might leave the idea of carry reset flags a property of the individual Budget, and there will be less to fiddle with in the BudgetPlanPanel. Maybe the detail view would be a good spot for this carry flag.

So on changing a budget plan, the application will reset all budgets items for all periods by removing them. Per your note in the review, we will lazily generate budget items. The BudgetView and associated classes will populate one line per expense, per period showing available data if any for that period. The oven will cook existing Budget entries per normal.

I do think it would be nice (useful to me at least) to leave budget records that existed prior or up to the start date of the new budget. This would be to answer questions like, "how good did we do with our budget up until we created a new budget plan?" This would be to done by only removing Budget objects that either overlap that new budget plan start date or start after. The budget view will materialize any budget item that did not exist before the new budget plan date (or have to do some other trickery). I'll save this "old record retention" for after things are working so as to not delay the v3 release, and not try to scope out too much.

I think that's enough to make a more solid plan and modify the tests and loader. I may incrementally ping you if I think there is something worth review again. I will definitely use the DateCounter and inc_date. I just wanted to get something down in a commit to sketch the idea.

@hsoft

This comment has been minimized.

Copy link
Owner

hsoft commented Jan 28, 2019

I was suggesting lazy budget generation, but that's in fact already what happens: Budget generates Spawn in the Oven. What I write are just guidelines, I don't have my head in that implementation, so what I write could be wrong. It's just that pre-generating Budget instances for 12 period cycles feels wrong.

The oven will cook existing Budget entries per normal.

I don't know about that. It seems that you started working on a system where we had one Budget instance per period, which isn't the case currently. Currently, we have one Spawn per period per Budget. If I understand your code right, your new Budget instance would only generate a single Spawn in all cases. You might want to untangle this before going further, you might be re-inventing a system (spawn system) that already exists.

I do think it would be nice (useful to me at least) to leave budget records that existed prior or up to the start date of the new budget.

I have a feeling that it could be a nest of complexity lurking there, waiting for the right moment to jump on us and devour us alive. A nice to have indeed, but I have to say that I'm happy to see your will to stay within scope :)

Don't hesitate to ping me (IRC might work better for quick questions), especially on the Budget/Spawn process which I think is important to understand well to continue further.

@brownnrl

This comment has been minimized.

Copy link
Contributor

brownnrl commented Jan 28, 2019

It's probably good to clear this up, it is either my misconception either about how it works or how we want budgets to work. As I had envisioned it originally, the amount for the next budget period wouldn't necessarily be the same amount in the subsequent period. It would always be zero by default. The user would then enter a different manual-input of an amount so that the spawns are only generated for their pro-rated nature within the same period. That is to say, they would reflect a transaction pro-rated to the end of the period in the net worth and P&L and not utilize the recurrence outside of that.

Difference reporting would be based on the budget amount as defined, rather than as pro-rated used for future forecasting in profit/loss and net worth.

What I think you are saying is that the system should work as it does in MG now, and all amounts entered into a single budget item are edited/allocated for the whole budget cycle and then divided equally into each budget period? If that is correct, I worry it might be too complicated to edit and update? For instance, if my carry is $10 low, in the next period I may allocate more. And if $10 high, I may allocate less. This would lend itself later to the idea of applying available cash to a budget in the next phase of this feature.

One middle ground (resulting in less manual input) would be if the last budget period that was actually defined by the user have a recurrence date equal to the period but no stop date. Then the views could use the spawns to fill in the last amount defined per period.

I apologize if I'm not understanding or belaboring. Just trying to get the intention clear.

@hsoft

This comment has been minimized.

Copy link
Owner

hsoft commented Jan 28, 2019

It's probably good to clear this up, it is either my misconception either about how it works or how we want budgets to work. As I had envisioned it originally, the amount for the next budget period wouldn't necessarily be the same amount in the subsequent period. It would always be zero by default. The user would then enter a different manual-input of an amount so that the spawns are only generated for their pro-rated nature within the same period. That is to say, they would reflect a transaction pro-rated to the end of the period in the net worth and P&L and not utilize the recurrence outside of that.

We get that clear. It's what I think too.

What I think you are saying is that the system should work as it does in MG now, and all amounts entered into a single budget item are edited/allocated for the whole budget cycle and then divided equally into each budget period? If that is correct, I worry it might be too complicated to edit and update? For instance, if my carry is $10 low, in the next period I may allocate more. And if $10 high, I may allocate less. This would lend itself later to the idea of applying available cash to a budget in the next phase of this feature.

No, that's not what I think.

So, I think we're in sync. Because I can't be behind your shoulder as you implement, I'm just trying to avoid wasted effort. In my head, the implementation of this new budget system would look like the "exceptions" and "global changes" system that we already have in Recurrence: Have a single Budget item per account (with no need to create it if the amount is zero), but record user input that happen after the initial budget amount in a "changes" dict and compute the effect of those changes on the fly, when needed.

But maybe you have a better idea than that, I don't know. I'm simply advising a gathering a good knowledge of the existing system.

@brownnrl

This comment has been minimized.

Copy link
Contributor

brownnrl commented Jan 30, 2019

I do see what you mean.

I'm going to try to revert the commit you made decoupling Budget from Recurrence. I'm going to re-use the transaction spawning mechanism and the budget period data from BudgetList. So budgets themselves won't be editable objects, the users will operate on the underlying spawned and "exception" transactions.

I was going to store the cycle reset values in these spawned instances, or keep a reference to the budget. But that's too coupled with the C code and I don't want to subclass transaction as I see you are moving away from that model. I'll store the cycle reset dates with the BudgetList initially making that a global only change.

brownnrl added a commit to brownnrl/moneyguru that referenced this issue Feb 6, 2019

(hsoft#391) Set recurrence data from budget list
Not a permanent change, but trying to get tests passing as a starting
point after the budget/recurrence revert.  Now all tests pass except
loader / saver.  Now need to start updating tests to reflect desired
behavior.

brownnrl added a commit to brownnrl/moneyguru that referenced this issue Feb 13, 2019

(hsoft#391) Getting toward "ideal" budget spawn record
Subclass _Transaction so that a spawn can record some values necessary
for the budget.  I don't think this approach will be permanent, but I
needed a spot for these records that still worked with recurrences.

Rather than keeping a separate association of the spawn / spawn data in
```.Budget``` I thought it would be better to put that in the spawn
itself.  Some thoughts are in comments.
@brownnrl

This comment has been minimized.

Copy link
Contributor

brownnrl commented Feb 15, 2019

I'm trying to get a complete understanding of the Recurrence mechanisms for recording exceptions. In what instances (if any) are values recorded in date2exceptions?

I've tried searching for instances in the tests, and debug runs of MG where I modify some attribute and select "Just this one". I can't seem to locate instances where some attributes are recorded within an exception. The loader suggests that it can happen. However, in practice it seems to be that the transaction is materialized and the exceptions are treated as a set for not spawning specific dates. But the date2exception being a map and some usages that surround it would suggest that a transaction at some point is recorded there. For example, in account reassignment or deletion MG will collect up all exception values. In that scenario, it culls out the None values. I just haven't been able to figure out where a non-None value in that map occurs.

A serialized exception within a recurrence is provided below. Is there any point where I would store something within an exception element?

	<recurrence every="1" type="monthly">
		<exception date="2019-05-15"/>
		<exception date="2019-07-15"/>
		<exception date="2019-09-15"/>
		<transaction date="2019-02-15" description="here" mtime="1550235503">
			<split account="income 1" amount="USD -1.00"/>
			<split account="A" amount="USD 1.00"/>
		</transaction>
	</recurrence>

Ultimately, I'm looking to try to re-use this mechanism for budgets to record the allocated budget_amounts and notes for a budget period that the user may want to enter. Or at least understand before I re-invent the wheel or break it. :)

@hsoft

This comment has been minimized.

Copy link
Owner

hsoft commented Feb 15, 2019

None values in date2exception are for deleted recurrences (which happens every time a spawn is materialized or deleted).

You are correct when you say that a "just this one" scenario will result in a normal "deletion + materialize". So no, it will not generate a "valued" exception (it used to, however, which is why the system might be a little bit over-engineered right now).

The only case where a "valued" exception occurs is when two things happen:

  1. A "for all future spawn" modification happens
  2. A spawn before the "all future spawn" modification still exist.

At the very core, I guess you've noticed that it happens at https://github.com/hsoft/moneyguru/blob/master/core/model/recurrence.py#L222 . If you grep the code for change_globally, you'll see that it's used at one place: https://github.com/hsoft/moneyguru/blob/master/core/document.py#L128 , that is, on a "global" spawn change.

Hum, now that I look at the code, it looks like my second condition listed above isn't correct. A "valued" exception will be created for every global change. I think that at some point I wanted to optimize global changes out, but it looks like I ended up never doing it.

Does it clear things up for you?

@hsoft

This comment has been minimized.

Copy link
Owner

hsoft commented Feb 15, 2019

Oh, there we go, I'm stupid, I didn't remember the two separate dicts! It seems that the date2exception dict, as of now, is used as a set, yes. It could be simplified. I think that I left things as-is to be able to load old moneyguru files. This could be removed now.

@brownnrl

This comment has been minimized.

Copy link
Contributor

brownnrl commented Feb 16, 2019

Actually, I think that will work for budgets (at least how I was thinking of using it). So my intention is to use the date2exception with a custom BudgetSpawn instance to record the specific user input values budget_amount, notes, and carry_reset. I'll use the reference date of the spawn to indicate the budget period.

I was just trying to get the lay of the land. That does help. Thank you.

brownnrl added a commit to brownnrl/moneyguru that referenced this issue Feb 22, 2019

(hsoft#391) Budgeting interface changes
All tests currently passing, slowly modifying the api.

brownnrl added a commit to brownnrl/moneyguru that referenced this issue Feb 22, 2019

(hsoft#391) Budget Periods
Removed the cull_nulls parameter in favor of a separate, explicit
property that returns all spawns for the budget.

brownnrl added a commit to brownnrl/moneyguru that referenced this issue Feb 25, 2019

(hsoft#391) Budget core model complete
Implemented budget amounts, differences, carry amounts, and carry resets.
@brownnrl

This comment has been minimized.

Copy link
Contributor

brownnrl commented Feb 25, 2019

@hsoft
Hey Virgil,

Do you have a time table for the next release? I just finished modifying the core model, implementing budget amounts, differences, carry and carry resets in the model to the point that I'm confident it's correct. I used direct APIs to create some test cases, and everything is working as expected. I plan on using the TestApp API while I integrate the model into the view / document / app. It was just a bit too much of a push to do the whole thing at once.

I store the budget amount and carry reset flags in the spawns themselves. I'm using the budget date (last date of the period) to reference the exception. I use a date range lookup for budget amounts in between or outside of the exception ranges, using 0 amounts for dates before the first exception and the last defined exception from a given budget date for amounts with no defined exception.

I cache the result of all budget spawns (past and future) and a different property to access the complete set which I'll use for editing the budget. What gets returned to the oven is as it works today so the transaction table will still have only have budget spawns which occur in the future and have a greater than zero amount.

Now that the model is complete, I'll start integrating into the view / app. I just wanted to make sure I wasn't slowing you down.

@hsoft

This comment has been minimized.

Copy link
Owner

hsoft commented Feb 25, 2019

I am waiting after you to continue with the release (I'll leave loader/savers alone for this release after all. too much work for now, so there's nothing left to do than budget + the subsequent C-ification of the Recurrence and budgets), but I wouldn't say your slowing me down. There's no rush. I have plenty of other things to do elsewhere meanwhile :)

Again, let me know if you need my involvement.

brownnrl added a commit to brownnrl/moneyguru that referenced this issue Mar 2, 2019

(hsoft#391) Expenses generate in the budget view
Starting to hook the view to the doc.  All tests pass.

Undo / Actions need attention, first pass implementation and need more
tests.
@hsoft

This comment has been minimized.

Copy link
Owner

hsoft commented Mar 24, 2019

As you told me privately @brownnrl, you lack availability to complete the budget feature in a reasonable time frame.

I was waiting after this for the v3 release and I'm not willing to do it myself. Because I wanted to use v3 as an opportunity to break with the old badly implemented budget system, I will strip budgets altogether for v3. We'll re-add it later, without the time pressure.

I'm adding a before-budget-strip tag in the git repo just before doing that. If we need to revive some of that code later, we'll know where it was.

hsoft added a commit that referenced this issue Mar 24, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.