Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multiple saved settings for one tree #299

Closed
Neriderc opened this issue Dec 10, 2022 · 33 comments · Fixed by #321
Closed

Multiple saved settings for one tree #299

Neriderc opened this issue Dec 10, 2022 · 33 comments · Fixed by #321
Labels
enhancement New feature or request
Milestone

Comments

@Neriderc
Copy link
Owner

As discussed in #92, it would be good to have the ability to save multiple different settings. We should:

  • Add a button to save the current settings
  • Allow a name to be chosen
  • Add the saved settings to a list of saved settings to choose from
  • Use the list style that the person selects use
  • Allow deleting saved settings by clicking the X
  • Save the settings in cookies

The list of settings could be saved in the "base" cookie used to save current settings. We could also save the details of the different settings within this base cookie (easier now with the change to JSON format), or we could create one cookie per set of settings.

@Neriderc Neriderc added the enhancement New feature or request label Dec 10, 2022
@hartenthaler
Copy link

The custom module https://github.com/huhwt/huhwt-wtlin has a very similar function. Maybe this can help to implement this issue.

But the technology behind is different.

@Neriderc
Copy link
Owner Author

Thanks, I'll take a look when I come to attempting this. I think I have a good idea of how I want to approach it based on the current setup in GVExport but I'm always keen to see how others have done it in case they have better ideas 🙂

@hartenthaler
Copy link

The LIN module saves the configuration to an external file or to IndexedDB (IDB). Writing to IDB is executed immediately, there is no explicit feedback. When reading IDB, an overview of the existing entries is shown, in this overview entries can also be removed to clean up the database.

@schuco
Copy link

schuco commented Dec 16, 2022

A proposal which may be more a naive question:
After I had invited my many cousins and other family members to use the webtree based family tree and even prepared user accounts for them, most of these only tried once and then no more. It seems that for those who are not specifically interested in family history the threshold to learn some basics to navigate and gain interesting information is too high.
This may have changed after GVE offers an easy and well documented entrance into the family history where newcomers can follow the paths between their parents to aunts and uncles of yet unknown relationship and get more information by just clicking into the diagram.
How could this entrance via GVE become even more attractive?

  1. Is there a way to make webtrees immediately open with a GVE diagram including the starting person of the users account?
  2. Could there be saved settings with maybe several starting persons tailored for the users family which will be used for the start described above?
  3. Could there be several such prepared setting, e.g. for the early family history, for the changes in the ownership of a specic farm or for the relation of the family to prominent persons, which can be loaded by any registered user? That means, these files should not be local files.

@hartenthaler
Copy link

  1. Yes.
    One possibility: sent them a direct link like http://ahnentest.hartenthaler.eu/index.php?route=%2Fmodule%2F_GVExport_%2FChart%2Fkennedy&xref=I179
    Another possibility: add an HTML block to the user my-page and place such a link there
  2. I have no idea.
  3. Ok, no local file, no Cookie, no IDB file. Then it has to be stored in the webtrees database or as a file in a cloud server (which can be accessed via an URL). No idea.

@Neriderc
Copy link
Owner Author

I couldn't find any setting for changing the first page you see on login, but in theory it should be possible to add as a feature. I'm not very familiar with using modules to interact with webtrees in this way, but I'm a lot more familiar than I was 6 months ago so I'll try some things out and see if it's feasible.

I think for saving several sets of saved settings, we could use the same system that is used for saving admin preferences. Posibly the easiest way would be to allow the admin to upload settings files to be saved into a list of settings. One issue I see if that functionality such as planned in this "issue" I think should be in the advanced section. It's confusing for new users. But if we want pre-saved settings for new users, these ideas are in conflict with each other. Perhaps a dropdown list of settings is shown in the non-advanced area, but only if there are saved settings. And saving new sets of settings would remain in the advanced section. I suggest a dropdown to save space compared to the person select style system I originally suggested, but either would work.

In regards to tailoring for specific users, are you thinking that perhaps with the above system of uploading settings to the admin panel, you could restrict certain settings to just certain users? It sounds feasible.

There is a lot in this suggestion that involves interacting with the webtrees backend in ways I'm not familiar, but I'm willing to give it a try if we're happy with how the user interaction would work.

@hartenthaler
Copy link

The requirement is -as far as I understood - to store a set of settings specific for a tree and a user. It should be possible to use the database for this. Is "save" restricted to admins or is a logged-in user able to store a set of settings, too?

The user interaction can be - correct me, please!

  1. Prepare as admin several settings for a new user in his default tree. Store them using a nice name for each of the settings.
  2. Prepare an HTML block in the my-page of the new user with a welcome message, a very short introduction and a link to GVExport, maybe calling already the first of the set of settings.
  3. invite the new user with userid/password.
  4. the new user logs in, is on his my-page, sees the prominent link clicks on it and is in GVExport using his default tree.
  5. then he can load another of the prepared settings using a dropdown-selection menu with all the prepared settings.

The new things:

  • store settings in the database instead of an external file
  • support an URL that loads a named setting
  • several security features to check if the user is logged in, has the correct tree assigned, and the settings are not manipulated, ...

@hartenthaler
Copy link

To make it a bit more complicated:
allow user-agnostic settings, ie. a setting for a tree and all users of this tree. So the user will see in the dropdown-selection box two types of settings for his tree: "global settings" and "my-settings".

@Neriderc
Copy link
Owner Author

The requirement is -as far as I understood - to store a set of settings specific for a tree and a user. It should be possible to use the database for this. Is "save" restricted to admins or is a logged-in user able to store a set of settings, too?

The module interface allows for saving module preferences - I'd think it's not designed for per-user, rather for the admin settings. But we could perhaps leverage this to allow the admin to set default saved settings per-user.

The user interaction can be - correct me, please!

1. Prepare as admin several settings for a new user in his default tree. Store them using a nice name for each of the settings.

2. Prepare an HTML block in the my-page of the new user with a welcome message, a very short introduction and a link to GVExport, maybe calling already the first of the set of settings.

3. invite the new user with userid/password.

4. the new user logs in, is on his my-page, sees the prominent link clicks on it and is in GVExport using his default tree.

5. then he can load another of the prepared settings using a dropdown-selection menu with all the prepared settings.

This seems like a good approach. I have had a little play, I find I can quite easily generate a link with the XREF of the starting person being set to the user's linked individual, but I'm not quite sure how to go about automatically redirecting webtrees to GVE (I tried using redirect($url) but it doesn't seem to work - perhaps there are protections stopping a home page block from redirecting the page).

The new things:

* store settings in the database instead of an external file

Is there an interface for saving per-user settings in webtrees that we can leverage?

* support an URL that loads a named setting

This shouldn't be too hard so long as we have either saved setting IDs or unique names.

* several security features to check if the user is logged in, has the correct tree assigned, and the settings are not manipulated, ...

I was hoping there might be saveUserPreference() and loadUserPreference() where webtrees would handle the details 🙂

To make it a bit more complicated: allow user-agnostic settings, ie. a setting for a tree and all users of this tree. So the user will see in the dropdown-selection box two types of settings for his tree: "global settings" and "my-settings".

Yes, we could have some settings that come via the admin panel, and some that the user has saved themselves. Perhaps some small marker to indicate if it is admin or user - or we could load from the admin preferences into cookie on first load, and then the user could remove them if they don't want them. A reset could restore the admin-set saved settings.

@Neriderc
Copy link
Owner Author

I've just made a change in #305 that lets you link to GVExport without providing an XREF in the URL. Previously this would have given you an error, but now it will search for an XREF with preference (first use what's in the URL, then use the user's XREF if linked, then use the default XREF if the other options aren't available).

With this change, you could use the built-in HTML block to provide a message to users. You can link to GVExport, just copy the URL and then remove the part after the ? that sets the XREF.

This works for the tree home page. I'm afraid I'm not sure how to do the same for the user pages. You can add an HTML block by default for new users, but you don't seem to be able to set the content of it (it's just blank waiting on them to edit it). Probably you guys know more about this than I do. If needed, we can add a block into GVExport. Perhaps we can use the GVE admin page to set text rather than being user edited. Let me know if you're interested in this.

@Neriderc
Copy link
Owner Author

Neriderc commented Dec 17, 2022

@hartenthaler are you familiar with $tree->setUserPreference($user, $setting_name, $setting_value)?

Any reason we can't/shouldn't use this? It seems we could use this to save per-user settings, and could use it to save settings for the user rather than in a cookie.

We could switch completely away from cookies, and use this instead. Then @schuco could "Masquerade as this user" to set up some default per-user settings, which would persist to other computers when logged in with the same account. In fact you could use this option to set up the HTML blocks on My Page using the latest version of GVE on the main branch, this could be done now.

@hartenthaler
Copy link

No. I never used this function.

I would prefer not to use cookies. It makes it easier regarding the "Privacy policy" in the footer. And it should work if the user switches to another device.

@Neriderc
Copy link
Owner Author

I've had a little play, we're not able to save all the settings in one field as the fields aren't long enough, so we'd have to do them separately (which is fine). My biggest concern is that we might accidentally overwrite a built in saved setting as there doesn't seem to be any separation of settings. I'm thinking it may be a good idea to ask for advice on the webtrees forum. It seems logical that we just start all our preferences with a prefix to separate our settings from the others, but I'd like to know what others do in this situation.

@Neriderc
Copy link
Owner Author

Neriderc commented Dec 18, 2022

Greg has replied, said it's fine to use, and also brought to my attention that there is a per-tree version as well. (Edit: I realised the function I mentioned above was the tree version, not sure how I missed this)

So as I understand it, specifically for @schuco's use case, if we move the settings to be stored by webtrees instead of in browser cookies then he could simply "Masquerade as this user" for each account, adding an HTML block to the home page of that user with a welcome message and a link to GVExport. He could then go into GVExport, and set up the diagram. When the user logs in and clicks the link to GVExport, they can then see what has been pre-prepared for them.

I think this is step 1, and I'll make a separate issue for it - migrate to using webtrees preference storage instead of cookies.

@Neriderc
Copy link
Owner Author

Then this issue is step 2 - add the ability to save multiple versions of the settings. This gets more complicated with the move away from cookies (where we could simply add a new cookie for each version). We now need a way to save individual settings in webtrees while also tracking which set of saved settings it relates to.

I'm going to do some thinking out loud here, anyone feel free to comment or make suggestions. We only have the ability to save a setting name and a value. So for example:
Ancestors: enabled
Ancestors-Max-Levels: 2
etc.

So to save different versions, we could require unique names, but I feel IDs would be better. For example:
Ancestors_1: enabled
Ancestors-Max-Levels_1: 2
Ancestors_2: enabled
Ancestors-Max-Levels_2: 2

But in either case, we need to know the name to be able to retrieve it. So we need to store a list of saved settings IDs. So we need one preference that is saved with a known name, that lists the others. But we are also limited in the length we can save, to about 250 character (one reason IDs are better than user-set names). This sounds like plenty, perhaps the best is simply comma separated:
Saved-settings: 1,2

This seems to make sense, we could fit quite a few here but we actually run out of space earlier than expected. By my maths this gives us 87 sets of settings. Seems fine, right? Probably no one will use that many at once. But as the ID length grows over time, this could come down to 50 when we hit 4 digit IDs and 41 when we hit 5 digit IDs. Most people will probably never hit this, but it also would not be hard to hit "save" a bunch of times and cause a webtrees error. Or simply never remember to delete any. So we need to gracefully catch this and show a message saying they can't save more settings until they delete some.

So the next issue is remembering the names of the saved settings. It could just be a list of the IDs but this isn't very user friendly, so it would be best to let the user choose a name, and only use IDs behind the scenes. But we can't use one field to save all the names like we do with IDs. as we will very quickly run out of space. So we could I guess save a separate preference per saved setting, e.g.:
Saved-settings_1: User entered name
Saved-settings_2: Another user entered name

We may still want to limit the length of the chosen name to make sure it fits in the UI.

The next thing is that by doing this, we are going to be creating a huge number of user preferences. It could easily be many hundreds or even thousands of preferences saved individually if you have a bunch of saved settings. When something changes in a future version, we have to consider what the upgrade path looks like. We don't seem to have the ability to get a list of preferences we saved, we have to remember the name of each one. If in the future we change something, we may end up abandoning saved settings that will stay in the database forevermore without a way to remove them or even know they are there. This is not so much a problem now, but in future if we change something we will need to incorporate a way to delete the preferences that aren't used anymore. Basically we will need to look for preferences that we used to use but don't anymore, and we will need to make sure this works if someone jumps versions instead of installing every release.

Does this seem like I've covered everything? @hartenthaler any suggestions on this? It seems like an overly complicated plan but I'm not sure we have a choice considering the 250 character limit.

@hartenthaler
Copy link

All of your analysis is correct as far as I understood it.

  1. If an ID is not only using 0...9 the limits are less important. You can use not only digits, but all characters. So an ID can be "AÄá". This allows thousands of versions of settings.
  2. Yes, IDs should be used only internally.
  3. Yes, you should add a database version number to the GVExport module. There are maybe in the future disrupting changes how to handle this new function. So if you introduce such a disrupting change, you have to increment this version number.
    3a. If a user installs an old version of GVExport there should be a function that says "version in the database is 2, version in GVExport is 1, so you cannot use this function in this old module version".
    3b. If the user installs a new version of GVExport after you introduced an disrupting change, there should be a function that detects: version in database is 1, version in GVExport is 2. Then this function has to do some migration activities to modify the settings in the database to the new format.
  4. There is a possibility to use SQL commands to get directly access to the entries in the database, not using the webtrees layer in between. Normally we should avoid such an access but if you need a function like "delete all settings in the database that look like 'GVE__' this can be done by a SQL command very easily.
  5. Maybe you can define a default value for all settings and only save a setting in the database if it is different to the default. This would save space in the database. For example default for DPI is 72. So only store "GVE_DPI_AÄá:300" but not store "GVE_DPI_AÄá:72".

@hartenthaler
Copy link

@schuco Regarding "1. Prepare as admin several settings for a new user in his default tree."

I would like to be more precise now. It is "Prepare as admin several settings for a new user in his trees."

A user can have access to more than one tree. You - as an admin - can "Masquerade as this user", visit all his trees and set up some default per-user settings using GVExport for him/her. Using the new function discussed in this issue will allow you to save several sets of settings for this user for each tree. You can add a html block on the my-page of this user with an introduction text and a list of URLs to all the settings in all trees of this user that you have prepared.

@schuco
Copy link

schuco commented Dec 18, 2022

@Neriderc, I am highly impressed by the way and speed you are developing and optimizing this issue. It will clearly be a better solution to store settings serverside within the webtees environment.
I have not been successful yet to call GVE via the Diagram-tag for a webtrees starting page or my page.
@hartenthaler kindly sent me the example of a link to directly open webtees with a GVE diagram. I am not very familiar with html. So could you please give an example how to put this link into a html-block?

@Neriderc
Copy link
Owner Author

All of your analysis is correct as far as I understood it.

1. If an ID is not only using 0...9 the limits are less important. You can use not only digits, but all characters. So an ID can be "AÄá". This allows thousands of versions of settings.

Good idea! Might need some care, I'm not sure how the database is storing text, perhaps it's 250 characters but it might be a limit of 250 bytes, in which case using 2-byte characters might have the opposite effect to what we are intending. ASCII does have some special characters, but probably sticking to 0-9, a-z, and A-Z would give us enough options that it doesn't matter anymore. If it does become an issue (which I'm sure it won't), we can make a system to roll the list of IDs across multiple saved settings (with an additional one to keep track), but I think this is unnecessary complexity, 80 or so is plenty, and anyone with a special use case that needs more would likely be better served using the option to download settings files.

3. Yes, you should add a database version number to the GVExport module. There are maybe in the future disrupting changes how to handle this new function. So if you introduce such a disrupting change, you have to increment this version number.
   3a. If a user installs an old version of GVExport there should be a function that says "version in the database is 2, version in GVExport is 1, so you cannot use this function in this old module version".
   3b. If the user installs a new version of GVExport after you introduced an disrupting change, there should be a function that detects: version in database is 1, version in GVExport is 2. Then this function has to do some migration activities to modify the settings in the database to the new format.

Good idea, I'll be sure to do this.

4. There is a possibility to use SQL commands to get directly access to the entries in the database, not using the webtrees layer in between. Normally we should avoid such an access but if you need a function like "delete all settings in the database that look like 'GVE____' this can be done by a SQL command very easily.

It's good to know this option exists in case we need to make a change in future that we know will leave abandoned records, but as you say it's best to use the webtrees layer where possible. It does raise a question, if someone removed the GVExport module, do we have a way of removing GVE settings from the database? Should we perhaps have an option in the control panel to clear any saved settings (across all users)?

5. Maybe you can define a default value for all settings and only save a setting in the database if it is different to the default. This would save space in the database. For example default for DPI is 72. So only store "GVE_DPI_AÄá:300" but not store "GVE_DPI_AÄá:72".

This thought also crossed my mind. This would significantly reduce the number of settings saved in the database, but does have a downside. If we change the defaults, then loading the settings would then not load the same settings as were saved. We could keep track of which defaults changed for each database version change, but it might get quite burdensome having to maintain code to migrate from various versions. Hopefully we don't need to change the database version too often, but probably in future we will need a plan for this.

@Neriderc
Copy link
Owner Author

Neriderc commented Dec 18, 2022

I have not been successful yet to call GVE via the Diagram-tag for a webtrees starting page or my page.
@hartenthaler kindly sent me the example of a link to directly open webtees with a GVE diagram. I am not very familiar with html. So could you please give an example how to put this link into a html-block?

You shouldn't need to be familiar with HTML, webtrees has a "WYSIWYG" editor, which stands for What You See Is What You Get. Here is a basic example. Go to your "My page", then choose the option to customise it:
image

Next, drag the HTML block to the top section (you can leave others there if you want, but probably put HTML at the top):
image

Save, then on your My Page, click the edit button that has tools in an X:
image

Write a message, highlight the part you want to be a link (also see the template list above the box as it shows you how to get in stats and other things), then click the icon that looks like a chain to the left of the flag in the top right:
image

Put your link in the URL box, making sure you remove the ?xref=XXX part off the end:
image

Click OK, and you'll find the text now has a link highlighted:
image

You could highlight the text and change the size, make it red, play with the settings if you'd like. Then save when you're done, and you'll end up with it on the My Page:
image

The default theme doesn't underline the link or colour it except when you hover, so it can be a little hard to tell it's a link. Luckily, you can set those in the editor, so you could highlight the "click here" in the editor and set it to underlined and change the colour (it will look underlined in the editor as it's a link, but you can still select it and click the underline button to force the underline). Make it bold or bigger or whatever you want to make it more visible:
image

Hopefully this helps :)

If you're getting an error like this when trying the link:
image

Then you need to update to the latest version from the main branch, I only made the change yesterday so it's not in a release yet.

@schuco
Copy link

schuco commented Dec 19, 2022

Thank you for this helpful lecture @Neriderc !

@Neriderc
Copy link
Owner Author

My pleasure!

You should find the main branch now saves user settings within webtrees when logged in, so if you masquerade as a user then when that user logs in they should see whatever settings you set (of course, they can change these). Also be mindful that if you want to set a starting individual, best to keep the xref=XXX part of the URL with that person's XREF in it (you should just be able to copy the URL in your browser when in webtrees with that person selected). Otherwise the default actions will apply - if one person selected was selected, they will be replaced by the default XREF. If Two were selected, then the default XREF will be added to the list along with the others. If the person's user account is linked to their webtrees record, and you ensure that record is part of the starting persons, then this shouldn't be an issue.

When I get a chance I think I'll write a reply here that summarises where the conversation got to, to make sure we're on the same page.

This is quite a big piece of work so I'm considering releasing what is on the main branch as a new release, and doing this after Christmas when I get a chance. So if you folks are using the main branch version, please let me know of any issues you find. I've been occasionally finding things not working right so it would be good to know of any other issues.

@Neriderc
Copy link
Owner Author

Updated based on above conversation:

  • Add a button to save the current settings
  • Allow a name to be chosen
  • Add the saved settings to a list of saved settings to choose from
  • Use the list style that the person selects use
  • Allow deleting saved settings by clicking the X
  • Save the settings in webtrees preferences, and keep them separate per tree
  • Add dropdown list in non-advanced section where you can select a saved setting

Here's a quick pic of what I've come up with for the UI:
image

Let me know if anything here isn't what you expect, or if I've missed something.

@Neriderc
Copy link
Owner Author

Neriderc commented Jan 9, 2023

It's been three weeks! Time for an update!

I am slowly working on it but it turned out to be a lot more work than I bargained for.

First, I didn't think about how the save button might send the settings to the server to be saved. All current communication with the server is done via submitting the form (the Download button). This includes browser render updates, which are a bit hacky - it basically switches the output type to DOT then submits the form (Download) to download the DOT file. It then catches the file before it's downloaded by the browser and then loads this into Viz.js. So all in all, we currently have no way of sending arbitrary requests to the server (like asking it to save some settings). So anyway, this part is now done. I've built what's effectively a basic API that lets you send a request to the server (get settings, save settings, delete settings) and have that processed.

Next, we have two states - logged in and logged out. Logged in is probably the most common one but I decided to start with logged out as cookies are a lot easier to fix if I mess it up. Unfortunately it turns out cookies aren't really suitable for this. There is a maximum cookie size (all together) of 4KB, which by the time webtrees saves its cookies and we save the settings for the form, we don't have a lot left for saving more versions of the settings. So we can't use cookies. Instead, we could use the browser's local storage. However, this has to be done client side. Currently all the data is processed on the server then the cookie is sent to the browser. Using local storage means having to do all this processing on the client side - which means writing a whole new process for the browser to process all the data, or more likely add to the API to send and retrieve the data. One day I might do this but for now I've paused work on this. Other options are to either not let logged out users use this feature, or use the webtrees session settings letting them use it but they will lose the settings when the tab is closed. Probably the first version of this will use one of these two options.

For now I've paused work on the logged out state, the last part is saving multiple settings when logged in. I've made better progress here. So currently I have it so you can click the save button to save a version of the settings, and have this sent to the server. It assigns an ID and saves these settings with the ID as a suffix into webtrees.

Next step is to load the setting, and to allow saving a name for the settings, then allow deleting the settings.

There's still a fair bit of work here but it's all starting to come together 🙂

@schuco
Copy link

schuco commented Jan 9, 2023

@Neriderc I have been happy having understood when GVE is processing serverside or clientside. Also I understand your goal saving complex settings and make them available in a flexible way. I am pretty confident to see a positve result in the end (as always with your projects), but I am far away from being able to give any help or advice .
Looking back the development of GVE since you took the initiative my expectations moved from getting a bit more stable module with less errors to perfectionating a most powerful, flexible and well documented tool to work with and presentate genealogical data. Respect and thank you.

@Neriderc
Copy link
Owner Author

Neriderc commented Jan 9, 2023

No advice needed! I've been slowly working through issues and have a path forward, just thought I'd give an update.

I guess from previous discussion we knew it wasn't going to be simple, but it's good that some of this work will help in future and I'm confident we will get version 2.1.17 released eventually 🙂

With regards to errors, I feel I've added more bugs in the past month than there has ever been before, so thanks for helping to catch those!

@Neriderc
Copy link
Owner Author

Neriderc commented Jan 15, 2023

So I now have a perfectly working version of this. Saving, loading, and deleting settings. All working great.

However... after further discussion with Greg, I'm not going to use this method. He has pointed out the module settings allows a lot more data to be saved than the webtrees user preferences. So I'm going to re-do it all using JSON to store the data in the module preferences instead. This solves a number of problems, such as:

  • There would be no practical limit on the number of settings that could be saved
  • Admins are prompted to delete module preferences after a module is removed, so we no longer need Add ability to clear settings to control panel #320
  • webtrees provides no way to delete user settings, because it doesn't normally make sense. Settings are normally always there, just with a different value. Our unique concept of adding different versions of settings doesn't fit nicely in this model, as we can't delete preferences once set. We can get around this by directly deleting from the database, but this isn't a great structure to have if we can avoid it.
  • We can save all settings for one version in one module preference. This should be a little more efficient for loading the settings, because we only need to get data once - though I'm doubtful it would be noticeable by users.

There are still a number of options of how to approach it, but I think the best option is to save all settings for one user/tree combination in one preference. Settings from other users don't affect the speed of loading your settings, but it also (mostly) solves the problem of not being able to delete a preference. If a user is deleted their preferences would stay in the module settings, but we do not have an issue where if a user saves a settings version then removes it that we can't delete that record from the database. Saving all settings versions on one preference means we can edit the preference to remove those settings and not have an orphaned database record.

This has a downside though, which is that if a user saves many settings versions, they are all loaded for each request. i.e. the page loads, which loads the main saved settings - to load these settings we need to load all settings versions for that user/tree combination. Luckily we can do this serverside, so the server gets all the data from the database, then identifies the settings version we are looking for and only sends that to the browser. I think this is efficient enough that any speed difference wouldn't be noticed. The slowest part is the connection between the browser and server and this avoids sending the extra data.

So anyway, I'll close #320 and work on moving this over to the new setup. I much prefer this way of storing settings, it feels less "janky", like a better structure.

I've also had a play with local storage and it doesn't seem to difficult to get this setup working from logged out users (saving settings in the browser using local storage - not Indexed DB as this seems like overkill for what is comparatively only a small amount of data).

I don't really have a timeline for this work but I'm hoping to make significant progress in the next week or two.

@Neriderc
Copy link
Owner Author

Ok so I think I'm done!

So when you're logged in, data is saved in the module settings.

There's one entry for admin settings (used to be one per setting, now it's one for all settings)

There's then one entry for each user/tree combination. This way all the data we need is loaded from one record, and we don't load data we don't need.

There's also an extra entry per tree that holds the list of IDs. This isn't strictly necessary now that all the data is in one entry, but I've left it there for quick allocating of new IDs.

The module settings fields are much larger than the $tree->setUserPreference() fields. They hold 4GB of data, so I suspect this will never be filled up by anyone. You can add as many saved settings as you like, previous talk of a limit no longer applies. It might make the page slow though, if you try to add hundreds, but the database won't cause an error.

If you have the module enabled for logged out users, it will save the settings in "local storage" within the browser. The main settings on the form are still saved in a cookie as it gives a nicer user experience.

I've written a bit more on the pull request, #321

Give it a go and let me know what you think! Especially report any errors. I have occasionally seen errors that I couldn't reproduce, so if you can reliably cause an error from a set of actions then please let me know!

This one has been quite the ride!

@schuco
Copy link

schuco commented Jan 24, 2023

It works fine, however it took some time for me to notice the change. Thanks again for your effort to improve GVE once more!

My observations:
1.
grafik
In my installation Graphviz IS installed

  1. Admin settings default for MCLIMIT seems to be 1. 50 or 100 would be better.

  2. When reloading an externaly saved setting file the previous starting person is added to the list of starting persons. This may be intended, however in my opinion it woud be easier to add this person manually if it is really wanted, than detecting and deleting this person from the list, if only the stored starting persons are to be displayed.

  3. The new saved settings stored in GVE works different and in my sense, i.e. the previous starting person is not included.

Still wishes:
If an admin or moderator has prepared settings with a selected group of starting persons, e.g. to explain relationship between two persons or to show, how the ownership for a farm changed between families, he might want to make these availiable for users not (yet) able to work with GVE. How can these be made accessable for users (e.g. include call in HTML-page)?

@Neriderc
Copy link
Owner Author

My observations: 1. In my installation Graphviz IS installed

Can you check Admin settings and see if GraphViz is disabled in the Debug section?
image
If it's enabled, try resetting the settings in the user interface.

2. Admin settings default for MCLIMIT seems to be 1. 50 or 100 would be better.

As per #163, we couldn't prove it really did anything. In later testing I've found it helps when set to big numbers, like 1,000. But this makes it very slow to load. Even 100 can be slow when the server is not powerful. Generating large diagrams can make this especially slow. And although the layout of the graph often changes with a setting like 50 or 100, I have not found it to have much of an effect in actually reducing the number of arrows crossing each other until the setting gets to 1,000 or even 10,000. So the idea was basically that we don't want everything to be slow for users, so we don't want to default to 1,000. But if setting it to 50 doesn't really help the diagram much, then might as well set it to 1 so it loads quicker. If you can prove me wrong I'm happy to change the default :)

3. When reloading an externaly saved setting file the previous starting person is added to the list of starting persons. This may be intended, however in my opinion it woud be easier to add this person manually if it is really wanted, than detecting and deleting this person from the list, if only the stored starting persons are to be displayed.

Interesting! This is not intended behaviour. I find that if I add three starting persons to the list, then load from a file that has three different people, it adds them in so we end up with 6! It seems the setting is loaded (if you check the XREF list in the advanced settings), but the list is not refreshed. Clicking the refresh button loads the correct people.

I had a quick look at the code but it's not obvious what causes this (and why it's different between loading a file and loading from the saved settings - when they both use the same code). I'll spend some time looking into this.

Still wishes: If an admin or moderator has prepared settings with a selected group of starting persons, e.g. to explain relationship between two persons or to show, how the ownership for a farm changed between families, he might want to make these availiable for users not (yet) able to work with GVE. How can these be made accessable for users (e.g. include call in HTML-page)?

Now that settings are saved in the webtrees settings instead of cookies, you can "Masquerade as this user" from the user list in the Control Panel:
image

Then you can set up some settings for them. Currently they would have to check the advanced settings to see the options. I would like to add saved settings to a drop-down near the top, but this is not done yet. Any suggestions on where to put this? I think it needs to be quite obvious. Perhaps there could be a new section above "People to be included" that just has a dropdown of these saved settings? We could add a toggle to the File settings advanced section where you can enable and disable it, because I'm sure it could get annoying, especially if you don't use this feature.

I'm thinking something like this? I think we would disable it by default, but it would be a user setting that you could enable on their behalf so when they went to GVExport it would show at the top for them.

image

@schuco
Copy link

schuco commented Jan 25, 2023

Can you check Admin settings and see if GraphViz is disabled in the Debug section?
I have disabled and enabled thereafter.Now there is no more comment on graphviz not being installed.

Thanks for your comments and hints.
You are right. In most case when I tried to change appearance of a chart by MCLIMIT I failed.

I will try to masquerade to offer prepared settings to users.

@Neriderc
Copy link
Owner Author

Great! I'm going to create a couple of new issues so we can continue the discussion on adding a dropdown list, and on the issue where individuals are added to the list instead of replaced when loading from file.

I have a suspicion of what the issue is, but haven't quite narrowed it down yet.

@Neriderc
Copy link
Owner Author

@schuco the issue should be resolved now if you update from the main branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

3 participants