-
-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Robot Settings Error Handling #184
Conversation
How important is is that the system carry on in the face of errors? Would rejecting the file (or invalid settings) and then stopping be a valid option? The current approach here does improve things, as there's now some diagnostic information, however it still feels relatively easy to miss that an error file has been created. Additionally, it looks like the error file will persist even when the settings read was successful which could complicate things on subsequent runs. It may be worth thinking through what sorts of scenarios can cause errors here. For example: could it be that someone has customised their WiFi config, had it working, then tweaked some other aspect of the config and now finds they can't connect to the WiFi due to the settings having all been regenerated? That sort of thing feels like it could end up in blind alleys around "why has my WiFi stopped working (but the robot still ran)" which miss the fact that the WiFi is actually working just with different credentials. Having the old config output as part of the error file feels like a good aspect here, however I'm not really convinced that the "regenerate and replace" approach is a good one to start with. (Am I missing something which makes that sort of approach unfeasible?) |
Thanks for the good questions, much appreciated :)
We don't have a separate LED we could use to indicate to failure. We could use the OK LED on the KCH to indicate this, perhaps a reuse of the red colour.
Excellent point. Would you suggest deleting the error file (if it exists) where the settings file has been successfully parsed?
There is an indicator on the robot to show that the WiFi is working. I'm not sure that's applicable to all potential settings though.
From an implementation perspective, this is potentially difficult as the settings file is parsed in a different process from where the code is executed. We would have to think carefully how to avoid the code starting when the settings file is invalid, the current service architecture just doesn't work well for that scnenario. I'd also be concerned about a robot failing to run in a competition match because the settings file became corrupted. Regenerating the file is the current behaviour for an error, but just completely silently. I think in the future there is an aim to have the settings editable from the web ui instead anyway, which would mitigate most of these issues. That is very much not in scope until SR2024 though. |
I wondered about this, however it feels like this could lead to undesirable things too. One example is that running the robot twice after breaking the settings file leaves no trace of the error and completely erases the original (broken) settings file.
Given the presence of the usercode entry point in the settings file I assume there's already a strict ordering to things though? What would happen if we just never provided the usercode entry point from the service that's parsing the settings? (It feels like that should have a similar effect -- the code never runs) For clarity: I also think it would be reasonable to only degrade the parts of the settings which aren't valid -- for example if the WiFi details aren't valid but the usercode entry point is, then running the code but disabling the WiFi might be a better solution than fully regenerating the file.
Good point. As the settings file includes the usercode entry point this already seems possible with the regeneration route? I hadn't spotted this (wrong code) scenario before, though this is the sort of thing I mean by the behaviour not being "obvious". It also feels a pretty compelling reason to me to avoid the regeneration route and focus on improving the feedback to the user that something is wrong. |
Good point. I think we should continue to leave it there, in the same manner as
Oh I wish. The settings file is actually read twice, both in @PeterJCLaw: Would you be okay with us leaving this as is for this year, documenting the changes we need and then introducing strict ordering for next competition year?
I'm somewhat regretting adding the
@raccube I'd be interested to hear your thoughts here please |
Is there an option to remove the regeneration of the file in cases where it already exists? If it's read twice then it sounds like the regeneration already has race conditions associated with it.
I'm not completely clear on what you mean by the "this" here. If you mean the case of running the user code even when the WiFi settings are broken, I think that's fine (perhaps even in the long term as long) as the feedback to the user is clear. |
a2b7099
to
bffdfd5
Compare
Not feasible without a re-architecture to handle the robot settings under a separate service. (i.e read in one place*) *: Technically it will still need to be read in two places, but one read is strictly ahead of the other as it takes place when disk type is determined.
Yes. There is a dodgy sleep statement that handles this at the moment. The above separate service would probably resolve these issues too.
I agree that this would be desirable. I don't think it's feasible to make the changes required mid-competition.
More so that this is strictly an improvement over the current behaviour, even if the behaviour is still very much imperfect. I think we can cover the entrypoint mess by adding a note in the docs. I'll add a PR shortly. Issue #187 created so that we do not forget about this. |
When a robot-settings.toml file is bad, generate a robot-settings-error.txt file on the USB drive also. This file contains a message to tell the user that their settings has been regenerated, a detailed error for why the settings were invalid and in most cases a copy of the old settings file. An error file is not generated if the file is missing, as this is the expected behaviour for a new USB drive.
Whilst we don't allow non-ascii chars to be specified by the user, technically we need to encode this to check it
bffdfd5
to
7a19368
Compare
For my understanding/posterity -- could you expand on what the failure mode is if we didn't regenerate the file? i.e: just left the "bad" one in place? |
With our current architecture, exceptions would be thrown in all Astoria daemons and no metadata would be processed. This would result in code never running because the mqtt messages get thrown away. It's a really bad idea. It's not even really possible either. Pydantic is used to parse and validate the file and then that struct is used everywhere. There can't really be invalid data. We will fix this, just not now. |
Ah, ok. I had thought I'd seen some checks for things like the WiFi name being the right shape (having a TLA) -- i.e: validation of the content of the data rather than parsing errors. Does that case behave differently? |
At the moment if a
robot-settings.toml
file is invalid, we silently regenerate it without any notice to the user.This PR adds a
robot-settings-error.txt
that is generated when an error occurs. This file contains a message to tell the user that their settings has been regenerated, a detailed error for why the settings were invalid and in most cases a copy of the old settings file (as long as the file is valid unicode).An error file is not generated if the file is missing, as this is the expected behaviour for a new USB drive.
This PR also adds better validation of the WiFi PSK.
Example error file:
Also fixes #175