Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Pull Request] Unit Test Server #55

Merged
merged 32 commits into from
Apr 16, 2022

Conversation

ztalbot2000
Copy link
Collaborator

@ztalbot2000 ztalbot2000 commented Apr 14, 2022


name: Pull Request
about:Resolve issues found through new AirConServer with all new unit tests
title: "[Pull Request]"
labels: pull-request
assignees: mitch7391


Is your pull request related to a problem or a new feature? Please describe:

AdvAir.sh does many things that avoid proper testing of the complete script in a test environment. in doing so many kludges were created to work around proper system behaviour; some which I have determined can cause a Pi to crash.

Describe the solution you'd have implemented:

  • With the new AirConServer that mimics the real thing means that Unit testing goes from 60% to a full 100% of the code. This is because before only the jq portion was tested and now the exact same curl commands that go to a real AirCon, Now go to the AirConServer.
  • All the weird Unit test files AdvAir.sh created or used disappear because the AirConServer makes AdvAir.sh behave 100% the same when unit testing.
  • For "Set" operations a getSystemData is always done first and last and for "Get" operations a cached getSystemData is always done first. There were multiple places that sometimes this was true and sometimes not. The inconsistencies were patched by doing another getSystemData which was not handled properly with file locking.
  • The file locking so getSystemData was fixed properly and the failure of returned bad data from the AirCon checked correctly.
  • The duplicate 12 hour getSystemData file for lights, etc was no longer required with a getSystemData always done first.
  • All unit tests were rewritten for the new AirConServer. The old checks with zones.sh and ezone.sh removed. They are a lot faster now as well.
  • The error logs now go to one file as when AdvAir.sh died, the excess log files could crash the Pi.

Do your changes pass local testing:

  • Yes

Additional context:

…the AirCon

This makes testing behave more like a real system and no kludges for Unit Testing
This update has only GetBrightness working, via npm run bats test/GetBrightness.bats
@mitch7391 mitch7391 changed the title B unit test server [Pull Request] Unit Test Server Apr 15, 2022
@mitch7391 mitch7391 self-requested a review April 15, 2022 16:11
@mitch7391
Copy link
Owner

Thanks for this @ztalbot2000, I will take some time to look over this when I can. I fear my biggest battle is going to be to get GitHub to like it as it has a number of conflicts due to the fact I went ahead with v3.3.0 before taking this on.

@uswong can you please review alongside me as well? Always best to get more eyes looking over this considering my level of experience vs the two of you.

@mitch7391 mitch7391 added enhancement New feature or request pull-request Merge your work! labels Apr 15, 2022
@ztalbot2000
Copy link
Collaborator Author

ztalbot2000 commented Apr 15, 2022 via email

@mitch7391
Copy link
Owner

Hmm that is strange then John! You can probably see this yourself, but it is showing me the following issues:

image

It states that these conflicts cannot be resolved in the web editor and it will require the command line to resolve. I am happy to leave this with you if you think you can get around it; I know you have done this a lot more than me haha...

@ztalbot2000
Copy link
Collaborator Author

ztalbot2000 commented Apr 15, 2022 via email

@ztalbot2000
Copy link
Collaborator Author

ztalbot2000 commented Apr 15, 2022 via email

@mitch7391
Copy link
Owner

Thanks John, let me know how you go or if you need me to do anything!

@ztalbot2000 ztalbot2000 merged commit 66fdddd into mitch7391:beta Apr 16, 2022
@uswong
Copy link
Contributor

uswong commented Apr 16, 2022

@uswong can you please review alongside me as well? Always best to get more eyes looking over this considering my level of experience vs the two of you.

I am super excited to see the creation of AirConServer. I have a go with it and it really behaves like a real small system. A big system is a lot more laggy.

I also have a good look over the AdvAir.sh and thanks JohnT for making the code a lot more elegant. However I have the following concerns for big MyPlace systems:

  1. If a lock file is detected, the use of iteration to getSystemData may not be enough for a big system. A big system takes between 1s to 60s (histogram distribution shown below) to getSystemData, ie the lock file can be there on average ~6s and occasionally can be up to 60s and beyond. The maximum time allowed within the iteration is 4.8s. As such, for a big system, ~50% of getSystemData will fail.

    The histogram below shows the time taken to getSystemData for a big MyPlace system. The overall average time taken is ~6s. It has ~6000 data points.

image

  1. For a big system with a lot of automations programmed within the control unit, the curl output of getSystemData to a variable directly may be fragmented. This can be resolved by outputting the curl getSystemData result to a file first.

  2. The "forced" getSystemData after every Set command will overwhelm the aircon system and will bring the control unit to a grounding halt if an user runs his automation to turn on/off multiple lights.

    JohnW has an automation set up to turn off all his 77 lights when he leaves home. That will certainly bring his aircon control unit to a grounding halt and not all lights will be turned off. Moreover, the fact that the aircon control unit may take up to 4s to populate the json body after each Set command will further aggravate the performance.

May I propose to leave the above concerns to me to resolve? And I will do a PR when done and allow John to look over it and perhaps can help to make it more elegant in the coding.

Cheers,
Ung Sing

@ztalbot2000
Copy link
Collaborator Author

ztalbot2000 commented Apr 16, 2022 via email

@mitch7391
Copy link
Owner

mitch7391 commented Apr 26, 2022

Hey @ztalbot2000 sorry for the late reply on this one. I would like @uswong to move forward with the changes he has in mind. I can see that the proof is in the pudding with the data presented and as a user, I have noticed that the overall performance of all of our work combined is the best it has ever been. Definitely always happy for your suggestions of how we can better do things as all of us are always still learning (me the most out of the three of us!), but I am happy with the explanation Ung Sing has provided and am not sure if we are getting lost in translation at the moment :)

@ztalbot2000
Copy link
Collaborator Author

ztalbot2000 commented Apr 30, 2022 via email

@mitch7391
Copy link
Owner

Hey @ztalbot2000, I am sorry if it has come across that way. That being said, a lot of Ung Sing’s hard work was removed in your PR as well; but that had been extensively tested with data to back the results. I am sorry if we have not given your PR the same courtesy.

I cannot speak for Ung Sing, but I did not have a chance to test your PR as my personal life has been really hectic lately (as you both know); but I think Ung Sing had tested it for me. His son JohnW is the only one we can rely on to test a larger system, but he is a very busy man and any setbacks require remote support from Ung Sing to rectify. This being said, with Ung Sing’s work that was removed in your PR; I had never seen cmd4 respond so quickly to our script. But I can only speak for a smaller system.

I am a little bit between a rock and a hard place if the two of you do not agree with each other’s work and do ask that you both keep an open mind with each other’s work and try to keep any emotion out of it. I highly respect both of you, your opinions and really love how much the two of you have made this project what it is, for myself and the collective of users I now have. It is all thanks to you both.

@ztalbot2000
Copy link
Collaborator Author

ztalbot2000 commented May 1, 2022 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request pull-request Merge your work!
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants