Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v3 Development - Calling All TypeScript Developers #98

Closed
mrjackyliang opened this issue Apr 2, 2023 · 130 comments
Closed

v3 Development - Calling All TypeScript Developers #98

mrjackyliang opened this issue Apr 2, 2023 · 130 comments
Labels
enhancement New feature or request

Comments

@mrjackyliang
Copy link
Owner

I'm looking to re-build the entire plugin base using TypeScript due to the massive amounts of support for this plugin. Because of the defunct platform that ADT currently has no plans improving, I'm not sure if the switch to it would be worth the investment.

If you think this plugin can be further improved, please let me know, and I'll see if I can make this crucial plugin work out.

@mrjackyliang mrjackyliang added the enhancement New feature or request label Apr 2, 2023
@mrjackyliang
Copy link
Owner Author

Tagging @Danimal4326 in for a discussion, see if you're interested.

@mrjackyliang
Copy link
Owner Author

The re-write versions will be in this branch and merged when everything is tested, documented, and stable.

@Danimal4326
Copy link
Contributor

I talked to ADT regarding upgrading my system to their newer platform (based on Alarm.com I believe) After discussing costs and benefits, I’ll be sticking with Pulse. They’ve upgraded my backed call box to LTE too so I don’t think Pulse is going anywhere, as they have a large installed base.

With that said, I’d love to see continued support for the plug-in as well as improvements

@Danimal4326
Copy link
Contributor

My ts skills are basic at best, as my day job is firmware development in C/C++ with Python for testing/support. But I’d love to help out in any way I can

@mrjackyliang
Copy link
Owner Author

My ts skills are basic at best, as my day job is firmware development in C/C++ with Python for testing/support. But I’d love to help out in any way I can

Yeah, I just had feelings that one day ADT might want to pull the plug, but it has been years and I'm sure I seem to like the way how everything works.

On the contrary, TypeScript is just JS with types haha, really nothing crazy about it. Plus, I'd convert this to ESM along the way.

@mrjackyliang
Copy link
Owner Author

Decided to put a pause on this for now. Will re-open when I decide it is worth doing.

@mrjackyliang
Copy link
Owner Author

mrjackyliang commented Nov 21, 2023

Began re-writing the API today.

Future versions will no longer auto-detect accessories due to the fragility of users losing data often when ADT makes a unsuspecting change. Before updating, please make a detailed note of all your sensors info.

  1. Name (the name you put in HomeKit, optional)
    • Normally I'd put in the Room and then the sensor this is. For example, if this was a kitchen sink right side window, I'd name it something like "Kitchen Counter Window".
  2. The EXACT name assigned for your ADT sensor
  3. Note the type of the sensor. Homebridge now separates doors and window sensors (so please take note of that)
  4. The zone number.

However, I may create a script for you to easily generate part of that configuration where you'd just need to focus on what to name your devices.

@mrjackyliang
Copy link
Owner Author

mrjackyliang commented Nov 24, 2023

I've noticed something odd with the newer portal versions, and that is the orb.jsp file never loads during the summary page load. I do see that there is a updateContent() function on the summary page.

That is not called (because orb.jsp doesn't show in the networks tab), and I have a feeling that they may remove those pages or make it inaccessible in the future.

If that happens, the only way we can get status is through the summary page or the system page (which is fine). Marking this here to document there may be breaking changes.

UPDATE: My findings were incorrect. The function is being used, but not during initial page load. Only when something updates live while you are on the page.

@mrjackyliang
Copy link
Owner Author

mrjackyliang commented Nov 26, 2023

I made some progress today and worked on reverse engineering the setArmState function and how it works.

Also, I forgot that I finally figured out why sometimes the sensors are blank and accidentally trigger the accessories reset. Nonetheless, having a manually configured sensors list is actually still the best just to protect ourselves from unwanted changes.

@sapireli
Copy link

is there any way to quickly export this?

Began re-writing the API today.

Future versions will no longer auto-detect accessories due to the fragility of users losing data often when ADT makes a unsuspecting change. Before updating, please make a detailed note of all your sensors info.

  1. Name (the name you put in HomeKit, optional)

    • Normally I'd put in the Room and then the sensor this is. For example, if this was a kitchen sink right side window, I'd name it something like "Kitchen Counter Window".
  2. The EXACT name assigned for your ADT sensor

  3. Note the type of the sensor. Homebridge now separates doors and window sensors (so please take note of that)

  4. The zone number.

However, I may create a script for you to easily generate part of that configuration where you'd just need to focus on what to name your devices.

@mrjackyliang
Copy link
Owner Author

is there any way to quickly export this?

Began re-writing the API today.

Future versions will no longer auto-detect accessories due to the fragility of users losing data often when ADT makes a unsuspecting change. Before updating, please make a detailed note of all your sensors info.

  1. Name (the name you put in HomeKit, optional)
  • Normally I'd put in the Room and then the sensor this is. For example, if this was a kitchen sink right side window, I'd name it something like "Kitchen Counter Window".
  1. The EXACT name assigned for your ADT sensor
  1. Note the type of the sensor. Homebridge now separates doors and window sensors (so please take note of that)
  1. The zone number.

However, I may create a script for you to easily generate part of that configuration where you'd just need to focus on what to name your devices.

Yes, I will include a script and instructions on how to easily export this. Might also add some recommendations on how to name the devices in case there is a need to reset the plugin (remembering which zone is which is a pain)

@mrjackyliang
Copy link
Owner Author

At the moment, working on the arm/disarm handlers. I found a few interesting arm states that are in the new version but never seen before in the older versions.

@tr1um9h
Copy link

tr1um9h commented Nov 27, 2023

Thanks for this! Any ETA on possible release?

@mrjackyliang
Copy link
Owner Author

Thanks for this! Any ETA on possible release?

I'm hoping in the next week or two. The plugin has to be completely rewritten.

@mrjackyliang
Copy link
Owner Author

Today, I finished the API portion of the plugin, and successfully completed testing. Going to make some optimizations before proceeding to the Homebridge plugin part.

@Danimal4326
Copy link
Contributor

Awesome!!! can't wait.

@mrjackyliang
Copy link
Owner Author

Today I spent some time refactoring the newly written API, and added some detection utilities to help with unexpected changes.

@mrjackyliang
Copy link
Owner Author

I've pushed the finished API along with the test files to the ts-rewrite branch.

@sapireli
Copy link

sapireli commented Dec 3, 2023

Jacky Liang coming in hot!

@hapinstance
Copy link

Wow! What an effort. Let me know if you need any help. I have some coding experience, and can assist in testing early versions if you like.

@mrjackyliang
Copy link
Owner Author

Wow! What an effort. Let me know if you need any help. I have some coding experience, and can assist in testing early versions if you like.

Will let you know, thank you!!

@mrjackyliang
Copy link
Owner Author

The fun part has come 👯

Screenshot 2023-12-08 at 8 37 43 AM

@chabapj
Copy link

chabapj commented Dec 16, 2023

It is working now - but it is a bit slow. The motion sensor takes 20-30 seconds to show motion, and then several minutes to reset itself to "no motion"?

The same for the door sensors. API polling interval?

@mrjackyliang
Copy link
Owner Author

It is working now - but it is a bit slow. The motion sensor takes 20-30 seconds to show motion, and then several minutes to reset itself to "no motion"?

The same for the door sensors. API polling interval?

I think I forgot to put in the force update after polling that's why it's taking forever.

@sapireli
Copy link

Ok was able to install, it’s working!! some issues:

  1. High cpu usage - this plugin gets my cpu usage up to 100% every few seconds running on a raspberry pi, prior version had negligible effect on cpu.
  2. As already reported updating sensors and security panel state takes forever.

Thank you for all the hard work @mrjackyliang

@mrjackyliang
Copy link
Owner Author

Ok was able to install, it’s working!! some issues:

  1. High cpu usage - this plugin gets my cpu usage up to 100% every few seconds running on a raspberry pi, prior version had negligible effect on cpu.

  2. As already reported updating sensors and security panel state takes forever.

Thank you for all the hard work @mrjackyliang

Oh high CPU? That's a first. I ran it on my system and it's working fine.

@chabapj
Copy link

chabapj commented Dec 17, 2023

I do not see any high CPU usage. I am running it on an 8-core beast, there is no difference between the plugin enabled or disabled.

@mrjackyliang
Copy link
Owner Author

@sapireli What are the specs to your Raspberry Pi? My CPU is running at the typical 2% right now. It's an i7

@sapireli
Copy link

@sapireli What are the specs to your Raspberry Pi? My CPU is running at the typical 2% right now. It's an i7

i'm using a trusty old raspi 1b, on the old plugin cpu usage would minimally increase 5-7% now its hitting 100%. i disabled it for now and its back down to 11%

@sapireli
Copy link

also why are you all using an i7 for homebridge? waste of $$ on electricity

@mrjackyliang
Copy link
Owner Author

also why are you all using an i7 for homebridge? waste of $$ on electricity

@sapireli I don't just use it for Homebridge. Would definitely be a waste if it really was for just that.

@mrjackyliang
Copy link
Owner Author

@sapireli What are the specs to your Raspberry Pi? My CPU is running at the typical 2% right now. It's an i7

i'm using a trusty old raspi 1b, on the old plugin cpu usage would minimally increase 5-7% now its hitting 100%. i disabled it for now and its back down to 11%

@sapireli That's odd, because the way how I coded the old plugin uses the same syncing methodology

@mrjackyliang
Copy link
Owner Author

I have a feeling it's because of OpenSSL and their EOL upgrade from 1.1.1 to 3.0.

I looked up Axios 100% high CPU just as a hunch and I'm thinking it's because of that. The old version used older ciphers and the depreciated request package, because some older Raspberry Pi's couldn't use the plugin, but Axios, unfortunately doesn't offer custom ciphers.

@mrjackyliang
Copy link
Owner Author

@sapireli Updated to v3.0.0-beta.6, you can now choose between 4 operational modes for your Raspberry Pi.

@sapireli
Copy link

sapireli commented Dec 18, 2023

last night i went down the rabbit hole of trying to upgrade the openssl library to 3.2, ended up destroying dietpi beyond recovery, and then spent way too many hours rebuilding / recreating automations. The homebridge backups weren't usable either. FML.

In any event, i ended up installing a bullseye version of dietpi instead of bookworm that has openssl 1.1.1. It does seem to be a lot faster now FYI.

@sapireli
Copy link

Ps. @mrjackyliang Thank you will try it tonight. You're the man.

@mrjackyliang
Copy link
Owner Author

last night i went down the rabbit hole of trying to upgrade the openssl library to 3.2, ended up destroying dietpi beyond recovery, and then spent way too many hours rebuilding / recreating automations. The homebridge backups weren't usable either. FML.

In any event, i ended up installing a bullseye version of dietpi instead of bookworm that has openssl 1.1.1. It does seem to be a lot faster now FYI.

Try to see if the 100% CPU comes back up even under a normal operational speed setting. I'm pretty sure it has to do with OpenSSL. That also was one of the knockers when the plugin was first released as well.

I do hope Axios can take care of it automatically because with request, I had to specify specific ciphers.

@sapireli
Copy link

New beta working well. After the downgrade to 'debian bullseye' i.e. Openssl 1.1.1, the CPU usage is back to normal. You were absolutely right.

The only two things are:

  1. Now I'm seeing this error in the logs:
    [12/18/2023, 10:59:32 AM] [ADT Pulse] Plugin is unable to initialize due to an invalid platform configuration. [ { "code": "invalid_union", "unionErrors": [ { "issues": [ { "code": "invalid_literal", "expected": "normal", "path": [ "mode" ], "message": "Invalid literal value, expected 'normal'" } ], "name": "ZodError" }, { "issues": [ { "code": "invalid_literal", "expected": "paused", "path": [ "mode" ], "message": "Invalid literal value, expected 'paused'" } ], "name": "ZodError" }, { "issues": [ { "code": "invalid_literal", "expected": "reset", "path": [ "mode" ], "message": "Invalid literal value, expected 'reset'" } ], "name": "ZodError" } ], "path": [ "mode" ], "message": "Invalid input" }, { "code": "invalid_union", "unionErrors": [ { "issues": [ { "code": "invalid_literal", "expected": 1, "path": [ "speed" ], "message": "Invalid literal value, expected 1" } ], "name": "ZodError" }, { "issues": [ { "code": "invalid_literal", "expected": 0.75, "path": [ "speed" ], "message": "Invalid literal value, expected 0.75" } ], "name": "ZodError" }, { "issues": [ { "code": "invalid_literal", "expected": 0.5, "path": [ "speed" ], "message": "Invalid literal value, expected 0.5" } ], "name": "ZodError" }, { "issues": [ { "code": "invalid_literal", "expected": 0.25, "path": [ "speed" ], "message": "Invalid literal value, expected 0.25" } ], "name": "ZodError" } ], "path": [ "speed" ], "message": "Invalid input" } ]
  2. and perhaps bump to the node version so this message doesnt show> 'The plugin "homebridge-adt-pulse" requires Node.js version of 18 which does not satisfy the current Node.js version of v20.10.0. You may need to upgrade your installation of Node.js - see https://homebridge.io/w/JTKEF'

Many thanks @mrjackyliang would love to buy you a drink i'm in NYC too.

@mrjackyliang
Copy link
Owner Author

mrjackyliang commented Dec 18, 2023

New beta working well. After the downgrade to 'debian bullseye' i.e. Openssl 1.1.1, the CPU usage is back to normal. You were absolutely right.

The only two things are:

  1. Now I'm seeing this error in the logs:

`[12/18/2023, 10:59:32 AM] [ADT Pulse] Plugin is unable to initialize due to an invalid platform configuration.

[

{

"code": "invalid_union",

"unionErrors": [

  {

    "issues": [

      {

        "code": "invalid_literal",

        "expected": "normal",

        "path": [

          "mode"

        ],

        "message": "Invalid literal value, expected 'normal'"

      }

    ],

    "name": "ZodError"

  },

  {

    "issues": [

      {

        "code": "invalid_literal",

        "expected": "paused",

        "path": [

          "mode"

        ],

        "message": "Invalid literal value, expected 'paused'"

      }

    ],

    "name": "ZodError"

  },

  {

    "issues": [

      {

        "code": "invalid_literal",

        "expected": "reset",

        "path": [

          "mode"

        ],

        "message": "Invalid literal value, expected 'reset'"

      }

    ],

    "name": "ZodError"

  }

],

"path": [

  "mode"

],

"message": "Invalid input"

},

{

"code": "invalid_union",

"unionErrors": [

  {

    "issues": [

      {

        "code": "invalid_literal",

        "expected": 1,

        "path": [

          "speed"

        ],

        "message": "Invalid literal value, expected 1"

      }

    ],

    "name": "ZodError"

  },

  {

    "issues": [

      {

        "code": "invalid_literal",

        "expected": 0.75,

        "path": [

          "speed"

        ],

        "message": "Invalid literal value, expected 0.75"

      }

    ],

    "name": "ZodError"

  },

  {

    "issues": [

      {

        "code": "invalid_literal",

        "expected": 0.5,

        "path": [

          "speed"

        ],

        "message": "Invalid literal value, expected 0.5"

      }

    ],

    "name": "ZodError"

  },

  {

    "issues": [

      {

        "code": "invalid_literal",

        "expected": 0.25,

        "path": [

          "speed"

        ],

        "message": "Invalid literal value, expected 0.25"

      }

    ],

    "name": "ZodError"

  }

],

"path": [

  "speed"

],

"message": "Invalid input"

}

]`

  1. and perhaps bump to the node version so this message doesnt show> 'The plugin "homebridge-adt-pulse" requires Node.js version of 18 which does not satisfy the current Node.js version of v20.10.0. You may need to upgrade your installation of Node.js - see https://homebridge.io/w/JTKEF'

Many thanks @mrjackyliang would love to buy you a drink i'm in NYC too.

@sapireli Ah yes, the configuration changed a bit. I consolidated the Pause and Reset modes and added a Speed setting. The new config shape should be in the readme!

For the Homebridge version part, I have only tested it on Node.js 18. I know it is the last supported version, but from my experience, the last supported versions are actually the ones that are most stable overall. v20 is optimal, should work, v21 a bit too new and possibly plagued with unknown bugs.

Awesomeness!! Would love to meet whenever we get the chance to😁

@knedd
Copy link

knedd commented Dec 18, 2023 via email

@mrjackyliang
Copy link
Owner Author

I had those initial errors when I upgraded from beta 5 to beta 6. Upgrading my configuration file with the new additional attributes fixed it for meKevinSent via iPhoneOn Dec 18, 2023, at 5:12 PM, Jacky Liang @.> wrote: New beta working well. After the downgrade to 'debian bullseye' i.e. Openssl 1.1.1, the CPU usage is back to normal. You were absolutely right. The only two things are: Now I'm seeing this error in the logs: [12/18/2023, 10:59:32 AM] [ADT Pulse] Plugin is unable to initialize due to an invalid platform configuration. [ { "code": "invalid_union", "unionErrors": [ { "issues": [ { "code": "invalid_literal", "expected": "normal", "path": [ "mode" ], "message": "Invalid literal value, expected 'normal'" } ], "name": "ZodError" }, { "issues": [ { "code": "invalid_literal", "expected": "paused", "path": [ "mode" ], "message": "Invalid literal value, expected 'paused'" } ], "name": "ZodError" }, { "issues": [ { "code": "invalid_literal", "expected": "reset", "path": [ "mode" ], "message": "Invalid literal value, expected 'reset'" } ], "name": "ZodError" } ], "path": [ "mode" ], "message": "Invalid input" }, { "code": "invalid_union", "unionErrors": [ { "issues": [ { "code": "invalid_literal", "expected": 1, "path": [ "speed" ], "message": "Invalid literal value, expected 1" } ], "name": "ZodError" }, { "issues": [ { "code": "invalid_literal", "expected": 0.75, "path": [ "speed" ], "message": "Invalid literal value, expected 0.75" } ], "name": "ZodError" }, { "issues": [ { "code": "invalid_literal", "expected": 0.5, "path": [ "speed" ], "message": "Invalid literal value, expected 0.5" } ], "name": "ZodError" }, { "issues": [ { "code": "invalid_literal", "expected": 0.25, "path": [ "speed" ], "message": "Invalid literal value, expected 0.25" } ], "name": "ZodError" } ], "path": [ "speed" ], "message": "Invalid input" } ] and perhaps bump to the node version so this message doesnt show> 'The plugin "homebridge-adt-pulse" requires Node.js version of 18 which does not satisfy the current Node.js version of v20.10.0. You may need to upgrade your installation of Node.js - see https://homebridge.io/w/JTKEF' Many thanks @mrjackyliang would love to buy you a drink i'm in NYC too. @sapireli Ah yes, the configuration changed a bit. I consolidated the Pause and Reset modes and added a Speed setting. The new config shape should be in the readme! Awesomeness!! Would love to meet whenever we get the chance to😁 —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you are subscribed to this thread.Message ID: @.>

@knedd I attest to the huge blocks of error messages for the invalid configuration somewhat hard to read. I did spend some time trying to add custom messages to the schema validator, but the messages were wayyy too dynamic.

I'm pretty sure I had to account for more than 40 different error messages just in that config. Not sure if I really want to put the effort in that. In the future if I were to update the config to add supported sensors or even another setting, that's like another 6 error messages I need to add.

@Danimal4326
Copy link
Contributor

I just upgraded to the beta.

Not sure if this was mentioned, but if "name" does not match "adtName" in the plugin config, the sensor shows as unavailable.

@mrjackyliang
Copy link
Owner Author

I just upgraded to the beta.

Not sure if this was mentioned, but if "name" does not match "adtName" in the plugin config, the sensor shows as unavailable.

@Danimal4326 Mmm, they don't have to match each other. I think the sensor shows as unavailable because of the slow updating.

@Danimal4326
Copy link
Contributor

Hmm I changed the name to match, and they immediately showed as available..

let me test again.

@mrjackyliang
Copy link
Owner Author

Hmm I changed the name to match, and they immediately showed as available..

let me test again.

@Danimal4326 Yeah the sensor's updating is slow, haven't worked on it yet. Yesterday I re-built the config UI, and that took an entire day

@sapireli
Copy link

tty sure I had to account for more than 40 different error messages just in that config. Not sure if I really want to put the effort in that. In the future if I were to update the config to add supported sensors or even another setting, that's like another 6 error messages I need to add.

That fixed it. i could have sworn i updated the config i guess i didn't save or something. Many thanks!

@sapireli
Copy link

beta 6 still going strong only recurring issue is some sensors aren't being matched although less with every restart.

take for example:

12/19/2023, 2:09:45 AM] [ADT Pulse] Attempted to add or update Window 1 Glass (zone: 25) accessory that does not exist on the portal.
[12/19/2023, 2:09:45 AM] [ADT Pulse] Attempted to add or update Window 2 Glass (zone: 27) accessory that does not exist on the portal.
[12/19/2023, 2:10:22 AM] [ADT Pulse] Attempted to add or update Window 1 Glass (zone: 25) accessory that does not exist on the portal.
[12/19/2023, 2:10:22 AM] [ADT Pulse] Attempted to add or update Window 2 Glass (zone: 27) accessory that does not exist on the portal.

all these zones are correct, and the spelling is exact.

@mrjackyliang
Copy link
Owner Author

beta 6 still going strong only recurring issue is some sensors aren't being matched although less with every restart.

take for example:

12/19/2023, 2:09:45 AM] [ADT Pulse] Attempted to add or update Window 1 Glass (zone: 25) accessory that does not exist on the portal.
[12/19/2023, 2:09:45 AM] [ADT Pulse] Attempted to add or update Window 2 Glass (zone: 27) accessory that does not exist on the portal.
[12/19/2023, 2:10:22 AM] [ADT Pulse] Attempted to add or update Window 1 Glass (zone: 25) accessory that does not exist on the portal.
[12/19/2023, 2:10:22 AM] [ADT Pulse] Attempted to add or update Window 2 Glass (zone: 27) accessory that does not exist on the portal.

all these zones are correct, and the spelling is exact.

@sapireli Does this happen all the time, or from time to time? If it is from time to time, then it is probably ADT wanting to act up again. Definitely need to figure out how to better support these errors.

@mrjackyliang
Copy link
Owner Author

Let's continue this conversation in #124, thanks! This thread is too long to scroll down to🥲

Repository owner locked and limited conversation to collaborators Dec 19, 2023
@mrjackyliang
Copy link
Owner Author

Also plugin has been updated to v3.0.0-beta.7

@mrjackyliang mrjackyliang unpinned this issue Jan 27, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

7 participants