Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Synchronise changes with WideSky's pyhaystack fork #95

Merged

Conversation

sjlongland
Copy link
Collaborator

This adds some features from the WideSky fork of pyhaystack back to the main-line:

  • Fixes to the Debian package builds so that dependencies are respected
  • Added support for user password changes using the updatePassword API
  • Added support for user impersonation for easier authorisation rule debugging by administrators

All code has been passed through the black code-reformatter to be consistent with the upstream project. This PR is a follow-on from PR #94.

@sjlongland sjlongland changed the base branch from master to develop September 2, 2020 02:25
@sjlongland sjlongland marked this pull request as ready for review September 2, 2020 22:41
@sjlongland sjlongland force-pushed the feature/widesky-sync branch 2 times, most recently from 2955a95 to 3602fda Compare September 2, 2020 23:06
sjlongland and others added 6 commits September 3, 2020 13:13
commit 5b4c422
Author: Stuart Longland <stuartl@vrt.com.au>
Date:   Mon Apr 15 12:10:35 2019 +1000

    VRT-1681: mkdeb.sh: Also depend on signalslot

commit f8ede3c
Author: Stuart Longland <stuartl@vrt.com.au>
Date:   Mon Apr 15 10:30:39 2019 +1000

    VRT-1681: mkdeb.sh: Force dependency on hszinc
Squashed commit of the following:

commit 0be998be9fedf46fddded47aa1847ea98798fb26
Author: Stuart Longland <stuartl@vrt.com.au>
Date:   Tue Nov 19 12:50:47 2019 +1000

    WC-2475: widesky ops: Use correct base class for password change.

    Using `BaseAuthOperation` will trigger an automatic authentication
    attempt instead of requiring the user to call `authenticate()` first.

commit 79bc1cb34374a6433ad43dc4b6757e6cef359309
Author: Stuart Longland <stuartl@vrt.com.au>
Date:   Mon Nov 18 14:34:05 2019 +1000

    WC-2475: setup.py: Drop parsimonious dependency

    That was an _old_ `hszinc` dependency, it is not required today.

commit d1c1907d6ed14395dd4608b24cd6bcc0c7de821e
Author: Stuart Longland <stuartl@vrt.com.au>
Date:   Mon Nov 18 10:10:20 2019 +1000

    WC-2475: widesky tests: Make sure password change is authenticated

commit d0540170c7a396a9700d9a68e6f07b79af8db44d
Author: Stuart Longland <stuartl@vrt.com.au>
Date:   Mon Nov 18 09:41:33 2019 +1000

    WC-2475: widesky tests: Add test case for change of password

commit 65920d81fb4c65352c23c87d01c7a38c21764fe7
Author: Stuart Longland <stuartl@vrt.com.au>
Date:   Mon Nov 18 09:27:01 2019 +1000

    WC-2475: client.widesky: Make password change an operation.

    Migrate Sam's code to a operation class in `client.ops.vendor.widesky`
    then reference that in the WideSky client class.

commit 7e68ecdcffaa19410cde3d6bcd5e77ed33e3b389
Author: SamuelToh <samuelt@vrt.com.au>
Date:   Fri Oct 18 14:57:36 2019 +1000

    WC-2475: Pass through user specified callback fn

commit 143dee9305b675c81124823b766cd46de156e33c
Author: SamuelToh <samuelt@vrt.com.au>
Date:   Fri Oct 18 12:35:07 2019 +1000

    WC-2475: Apply review changes

commit 6ce42c55eb47630a002adfbbe34044cfcff5faf0
Author: SamuelToh <samuelt@vrt.com.au>
Date:   Tue Oct 8 16:03:49 2019 +1000

    WC-2475: simply return type as boolean

    Since the async exception is not easily accessible to the caller

commit 270515da4916d021efed325247b6e2ca2d0e432a
Author: SamuelToh <samuelt@vrt.com.au>
Date:   Sat Oct 5 22:18:39 2019 +1000

    WC-2475: added test for it

commit 9e8e867f84254b4ef858cf2ab313d1856b68a62c
Author: SamuelToh <samuelt@vrt.com.au>
Date:   Sat Oct 5 21:53:37 2019 +1000

    WC-2475: new password mixin for WS
- Remove `pandas` from the requirements, make it a "suggested" package
  instead.
- Base the version string off the git tag instead of whatever is in the
  package.
- Add `dh-python` to build-depends as per suggestions from the Debian
  build system.
Result of command: `black -t py27 .`
Package used: `dev-python/black-19.10_beta0::gentoo`
@ChristianTremblay ChristianTremblay merged commit 4719b13 into ChristianTremblay:develop Sep 3, 2020
@coveralls
Copy link

Coverage Status

Coverage remained the same at 39.262% when pulling c23ddf1 on vrtsystems:feature/widesky-sync into 13304c0 on ChristianTremblay:develop.

ChristianTremblay added a commit that referenced this pull request Sep 3, 2020
* Syncing master with develop

Signed-off-by: Christian Tremblay <christian.tremblay@servisys.com>

* Quick fix to look for exception in scram login

* cleanup / fix setup.py, and add license (#69)

`re` and `requests` weren't used in this file, and weren't declared in
`setup_requires` causing some build issues. Removed these imports.

Removed mixed tabs & spaces.

Explicitly added the Apache License to list of Classifiers so this
project can be identified as Apache Licensed at pypi.org. This matches
the license file.

* Synchronise VRT's Pyhaystack tree (#71)

* WC-847: client.widesky: Expect to see 404 messages.

* WC-847: client.ops.entity: Handle "not found" errors.

If we get back a `HaystackError` of the form `HNotFoundError: …`, then
the entity does not exist.

* WC-847: client.mixins.vendor.widesky.crud: Handle 200/400/404 error codes.

- 200: the "happy path"
- 400: bad request
- 404: not found

* WC-621: client.ops.his: Handle rng=slice object case.

MeterMaster passes in a `slice` object with two `datetime.datetime`
objects when it wishes to do a read operation.  It seems this construct
causes some grief with `pyhaystack` and `hszinc` which shouldn't be an
issue.

This fixes the issue by converting the slice object into a
comma-separated string: standard Project Haystack format.

* WC-1534: fix for creating entity without specifing 'id' tag

* WC-1534: add api server version check for uuid support

* VRT-1681: mkdeb.sh: Add Debian package build script.

* VRT-1681: Drop out-of-date Debian package files.

The script `mkdeb.sh` just needs `stdeb` and should remain up-to-date.

* WC-847: client.session: Wrap low-level function calls

This lets a subclass easily insert options into the `_get` or `_post`
request passed to the HTTP client, e.g. to expect certain HTTP error
codes.

* VRT-1681: client.widesky: Clean up duplication from merge.

* Added a helper function to translate niagara ~2d, etc characters and make more readable strings

* Run black over the files.

* Update the company name of Widesky IoT platform (#78)

* cleanup / fix setup.py, and add license

`re` and `requests` weren't used in this file, and weren't declared in
`setup_requires` causing some build issues. Removed these imports.

Removed mixed tabs & spaces.

Explicitly added the Apache License to list of Classifiers so this
project can be identified as Apache Licensed at pypi.org. This matches
the license file.

* Update the company name of Widesky IoT platform

As of July 2019 Widesky.cloud is now the company responsible for developing Widesky.

* Added forgotten file

* Modification in his.py to cover nan in series. Better implementation for JSON by default for Niagara4

* Added notice that Niagara4 works better with JSON

* Issue 82: client.ops.grid: Clean up whitespace.

* Issue 82: client.ops.grid: Remove check on raw_request.

Regardless of something being a raw request or not, we should still be
setting the `Accept` header accordingly.  Can't recall why this check
was done and can't see a good reason for doing it, so out it goes.

* simplifying grid_format parameter (#81)

* Setting dev on version...

* SkysparkScramHaystackSession can now be used as a context manager (#90)

* following the work of @criverc in PR #90, we implement context manager for all implementations.

* following the work of @criverc in PR #90, we implement context manager for all implementations.

* An attempt to make it work

* # client.session: Use POST request to perform `hisRead`

SkySpark apparently do more than just "retrieve the data" when you do a
`hisRead`, and so have decided that it is not appropriate to allow `GET`
requests to perform a `hisRead`.
(Annoying, because it was handy to see the read range being fetched by a
client in HTTP server logs.)

# client.ops.grid: Allow pass-through of exclude_cookies.

SkySpark gets its knickers twisted if we "round-trip" cookies it sent
us.  So we need the ability to exclude the use of cookies in some cases.
# client.skyspark: Suppress cookies for SCRAM-based SkySpark
SkySpark really gets confused if you follow HTTP standards and hand its
cookies back to it.

# client.http.sync: Drop use of requests `Session`
This session object is responsible for doing things like
managing and expiring cookies, which normally is a desirable thing for
HTTP clients.
SkySpark gets horribly confused when you do this.  It will demand an
attestation key if it sees a cookie round-tripped by the HTTP client.
We actually round-trip things like cookies at a higher level, so it's
redundant for `requests` to do it for us, and plus it's harmful in this
case, we actually don't want to store the cookie in this situation.

* Minor black reformat

* Bumping develop version

* Probably needed when using get for his_read

* Niagara needs the Sessions... adding an http_args flag that will allow us to use it or now depending on the implementation

* http_args is setup for turning off requests.Session for Skyspark automatically.
Bumping develop version

* Some work on point_write. nhaystack can'T accept post (at least the way I did it) So i'll wait.

Also created a helper to build a grid from a dict

On write, if who is not provided, we'll use session username

Version bump

* Fix breakage due to hszinc changes (#94)

* WC-3135: client.ops.grid: Always request multi-grid parsing

* WC-3135: client unit tests: Parse single grids

* post_format didn't pass well to _post_grid making invoke action fail on Niagara if default mode is JSON (for reading speed) not all post action support JSON. So until explicitly specified, post default format should be ZINC.

* Black formatting

* Bumping version of develop for fun.... next version will be 3 to fit with haystack version supported. Will be better than 0.xxx

* Removing print statements

* Synchronise changes with WideSky's pyhaystack fork (#95)

* WC-3136: mkdeb.sh: Fix dependencies in Debian package.

commit 5b4c422
Author: Stuart Longland <stuartl@vrt.com.au>
Date:   Mon Apr 15 12:10:35 2019 +1000

    VRT-1681: mkdeb.sh: Also depend on signalslot

commit f8ede3c
Author: Stuart Longland <stuartl@vrt.com.au>
Date:   Mon Apr 15 10:30:39 2019 +1000

    VRT-1681: mkdeb.sh: Force dependency on hszinc

* WC-2463: Added support for impersonate

* WC-3136: Merge password change feature

Squashed commit of the following:

commit 0be998be9fedf46fddded47aa1847ea98798fb26
Author: Stuart Longland <stuartl@vrt.com.au>
Date:   Tue Nov 19 12:50:47 2019 +1000

    WC-2475: widesky ops: Use correct base class for password change.

    Using `BaseAuthOperation` will trigger an automatic authentication
    attempt instead of requiring the user to call `authenticate()` first.

commit 79bc1cb34374a6433ad43dc4b6757e6cef359309
Author: Stuart Longland <stuartl@vrt.com.au>
Date:   Mon Nov 18 14:34:05 2019 +1000

    WC-2475: setup.py: Drop parsimonious dependency

    That was an _old_ `hszinc` dependency, it is not required today.

commit d1c1907d6ed14395dd4608b24cd6bcc0c7de821e
Author: Stuart Longland <stuartl@vrt.com.au>
Date:   Mon Nov 18 10:10:20 2019 +1000

    WC-2475: widesky tests: Make sure password change is authenticated

commit d0540170c7a396a9700d9a68e6f07b79af8db44d
Author: Stuart Longland <stuartl@vrt.com.au>
Date:   Mon Nov 18 09:41:33 2019 +1000

    WC-2475: widesky tests: Add test case for change of password

commit 65920d81fb4c65352c23c87d01c7a38c21764fe7
Author: Stuart Longland <stuartl@vrt.com.au>
Date:   Mon Nov 18 09:27:01 2019 +1000

    WC-2475: client.widesky: Make password change an operation.

    Migrate Sam's code to a operation class in `client.ops.vendor.widesky`
    then reference that in the WideSky client class.

commit 7e68ecdcffaa19410cde3d6bcd5e77ed33e3b389
Author: SamuelToh <samuelt@vrt.com.au>
Date:   Fri Oct 18 14:57:36 2019 +1000

    WC-2475: Pass through user specified callback fn

commit 143dee9305b675c81124823b766cd46de156e33c
Author: SamuelToh <samuelt@vrt.com.au>
Date:   Fri Oct 18 12:35:07 2019 +1000

    WC-2475: Apply review changes

commit 6ce42c55eb47630a002adfbbe34044cfcff5faf0
Author: SamuelToh <samuelt@vrt.com.au>
Date:   Tue Oct 8 16:03:49 2019 +1000

    WC-2475: simply return type as boolean

    Since the async exception is not easily accessible to the caller

commit 270515da4916d021efed325247b6e2ca2d0e432a
Author: SamuelToh <samuelt@vrt.com.au>
Date:   Sat Oct 5 22:18:39 2019 +1000

    WC-2475: added test for it

commit 9e8e867f84254b4ef858cf2ab313d1856b68a62c
Author: SamuelToh <samuelt@vrt.com.au>
Date:   Sat Oct 5 21:53:37 2019 +1000

    WC-2475: new password mixin for WS

* WC-3136: widesky tests: Fix comparison of bytestring

* WC-2627: mkdeb.sh: Tweak package builds

- Remove `pandas` from the requirements, make it a "suggested" package
  instead.
- Base the version string off the git tag instead of whatever is in the
  package.
- Add `dh-python` to build-depends as per suggestions from the Debian
  build system.

* WC-3136: Re-format through `black` code formatter

Result of command: `black -t py27 .`
Package used: `dev-python/black-19.10_beta0::gentoo`

Co-authored-by: SamuelToh <samuelt@vrt.com.au>

* Clean up differences between WideSky and official pyhaystack trees. (#96)

* WC-3138: README.rst: Tweak README.rst formatting

* WC-3138: docs: Document `impersonate`

* WC-3138: tests: Clean up issues spotted by `pyflakes`

* Minor correction to log error in logout for niagara. Black formatting

* Bumpîng version to 3.0

This version will require hszonc 1.3.0+ to work and is compatible with haystack version 3

Skysparks users are now using post requests tfor his_read (no more get with side effect)

A context manager is available for Niagara and Skyspark

A lot of bugs have been wiped out.

Co-authored-by: Stuart Longland <stuartl@vrt.com.au>
Co-authored-by: Samuel <samuelt@vrt.com.au>
Co-authored-by: Eduardo S. Klein <duduklein@gmail.com>
Co-authored-by: criverc <carlos@superkaos.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants