-
Notifications
You must be signed in to change notification settings - Fork 19
Trigger automated checks to keep python-podman aligned with libpod defined interfaces #70
Comments
A quick-and-dirty option would just be to pull out all the method names from the varlink schema and run an identifier search over the python source (especially If the identifier doesn't appear, you have pretty high confidence the feature isn't implemented. (Since afaik you're not doing dynamic things.) If it does appear, you can be mildly certain it is implemented. You could probably hack an initial version with a regex and a pile of greps. Unless you have a varlink schema library lying around. |
I think we should have a discussion on these features. I love seeing python-podman further develop. |
Yep I agree it can be a first version quickly released, however, we still don't check if defined and implemented signatures and we risk to face this kind of issue #64, so we need to move further in a second time to cover this part too. |
Thanks Daniel for the heads up, I will take a look to the APIv2 to avoid to spend time on something who will be deprecated in the next releases. |
libpod Related to containers#70 Summarize --------- This tools allow us to compare branches or revision to check if python-podman is unsynchronized with libpod. Design ------ These changes only check if method exist in python-podman souce code. They don't yet check method signature. If a method doesn't exist it will return 1 so we can call it in CI/CD and catch errors to append missing methods. It's possible to force to return a code 0 to avoid to polluate our CI/CD in the first time with a lot of well know issues related to unsynchronized projects. Usage ----- ```shell $ # Check if python-podman is properly synchronized with libpod master $ ./tools/synchronize.sh Downloading the libpod's defined interface https://raw.githubusercontent.com/containers/libpod/master/cmd/podman/varlink/io.podman.varlink % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 52112 100 52112 0 0 49725 0 0:00:01 0:00:01 --:--:-- 49725 Method Reset seems not yet implemented Method GetContainersByStatus seems not yet implemented Method HealthCheckRun seems not yet implemented Method GetContainersByContext seems not yet implemented Method GetContainersLogs seems not yet implemented Method GetContainerStatsWithHistory seems not yet implemented ... Method GetLayersMapWithImageInfo seems not yet implemented Method BuildImageHierarchyMap seems not yet implemented 42 error(s) found Comparison finished...bye! $ # Check if python-podman is properly synchronized with libpod version $ # 1.6.2 (which refer to commit ID f3ffda1e08f19e9a6a88484136b5eed76533f21a) $ ./tools/synchronize.sh -b=f3ffda1e08f19e9a6a88484136b5eed76533f21a Downloading the libpod's defined interface https://raw.githubusercontent.com/containers/libpod/f3ffda1e08f19e9a6a88484136b5eed76533f21a/cmd/podman/varlink/io.podman.varlink % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 50588 100 50588 0 0 51515 0 --:--:-- --:--:-- --:--:-- 51462 Method GetContainersByStatus seems not yet implemented Method HealthCheckRun seems not yet implemented ... Method GetEvents seems not yet implemented Method GetLayersMapWithImageInfo seems not yet implemented Method BuildImageHierarchyMap seems not yet implemented 37 error(s) found Comparison finished...bye! ```
See my proposed changes #71 |
libpod Related to containers#70 Summarize --------- This tools allow us to compare branches or revision to check if python-podman is unsynchronized with libpod. Design ------ These changes only check if method exist in python-podman souce code. They don't yet check method signature. If a method doesn't exist it will return 1 so we can call it in CI/CD and catch errors to append missing methods. It's possible to force to return a code 0 to avoid to polluate our CI/CD in the first time with a lot of well know issues related to unsynchronized projects. Usage ----- ```shell $ # Check if python-podman is properly synchronized with libpod master $ ./tools/synchronize.sh Downloading the libpod's defined interface https://raw.githubusercontent.com/containers/libpod/master/cmd/podman/varlink/io.podman.varlink % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 52112 100 52112 0 0 49725 0 0:00:01 0:00:01 --:--:-- 49725 Method Reset seems not yet implemented Method GetContainersByStatus seems not yet implemented Method HealthCheckRun seems not yet implemented Method GetContainersByContext seems not yet implemented Method GetContainersLogs seems not yet implemented Method GetContainerStatsWithHistory seems not yet implemented ... Method GetLayersMapWithImageInfo seems not yet implemented Method BuildImageHierarchyMap seems not yet implemented 42 error(s) found Comparison finished...bye! $ # Check if python-podman is properly synchronized with libpod version $ # 1.6.2 (which refer to commit ID f3ffda1e08f19e9a6a88484136b5eed76533f21a) $ ./tools/synchronize.sh -b=f3ffda1e08f19e9a6a88484136b5eed76533f21a Downloading the libpod's defined interface https://raw.githubusercontent.com/containers/libpod/f3ffda1e08f19e9a6a88484136b5eed76533f21a/cmd/podman/varlink/io.podman.varlink % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 50588 100 50588 0 0 51515 0 --:--:-- --:--:-- --:--:-- 51462 Method GetContainersByStatus seems not yet implemented Method HealthCheckRun seems not yet implemented ... Method GetEvents seems not yet implemented Method GetLayersMapWithImageInfo seems not yet implemented Method BuildImageHierarchyMap seems not yet implemented 37 error(s) found Comparison finished...bye! ```
libpod Related to containers#70 Summarize --------- This tools allow us to compare branches or revision to check if python-podman is unsynchronized with libpod. Design ------ These changes only check if method exist in python-podman souce code. They don't yet check method signature. If a method doesn't exist it will return 1 so we can call it in CI/CD and catch errors to append missing methods. It's possible to force to return a code 0 to avoid to polluate our CI/CD in the first time with a lot of well know issues related to unsynchronized projects. Usage ----- ```shell $ # Check if python-podman is properly synchronized with libpod master $ ./tools/synchronize.sh Downloading the libpod's defined interface https://raw.githubusercontent.com/containers/libpod/master/cmd/podman/varlink/io.podman.varlink % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 52112 100 52112 0 0 49725 0 0:00:01 0:00:01 --:--:-- 49725 Method Reset seems not yet implemented Method GetContainersByStatus seems not yet implemented Method HealthCheckRun seems not yet implemented Method GetContainersByContext seems not yet implemented Method GetContainersLogs seems not yet implemented Method GetContainerStatsWithHistory seems not yet implemented ... Method GetLayersMapWithImageInfo seems not yet implemented Method BuildImageHierarchyMap seems not yet implemented 42 error(s) found Comparison finished...bye! $ # Check if python-podman is properly synchronized with libpod version $ # 1.6.2 (which refer to commit ID f3ffda1e08f19e9a6a88484136b5eed76533f21a) $ ./tools/synchronize.sh -b=f3ffda1e08f19e9a6a88484136b5eed76533f21a Downloading the libpod's defined interface https://raw.githubusercontent.com/containers/libpod/f3ffda1e08f19e9a6a88484136b5eed76533f21a/cmd/podman/varlink/io.podman.varlink % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 50588 100 50588 0 0 51515 0 --:--:-- --:--:-- --:--:-- 51462 Method GetContainersByStatus seems not yet implemented Method HealthCheckRun seems not yet implemented ... Method GetEvents seems not yet implemented Method GetLayersMapWithImageInfo seems not yet implemented Method BuildImageHierarchyMap seems not yet implemented 37 error(s) found Comparison finished...bye! ```
libpod Related to containers#70 Summarize --------- This tools allow us to compare branches or revision to check if python-podman is unsynchronized with libpod. Design ------ These changes only check if method exist in python-podman souce code. They don't yet check method signature. If a method doesn't exist it will return 1 so we can call it in CI/CD and catch errors to append missing methods. It's possible to force to return a code 0 to avoid to polluate our CI/CD in the first time with a lot of well know issues related to unsynchronized projects. Usage ----- ```shell $ # Check if python-podman is properly synchronized with libpod master $ ./tools/synchronize.sh Downloading the libpod's defined interface https://raw.githubusercontent.com/containers/libpod/master/cmd/podman/varlink/io.podman.varlink % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 52112 100 52112 0 0 49725 0 0:00:01 0:00:01 --:--:-- 49725 Method Reset seems not yet implemented Method GetContainersByStatus seems not yet implemented Method HealthCheckRun seems not yet implemented Method GetContainersByContext seems not yet implemented Method GetContainersLogs seems not yet implemented Method GetContainerStatsWithHistory seems not yet implemented ... Method GetLayersMapWithImageInfo seems not yet implemented Method BuildImageHierarchyMap seems not yet implemented 42 error(s) found Comparison finished...bye! $ # Check if python-podman is properly synchronized with libpod version $ # 1.6.2 (which refer to commit ID f3ffda1e08f19e9a6a88484136b5eed76533f21a) $ ./tools/synchronize.sh -b=f3ffda1e08f19e9a6a88484136b5eed76533f21a Downloading the libpod's defined interface https://raw.githubusercontent.com/containers/libpod/f3ffda1e08f19e9a6a88484136b5eed76533f21a/cmd/podman/varlink/io.podman.varlink % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 50588 100 50588 0 0 51515 0 --:--:-- --:--:-- --:--:-- 51462 Method GetContainersByStatus seems not yet implemented Method HealthCheckRun seems not yet implemented ... Method GetEvents seems not yet implemented Method GetLayersMapWithImageInfo seems not yet implemented Method BuildImageHierarchyMap seems not yet implemented 37 error(s) found Comparison finished...bye! ```
Summarize
In the sake of compatibilty in
libpod
andpython-podman
I propose to design and implement automated checks to verify if python-podman implementation is fully compatible with libpod varlink defined interface.The goal of this ticket is to discuss about this feature design and watch how we could implement this and the needs behind this (impact on project workflow, how to proceed etc...).
Draft design
Project lifecycle
I think we need to create a git branch dedicated for each libpod version that we want to maintain to avoid to introduce future features into wrong branches and python-podman releases, in other word we need to isolate development versions.
The master branch will only reflect the libpod master and will help us to synchronize changes between future latest versions only.
I think we need as many version of python-podman than existing libpod versions to keep the python-api compatible with libpod defined interface.
Triggering automated checks
The automated check can be trigger from:
The automated check need to reflect the relation between version on the both projects, we only need to compare the varlink defined interface in the right libpod tag version and on the right development branch version on python-podman.
If we choose to trigger automated checks on
python-podman
then we need to pull from the right libpod tag/branch the raw varlink defined interface file (by using http request) and then compare it the python-podman implementation.How to compare interface with implementation
For the moment I see some solutions to compare if these projects are synchronized, lets describe them.
Maybe you will see other/better solutions to manage this.
Don't forget that the main goal of this issue is to discuss with you if we can do this and how to do this before starting to spend time on it. Also I want to raise pro and con that I may miss.
First solution
The main idea in this solution is to compare function signatures.
Steps needed in order:
python-podman
The main difficulty here is how to ignore perifical function which are part of the python-podman mechanisms and not related to interface implementation... Since python-podman is only compatible with python3 we could by example use the powerful python
inspect
module coupled with a standardized mechanism which will allow us to analyze keywords in functions's docstrings to detect if the function is part of the interface definition, determine which interface the implementation is linked and if signature need to be compared with interface definition.All these things can be done in pure python without extra module and dependencies (or just to some mainstream requirements).
We don't need to compile and install libpod here, we just need to refer to the related interface.
I think this one is the more preferable solution and the more safer and the less painful.
Second solution
On travis's instance compile and install the right libpod version and then start its varlink server, and then run unit tests against but I think it's more likely that libpod's varlink defined interface will be ahead of the
python-podman
implementation. In this case it can be difficult to witness raised exceptions from thepython-podman
side because thepython-podman
would not pass undefined params (except if some params are removed from libpod and not yet removed from python-podman).Maybe I'm wrong by thinking this way, I'm not sure if all params are mandatory and if exceptions can be raised by doing this. If exception are raised then it's can be a way to identify non synchonized interface implemention in python-podman.
My main concern about this solution is how to properly manage travis instance, libpod rights (sudo etc...), rootless needs, etc... on a travis's instance image which could have some limitations. I think it can be painful to follow this way and our goal is not to test libpod but just to observe if implementation is rightly synchronized, I don't want to fight with windmills here.
Third solution
By adding new capabilities to the varlink's mocking feature it will allow us to generate a fake service by lookup form the downloaded raw interface to generate some fake python classes (rather than generating fake interface definition from fake python class as is currently the case).
Then we can mix solution 1 and solution 2 approachs (if enough unit test exists for solution 2), and by example compare generated function's signatures and real implementation signatures.
Like has explained in the second solution I'm not sure if 100 percent of the args are mandatories and if some are missing I'm not sure we will catch exception and error. Anyway with this approach libpod stay ahead of the python-podman implementation so this is the main risk.
This is a cross project solution which ask to us to adding new feature on the varlink side and in a second time to implement the check on python-podman.
The main advantage here is to provide feature which can be reused outside of the scope podman (the feature added to varlink).
Conclusion
I think we need this kind of feature to avoid to spend time fix issue unsynchronized implementation.
Also I think these changes can help us to better handle the support of libpod's versions on python-podman and to more often and more merely release new versions of this API.
Do not hesitate to react to my proposal by sending comments here.
Possibly related to
The text was updated successfully, but these errors were encountered: