Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
RequestSplitter and staging selection proposals #637
While working on
The goal is threefold:
I believe the first two are achieved quite well by this and much of the third.
The following is taken from the included documentation.
Hopefully, from that you have some ideas on how to utilize this.
One note is that
I made the last set of staging selections for
A new command, perhaps
Currently, when a list of stagings to consider is provided the list is merged with available stagings. For example, if A, B, and C are available and C, and D are specified only C will be considered. After thinking about the use-cases I believe dropping the merge and simply taking the input as is will be the best choice. That said having merge logic in place would probably make for a better proposal as otherwise it would consider full stagings the same as empty ones. For now then I'll leave as is.
I have a few small changes to provide more specific error output and handling. After that I'll need to fix the tests which currently fail due to missing
For those looking to try it out, be sure to set
osc staging -p openSUSE:Leap:42.3 select
all: bootstrap_required: true requests: 450181: qemu 450182: gdk-pixbuf 450183: bash-completion staging: A
osc staging -p openSUSE:Leap:42.3 select --group-by='./action/source/@devel_project'
GNOME:Factory: bootstrap_required: false requests: 450182: gdk-pixbuf staging: E Virtualization: bootstrap_required: false requests: 450181: qemu staging: B shells: bootstrap_required: true requests: 450183: bash-completion staging: A
Note that at the time of running this command C and D are occupied and as such
Another cool example is selecting request based on text of ignore message. So in the case where multiple requests are ignored for the same reason they can all be staged as soon as that problem is solved.
osc staging -p openSUSE:Leap:42.3 select --filter-by='contains(@ignored, "augeas-lenses")'
all: bootstrap_required: false requests: 449500: yast2-ntp-client staging: E
As of now, I have to comment
Or just exclude a single request from proposal
Also note the
From today then, no need for interactive.
$ osc staging -p openSUSE:Leap:42.3 select
To speak a bit towards the third goal, I'd like to stack on top of this a set of strategies for creating a proposal and a higher level bit for choosing the most effective strategy given the state of stagings and requests. That way at some point in the future we might even be able to have a cronjob run select command and
In general this sounds pretty cool. The xpath filtering seems to be a bit too much though. I at least wouldn't be able to remember xpath queries. So IMO it would be better to provide the most common use cases pre-defined (like by devel) and leave the rest to $EDITOR.
@lnussel Yes, I agree the common filters/groups should have shorthand versions which I noted is something which can be added. Using the xpath under the hood makes the code much simpler, cleaner, and flexible in that it supports just about anything should we want/need it in the future. Since the lxml seems alright I'll look into the tests failing and polishing this up.
@nilxam Yeah, I imagine there will always be manual moving around, but I also would guess that packages will also be either staged correctly or staged incorrectly once which could be done and manually move those that are wrong. I figured having a bot keep track of pairings or analyze dependency tree might allow for rather solid proposals. Either way, I would expect the filter/grouping to aid in manually staging, but I understand that the Tumbleweed case is much more busy/complicated than what I typically deal with in Leap.
I spent some time to ensure the edge-cases and previous functionality were all there in addition to using this branch for daily work for last couple weeks.
The previous code supported selecting to adi stagins if the full project name was used, but does not appear to have a short-hand method. Since the proposal generation wants to be able to support multiple stagings and requests being passed in at the same time this complicates differentiating the arguments. When considering Factory has special stagings like Gcc6 of PIE that break the letter rule checking the length of a staging name is not sufficient either. I noticed that the special stagings at least seem to start with a capitol letter which is handy for making them unique when compared to packages like Gcc6 and gcc6.
Given the above I wanted to achieve the following.
In order for adi stagings to not be confused with request numbers (aside from cheating since request IDs are much larger, but could do that) they follow the form
For example, the following work.
Given that special stagings are currently unique and that seems like a good practice the duplicate argument case should not normally be reached.
The proposal mode does not support adi stagings (ie multiple adi stagings or mimicing adi command, but this could be considered down the road as it isn't too much of a change).
Multiple filter-by and group-by options are now exposed to CLI.
Resolved the case of listing multiple stagings that are already filled. The assumption is if a specific list of stagings is provided they are used in proposal regardless of state. From usage this seems to make the most sense.
One the topic of short-hand for the xpaths, that seems to make sense, but should write up what common usages are (or perhaps figure out how to log usage) to determine what makes the most sense. For now the common ones can be copy/pasted from help text. All previous options are still supported like
The tests have also been fixed and the one warning related to
Given the above, this seems ready to merge from my perspective. Follow-up work can be done fairly easily on top of this base.
The inner diff of changes: https://gist.github.com/jberry-suse/c0106957fac0b64453964ee96de786df
- re-implement list and adi commands using RequestSplitter - numerous small cleanups and clarity improvements - notably, adi now prints similar output to select when adding requests - lxml is needed to provide more fully-featured xpath implementation
As a note, I did a fair bit of the revamp work today so that negates some of my testing over the last couple weeks. As such if anyone gets the chance to use it a bit that would probably be good. Otherwise, I imagine I'll be able to use adi and select commands properly tomorrow.
One area I was reminded of today which I would like some input. The difference between
To be clear it seems wrong that
devel_project = self.api.get_devel_project(source_project, source_package) # try target pacakge in Factory # this is a bit against Leap development in case submissions is from Update, # or any other project than Factory if devel_project is None and self.api.project.startswith('openSUSE:'): devel_project = self.api.get_devel_project('openSUSE:Factory', target_package) if devel_project is not None: source_project = devel_project
devel = self.api.get_devel_project("openSUSE:Factory", target_package)
Either way I imagine it works fine, just seems like needless requests for Factory and not sure what cases benefit from checking source project.