Pulling all matching packages from a snapshot #70

Merged
merged 3 commits into from Jul 9, 2014

Conversation

Projects
None yet
2 participants
@queeno
Contributor

queeno commented Jun 27, 2014

When performing an aptly snapshot pull, users might list dependency
versions that can potentially match multiple packages in the source snapshot. However,
the current implementation of the 'snapshot pull' command only allows one package
to be pulled from a snapshot at a time for a given dependency.

The newly implemented all-matches flag allows users to pull all the
matching packages from a source snapshot, provided that they satisfy the
version requirements indicated by the dependencies.

The all-matches flag defaults to false and only produces the described
behaviour when it is explicitly set to true.

For example, I created a source_snap snapshot from a sensu mirror:

root@sim-tst-lb1:~# aptly mirror create sensu http://repos.sensuapp.org/apt sensu
...
root@sim-tst-lb1:~# aptly mirror update sensu
...
root@sim-tst-lb1:~# aptly snapshot create source_snap from mirror sensu
...
root@sim-tst-lb1:~# aptly snapshot create empty empty
...

The current behaviour of aptly snapshot pull is shown below.
Only one package satisfying dependency sensu > 0.11 is added to dest_snap.

root@sim-tst-lb1:~# aptly snapshot pull -architectures=amd64 --dry-run empty sensu-snap dest_snap 'sensu (> 0.11)'
Dependencies would be pulled into snapshot:
    [empty]: Created as empty
from snapshot:
    [sensu-snap]: Snapshot from mirror [sensu]: http://repos.sensuapp.org/apt/ sensu
and result would be saved as new snapshot dest_snap.
Loading packages (154)...
Building indexes...
[+] sensu_0.12.1-1_amd64 added

Not creating snapshot, as dry run was requested.

The proposed all-matches flag behaviour is shown below.
All the packages that satisfy dependency sensu > 0.11 are added to dest_snap.

root@sim-tst-lb1:~# aptly snapshot pull -architectures=amd64 --all-matches --dry-run empty sensu-snap dest_snap 'sensu (> 0.11)'
Dependencies would be pulled into snapshot:
    [empty]: Created as empty
from snapshot:
    [sensu-snap]: Snapshot from mirror [sensu]: http://repos.sensuapp.org/apt/ sensu
and result would be saved as new snapshot dest_snap.
Loading packages (154)...
Building indexes...
[+] sensu_0.11.0.beta.3-1_amd64 added
[+] sensu_0.11.0.beta.2-1_amd64 added
[+] sensu_0.11.3-1_amd64 added
[+] sensu_0.11.0-1_amd64 added
[+] sensu_0.11.0.beta.4-1_amd64 added
[+] sensu_0.11.0.beta.1-1_amd64 added
[+] sensu_0.12.0-1_amd64 added
[+] sensu_0.12.3-1_amd64 added
[+] sensu_0.11.1-1_amd64 added
[+] sensu_0.12.1-1_amd64 added
[+] sensu_0.12.4-1_amd64 added
[+] sensu_0.12.6-4_amd64 added
[+] sensu_0.12.2-1_amd64 added
[+] sensu_0.12.6-5_amd64 added
[+] sensu_0.12.5-1_amd64 added
[+] sensu_0.12.6-2_amd64 added
[+] sensu_0.12.6-3_amd64 added
[+] sensu_0.12.6-1_amd64 added
[+] sensu_0.11.0.beta-1_amd64 added

Not creating snapshot, as dry run was requested.

When the dependency listed is:

  • equal to a set version, for example (= 0.11)
    and
  • the all-matches flag is set to true

under the proposed changes aptly returns all the versions between 0.11 and 0.12 (effectively 0.11.*):

root@sim-tst-lb1:~# aptly snapshot pull -architectures=amd64 --all-matches --dry-run empty sensu-snap dest_snap 'sensu (= 0.11)'
Dependencies would be pulled into snapshot:
    [empty]: Created as empty
from snapshot:
    [sensu-snap]: Snapshot from mirror [sensu]: http://repos.sensuapp.org/apt/ sensu
and result would be saved as new snapshot dest_snap.
Loading packages (154)...
Building indexes...
[+] sensu_0.11.0.beta.4-1_amd64 added
[+] sensu_0.11.0.beta.1-1_amd64 added
[+] sensu_0.11.0.beta.2-1_amd64 added
[+] sensu_0.11.1-1_amd64 added
[+] sensu_0.11.0.beta-1_amd64 added
[+] sensu_0.11.0.beta.3-1_amd64 added
[+] sensu_0.11.0-1_amd64 added
[+] sensu_0.11.3-1_amd64 added

Not creating snapshot, as dry run was requested.
Implementation of all-matches functionality + tests
When performing an *aptly snapshot pull*, users might list dependency
versions that can potentially match multiple packages in the source
snapshot. However, the current implementation of the 'snapshot pull'
command only allows one package to be pulled from a snapshot at a time
for a given dependency.

The newly implemented all-matches flag allows users to pull all the
matching packages from a source snapshot, provided that they satisfy the
version requirements indicated by the dependencies.

The all-matches flag defaults to false and only produces the described
behaviour when it is explicitly set to true.

@smira smira added the enhancement label Jun 27, 2014

@smira smira added this to the v0.7 milestone Jun 27, 2014

smira and others added some commits Jun 27, 2014

@smira

This comment has been minimized.

Show comment
Hide comment
@smira

smira Jul 9, 2014

Member

@simonaquino, thanks for your contribution!

I'm working right now on "queries", searching, filtering (following reprepro query language).. I like all-matches behavior, but I believe the part that does magic with (= 0.11) and pulling many packages from equal operator is not great idea. I'm going to merge the main part that makes Search return many results.

When query stuff is finished, we can do (= 0.11) trick with query like package (>= 0.11), package (<= 0.12).

Once again, thanks a lot for your help!

Member

smira commented Jul 9, 2014

@simonaquino, thanks for your contribution!

I'm working right now on "queries", searching, filtering (following reprepro query language).. I like all-matches behavior, but I believe the part that does magic with (= 0.11) and pulling many packages from equal operator is not great idea. I'm going to merge the main part that makes Search return many results.

When query stuff is finished, we can do (= 0.11) trick with query like package (>= 0.11), package (<= 0.12).

Once again, thanks a lot for your help!

@queeno

This comment has been minimized.

Show comment
Hide comment
@queeno

queeno Jul 9, 2014

Contributor

Hi @smira , no problem at all, thanks for your help! It's always fun to develop new features for Aptly.

You're right, pulling many packages using the equal operation is not a great idea. I'd be very happy to look into how to pull packages using more than one condition once you're done with your queries.

I would think that, at the moment, if you run aptly snapshot pull -all-matches package (>= 0.11), package (<= 0.12) the first set of packages will get removed in favour of the second. Perhaps we should have a think about how that should interact with the -no-remove flag.

Anyway, if you could merge the rest of my pull request it would be great! :)

Simon

Contributor

queeno commented Jul 9, 2014

Hi @smira , no problem at all, thanks for your help! It's always fun to develop new features for Aptly.

You're right, pulling many packages using the equal operation is not a great idea. I'd be very happy to look into how to pull packages using more than one condition once you're done with your queries.

I would think that, at the moment, if you run aptly snapshot pull -all-matches package (>= 0.11), package (<= 0.12) the first set of packages will get removed in favour of the second. Perhaps we should have a think about how that should interact with the -no-remove flag.

Anyway, if you could merge the rest of my pull request it would be great! :)

Simon

@smira

This comment has been minimized.

Show comment
Hide comment
@smira

smira Jul 9, 2014

Member

That would be single query that would match all those packages, so there should be no problem with removing (I hope :)

Member

smira commented Jul 9, 2014

That would be single query that would match all those packages, so there should be no problem with removing (I hope :)

smira added a commit to aptly-dev/aptly-dev.github.io that referenced this pull request Jul 9, 2014

@smira smira merged commit e1348ab into aptly-dev:master Jul 9, 2014

1 check passed

continuous-integration/travis-ci The Travis CI build passed
Details

smira added a commit that referenced this pull request Jul 9, 2014

Revert changes related to NextVersion. #70
It would be done in a bit different way: by introducing powerful search queries.

smira added a commit that referenced this pull request Jul 9, 2014

smira added a commit that referenced this pull request Jul 9, 2014

smira added a commit that referenced this pull request Jul 9, 2014

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment