Make local pacman use flexo's cache

[codeswhite]

It would be really great to somehow make local pacman use flexo's cache as a cache directory for local system updates;

So for example:

• Suppose I have a certain package in my flexo cache and I wish to pacman -S that package,
  My suggestion is to not indulge pacman to download this certain package from localhost flexo service (thus creating a wasteful copy of the file),
  But instead to directly install it from flexo's cache!

I've though of few approaches:

1. Encourage pacman to officially support storing and using its' system cache in flexo's format (aka the mirror-able tree structure).
2. Include a configurable feature in flexo that will create a symlink for every package;
   • In /var/cache/flexo/pkg/ create a new directory (lets call it ./pacman/ for demonstration);
   • Symlink all latest versions of all packages into /var/cache/flexo/pkg/pacman/*
   • Point into the new cache directory in your local /etc/pacman.conf

Edit:
I would gladly contribute but unfortunately i don't know Rust (yet!?).

[nroi]

So far, I've dealt with this problem by setting the CacheDir of pacman to /tmp/pacman_cache. This means files are still stored redundantly, but are removed after each reboot due to /tmp being mounted as tmpfs. Also, it's all stored in RAM, so you don't have any additional wear-and-tear on your SSD. But I understand that this is not ideal if you don't have enough RAM to waste.

It might be possible to solve this problem without making any changes to Flexo: according to man 5 pacman.conf, it's possible to set multiple directories as CacheDir:

Multiple cache directories can be specified, and they are tried in the order they are listed in the config file.

However, currently it does not work to just set something like:

CacheDir  = /tmp/pacman_cache /var/cache/flexo/pkg/community/os/x86_64 /var/cache/flexo/pkg/core/os/x86_64 /var/cache/flexo/pkg/extra/os/x86_64 /var/cache/flexo/pkg/multilib/os/x86_64

But this seems to be due to the fact that Flexo currently does not cache .sig files. So, if pacman doesn't find the .sig files (along with the actual package file) in the local file system, it fetches both the .sig file and the package file via HTTP.

I've created a separate issue for this (#65). As soon as Flexo caches .sig files, I'll have a closer look at this.

[codeswhite]

Thanks for the reply!
Firstly I learned something new today, didn't knew I could specify multiple cache directories in pacman.conf.

You wrote:

However, currently it does not work to just set something like:

CacheDir  = /tmp/pacman_cache /var/cache/flexo/pkg/community/os/x86_64 /var/cache/flexo/pkg/core/os/x86_64 /var/cache/flexo/pkg/extra/os/x86_64 /var/cache/flexo/pkg/multilib/os/x86_64

I found out that using it as:

CacheDir = /tmp/pacman_cache
CacheDir = /var/cache/flexo/pkg/core/os/x86_64
CacheDir = /var/cache/flexo/pkg/extra/os/x86_64
... etc ...

Is actually a valid solution for the local pacman.. !

I've tested it with a single package that I've downloaded from remote machine that used Flexo as its mirror, and then tried to install that same package on the Flexo server, it worked thus it did not seem that .sig was required...
Although I've seen in /var/cache/pacman that only some packages are being fetched with .sig files alongside, which makes me wonder whether you are right about pacman falling-back to downloading when it will not find a .sig alongside a particular package.

[nroi]

Sounds good! I haven't investigated if and why pacman was really falling back to downloading the entire file when the .sig file is missing, but either way, with the newest release, Flexo caches .sig files just like normal package files, so this should not be a problem anymore.

Just to reiterate: If you want pacman to directly fetch existing files from the file system instead of downloading them via HTTP and storing a redundant copy, modify your /etc/pacman.conf to use multiple CacheDir entries, for example:

CacheDir = /var/cache/pacman/pkg
CacheDir = /var/cache/flexo/pkg/community/os/x86_64
CacheDir = /var/cache/flexo/pkg/core/os/x86_64
CacheDir = /var/cache/flexo/pkg/extra/os/x86_64
CacheDir = /var/cache/flexo/pkg/multilib/os/x86_64

If you use any additional repositories, you need to add more CacheDir entries.

You might also want to replace the first CacheDir entry with:

CacheDir = /tmp/pacman_cache

to avoid redundant package files: /tmp is usually mounted as tmpfs, so the files disappear after reboot, which is fine since the files are still stored inside /var/cache/flexo.