Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ca-derivations support #21

Draft
wants to merge 3 commits into
base: master
Choose a base branch
from
Draft

Conversation

thufschmitt
Copy link

Add a /realisations endpoint to support content-addressed derivations.

This required updating Nix (currently using a custom branch even) to access a new queryRawRealisation perl function that’s needed for that.

As part of this, I also had to upgrade nixpkgs to match the one used by the Nix flake, which caused an update of perl which in turn broke things:

  • For some reason, the /nix-cache-info endpoint doesn’t properly interpolate the store dir anymore
  • The narinfo files now have an extra newline at the end, which breaks the Nix parser

I’ll try to build everything with the old nixpkgs, seeing whether that fixes the issues for the time being

Fix #20

/cc @Mic92 @zseri @SuperSandro2000

Exporting the `queryRawRealisation` perl method
Otherwise the raw string `$Nix::Config::storeDir` is returned rather
than its value. No idea why, maybe that’s linked to the perl update
@@ -13,7 +14,7 @@
nix-serve = with final; stdenv.mkDerivation {
name = "nix-serve-${self.lastModifiedDate}";

buildInputs = [ perl nix.perl-bindings perlPackages.Plack perlPackages.Starman perlPackages.DBDSQLite ];
buildInputs = [ perl nix.defaultPackage.${super.system}.passthru.perl-bindings perlPackages.Plack perlPackages.Starman perlPackages.DBDSQLite ];
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it doesn't seem like super.system is correct. I suppose you meant prev.system?

@@ -15,7 +15,14 @@ my $app = sub {
my $path = $env->{PATH_INFO};

if ($path eq "/nix-cache-info") {
return [200, ['Content-Type' => 'text/plain'], ["StoreDir: $Nix::Config::storeDir\nWantMassQuery: 1\nPriority: 30\n"]];
return [200, ['Content-Type' => 'text/plain'], ["StoreDir: /nix/store\nWantMassQuery: 1\nPriority: 30\n"]];
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe you could try the following (see also):

Suggested change
return [200, ['Content-Type' => 'text/plain'], ["StoreDir: /nix/store\nWantMassQuery: 1\nPriority: 30\n"]];
return [200, ['Content-Type' => 'text/plain'], ["StoreDir: ${Nix::Config::storeDir}\nWantMassQuery: 1\nPriority: 30\n"]];

@Mic92
Copy link
Contributor

Mic92 commented Jul 31, 2021

I also started adding CI here: #22

@@ -1,7 +1,8 @@
{
inputs.nixpkgs.url = "nixpkgs/nixos-20.09";
inputs.nixpkgs.follows = "nix/nixpkgs";
inputs.nix.url = "github:nixos/nix/ca/queryRealisation-perl";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was merged into master now.

@thufschmitt
Copy link
Author

Just saw the rewrite at https://github.com/Mic92/nix-serve , which seems to be a much nicer alternative. @Mic92 do you need any help with it?

@Mic92
Copy link
Contributor

Mic92 commented Aug 30, 2021

I am not so happy with the performance of the used http server library yet: yhirose/cpp-httplib#1023
It's probably faster than the current perl implementation... I am currently rewriting this using libh2o, which gives me also http2 - none of the c++ libraries I found support chunking, which is needed for large nars.

@SuperSandro2000
Copy link
Contributor

SuperSandro2000 commented Dec 4, 2023

Ported this to hydra here SuperSandro2000/hydra@9e528b2

@thufschmitt could you finish that anyway just for completeness?

@Mic92
Copy link
Contributor

Mic92 commented Dec 4, 2023

I am not so happy with the performance of the used http server library yet: yhirose/cpp-httplib#1023 It's probably faster than the current perl implementation... I am currently rewriting this using libh2o, which gives me also http2 - none of the c++ libraries I found support chunking, which is needed for large nars.

These days I am just using harmonia: https://github.com/nix-community/harmonia/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Does not work with nix ca-derivations enabled
4 participants