Add comby to Nixpkgs

[rvantonder]

This issue tracks the build instructions or changes for packaging comby in nixpkgs. This would be nice to have--related PRs and work in progress are in #136, #137, #138. This issue tracks the discussion to for getting things to work with NixOS.

cc thanks @strager, who prompted this.

[rvantonder]

Related to #89 (comment)

[rvantonder]

@strager: progress I've made on packaging/reproducing from your branch:

I'm blocked by NixOS/nix#2925. From the comment thread it does not look like an easy fix. So cannot actually check that everything builds correctly. The changes needed in this branch are at least these:

For #137 - Using Core 0.12.4 and not 0.13.0 should make this go away I think. I can't easily convince opam to pick the right pakcages to compile against 0.13.0 to verify that this is what's causing the compile errors though.
For #138 - We can add this to the dune file, we don't need to add it in the opam file, since the nix packaging doesn't use the opam
For #136 - I will look into configuring the dune scripts so that dune build works without the bisect_ppx dependency, this shouldn't be a blocker.

The following are blockers though (that I can't verify, but presume they need to be addressed):

The libraries rvantonder/lwt and rvantonder/patdiff have some special modifications for comby to work. comby will compile against ocsigen/lwt but without rvantonder/lwt it will crash at runtime.

We can track ocsigen/lwt once lwt 5.0.0 is released, in about a week.

patdiff is a different and recent issue that needs the file below if > 0.11.0 should to be supported. <= 0.11.0 is fine without it and we can use janestreet/patdiff.

For these two dependencies, the following is needed:

pkgs/development/ocaml-modules/lwt/comby.nix:

{ lib, fetchFromGitHub, pkgconfig, ncurses, libev, buildDunePackage, ocaml                            
, cppo, ocaml-migrate-parsetree, ppx_tools_versioned, result                                          
, bisect_ppx                                                                                          
, mmap, seq                                                                                           
}:                                                                                                    
                                                                                                      
buildDunePackage rec {
  pname = "lwt";
  version = "4.3.0";

  minimumOCamlVersion = "4.02"; # @@@

  src = fetchFromGitHub { 
    owner = "rvantonder"; 
    repo = "lwt";
    rev = "f5d6d3c94996072960207da1b4930dabd9b173c8";
    sha256 = "57267288953c663ed59b80fd982b637";
  };
  nativeBuildInputs = [ pkgconfig ];
  buildInputs = [ cppo ocaml-migrate-parsetree ppx_tools_versioned ]
   ++ optional (!versionAtLeast ocaml.version "4.07") ncurses;
  propagatedBuildInputs = [ libev mmap seq result ];

  meta = {
    homepage = "https://ocsigen.org/lwt/";
    description = "A cooperative threads library for OCaml";
    maintainers = [ lib.maintainers.vbgl ];
    license = lib.licenses.mit;
  };
}

pkgs/development/ocaml-modules/patdiff/comby.nix:

# @@@                                                                                                 
{ stdenv, fetchFromGitHub, buildDunePackage, cstruct, ppx_deriving, core, core_kernel_p4, patience_dif
f, ppx_jane, dune, ocaml_pcre, re }:                                                                  

buildDunePackage rec {
  pname = "patdiff";
  version = "0.13.0";

  minimumOCamlVersion = "4.02"; # @@@

  src = fetchFromGitHub { 
    owner = "rvantonder"; 
    repo = "patdiff";
    rev = "b26da18bfcd0ff416547a7cfd271960265304938"
    sha256 = "c3c189b1527581050d3cf9fbf777589d13750";
  };

  buildInputs = [ core core_kernel_p4 patience_diff ppx_jane dune ocaml_pcre re ];
  #propagatedBuildInputs = [ ];

  meta = {
  # whatever seems reasonable
  };
}

^ The important parts are the revisions and sha's above. The changes for those do not need any extra dependencies.

[ghuntley]

Good news, NixOS/nix#2925 has been resolved / is no longer blocking this issue!

[JohnAZoidberg]

Hmm, the derivations you provide don't build (anymore).
Are your changes upstreamable? That would be preferred instead of putting the patches or forked versions into nixpkgs.

I tried to use patches but patdiff doesn't merge cleanly: rvantonder/patdiff@master...rvantonder:0.13.0-patch-compatible-diffs

Check out the progress at #76828 @rvantonder

[jtrakk]

I tried using opam2nix but got some errors at build time.

building
+ /nix/store/xfxhfj8vc3x0gbpk3x57dn4mgkr3j9pv-ocaml4.10.0-opam2nix-1.1.0/bin/opam2nix invoke build
Loading comby.opam
creating /nix/store/awi6sz59m8lir7sj6p8bj0k4vdfciz2i-comby-development/bin
creating /nix/store/awi6sz59m8lir7sj6p8bj0k4vdfciz2i-comby-development/lib/ocaml/4.08.1/site-lib/stublibs
 + dune build -p comby -j 24 @install
  ocamlmklib lib/app/vendored/camlzip/dllcamlzip_stubs.so,lib/app/vendored/camlzip/libcamlzip_stubs.a
Unknown option -I/nix/store/chqfzwv01s07g7hi96bgcph2vsbnhwpg-zlib-1.2.11-dev/include
      ocamlc lib/app/.comby.objs/byte/comby.{cmo,cmt} (exit 2)
(cd _build/default && /nix/store/kwclc59l4dplbfmv3viwbqkx21qilpzp-ocaml-4.08.1/bin/ocamlc.opt -w -40 -w A-3-4-32-34-39-40-41-42-44-45-48-49-50-57-60-66-67 -g -bin-annot -I lib/app/.comby.objs/byte -I /nix/store/0ax00wghk6p9nqqlmnz7imfld0byfr7p-yojson-1.7.0/lib/ocaml/4.08.1/site-lib/yojson -I />
File "lib/app/comby.ml", line 28, characters 12-20:
28 |   let sat = Rule.sat
                 ^^^^^^^^
Error: Unbound value Rule.sat
      ocamlc lib/app/configuration/.configuration.objs/byte/configuration__Command_configuration.{cmo,cmt} (exit 2)
(cd _build/default && /nix/store/kwclc59l4dplbfmv3viwbqkx21qilpzp-ocaml-4.08.1/bin/ocamlc.opt -w -40 -w A-3-4-32-34-39-40-41-42-44-45-48-49-50-57-60-66-67 -g -bin-annot -I lib/app/configuration/.configuration.objs/byte -I /nix/store/0ax00wghk6p9nqqlmnz7imfld0byfr7p-yojson-1.7.0/lib/ocaml/4.08.>
File "lib/app/configuration/command_configuration.ml", line 108, characters 27-38:
108 |   match Option.map rule ~f:Rule.create with
                                 ^^^^^^^^^^^
Error: Unbound value Rule.create
      ocamlc lib/app/pipeline/.pipeline.objs/byte/pipeline.{cmo,cmt} (exit 2)
(cd _build/default && /nix/store/kwclc59l4dplbfmv3viwbqkx21qilpzp-ocaml-4.08.1/bin/ocamlc.opt -w -40 -w A-3-4-32-34-39-40-41-42-44-45-48-49-50-57-60-66-67 -g -bin-annot -I lib/app/pipeline/.pipeline.objs/byte -I /nix/store/0ax00wghk6p9nqqlmnz7imfld0byfr7p-yojson-1.7.0/lib/ocaml/4.08.1/site-lib>
File "lib/app/pipeline/pipeline.ml", line 35, characters 21-31:
35 |       let sat, env = Rule.apply ?metasyntax ~fresh ~substitute_in_place ~matcher rule environment in
                          ^^^^^^^^^^
Error: Unbound value Rule.apply
      ocamlc test/alpha/.alpha_test_integration.objs/byte/alpha_test_integration__Test_custom_metasyntax.{cmi,cmo,cmt} (exit 2)
(cd _build/default && /nix/store/kwclc59l4dplbfmv3viwbqkx21qilpzp-ocaml-4.08.1/bin/ocamlc.opt -w -40 -w A-3-4-32-34-39-40-41-42-44-45-48-49-50-57-60-66-67 -g -bin-annot -I test/alpha/.alpha_test_integration.objs/byte -I /nix/store/0ax00wghk6p9nqqlmnz7imfld0byfr7p-yojson-1.7.0/lib/ocaml/4.08.1/>
File "test/alpha/test_custom_metasyntax.ml", line 172, characters 24-44:
172 |       [ Hole (Alphanum, Reserved_identifiers ["α"; "β"])
                              ^^^^^^^^^^^^^^^^^^^^
Error: This variant expression is expected to have type
         Matchers.Metasyntax.hole_definition
       The constructor Reserved_identifiers does not belong to type Matchers.Metasyntax.hole_definition
      ocamlc test/common/.common_test_integration.objs/byte/common_test_integration__Test_helpers.{cmi,cmo,cmt} (exit 2)
(cd _build/default && /nix/store/kwclc59l4dplbfmv3viwbqkx21qilpzp-ocaml-4.08.1/bin/ocamlc.opt -w -40 -w A-3-4-32-34-39-40-41-42-44-45-48-49-50-57-60-66-67 -g -bin-annot -I test/common/.common_test_integration.objs/byte -I /nix/store/0ax00wghk6p9nqqlmnz7imfld0byfr7p-yojson-1.7.0/lib/ocaml/4.08.>
File "test/common/test_helpers.ml", line 31, characters 19-39:
31 |     | Some rule -> Language.Rule.create rule |> Or_error.ok_exn
                        ^^^^^^^^^^^^^^^^^^^^
Error: Unbound value Language.Rule.create
      ocamlc test/common/.common_test_integration.objs/byte/common_test_integration__Test_parse_rewrite_template.{cmi,cmo,cmt} (exit 2)
(cd _build/default && /nix/store/kwclc59l4dplbfmv3viwbqkx21qilpzp-ocaml-4.08.1/bin/ocamlc.opt -w -40 -w A-3-4-32-34-39-40-41-42-44-45-48-49-50-57-60-66-67 -g -bin-annot -I test/common/.common_test_integration.objs/byte -I /nix/store/0ax00wghk6p9nqqlmnz7imfld0byfr7p-yojson-1.7.0/lib/ocaml/4.08.>
File "test/common/test_parse_rewrite_template.ml", line 8, characters 31-35:
8 |   let module Template_parser = Make(M) in
                                   ^^^^
Error: Unbound module Make
      ocamlc test/common/.common_test_integration.objs/byte/common_test_integration__Test_parse_rule.{cmi,cmo,cmt} (exit 2)
(cd _build/default && /nix/store/kwclc59l4dplbfmv3viwbqkx21qilpzp-ocaml-4.08.1/bin/ocamlc.opt -w -40 -w A-3-4-32-34-39-40-41-42-44-45-48-49-50-57-60-66-67 -g -bin-annot -I test/common/.common_test_integration.objs/byte -I /nix/store/0ax00wghk6p9nqqlmnz7imfld0byfr7p-yojson-1.7.0/lib/ocaml/4.08.>
File "test/common/test_parse_rule.ml", line 6, characters 8-19:
6 |   match Rule.create rule with
            ^^^^^^^^^^^
Error: Unbound value Rule.create
      ocamlc test/common/.common_test_integration.objs/byte/common_test_integration__Test_script.{cmi,cmo,cmt} (exit 2)
(cd _build/default && /nix/store/kwclc59l4dplbfmv3viwbqkx21qilpzp-ocaml-4.08.1/bin/ocamlc.opt -w -40 -w A-3-4-32-34-39-40-41-42-44-45-48-49-50-57-60-66-67 -g -bin-annot -I test/common/.common_test_integration.objs/byte -I /nix/store/0ax00wghk6p9nqqlmnz7imfld0byfr7p-yojson-1.7.0/lib/ocaml/4.08.>
File "test/common/test_script.ml", line 3, characters 10-31:
3 |     match Language.Script.parse input with
              ^^^^^^^^^^^^^^^^^^^^^
Error: Unbound module Language.Script
      ocamlc test/common/.common_test_integration.objs/byte/common_test_integration__Test_substitute.{cmi,cmo,cmt} (exit 2)
(cd _build/default && /nix/store/kwclc59l4dplbfmv3viwbqkx21qilpzp-ocaml-4.08.1/bin/ocamlc.opt -w -40 -w A-3-4-32-34-39-40-41-42-44-45-48-49-50-57-60-66-67 -g -bin-annot -I test/common/.common_test_integration.objs/byte -I /nix/store/0ax00wghk6p9nqqlmnz7imfld0byfr7p-yojson-1.7.0/lib/ocaml/4.08.>
File "test/common/test_substitute.ml", line 9, characters 31-35:
9 |   let module Template_parser = Make(M) in
                                   ^^^^
Error: Unbound module Make
    ocamlopt lib/app/configuration/.configuration.objs/native/configuration__Command_configuration.{cmx,o} (exit 2)
(cd _build/default && /nix/store/kwclc59l4dplbfmv3viwbqkx21qilpzp-ocaml-4.08.1/bin/ocamlopt.opt -w -40 -w A-3-4-32-34-39-40-41-42-44-45-48-49-50-57-60-66-67 -O3 -I lib/app/configuration/.configuration.objs/byte -I lib/app/configuration/.configuration.objs/native -I /nix/store/0ax00wghk6p9nqqlm>
File "lib/app/configuration/command_configuration.ml", line 108, characters 27-38:
108 |   match Option.map rule ~f:Rule.create with
                                 ^^^^^^^^^^^
Error: Unbound value Rule.create
Command failed.
builder for '/nix/store/40v5yj7ailm008yp8cb9vlbhw559m3c6-comby-development.drv' failed with exit code 1
error: build of '/nix/store/40v5yj7ailm008yp8cb9vlbhw559m3c6-comby-development.drv' failed

[rvantonder]

I'm working on some changes that might fix this. I hadn't come across opam2nix so might give it a try after I merged the changes.

[jtrakk]

There is also another one https://github.com/serokell/opam-nix which I haven't tried.

[jtrakk]

Comby is in Nixpkgs 21.11.