Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update dependency packages to use dune.2.6.0 and Multicore OCaml 4.10.0 #132

Merged
merged 11 commits into from Jul 10, 2020

Conversation

shakthimaan
Copy link
Contributor

@shakthimaan shakthimaan commented Jun 24, 2020

The attached patch updates Sandmark to use dune 2.6.0 for Multicore OCaml 4.10.0.

  • The dependencies/packages/ have been cleaned up so that only the latest version is retained.
  • Additional dependencies for newer releases have been added.
  • Both serial and parallel benchmarks run fine (except for frama-c, which is already a known issue. See frama-c benchmarks fail on multicore (build and run) #17)

In order to test, you need to first update dune locally on your system using:

$ opam install dune.2.6.0

You will need to apply this patch and the PR from #131 as well in order to build and test. You can obtain the raw patch from a PR by suffixing with .patch.
For example: https://patch-diff.githubusercontent.com/raw/ocaml-bench/sandmark/pull/131.patch.

You can then proceed to build the dependencies and run the serial benchmarks using the following command:

$ make ocaml-version/4.10.0+multicore.bench

In order to run the parallel benchmarks, you can use the following in run_all_parallel.sh, and execute the script:

make multicore_parallel_run_config_macro.json

BUILD_BENCH_TARGET=multibench_parallel \
	RUN_CONFIG_JSON=multicore_parallel_run_config_macro.json \
	make ocaml-versions/4.10.0+multicore.bench

This was referenced Jun 24, 2020
Makefile Outdated Show resolved Hide resolved
@kayceesrk
Copy link
Contributor

dune.2.6.0 should work now thanks to ocaml-multicore/multicore-opam#22

@shakthimaan shakthimaan changed the title Update dependency packages to use dune.2.5.1 and Multicore OCaml 4.10.0 Update dependency packages to use dune.2.6.0 and Multicore OCaml 4.10.0 Jun 25, 2020
@shakthimaan
Copy link
Contributor Author

The following dependency packages have been upgraded to build with dune.2.6.0:

  • dune.2.6.0
  • dune-private-libs.2.6.0
  • dune-configurator.2.6.0
  • digestif.0.8.1
  • checkseum.0.2.1

@kayceesrk
Copy link
Contributor

The build is still failing. Should we wait until it turns green for reviewing?

@shakthimaan
Copy link
Contributor Author

shakthimaan commented Jun 26, 2020

The CI is using ocaml/opam2 Docker container from DockerHub:

  1. The ocaml/opam2:{latest, 4.12, 4.10} images do not have the required dune.2.6.0 version.
  2. The packages are pulled from git://github.com/ocaml/ocaml-beta-repository, while dune.2.6.0 version is available only from https://opam.ocaml.org.
    Hence, the CI build is failing.
    Also, the following patch from Update decompress benchmarks #131 is required to successfully run the decompress benchmarks.
    @avsm Is it possible to allow ocaml/opam2 to use https://opam.ocaml.org and provide a Docker image with dune.2.6.0?

@shakthimaan
Copy link
Contributor Author

I have updated .drone.yml to use ocurrent/opam:debian-10-ocaml-4.10 Docker image that can install dune.2.6.0. Since this PR also requires #131, I have temporarily applied that in the YAML file for the CI to pass.

- wget https://github.com/ocaml-bench/sandmark/pull/131.patch
- patch -p1 < 131.patch

When both #131 and this PR are merged, I shall create a PR to remove the patch application from the .drone.yml. file.

.drone.yml Outdated
- make ocaml-versions/4.07.1.bench
- wget https://github.com/ocaml-bench/sandmark/pull/131.patch
- patch -p1 < 131.patch
- make ocaml-versions/4.10.0+multicore.bench
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we able to keep a CI build against stock OCaml 4.10 in here as well?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can create multiple exec pipelines in the same .drone.yml. Any specific reason to have it as part of this code base, rather than in stock OCaml 4.10 or our clone of the repository? The status of the build is determined by the success of both the pipeline runs though.
Source: https://docs.drone.io/pipeline/configuration/#multiple-pipelines

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm happy for it to be in the same run or in another run (I think in the past we had two runs, one for stock and one for multicore). I'm just keen to have CI coverage in case we accidentally make a change to packages which works with multicore but not with stock.

@kayceesrk
Copy link
Contributor

As discussed, we need 3 pipelines,

  • 4.10.0 & serial benchmarks
  • 4.10.0+multicore & serial benchmarks
  • 4.10.0+multicore & parallel benchmarks

@shakthimaan
Copy link
Contributor Author

shakthimaan commented Jun 30, 2020

The 4.10.0+stock build got killed because it reached the 60 minute default timeout setting. I tested the build on another machine and the benchmarks ran fine though. Since we are using the cloud hosted drone version, the default timeout is limited to 60 minutes. Only if we self-host will we have administrator access and the Drone configuration timeout can be increased.
Reference:

@kayceesrk
Copy link
Contributor

@ctk21 Do you have admin access to Drone?

@ctk21
Copy link
Member

ctk21 commented Jul 1, 2020

@ctk21 Do you have admin access to Drone?

We don't have admin access. It is provided as a cloud service by Drone. I think we need to find a way for the run to be under 60mins (which is likely going to be a headache anyhow).

Do we know what is taking the time? Maybe we need a cut-down config that can give us coverage of the places most likely to go wrong; this feels like the build stages of the benchmarks rather than the execution of all the benchmarks.

@shakthimaan
Copy link
Contributor Author

shakthimaan commented Jul 2, 2020

I have now added a DRY_RUN parameter to the Makefile:

  • When its value is set to 1, it runs everything except running the benchmarks.
  • By default (if no value is set), the benchmarks are executed.

@kayceesrk
Copy link
Contributor

@shakthimaan could you summarize what is pending to get this PR merged?

@shakthimaan
Copy link
Contributor Author

The PR can be merged to master:

  • We have tested the same for building Coq, and it works fine for dune.2.6.0 and Multicore OCaml 4.10.0.
  • I have now updated the branch to reflect the README and Makefile changes from master, and the CI build is clean.
    This PR does require the following PR Update decompress benchmarks to be merged first as a pre-requisite.
    Update decompress benchmarks #131

@kayceesrk kayceesrk merged commit 59b14b6 into ocaml-bench:master Jul 10, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants