Skip to content

FM'26 Tutorial#2012

Open
michael-schwarz wants to merge 28 commits into
masterfrom
fm26-tutorial
Open

FM'26 Tutorial#2012
michael-schwarz wants to merge 28 commits into
masterfrom
fm26-tutorial

Conversation

@michael-schwarz
Copy link
Copy Markdown
Member

@michael-schwarz michael-schwarz commented Apr 27, 2026

This is the first half of the tutorial for FM'26 and probably as far as we will go for the FOCS version.

This includes:

  • An interval analyses treating all globals as top where a local transfer function should be fixed
  • Extending this analysis so it uses global store widening
  • Turning into a mixed-flow sensitive analysis that tracks values locally for the owning thread if there is one.

@michael-schwarz
Copy link
Copy Markdown
Member Author

michael-schwarz commented Apr 27, 2026

For the longer session at FM'26 the idea would be to additionally ask for an analysis that detects which global variables are effectively thread-local, i.e., accessed only by one thread (like #1966 and https://github.com/goblint/analyzer/blob/ghosts/src/analyses/phaseGhost.ml) and interact with the gStoreWidening analysis via queries to treat such globals flow-sensitively.

Ideally, we could even distinguish between writes and reads and have both a flow-sensitive treatment within the owner, and a flow-insensitive one for other threads. That would really make it mixed flow-sensitive.

@michael-schwarz
Copy link
Copy Markdown
Member Author

Somehow opening the dev container gets stuck at

=> [2/2] RUN eval $(sed -n "s/opam:[^:]:([^:]):([^:]):[^:]:([^ 14.7s

for over 10 minutes. I'm not sure where that command is even from? Any idea how to debug this @arkocal @sim642? 🙏

@sim642
Copy link
Copy Markdown
Member

sim642 commented May 8, 2026

I have no idea where that command even is from. It looks like the build of some Dockerfile but I don't know where it might be from. Is that from some VSCode log that has more context?

@michael-schwarz
Copy link
Copy Markdown
Member Author

This is the complete log file

[78 ms] Dev Containers 0.460.0 in VS Code 1.120.0-insider (89bab60dbb6cdd0d1394f011e110457d885b9fa3).
[75 ms] Start: Resolving Remote
[113 ms] Setting up container for folder or workspace: /home/michael/Documents/analyzer-devcontainer/analyzer
[119 ms] Start: Check Docker is running
[120 ms] Start: Run: docker version
[156 ms] Client:
 Version:           29.1.3
 API version:       1.52
[157 ms]  Go version:        go1.24.4
 Git commit:        29.1.3-0ubuntu3~22.04.2
 Built:             Wed Apr 29 22:18:59 2026
 OS/Arch:           linux/amd64
 Context:           default

Server:
 Engine:
  Version:          29.1.3
  API version:      1.52 (minimum version 1.44)
  Go version:       go1.24.4
  Git commit:       29.1.3-0ubuntu3~22.04.2
  Built:            Wed Apr 29 22:18:59 2026
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          2.2.1
  GitCommit:        
 runc:
  Version:          1.3.4-0ubuntu1~22.04.1
  GitCommit:        
 docker-init:
  Version:          0.19.0
  GitCommit:        
[163 ms] Start: Run: docker volume ls -q
[190 ms] Start: Run: docker ps -q -a --filter label=vsch.local.folder=/home/michael/Documents/analyzer-devcontainer/analyzer --filter label=vsch.quality=insider
[214 ms] Start: Run: docker ps -q -a --filter label=devcontainer.local_folder=/home/michael/Documents/analyzer-devcontainer/analyzer --filter label=devcontainer.config_file=/home/michael/Documents/analyzer-devcontainer/analyzer/.devcontainer/devcontainer.json
[237 ms] Start: Run: docker ps -q -a --filter label=devcontainer.local_folder=/home/michael/Documents/analyzer-devcontainer/analyzer
[259 ms] Start: Run: docker ps -q -a --filter label=devcontainer.local_folder=/home/michael/Documents/analyzer-devcontainer/analyzer
[282 ms] Running Dev Containers CLI:   up --user-data-folder /home/michael/.config/Code - Insiders/User/globalStorage/ms-vscode-remote.remote-containers/data --container-session-data-folder /tmp/devcontainers-6e9677ae-78c0-4fa6-a21b-42add38c5d991778250218397 --workspace-folder /home/michael/Documents/analyzer-devcontainer/analyzer --workspace-mount-consistency cached --gpu-availability detect --id-label devcontainer.local_folder=/home/michael/Documents/analyzer-devcontainer/analyzer --id-label devcontainer.config_file=/home/michael/Documents/analyzer-devcontainer/analyzer/.devcontainer/devcontainer.json --log-level debug --log-format json --config /home/michael/Documents/analyzer-devcontainer/analyzer/.devcontainer/devcontainer.json --default-user-env-probe loginInteractiveShell --experimental-lockfile --mount type=volume,source=vscode,target=/vscode,external=true --mount type=bind,source=/run/user/1001/wayland-0,target=/tmp/vscode-wayland-aa000a6d-bc9a-4690-b9fd-86b92aaf1218.sock --skip-post-create --update-remote-user-uid-default on --mount-workspace-git-root --include-configuration --include-merged-configuration
[283 ms] Start: Run: /snap/code-insiders/2385/usr/share/code-insiders/code-insiders /home/michael/.vscode-insiders/extensions/ms-vscode-remote.remote-containers-0.460.0/dist/spec-node/devContainersSpecCLI.js up --user-data-folder /home/michael/.config/Code - Insiders/User/globalStorage/ms-vscode-remote.remote-containers/data --container-session-data-folder /tmp/devcontainers-6e9677ae-78c0-4fa6-a21b-42add38c5d991778250218397 --workspace-folder /home/michael/Documents/analyzer-devcontainer/analyzer --workspace-mount-consistency cached --gpu-availability detect --id-label devcontainer.local_folder=/home/michael/Documents/analyzer-devcontainer/analyzer --id-label devcontainer.config_file=/home/michael/Documents/analyzer-devcontainer/analyzer/.devcontainer/devcontainer.json --log-level debug --log-format json --config /home/michael/Documents/analyzer-devcontainer/analyzer/.devcontainer/devcontainer.json --default-user-env-probe loginInteractiveShell --experimental-lockfile --mount type=volume,source=vscode,target=/vscode,external=true --mount type=bind,source=/run/user/1001/wayland-0,target=/tmp/vscode-wayland-aa000a6d-bc9a-4690-b9fd-86b92aaf1218.sock --skip-post-create --update-remote-user-uid-default on --mount-workspace-git-root --include-configuration --include-merged-configuration
[545 ms] @devcontainers/cli 0.86.1. Node.js v22.22.1. linux 5.15.0-176-generic x64.
[545 ms] Start: Run: docker buildx version
[753 ms] github.com/docker/buildx 0.30.1 0.30.1-0ubuntu1~22.04.1
[754 ms] 
[754 ms] Start: Run: docker version --format {{.Server.Version}}
[794 ms] 29.1.3
[794 ms] 
[794 ms] Start: Run: docker -v
[825 ms] Start: Resolving Remote
[833 ms] Start: Run: git rev-parse --show-cdup
[848 ms] Loading 146 extra certificates from /etc/ssl/certs/ca-certificates.crt.
[1024 ms] Start: Run: docker ps -q -a --filter label=devcontainer.local_folder=/home/michael/Documents/analyzer-devcontainer/analyzer --filter label=devcontainer.config_file=/home/michael/Documents/analyzer-devcontainer/analyzer/.devcontainer/devcontainer.json
[1093 ms] Start: Run: docker inspect --type image ghcr.io/goblint/analyzer-devcontainer:fm26-tutorial
[1483 ms] Start: Run: docker-credential-secret get
[3796 ms] Error fetching image details: 
[3797 ms] Start: Run: docker pull ghcr.io/goblint/analyzer-devcontainer:fm26-tutorial
fm26-tutorial: Pulling from goblint/analyzer-devcontainer
Digest: sha256:4de03ce036914f2fdb795987e895d994de9461065607571bdf6c813f0dae1767
Status: Downloaded newer image for ghcr.io/goblint/analyzer-devcontainer:fm26-tutorial
ghcr.io/goblint/analyzer-devcontainer:fm26-tutorial
[4994 ms] Start: Run: docker inspect --type image ghcr.io/goblint/analyzer-devcontainer:fm26-tutorial
[5045 ms] Start: Run: docker build -f /tmp/devcontainercli-michael/updateUID.Dockerfile-0.86.1 -t vsc-analyzer-f4302c0a1788c4f90215b9e918c174e776ba6e16074bf41ce26823fdb3bf1988-uid --platform linux/amd64 --build-arg BASE_IMAGE=ghcr.io/goblint/analyzer-devcontainer:fm26-tutorial --build-arg REMOTE_USER=opam --build-arg NEW_UID=1001 --build-arg NEW_GID=140 --build-arg IMAGE_USER=opam /home/michael/.config/Code - Insiders/User/globalStorage/ms-vscode-remote.remote-containers/data/empty-folder
[+] Building 15.6s (4/5)                                         docker:default
 => [internal] load build definition from updateUID.Dockerfile-0.86.1      0.0s
 => => transferring dockerfile: 1.36kB                                     0.0s
 => WARN: InvalidDefaultArgInFrom: Default value for ARG $BASE_IMAGE resu  0.0s
 => [internal] load metadata for ghcr.io/goblint/analyzer-devcontainer:fm  0.0s
 => [internal] load .dockerignore                                          0.0s
 => => transferring context: 2B                                            0.0s
 => CACHED [1/2] FROM ghcr.io/goblint/analyzer-devcontainer:fm26-tutorial  0.0s
 => => resolve ghcr.io/goblint/analyzer-devcontainer:fm26-tutorial@sha256  0.0s
 => [2/2] RUN eval $(sed -n "s/opam:[^:]*:\([^:]*\):\([^:]*\):[^:]*:\([^  15.5s

@sim642
Copy link
Copy Markdown
Member

sim642 commented May 8, 2026

It's using the given prebuilt image as a base for some custom container with

docker build -f /tmp/devcontainercli-michael/updateUID.Dockerfile-0.86.1 [...]

Maybe it's possible look at that temporary Dockerfile to figure out something more. "updateUID" sounds like some general thing which as nothing to do with opam specifically. I guess it's trying to adapt something to the "remoteUser": "opam".

@michael-schwarz
Copy link
Copy Markdown
Member Author

Thanks for the help! The file is:

ARG BASE_IMAGE
FROM $BASE_IMAGE

USER root

ARG REMOTE_USER
ARG NEW_UID
ARG NEW_GID
SHELL ["/bin/sh", "-c"]
RUN eval $(sed -n "s/${REMOTE_USER}:[^:]*:\([^:]*\):\([^:]*\):[^:]*:\([^:]*\).*/OLD_UID=\1;OLD_GID=\2;HOME_FOLDER=\3/p" /etc/passwd); \
        eval $(sed -n "s/\([^:]*\):[^:]*:${NEW_UID}:.*/EXISTING_USER=\1/p" /etc/passwd); \
        eval $(sed -n "s/\([^:]*\):[^:]*:${NEW_GID}:.*/EXISTING_GROUP=\1/p" /etc/group); \
        if [ -z "$OLD_UID" ]; then \
                echo "Remote user not found in /etc/passwd ($REMOTE_USER)."; \
        elif [ "$OLD_UID" = "$NEW_UID" -a "$OLD_GID" = "$NEW_GID" ]; then \
                echo "UIDs and GIDs are the same ($NEW_UID:$NEW_GID)."; \
        elif [ "$OLD_UID" != "$NEW_UID" -a -n "$EXISTING_USER" ]; then \
                echo "User with UID exists ($EXISTING_USER=$NEW_UID)."; \
        else \
                if [ "$OLD_GID" != "$NEW_GID" -a -n "$EXISTING_GROUP" ]; then \
                        echo "Group with GID exists ($EXISTING_GROUP=$NEW_GID)."; \
                        NEW_GID="$OLD_GID"; \
                fi; \
                echo "Updating UID:GID from $OLD_UID:$OLD_GID to $NEW_UID:$NEW_GID."; \
                sed -i -e "s/\(${REMOTE_USER}:[^:]*:\)[^:]*:[^:]*/\1${NEW_UID}:${NEW_GID}/" /etc/passwd; \
                if [ "$OLD_GID" != "$NEW_GID" ]; then \
                        sed -i -e "s/\([^:]*:[^:]*:\)${OLD_GID}:/\1${NEW_GID}:/" /etc/group; \
                fi; \
                chown -R $NEW_UID:$NEW_GID $HOME_FOLDER; \
        fi;

ARG IMAGE_USER
USER $IMAGE_USER

@michael-schwarz
Copy link
Copy Markdown
Member Author

I tied again today, and this time it worked, maybe it has to do something with the reboot. However, I noticed another issue

For make test:

  • qcheck-ounit seems not installed
  • Missing os gem (install with: gem install os)

For ./regtest.sh:

  • pygmentize: not found
  • dot: not found

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces the first half of the FM’26 tutorial material by adding a new simplified-analysis interface (and a lifter to the standard Analyses.Spec), along with tutorial analyses and regression tests demonstrating global-store widening and related concepts.

Changes:

  • Added SimplifiedAnalysis and SimplifiedLifter to enable writing analyses with an explicit local state argument and a single combine hook.
  • Added tutorial analyses (gStoreWidening, effectivelyLocal) plus a provided solution (gStoreWideningSol) and a new tutorial query TutorialEffectivelyLocal.
  • Added two regression tests for the tutorial and adjusted devcontainer setup to use a prebuilt image.

Reviewed changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
tests/regression/99-tutorials/06-gstore-thread.c Adds multithreaded tutorial regression test for global-store widening behavior.
tests/regression/99-tutorials/05-gstore-zero.c Adds single-thread tutorial regression test for dead-code via branching precision.
src/goblint_lib.ml Exposes new tutorial and simplified-framework modules via the public library.
src/framework/simplifiedLifter.ml Implements adapter from simplified spec interface to standard Analyses.Spec.
src/framework/simplifiedAnalysis.ml Defines the simplified analysis interface and manager record passed to transfer functions.
src/dune Adjusts library module list (notably around memtrace module inclusion).
src/domains/queries.ml Adds TutorialEffectivelyLocal query (MustBool) for tutorial helper analysis.
src/cdomain/value/cdomains/int/intervalSetDomain.ml Removes unused helper (interval_to_bool).
src/analyses/tutorials/gStoreWideningSol.ml Adds solution implementation for the tutorial analysis.
src/analyses/tutorials/gStoreWideningHelper.ml Adds helper utilities (interval domain wrapper, tracking predicates, casting helpers).
src/analyses/tutorials/gStoreWidening.ml Adds tutorial starter implementation with TODO steps and a helper analysis skeleton.
src/analyses/mCPRegistry.ml Adds registered_simplified_analysis to register simplified analyses via the lifter.
src/analyses/assert.ml Removes unused opens.
.devcontainer/Dockerfile Adds caching layer for make setup and installs dev tools when building the image.
.devcontainer/devcontainer.json Switches to prebuilt devcontainer image and changes post-create setup command.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/dune
Comment on lines 6 to 10
(library
(name goblint_lib)
(public_name goblint.lib)
(modules :standard \ goblint goblint_memtrace privPrecCompare apronPrecCompare messagesCompare)
(modules :standard \ goblint privPrecCompare apronPrecCompare messagesCompare)
(libraries goblint.sites goblint.build-info goblint-cil goblint-cil.pta goblint-cil.syntacticsearch batteries.unthreaded qcheck-core.runner sha json-data-encoding jsonrpc cpu arg-complete fpath yaml yaml.unix uuidm goblint_timing catapult goblint_backtrace fileutils goblint_std goblint_config goblint_common goblint_domain goblint_constraint goblint_solver goblint_library goblint_cdomain_value goblint_incremental goblint_tracing goblint_logs domain_shims
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was now removed in 8a62196 but it causes compilation to fail.

There is a related dune regression (ocaml/dune#14085) but that doesn't affect locked dependencies and the unlocked ones have a conflict, so there should be no reason to remove this.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it is kept there, make fails.

Comment thread .devcontainer/devcontainer.json Outdated
"image": "ghcr.io/goblint/analyzer-devcontainer:fm26-tutorial",
"remoteUser": "opam",
"postCreateCommand": "make setup; make dev",
"postCreateCommand": "ln -s /home/opam/docker/analyzer/_opam .",
Comment thread src/analyses/tutorials/gStoreWideningHelper.ml Outdated
Comment thread src/analyses/tutorials/gStoreWidening.ml Outdated
Comment thread tests/regression/99-tutorials/06-gstore-thread.c
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
michael-schwarz and others added 3 commits May 9, 2026 23:01
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@michael-schwarz michael-schwarz marked this pull request as ready for review May 11, 2026 06:02
Copilot AI review requested due to automatic review settings May 11, 2026 06:02
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 16 out of 16 changed files in this pull request and generated 4 comments.

Comment thread tests/regression/99-tutorials/06-gstore-thread.c
Comment thread tests/regression/99-tutorials/07-gstore-mixed.c
Comment thread .devcontainer/devcontainer.json Outdated
"image": "ghcr.io/goblint/analyzer-devcontainer:fm26-tutorial",
"remoteUser": "opam",
"postCreateCommand": "make setup; make dev",
"postCreateCommand": "ln -s /home/opam/docker/analyzer/_opam .",
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is supposed to fail when _opam already exists outside of the devcontainer in the repository root. This doesn't happen for fresh users who clone the repository and use the devcontainer.

It does happen if you're already a Goblint developer and try to use the devcontainer when you have an existing switch on the host at the same location. The -f is very evil: it will simply quietly wipe your _opam repository on the host by replacing it with a symlink.

Also, this is postCreateCommand, not postStartCommand/postAttachCommand, so this doesn't re-execute when reopening the devcontainer. It only happens once when creating it.
The currently committed change for this in 7a069b6 is from #2012 (comment) but how is it a problem in the first place?

That variant also leads to strange things for those who have an _opam directory on the host: if it exists and the symlink isn't added, then the devcontainer will not use the packages installed into the container but the ones installed on the host, because they happen to be in the _opam directory.

Comment thread src/framework/simplifiedAnalysis.ml Outdated
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings May 11, 2026 06:35
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 16 out of 16 changed files in this pull request and generated 3 comments.

Comment thread src/framework/simplifiedLifter.ml
Comment thread .devcontainer/devcontainer.json Outdated
"image": "ghcr.io/goblint/analyzer-devcontainer:fm26-tutorial",
"remoteUser": "opam",
"postCreateCommand": "make setup; make dev",
"postCreateCommand": "ln -s /home/opam/docker/analyzer/_opam .",
Comment thread .devcontainer/Dockerfile Outdated
&& make setup \
&& eval $(opam env) \
&& opam install -y utop ocaml-lsp-server ocp-indent \
&& sudo gem install parallel
@michael-schwarz
Copy link
Copy Markdown
Member Author

@sim642: Can you push Ali's container to the registry? It seems he lacks the rights to do it.

"postCreateCommand": "ln -s /home/opam/docker/analyzer/_opam .",
"updateRemoteUserUID": true,

"runArgs": ["--init", "--env-file", ".devcontainer/devcontainer.env"], // TODO: why --init added by default?
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is the environment now removed? I think the point of it is that no eval $(opam env) is needed after opening the devcontainer to make things convenient. Also, I think it's needed for the OCaml plugin to work correctly because there's no way to run VSCode with the right opam switch when opening the devcontainer otherwise.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was not intentional, reverting.

Comment thread .devcontainer/devcontainer.json Outdated
"image": "ghcr.io/goblint/analyzer-devcontainer:fm26-tutorial",
"remoteUser": "opam",
"postCreateCommand": "ln -s /home/opam/docker/analyzer/_opam .",
"updateRemoteUserUID": true,
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But this is true by default. Was there some intermediate attempt to disable it to fix the issue @michael-schwarz had?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, specifically with permissions. I did not realize this was the default and assumed this has fixed the issue, removing

Comment thread .devcontainer/Dockerfile
# otherwise perl (not ruby!) complains during regression testing
ENV LC_ALL=C.UTF-8

USER opam
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this added here? The ocaml/opam:ubuntu-22.04-opam base image already does this.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this Dockerfile is accompanying the dev-container and I had issues with mounts/users; it does not hurt to make it explicit. (since we do not see the base image Dockerfile from here)

Comment thread .devcontainer/Dockerfile Outdated
&& opam -y install ocaml-lsp-server ocamlformat \
&& sudo gem install parallel os

RUN sudo apt-get install -y python3-pygments graphviz
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These should probably be moved up to the existing apt-get install to avoid an extra layer which increases image size.

Comment thread .devcontainer/Dockerfile Outdated
Comment on lines +27 to +29
&& make setup

RUN cd /home/opam/docker/analyzer \
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the purpose of having this extra layer which increases image size?

Copilot AI review requested due to automatic review settings May 13, 2026 13:46
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review is ineligible. To be eligible to request a review, you need a paid Copilot license, or your organization must enable Copilot code review.

@sim642 sim642 mentioned this pull request May 15, 2026
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants