Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

independently cache @cImports and C object files #2015

Closed
andrewrk opened this Issue Feb 27, 2019 · 0 comments

Comments

Projects
None yet
1 participant
@andrewrk
Copy link
Member

commented Feb 27, 2019

My changes in #2005 integrated @cImport with the caching system, so that if a header that is indirectly used changes, Zig knows to repeat a compilation. That's for the entire compilation though. @cImport and objects built with --c-source represent an independent compilation that can and should be independently cached. Caching for both features works the same way:

  • The compiler id hash uses the existing global cache mechanism
  • The project-local zig-cache directory is used to store hashes of the inputs:
    • The global compiler id hash
    • The command line which is given to clang
    • Any file paths which are part of the command line are "file hashed" (the caching system takes into account their contents)
  • -MD -MF foo.d args are inserted by zig and used to collect a list of files used when doing the compilation. These are "file hashed".
  • The project-local zig-cache directory is also used to store the outputs:
    • in the case of @cImport, the rendered .zig file translated from C. This solves a whole bunch of problems, for example debug info not being able to use a source file due to compiling straight from in-memory translated AST. It also makes it easier to introspect when troubleshooting.
    • in the case of compiling C source into an object, the object file produced by clang.

This makes sense to do in both stage1 and stage2 because although Zig will be doing incremental builds of zig code, compiling C code and importing .h files will always be at this "object" level, and will always make sense to cache this way.

Once implemented, this will make @cImport and objects built with --c-source near-instantaneous on subsequent builds, when none of the relevant C source files have changed.

@andrewrk andrewrk added this to the 0.5.0 milestone Feb 27, 2019

andrewrk added a commit that referenced this issue Mar 9, 2019

breaking changes to zig build API and improved caching
 * in Zig build scripts, getOutputPath() is no longer a valid function
   to call, unless setOutputDir() was used, or within a custom make()
   function. Instead there is more convenient API to use which takes
   advantage of the caching system. Search this commit diff for
   `exe.run()` for an example.
 * Zig build by default enables caching. All build artifacts will go
   into zig-cache. If you want to access build artifacts in a convenient
   location, it is recommended to add an `install` step. Otherwise
   you can use the `run()` API mentioned above to execute programs
   directly from their location in the cache. Closes #330.
   `addSystemCommand` is available for programs not built with Zig
   build.
 * Please note that Zig does no cache evicting yet. You may have to
   manually delete zig-cache directories periodically to keep disk
   usage down. It's planned for this to be a simple Least Recently
   Used eviction system eventually.
 * `--output`, `--output-lib`, and `--output-h` are removed. Instead,
   use `--output-dir` which defaults to the current working directory.
   Or take advantage of `--cache on`, which will print the main output
   path to stdout, and the other artifacts will be in the same directory
   with predictable file names. `--disable-gen-h` is available when
   one wants to prevent .h file generation.
 * `@cImport` is always independently cached now. Closes #2015.
   It always writes the generated Zig code to disk which makes debug
   info and compile errors better. No more "TODO: remember C source
   location to display here"
 * Fix .d file parsing. (Fixes the MacOS CI failure)
 * Zig no longer creates "temporary files" other than inside a
   zig-cache directory.

This breaks the CLI API that Godbolt uses. The suggested new invocation
can be found in this commit diff, in the changes to `test/cli.zig`.

@andrewrk andrewrk modified the milestones: 0.5.0, 0.4.0 Apr 8, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.