Skip to content

Commit

Permalink
feat: Caching the project model, so it can be loaded as a cached enti…
Browse files Browse the repository at this point in the history
…ty instead of mapping the target cfg to the project model on every iteration. Closes #120
  • Loading branch information
TheRustifyer committed Jul 14, 2024
1 parent 2f8a9d4 commit 2800fe7
Show file tree
Hide file tree
Showing 17 changed files with 186 additions and 136 deletions.
54 changes: 26 additions & 28 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ We recommend installing it in `/usr/bin`.

### macOS and another platforms

We currently don't provide installers or precompiled binaries for other operating systems.
We currently don't provide installers or precompiled binaries for other operating systems.

You can build `Zork++` manually if your platform is a supported `Rust` target.
You can check out [the list of available targets here](https://doc.rust-lang.org/nightly/rustc/platform-support.html).
Expand Down Expand Up @@ -179,10 +179,10 @@ What happened here?
See [The zork.toml config file](#zork_conf) section to have a better understanding on how to write the configuration file and your project.

> [!NOTE]
>
>
> This structure is just a guideline. You may prefer to organize your files in a completely different way. We are just providing a predefined layout, so you can quickly start working on your project.
>`Zork++` comes with this basic example by default, where is based on some driver code on the main file, a couple of module interfaces and module implementations. By passing `--template partitions` as a command line argument, you will have access to a more complex example, where module partitions and other module stuff appears, being a more sophisticated C++ modules project example.
>`Zork++` comes with this basic example by default, where is based on some driver code on the main file, a couple of module interfaces and module implementations. By passing `--template partitions` as a command line argument, you will have access to a more complex example, where module partitions and other module stuff appears, being a more sophisticated C++ modules project example.

## Let's explore the `out` directory a little
Expand Down Expand Up @@ -242,8 +242,8 @@ sources = [

[modules]
base_ifcs_dir = "./github-example/ifc"
interfaces = [
{ file = 'math.cppm' }
interfaces = [
{ file = 'math.cppm' }
]
base_impls_dir = "./github-example/src"
implementations = [
Expand Down Expand Up @@ -294,12 +294,12 @@ The optional `base_path` property allows you to specify a path where `Zork++` lo
> Whenever you declare a module interface or a module implementation in the configuration file, you must take in consideration that sometimes modules (both interfaces or implementations) depend on other modules. Dependencies of one or more modules are declared as shown below:
```toml
interfaces = [
{ file = 'math.cppm' }
interfaces = [
{ file = 'math.cppm' }
]
implementations = [
{ file = 'math.cpp' }, # Direct mapping with the interface `math`
{ file = 'math2.cpp', dependencies = ['math'] }
{ file = 'math2.cpp', dependencies = ['math'] }
# math2 isn't the same as math, so we need to specify the `math` dependency.
]
```
Expand All @@ -322,12 +322,12 @@ One thing that we haven't discussed are `module partitions`. As described by the

```toml
[modules]
interfaces = [
interfaces = [
{ file = 'interface_partition.cppm', partition = { module = 'partitions' } },
{ file = 'internal_partition.cpp', partition = { module = 'partitions', partition_name = 'internal_partition', is_internal_partition = true } },
{ file = 'partitions.cppm' }
]
```
```
*A closer look on how to work with module partitions within Zork++*

We included `partitions` inside the `interfaces` key because, most of the time, other module interfaces will require some partition, and having a separate key for them will break the way of letting you decide in which order the translation units must be processed.
Expand All @@ -342,7 +342,7 @@ Some peculiarities by compiler at the time of writing:
This means that you aren't obligated to explicitly declare module names or module partition names... But, there's a specific case: `internal module partitions`. So, whenever you have an internal module partition, you must declare your translation unit as `partition`, and then provide at least `module` and `is_internal_partition` in order to make it work

> [!NOTE]
>
>
> In future releases, things about module partitions may change drastically (or not!). For example, we are expecting Clang to implement a good way of making implicit declarations but having the opportunity to specify a concrete output directory, among other things in other compilers too.
## The sys_modules property
Expand All @@ -369,14 +369,14 @@ ZorkConfigFile {
tests: Option<TestsAttribute>,
}

/// The [project] key
/// The [project] key
ProjectAttribute {
name: &'a str
authors: Option<Vec<str>>,
compilation_db : bool
}

/// The [compiler] key
/// The [compiler] key
CompilerAttribute {
cpp_compiler: CppCompiler, // clang, msvc or gcc
driver_path: Option<str>, // The invokable name for the compiler's binary
Expand All @@ -385,7 +385,7 @@ CompilerAttribute {
extra_args: Option<Vec<str>>
}

/// The [build] key
/// The [build] key
BuildAttribute {
output_dir: Option<str>,
}
Expand All @@ -404,14 +404,12 @@ ExecutableAttribute {
/// * `base_impls_dir` - Base directory. So you don't have to specify the full path of the implementation files
/// * `implementations` - A list to define the module interface translation units for the project
/// * `sys_modules` - An array field explicitly declare which system headers must be precompiled
/// * `extra_args` - Extra arguments that will be added to the generated command lines
ModulesAttribute {
base_ifcs_dir: Option<str>,
interfaces: Option<Vec<ModuleInterface>>,
base_impls_dir: Option<str>,
implementations: Option<Vec<ModuleImplementation>>,
sys_modules: Option<Vec<str>>,
extra_args: Option<Vec<str>>,
}

/// The [tests] key
Expand All @@ -420,7 +418,7 @@ TestsAttribute {
sources_base_path: Option<str>,
sources: Option<Vec<str>>,
extra_args: Option<Vec<str>>,
}
}
```

## A closer look on the `ModulesAttribute` key
Expand All @@ -429,19 +427,19 @@ TestsAttribute {
/// [`ModuleInterface`] - A module interface structure for dealing
/// with the parse work of prebuilt module interface units
///
/// * `file`- The path of a primary module interface
/// * `file`- The path of a primary module interface
/// (relative to base_ifcs_path if applies)
///
/// * `module_name` - An optional field for make an explicit
/// declaration of the C++ module on this module interface
/// declaration of the C++ module on this module interface
/// with the `export module 'module_name' statement.
/// If this attribute isn't present, Zork++ will assume that the
/// C++ module declared within this file is equals
/// C++ module declared within this file is equals
/// to the filename
///
/// * `partition` - Whenever this attribute is present,
/// * `partition` - Whenever this attribute is present,
/// we are telling Zork++ that the actual translation unit
/// is a partition, either an interface partition
/// is a partition, either an interface partition
/// or an implementation partition unit
///
/// * `dependencies` - An optional array field for declare the module interfaces
Expand All @@ -460,11 +458,11 @@ ModuleInterface {
///
/// * `partition_name` - An optional field for explicitly declare the name of a module interface
/// partition, or a module implementation partition.
/// Currently this requirement is only needed if your partitions
/// Currently this requirement is only needed if your partitions
/// file names aren't declared as the modules convention,
/// that is `module_name-partition_name.extension`
///
/// * `is_internal_partition` - Optional field for declare that
/// * `is_internal_partition` - Optional field for declare that
/// the module is actually a module for hold implementation
/// details, known as module implementation partitions.
/// This option only takes effect with MSVC
Expand Down Expand Up @@ -557,15 +555,15 @@ But this is not available in every compiler using `C++20`, and at the time of wr
In `Zork++`, you have this feature enabled if:

- You're working with `Clang` because the `modulemap` feature of `Clang`. So, in your project, you're able to:

- `import std;` This our preferred way, in line with the C++23 feature. Under *Windows*, this is made automatically, because we manually generate a `module.modulemap` file that takes care to include the need system headers under the `import std;` statement. In *Unix* kind of operating systems, this is automatically passed as a requirement to `Clang` with a requirement. `libc++` must be installed in your machine. If there's no `libc++` or `libc++-dev` library installed in your computer, you will see some error like: `import std; --> Error, module not found`
So, make sure that you installed the `Clang's` implementation of the *standard library* to take advantage of this feature. On *Debian* based systems, you can just use `$ sudo apt install libc++-dev`. On *Arch* systems, just `$ sudo pacman -Sy libc++`.

> In any case, make sure that you enabled *libc++* as your standard library in your **zork.toml** configuration file.
- As alternative, you can use `import <system_header_name>;` This is, individually import some specific system header as a module.
- As alternative, you can use `import <system_header_name>;` This is, individually import some specific system header as a module.
Needs an explicit pre-compilation process. This is supported by `Clang` and `GCC` (since we are not able to do an `import std` for `GCC` builds).


- You're working with `MSVC`, you are able to use `import std.core`, as a compiler specific feature. But this will allow you to use import statements instead of `#include` directives.
In upcoming releases will we adapt to the real way on how Microsoft's compiler deals with this feature, so `Zork++` users will be able to correctly use `import std;` in their codebases with *MSVC*, not the workarounds existing up until this point.
Expand Down
41 changes: 21 additions & 20 deletions zork++/src/lib/cache/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ use std::{
path::{Path, PathBuf},
};

use crate::config_file::ZorkConfigFile;
use crate::domain::translation_unit::{TranslationUnit, TranslationUnitKind};
use crate::project_model::sourceset::SourceFile;
use crate::utils::constants::CACHE_FILE_EXT;
Expand All @@ -31,9 +32,17 @@ use crate::project_model::compiler::StdLibMode;

/// Standalone utility for load from the file system the Zork++ cache file
/// for the target [`CppCompiler`]
pub fn load<'a>(program_data: &'a ZorkModel<'_>, cli_args: &CliArgs) -> Result<ZorkCache<'a>> {
let compiler = program_data.compiler.cpp_compiler;
let cache_path = &program_data.build.output_dir.join("zork").join("cache");
pub fn load<'a>(config: &ZorkConfigFile<'a>, cli_args: &CliArgs) -> Result<ZorkCache<'a>> {
let compiler: CppCompiler = config.compiler.cpp_compiler.into();
let cache_path = Path::new(
&config
.build
.as_ref()
.and_then(|build_attr| build_attr.output_dir)
.unwrap_or("out"),
)
.join("zork")
.join("cache");

let cache_file_path = cache_path
.join(compiler.as_ref())
Expand All @@ -42,16 +51,16 @@ pub fn load<'a>(program_data: &'a ZorkModel<'_>, cli_args: &CliArgs) -> Result<Z
// TODO: analyze if the clear cache must be performed by target and/or active cfg file(s)
// TODO: should we just have a cache dir with the <compiler>_<cfg_file>_<target>.json or similar?
// Or just .../<compiler>/<cfg_file>_<target>.json
let mut cache = if !Path::new(&cache_file_path).exists() {
let cache = if !cache_file_path.exists() {
File::create(&cache_file_path).with_context(|| "Error creating the cache file")?;
helpers::initialize_default_cache(compiler, cache_file_path)?
} else if Path::new(cache_path).exists() && cli_args.clear_cache {
fs::remove_dir_all(cache_path).with_context(|| "Error cleaning the Zork++ cache")?;
helpers::initialize_default_cache(cache_file_path)?
} else if cache_path.exists() && cli_args.clear_cache {
fs::remove_dir_all(&cache_path).with_context(|| "Error cleaning the Zork++ cache")?;
fs::create_dir(cache_path)
.with_context(|| "Error creating the cache subdirectory for {compiler}")?;
File::create(&cache_file_path)
.with_context(|| "Error creating the cache file after cleaning the cache")?;
helpers::initialize_default_cache(compiler, cache_file_path)?
helpers::initialize_default_cache(cache_file_path)?
} else {
log::trace!(
"Loading Zork++ cache file for {compiler} at: {:?}",
Expand All @@ -61,16 +70,11 @@ pub fn load<'a>(program_data: &'a ZorkModel<'_>, cli_args: &CliArgs) -> Result<Z
.with_context(|| "Error loading the Zork++ cache")?
};

cache
.run_tasks(program_data)
.with_context(|| "Error running the cache tasks")?;

Ok(cache)
}

#[derive(Serialize, Deserialize, Default)]
pub struct ZorkCache<'a> {
pub compiler: CppCompiler,
pub compilers_metadata: CompilersMetadata<'a>,
pub generated_commands: Commands<'a>,
pub metadata: CacheMetadata,
Expand Down Expand Up @@ -195,8 +199,8 @@ impl<'a> ZorkCache<'a> {

/// Method that returns the HashMap that holds the environmental variables that must be passed
/// to the underlying shell
pub fn get_process_env_args(&'a mut self) -> &'a EnvVars {
match self.compiler {
pub fn get_process_env_args(&'a mut self, compiler: CppCompiler) -> &'a EnvVars {
match compiler {
CppCompiler::MSVC => &self.compilers_metadata.msvc.env_vars,
CppCompiler::CLANG => &self.compilers_metadata.clang.env_vars,
CppCompiler::GCC => &self.compilers_metadata.gcc.env_vars,
Expand Down Expand Up @@ -398,12 +402,8 @@ mod helpers {
use super::*;
use std::path::PathBuf;

pub(crate) fn initialize_default_cache<'a>(
compiler: CppCompiler,
cache_file_path: PathBuf,
) -> Result<ZorkCache<'a>> {
pub(crate) fn initialize_default_cache<'a>(cache_file_path: PathBuf) -> Result<ZorkCache<'a>> {
let default_initialized = ZorkCache {
compiler,
metadata: CacheMetadata {
cache_file_path: cache_file_path.clone(),
..Default::default()
Expand All @@ -413,6 +413,7 @@ mod helpers {

utils::fs::serialize_object_to_file(&cache_file_path, &default_initialized)
.with_context(move || "Error saving data to the Zork++ cache")?;

Ok(default_initialized)
}
}
2 changes: 1 addition & 1 deletion zork++/src/lib/compiler/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -254,7 +254,7 @@ fn process_kind_translation_unit<'a, T: TranslationUnit<'a>>(
translation_unit: &'a T,
for_kind: &TranslationUnitKind,
) {
let compiler = cache.compiler;
let compiler = model.compiler.cpp_compiler;
let lpe = cache.metadata.last_program_execution;

if let Some(generated_cmd) = cache.get_cmd_for_translation_unit_kind(translation_unit, for_kind)
Expand Down
2 changes: 1 addition & 1 deletion zork++/src/lib/config_file/compiler.rs
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ pub struct CompilerAttribute<'a> {
}

/// The C++ compilers available within Zork++
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq, Default)]
#[derive(Serialize, Deserialize, Debug, Clone, Copy, PartialEq, Default)]
pub enum CppCompiler {
#[serde(alias = "CLANG", alias = "Clang", alias = "clang")]
#[default]
Expand Down
4 changes: 0 additions & 4 deletions zork++/src/lib/config_file/modules.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,6 @@ use serde::{Deserialize, Serialize};
/// * `implementations` - A list to define the module interface translation units for the project
/// * `sys_modules` - An array field explicitly declare which system headers
/// must be precompiled in order to make the importable translation units
/// * `extra_args` - Extra arguments that will be added to the generated command lines
///
/// ### Tests
///
Expand All @@ -25,7 +24,6 @@ use serde::{Deserialize, Serialize};
/// { file = 'math.cpp' }, { file = 'some_module_impl.cpp', dependencies = ['iostream'] }
/// ]
/// sys_modules = ['iostream', 'vector', 'string', 'type_traits', 'functional']
/// extra_args = ['-Wall']
/// "#;
///
/// let config: ModulesAttribute = toml::from_str(CONFIG_FILE_MOCK)
Expand Down Expand Up @@ -72,8 +70,6 @@ pub struct ModulesAttribute<'a> {
pub implementations: Option<Vec<ModuleImplementation<'a>>>,
#[serde(borrow)]
pub sys_modules: Option<Vec<&'a str>>,
#[serde(borrow)]
pub extra_args: Option<Vec<&'a str>>,
}

/// [`ModuleInterface`] - A module interface structure for dealing
Expand Down
Loading

0 comments on commit 2800fe7

Please sign in to comment.