Skip to content

Latest commit

 

History

History
214 lines (143 loc) · 17.6 KB

extended-tour.md

File metadata and controls

214 lines (143 loc) · 17.6 KB
Raw

Extended Tour

Welcome to the extended tour of node-cpp-skel. This documentation is especially handy if you're just starting out with building Node Addon-ons or writing C++ 🎉

Table of Contents:

Walkthrough example code

This skeleton includes a few examples of how you might design your application, including standalone functions and creating Objects/Classes. Both the synchronous and asynchronous versions of each are included to give an iterative example of what the progression from sync to async looks like. Let's run through reasons why you'd design your code in these ways:

  1. Standalone function
  2. Standalone asynchronous function
  3. Object/Class
  4. Asynchronous Object/Class

When would you use a standalone function?

A standalone function is a function that exists at the top level of the module scope rather than as a member of an object that is instantiated. So if your module is wonderful, a standalone function would be called like wonderful.callme().

A standalone function makes sense when the only data needed by the function can be easily passed as arguments. When it is not easy or clean to pass data as arguments then you should consider encapsulation, for example, exposing a function as a member of an object. One of the benefits of creating a standalone function is that it can help separate concerns, which is mainly a stylistic design decision.

Example:

  • vtinfo: A synchronous standalone function that is simply getting info about a single vector tile buffer.

When would you use an object/class?

Create an object/class when you need to do some kind of data preprocessing before going into the thread pool. It's best to write your code so that the preprocessing happens once as a separate operation, then continues through to the thread pool after the preprocessing is finished and the object is ready. Examples:

  • node-mapnik: we create a Map object once, then use the object multiple times for rendering each vector tile.
  • node-addon-examples for another example of what an object-focused Addon looks like.

Objects/Classes are useful when you start doing more complex operations and need to consider performance more heavily, when performance constraints start to matter. Use classes to compute the value of something once, rather than every time you call a function.

One step further is using an asynchronous object or class, which enables you to pass data into the threadpool. The aim is to process data in the threadpool in the most efficient way possible. Efficiency and high performance are the main goals of the HelloObjectAsync example in this skel. The HelloObjectAsync example demonstrates high load, when many objects in many threads are being allocated. This scenario is where reducing unnecessary allocations really pays off, and is also why this example uses "move semantics" for even better performance.

  • Move semantics: move semantics avoid data being copied (allocating memory), to limit unnecessary memory allocation, and are most useful when working with big data.

Other thoughts related to move semantics:

  • Always best to use move semantics instead of passing by reference, espeically with objects like std::string, which can be expensive.
  • Relatedly, best to use std::unique_ptr instead of using std::shared_ptr because std::unique_ptr is non-copyable. So you're forced to avoid copying, which is a good practice.

Builds

What does "build" mean?

When you "build" your module, you are compiling and linking your C++ source code to produce a binary file that allows Node.js to load and run your module. Once loaded your C++ module can be used as if it were a Javascript module. A large list of tools come together to make this possible. In the below section we'll describe:

Software build components

The primary components involved in the build

node-pre-gyp

node-pre-gyp is a javascript command line tool used in this project as a front end to node-gyp to either compile your code or install it from binaries.

It is installed as a dependency in the package.json.

Learn more about node-pre-gyp here

node-gyp

node-gyp is a javascript command line tool used in this project as a front end to gyp.

node-gyp is bundled inside npm and does not need to be installed separately. Although, if installed in package.json, that version will be used by node-pre-gyp.

Learn more about node-gyp here

gyp

gyp is a python command line tool used in this project as a front end to make.

Learn more about gyp here

make

make is a command line tool, written in C, that is installed by default on most unix systems. It is used in this project in two ways:

  • We provide a Makefile that acts as a simple entry point for developers wanting to source compile node-pre-gyp
  • When node-gyp is run, it generates a set of Makefiles automatically which are used to call out to the compiler and linker to assemble your binary C++ module.

Learn more about make here

compiler

The command line program able to compile C++ source code, in this case clang++.

Learn more about what a compiler is here

linker

The command line program able to link C++ source code, in this case also clang++, which acts as a front end to the system linker

Configuration files

Files you will find inside this repo and their purpose. For more info look inside each file for detailed comments.

  • Makefile - entry point to building from source. This is invoked when you type make in the root directory. By default the default target is run which maps to the release target. See the comments inside the Makefile for more detail.
  • binding.gyp - JSON configuration file for node-gyp. Must be named binding.gyp and present in the root directory so that npm detects it. Will be passed to gyp by node-gyp. Because gyp is python and has less strict JSON parsing rules, code comments with # are allowed (this would not be the case if parsed with node.js).
  • common.gypi - "gypi" stands for gyp include file. This is referenced by the binding.gyp
  • package.json - configuration file for npm. But it also contains a custom binary object that is the configuration for node-pre-gyp.
  • lib/index.js - entry point for the javascript module. Referenced in the main property in the package.json. This is the file that is run when the module is loaded by require from another module.
  • scripts/setup.sh - script used to 1) install Mason and clang++ and 2) create a local.env that can be sourced in bash in order to set up Mason and clang++ on your PATH.
  • scripts/install_deps.sh - script that invokes Mason to install mason packages
  • scripts/publish.sh - script to publish the C++ binary module with node-pre-gyp. Designed to be run on travisci.org
  • .travis.yml - configuration for this module on travisci.org. Used to test the code and publish binaries for various node versions and compiler options.

Note: the binding.gyp file also inherits from two files that you will not find inside the node-cpp-skel repo. These are gyp include files that come from node core and node-gyp and are invoked when node-gyp configure is called:

Autogenerated files

Files you will notice are created when you build from source by running make:

  • build/ - a directory created by node-gyp to hold a variety of autogenerated Makefiles, gyp files, and binary outputs.
  • build/Release/ - directory created to hold binary files for a Release build. A Release build is the default build when you run make. For more info on release builds see this definition
  • build/Release/module.node - The C++ binary module, for a `Release build, in the form of a loadable module. This file was ultimately created by the linker and ended up at this path thanks to node-gyp.
  • lib/binding/module.node - the final resting place for the C++ binary module. Copied to this location from either build/Release/module.node or build/Debug/module.node. This location is configured via the module_path variable in the package.json and is used by node-pre-gyp when assembling a package to publish remotely (to allow users to install via binaries).
  • build/Release/obj.target/module/src/module.o: the object file that corresponds to the src/module.cpp.
  • build/Release/obj.target/module/standalone/hello.o: the object file that corresponds to the src/standalone/hello.cpp.
  • build/Debug/ - directory created to hold binary files for a Debug build. A Debug build is trigged when you run make debug. For more info on debug builds see this definition
  • build/Debug/module.node - The C++ binary module, for a `Debug build, in the form of a loadable module. This file was ultimately created by the linker and ended up at this path thanks to node-gyp.

Overall flow

The overall flow in terms of software components is:

make -> node-pre-gyp -> node-gyp -> gyp -> make -> compiler/linker

The overall flow, including operations, is:

Developer (you) runs 'make'
-> make reads Makefile
 -> Makefile has custom line that calls out to 'node-pre-gyp configure build'
  -> node-pre-gyp passes variables in the package.json along to 'node-gyp rebuild'
   -> node-gyp finds the `binding.gyp` and passes it to gyp along with other .gyp includes (addon.gypi and a common.gypi from node core)
    -> gyp loads the `binding.gyp` and the `common.gypi` and generates Makefiles inside `build/` directory
     -> node-gyp runs make in the `build/` directory
      -> make runs all the targets in the makefiles generated by gyp in the `build/` directory
       -> these makefiles run the `install_deps.sh` and invoke the compiler and linker for all sources to compile
        -> `install_deps.sh` puts mason packages in `/mason_packages` directory
         -> compiler outputs object files in `build/`
          -> linker outputs the loadable module in `build/`
           -> make copies the 'module.node` from `build/Release` to `lib/binding`

Then the module is ready to use. What happens when it is used is:

User of your module runs 'npm install'
-> npm fetches your module
 -> npm notices an 'install' target that calls out to node-pre-gyp
  -> node-pre-gyp downloads the C++ binary from remote url (as specified in the node-pre-gyp config in the package.json)
   -> node-pre-gyp places the C++ binary at the ['module_path']('../lib/binding/module.node')
    -> the index.js reads '../lib/binding/module.node'

This binary file ../lib/binding/module.node is what require() points to within Node.js.

Clang Tools

This skeleton uses two clang/llvm tools for automated formatting and static fixes.

Each of these tools run within their own Travis jobs. You can disable these Travis jobs if you'd like, by commenting them out in .travis.yml. This may be necessary if you're porting this skel into an already-existing project that triggers tons of clang-tidy errors, and you'd like to work through them gradually while still actively iterating on other parts of your code.

clang-format

Automates coding style enforcement, for example whitespace, tabbing, wrapping.

To run

make format

clang-tidy

Lint framework that can do very powerful checks, for example bugs that can be deduced via static analysis, unneeded copies, inefficient for-loops, and improved readability. In many cases, it can fix and modernize your code.

To run

make tidy
  • The clang-tidy tool has a large set of checks and when one fails, the error message contains the shorthand for the check name. See llvm's docs for details about what failed check is referring to. For example, see "performance-unnecessary-copy-initialization".
  • Set tidy preferences in .clang-tidy. When you run clang-tidy, you'll see output of a number of specific checks that tidy runs. You can add/remove any of these checks via the .clang-tidy file, configure these checks, and specify the files you'd like to run these checks against. The skel runs clang-tidy against everything within the /src directory.
  • This skel runs clang-tidy with the -fix option, which will automatically apply fixes to your code. Though some warnings will need to be fixed manually.
  • Installed and run via /scripts/clang-tidy.sh
  • NOLINT: In the case that clang-tidy throws for code that you either want to keep as-is or can't control, you can use NOLINT to silent clang-tidy.
  • Since clang-tidy must compile the code to do its checks, skel generates a JSON file that tells clang-tidy which files to run against and what compile command to run. This newly generated file, /build/compile_command.json, is created when running make tidy. See clang's JSON compilation format docs for more info.

Xcode

npm-test scheme in Xcode

If you're developing on macOS and have Xcode installed, you can also type make xcode to generate and open an Xcode project. In the dropdown, choose npm test to run the npm tests. You can also add more targets by adding the appropriate lines in Makefile, and rerunning make xcode. If you are modifying binding.gyp, e.g. by adding more source files, make sure to rerun make xcode so that Xcode knows about the new source files.