CI and package management (#39)

The compiler can now be built standalone much more easily using the
Conan package manager for dependencies.

### Dependencies and packaging
A `conanfile.py` and corresponding `conandata.yml` have been added at
the repo root. These files are used by the Conan package manager to
determine which external libraries are needed to build the compiler. In
the build steps outlined below, `conan install` is used to instruct
Conan to fetch and build necessary libraries. The `conanfile.py` also
creates a Conan package for the compiler itself, called `qss-compiler`,
which can be installed by other projects.

All external libraries exist on [Conan
center](https://conan.io/center/), except for:
- llvm-core
- clang-tools-extra
- qasm

For these libraries, we provide in-tree Conan recipes, which must be
registered with your host's Conan installation. To do this, run
`conan_deps.sh`. Run this any time you modify any of these in-tree
recipes.

There are two primary reasons we maintain these recipes in-tree:
1. They aren't available in Conan Center.
2. We update them more quickly than would be suitable for Conan Center
(e.g. it takes [months to
years](conan-io/conan-center-index#7613) to get
folks on Conan Center to agree on `llvm-core` version bumps).

### Building
Follow these steps to install dependencies with Conan and build the
compiler.

1. Run `conan_deps.sh` to export local Conan recipes for `llvm-core`,
`clang-tools-extra` and `qasm`.
3. Create a `build` directory and enter it.
4. Run `conan install --build=outdated <path to repo root>`
5. Run `conan build <path to repo root>` or use CMake to generate your
favorite build system and invoke that.
6. Run tests with `ninja check-tests` (build system is `ninja` by
default, use your own if relevant).

If you don't want to use Conan, make sure that you configure your system
with the necessary dependencies and tell CMake where to find
corresponding `find_package` modules (e.g. by specifying
`-DCMAKE_TOOLCHAIN_FILE=<path to toolchain>`).

### CI
CI is now available for Linux x64 to validate builds, unit tests, and
LLVM LIT testing for QASM and the mock target. This project builds both
LLVM Core and Clang Tools Extra (via Conan recipes), which takes several
hours. To accelerate development, a GitHub Cache Action is used to cache
the resulting library binaries along with all other Conan dependencies.
Due to GitHub's cache size limit, there's only enough space to cache
builds on the `main` branch. However, PR builds will load the Conan
cache from `main`, so they should finish in around 10 minutes. PRs that
change the LLVM recipes will need to wait for a full build (~5 hours).
Once a PR is merged to main, any changes to Conan files in the
repository will trigger a full build / cache rebuild to ensure the cache
is cleared of any stale dependencies (to prevent unbounded growth).

.github/workflows/ci.yml

@@ -0,0 +1,67 @@
+name: Continuous Integration
+on: [push]
+jobs:
+  Build:
+    runs-on: ubuntu-latest
+    env:
+      CONAN_USER_HOME: ${{ github.workspace }}
+    steps:
+      - uses: actions/checkout@v3
+        with:
+          fetch-depth: 0
+      # -- REMOVE ONCE OPEN SOURCE
+      # Needed until Qiskit/qss-qasm is public.
+      - uses: webfactory/ssh-agent@v0.7.0
+        with:
+          ssh-private-key: ${{ secrets.SSH_PRIVATE_KEY }}
+      - name: Set up Python
+        uses: actions/setup-python@v4
+        with:
+          python-version: '3.10'
+      - name: Install pip packages
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r requirements-dev.txt
+      - name: Create Conan default profile
+        run: |
+          conan profile new default --detect
+          conan profile update settings.compiler.libcxx=libstdc++11 default
+      - name: Export QSSC_VERSION from Git version tag
+        run: |
+          version=`python -c "from setuptools_scm import get_version; print(get_version())"`
+          echo "QSSC_VERSION=$version" >> $GITHUB_ENV
+      - name: Try load Conan cache
+        id: cache
+        uses: actions/cache/restore@v3
+        with:
+          path: .conan
+          key: conan-${{ runner.os }}-${{ hashFiles('conandata.yml', 'conanfile.py', 'conan/**/*.py', 'conan/**/*.yml') }}
+          restore-keys: conan-${{ runner.os }}
+      - name : Print cache hit/miss
+        run: |
+          echo "Cache was hit: ${{ steps.cache.outputs.cache-hit }}"
+      # If we have a cache miss on 'main', clear the cache.
+      # A dependency was updated, so we need to drop the old one
+      # to prevent unbounded cache growth over time.
+      - name : Clear Conan cache
+        if: github.event_name == 'push' && github.ref == 'refs/heads/main' && steps.cache.outputs.cache-hit != 'true'
+        run: |
+          rm -rf ./.conan
+      - name: Add QASM, LLVM, and clang tools recipes to Conan cache.
+        run: ./conan_deps.sh
+      - name: Build QSS compiler Conan package
+        id: build
+        run: |
+          export CONAN_LLVM_GIT_CACHE="${{ runner.temp }}/llvm-project"
+          mkdir build && cd build
+          conan install --build=outdated ..
+          conan build ..
+          ninja check-tests
+      # On 'main' branch, always save the cache if Conan build succeeded.
+      # Note: we only update the cache from 'main' to avoid "cache thrashing", which would result in the 'main'
+      #       cache getting LRU-evicted for every PR, since a single run uses most of the 10GB repo limit.
+      - uses: actions/cache/save@v3
+        if: github.event_name == 'push' && github.ref == 'refs/heads/main' && steps.build.outcome == 'success'
+        with:
+          path: .conan
+          key: conan-${{ runner.os }}-${{ hashFiles('conandata.yml', 'conanfile.py', 'conan/**/*.py', 'conan/**/*.yml') }}

CMakeLists.txt

@@ -38,6 +38,10 @@ endif()
 
 enable_language(C CXX ASM)
 
+# Add build directory to CMake module search path.
+# Some package managers like Conan generate CMake modules here.
+list(APPEND CMAKE_MODULE_PATH "${CMAKE_BINARY_DIR}")
+
 # Find external packages
 find_package(Python COMPONENTS Interpreter Development)
 find_package(qasm REQUIRED)
@@ -47,6 +51,8 @@ find_package(gmp REQUIRED)
 find_package(nlohmann_json REQUIRED)
 find_package(libzip REQUIRED)
 find_package(GTest REQUIRED)
+find_package(llvm REQUIRED)
+find_package(clang-tools-extra REQUIRED)
 
 if(DEFINED ENV{VERSION_STRING})
     set(VERSION_STRING $ENV{VERSION_STRING})
@@ -322,8 +328,8 @@ find_package(pybind11 REQUIRED)
 include_directories(SYSTEM ${nlohmann_json_INCLUDE_DIRS})
 
 # Boost -------------------------------------------------------------------
-find_package(boost 1.77.0 REQUIRED)
-include_directories(SYSTEM ${boost_INCLUDE_DIRS})
+find_package(Boost 1.77.0 REQUIRED)
+include_directories(SYSTEM ${Boost_INCLUDE_DIRS})
 
 # ------------------------------------------------------------------------------
 # QSS-COMPILER

conan/clang-tools-extra/conanfile.py

@@ -0,0 +1,163 @@
+from typing import Generator
+from conans.errors import ConanInvalidConfiguration
+from conans import ConanFile, CMake, tools
+
+from collections import defaultdict
+import json
+import re
+import os.path
+import os
+import shutil
+
+LLVM_TAG = "llvmorg-14.0.6"
+
+
+class ClangToolsExtraConan(ConanFile):
+    name = 'clang-tools-extra'
+    version = "14.0.6"
+    description = (
+        'A toolkit for analysis of c++ projects.'
+    )
+    license = 'Apache-2.0 WITH LLVM-exception'
+    topics = ('conan', 'llvm', 'clang-tools-extra')
+    homepage = 'https://github.com/llvm/llvm-project/tree/master/llvm'
+    url = 'https://github.com/conan-io/conan-center-index'
+
+    settings = ('os', 'arch', 'compiler', 'build_type')
+    options = {
+        'shared': [True, False],
+        'fPIC': [True, False],
+    }
+    default_options = {
+        'shared': False,
+        'fPIC': True,
+    }
+
+    generators = ['cmake']
+    no_copy_source = True
+    exports_sources = "llvm-project/*"
+
+    def source(self):
+        git_cache = os.environ.get("CONAN_LLVM_GIT_CACHE")
+        cache_hit = lambda: os.path.exists(f"{git_cache}/.git")
+        cache_arg = f" --reference-if-able '{git_cache}' " if git_cache else ""
+
+        if git_cache and cache_hit():
+            self.output.info(f"Cache hit! Some Git objects will be loaded from '{git_cache}'.")
+
+        self.run(f"git clone {cache_arg} -b {LLVM_TAG} --single-branch https://github.com/llvm/llvm-project.git")
+
+        if git_cache and not cache_hit():
+            # Update cache.
+            self.output.info(f"Updating cache at '{git_cache}'.")
+            self.run(f"cp -r llvm-project '{git_cache}'")
+
+    @property
+    def _source_subfolder(self):
+        return 'llvm-project/llvm'
+
+    def _supports_compiler(self):
+        compiler = self.settings.compiler.value
+        version = tools.Version(self.settings.compiler.version)
+        major_rev, minor_rev = int(version.major), int(version.minor)
+
+        unsupported_combinations = [
+            [compiler == 'gcc', major_rev < 8],
+            [compiler == 'clang', major_rev < 10],
+            [compiler == 'apple-clang', major_rev < 11],
+        ]
+        if any(all(combination) for combination in unsupported_combinations):
+            message = 'unsupported compiler: "{}", version "{}"'
+            raise ConanInvalidConfiguration(message.format(compiler, version))
+
+    def _configure_cmake(self):
+        cmake = CMake(self, generator="Ninja")
+        cmake.definitions["LLVM_ENABLE_PROJECTS"] = 'clang;clang-tools-extra'
+        cmake.definitions['BUILD_SHARED_LIBS'] = False
+        cmake.definitions['CMAKE_SKIP_RPATH'] = True
+        cmake.definitions['CMAKE_BUILD_TYPE'] = "Release"
+        cmake.definitions['CMAKE_POSITION_INDEPENDENT_CODE'] = \
+            self.options.get_safe('fPIC', default=False) or self.options.shared
+
+        cmake.definitions['LLVM_TARGET_ARCH'] = 'host'
+        cmake.definitions['LLVM_TARGETS_TO_BUILD'] = ''
+        cmake.definitions['LLVM_ENABLE_PIC'] = \
+            self.options.get_safe('fPIC', default=False)
+
+        cmake.definitions['LLVM_ABI_BREAKING_CHECKS'] = 'WITH_ASSERTS'
+        cmake.definitions['LLVM_ENABLE_WARNINGS'] = True
+        cmake.definitions['LLVM_ENABLE_PEDANTIC'] = True
+        cmake.definitions['LLVM_ENABLE_WERROR'] = False
+
+        cmake.definitions['LLVM_USE_RELATIVE_PATHS_IN_DEBUG_INFO'] = False
+        cmake.definitions['LLVM_BUILD_INSTRUMENTED_COVERAGE'] = False
+        cmake.definitions['LLVM_REVERSE_ITERATION'] = False
+        cmake.definitions['LLVM_ENABLE_BINDINGS'] = False
+        cmake.definitions['LLVM_CCACHE_BUILD'] = False
+
+        cmake.definitions['LLVM_INCLUDE_EXAMPLES'] = False
+        cmake.definitions['LLVM_INCLUDE_TESTS'] = False
+        cmake.definitions['LLVM_INCLUDE_BENCHMARKS'] = False
+        cmake.definitions['LLVM_APPEND_VC_REV'] = False
+        cmake.definitions['LLVM_BUILD_DOCS'] = False
+        cmake.definitions['LLVM_ENABLE_IDE'] = False
+
+        cmake.definitions['LLVM_ENABLE_EH'] = True
+        cmake.definitions['LLVM_ENABLE_RTTI'] = True
+        cmake.definitions['LLVM_ENABLE_THREADS'] = False
+        cmake.definitions['LLVM_ENABLE_LTO'] = False
+        cmake.definitions['LLVM_STATIC_LINK_CXX_STDLIB'] = False
+        cmake.definitions['LLVM_ENABLE_UNWIND_TABLES'] = False
+        cmake.definitions['LLVM_ENABLE_EXPENSIVE_CHECKS'] = False
+        cmake.definitions['LLVM_ENABLE_ASSERTIONS'] = False
+        cmake.definitions['LLVM_USE_NEWPM'] = False
+        cmake.definitions['LLVM_USE_OPROFILE'] = False
+        cmake.definitions['LLVM_USE_PERF'] = False
+        cmake.definitions['LLVM_USE_SANITIZER'] = ''
+        cmake.definitions['LLVM_ENABLE_Z3_SOLVER'] = False
+        cmake.definitions['LLVM_ENABLE_LIBPFM'] = False
+        cmake.definitions['LLVM_ENABLE_LIBEDIT'] = False
+        cmake.definitions['LLVM_ENABLE_FFI'] = False
+        cmake.definitions['LLVM_ENABLE_ZLIB'] = False
+        cmake.definitions['LLVM_ENABLE_LIBXML2'] = False
+        return cmake
+
+    def config_options(self):
+        if self.settings.os == 'Windows':
+            del self.options.fPIC
+
+    def configure(self):
+        self._supports_compiler()
+
+    def build(self):
+        cmake = self._configure_cmake()
+        cmake.configure(source_folder=self._source_subfolder)
+        cmake.build()
+
+    def package(self):
+        cmake = self._configure_cmake()
+        cmake.install()
+        lib_path = os.path.join(self.package_folder, 'lib')
+        share_path = os.path.join(self.package_folder, 'share/clang')
+        bin_path = os.path.join(self.package_folder, 'bin')
+
+        patterns = ["tidy", "format", "apply"]
+        if os.path.exists(bin_path):
+            for name in os.listdir(bin_path):
+                if not any(pattern in name for pattern in patterns):
+                    os.remove(os.path.join(bin_path, name))
+
+        if os.path.exists(lib_path):
+            for name in os.listdir(lib_path):
+                file = os.path.join(lib_path, name)
+                if not os.path.isdir(file):
+                    os.remove(file)
+
+        self.copy("*.py", dst="bin", src=share_path)
+        tools.rmdir(os.path.join(self.package_folder, 'include'))
+        tools.rmdir(os.path.join(self.package_folder, 'share'))
+        tools.rmdir(os.path.join(lib_path, 'cmake'))
+
+    def package_id(self):
+        self.info.include_build_settings()
+        del self.info.settings.build_type