Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CAS] LLVMCAS implementation #68448

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Commits on Oct 9, 2023

  1. [ADT] Add TrieRawHashMap

    Implement TrieRawHashMap which stores objects into a Trie based on the
    hash of the object.
    
    User needs to supply the hashing function and guarantees the uniqueness of
    the hash for the objects to be inserted. Hash collision is not
    supported
    cachemeifyoucan committed Oct 9, 2023
    Configuration menu
    Copy the full SHA
    c3ce218 View commit details
    Browse the repository at this point in the history
  2. [CAS] Add LLVMCAS library with InMemoryCAS implementation

    Add llvm::cas::ObjectStore abstraction and InMemoryCAS as a in-memory
    CAS object store implementation.
    
    The ObjectStore models its objects as:
    * Content: An array of bytes for the data to be stored.
    * Refs: An array of references to other objects in the ObjectStore.
    And each CAS Object can be idenfied with an unqine ID/Hash.
    
    ObjectStore supports following general action:
    * Expected<ID> store(Content, ArrayRef<Ref>)
    * Expected<Ref> get(ID)
    
    It also introduces following types to interact with a CAS ObjectStore:
    * CASID: Hash representation for an CAS Objects with its context to help
      print/compare CASIDs.
    * ObjectRef: A light-weight ref for an object in the ObjectStore. It is
      implementation defined so it can be optimized for
      read/store/references depending on the implementation.
    * ObjectHandle: A CAS internal light-weight handle to an loaded object in the
      ObjectStore. Underlying data for the object is guaranteed to be
      available and no error handling is required to access data. This is
      not exposed to the users of CAS from ObjectStore APIs.
    * ObjectProxy: A proxy for the users of CAS to interact with the data
      inside CAS Object. It bundles a ObjectHandle and an ObjectStore
      instance.
    
    Differential Revision: https://reviews.llvm.org/D133716
    cachemeifyoucan committed Oct 9, 2023
    Configuration menu
    Copy the full SHA
    3166cc0 View commit details
    Browse the repository at this point in the history
  3. [FileSystem] Allow exclusive file lock

    Add parameter to file lock API to allow exclusive file lock. Both Unix
    and Windows support lock the file exclusively for write for one process
    and LLVM OnDiskCAS uses exclusive file lock to coordinate CAS creation.
    cachemeifyoucan committed Oct 9, 2023
    Configuration menu
    Copy the full SHA
    6c7d348 View commit details
    Browse the repository at this point in the history

Commits on Oct 13, 2023

  1. [CAS] Add implementation for current OnDisk CAS + abstractions

    Add current downstream cas API and implementation that includes
    OnDiskCAS implementation, different level of abstractions for CAS,
    different utilities.
    cachemeifyoucan committed Oct 13, 2023
    Configuration menu
    Copy the full SHA
    dbb48a0 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    8c58a0f View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    34cbc76 View commit details
    Browse the repository at this point in the history
  4. [CAS] Add support for PluginCAS

    Allow loading external CAS implementation via PluginCAS. In this patch,
    it adds:
    * C APIs that can be implemented by plugin, from which LLVM can load the
      dylib to use external CAS implementation
    * A PluginCAS, that implements vending external CAS implementation as
      llvm ObjectStore and ActionCache class.
    * A libCASPluginTest dylib, that provides example external CAS
      implementation that wraps LLVM CAS for testing purpose.
    * Add a unified way to load external CAS implementation.
    cachemeifyoucan committed Oct 13, 2023
    Configuration menu
    Copy the full SHA
    8442410 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    1cbcebd View commit details
    Browse the repository at this point in the history

Commits on Oct 16, 2023

  1. Configuration menu
    Copy the full SHA
    281d178 View commit details
    Browse the repository at this point in the history