Skip to content

Community Meeting Minutes

Jump to bottom
Sebastian Becker edited this page Nov 6, 2023 · 98 revisions

06/11/2023

General

  • SAP has joined the Carbyne Stack initiative as a user and contributor.
    The first topics planned for contributions are:
    • adding support for the deployment of Carbyne Stack on AWS
    • the secure and convenient interaction with Carbyne Stack Services via a browser.
  • CarbyneStackCon'23 - the second edition of our annual community gathering - will take place on November 30 in Renningen, Germany.
    Do not miss your chance to register here.

    ❗ Important
    Please use the eventcode CSC23 to register on the EventXP platform. On the desktop version of the EventXP website, you must enter it in the the upper right corner of the UI. For the iOS mobile version, you will find a button at the very top. For Android use the hamburger menu button (≡).

07/08/2023 + 04/09/2023

General

WIP

03/07/2023 + 19/06/2023

General

WIP

  • Ephemeral #31: computations forcibly terminated with DeadlineExeeded

    • ✔️ Updates and Improvements:
      • The state machine now distinguishes between initialization and computation phases, with two separate timeouts that can be configured
      • The computation is no longer interrupted after 60s, but instead terminates after a user-defined timeout, defined either
        • Globally via the discovery service configuration
        • Locally, but Discovery requires local configuration
      • Local computations are now terminated on external events, e.g.
        • Global computation timeout published by discovery service (shared game state).
        • Game errors at other parties propagated through the discovery service.
      • Knative activation timeout is now configurable per ephemeral instance
        • Previously set to default (300s).
      • The CS Getting started guide describes how to configure the maximum activation time allowed for Knative
        • Previously set to default (600s)
      • Fixed TupleStreamer panic on broken pipe due to bad log formatting.
    • ✔️ Test with computation time of 29min 55s successful (total execution time including setup and compilation ~32min 50s)
      2 Parties,700000 input values each, computing modulo on the sum of the individual values
      Test performed on a local, 2 Kind cluster setting
      Time = 1773.06 seconds, Data sent = 324.802 MB in ~7000085 rounds (party 0), Global data sent = 649.604 MB (all parties)
    • ❗ PR to be available soon

  • Klyshko Secure Offline Phase #65:

    • Networking
    • MP-SPDZ secure CRG
      • Cowgear-based CRG has been implemented

  • 1st versions of Caliper has been open sourced

  • IaC (kind/Azure) currently under review / OSS release process

15/05/2023

General

  • Jonas Eppard is new student team member at Bosch working on combining TEEs with MPC (details to be pusblished in the coming weeks) for his bachelor thesis

WIP

  • Couple of things to land in master in the coming weeks
    • Nettle PoC (Flower-based FL w/ secure aggregation based on CS) (R: Sven)
    • Iac for local kind-based deployment of Carbyne Stack (whole stack including kind clusters) (R: Alessandro)
    • Caliper LTaC based on Gatling with first set of tests (R: Julian)
  • Also in the pipeline
    • Thymus PoC for authentication based on Ory Kratos, Hydra, and Istio

17/04/2023

General

WIP

  • Amphora

    • Add implementation to use Output Delivery Protocol from Damgård et al. for tuple provisioning
      (see amphora#45 reported by iko4)
      • ✔️ Implementation finished and ready for review amphora#46
        • Costs for sharing secrets increased from 1 input mask per secret (word) to 3 input masks and 2 multiplication triples
      • ❌ Update CLI to use latest Amphora Client version
      • ❌ Update carbynestack/carbynestack Amphora service helmfile

  • Klyshko

    • Bugfixes
      • ✔️ klyshko#60 - Scheduler fails generating new TupleGenerationJobs

03/04/2023

General

  • Working student to start mid April; will focus on Azure IaC first

WIP

  • First Bosch-internal Thymus PoC for authentication based on Ory Kratos, Hydra, and Istio available, under internal review currently

20/03/2023

General

  • Klyshko has been released (LinkIn Post) and is now available as part of the SDK
  • Will be extended by secure CRGs (MP-SPDZ, CC-based, Silent MPC) throughout the year

WIP

  • Two new (private for now) repositories for Thymus (Authn & Authz) and Caliper (Load Testing)
  • Working on a Castor Mock that allows for easy testing of CS without using Klyshko for tuple generation

06/03/2023

General

  • Submission of Carbyne Stack to Stuttgarter Innovationspreis 2023
  • Submission of Nettle to Flower Summit 2023 and Bosch AI Days 2023
  • New student team members at Bosch
    • Julian Grewe will work on scalability/load testing of Carbyne Stack.
    • Enrico Sorbera will work with Vincent Rieder on Silent MPC.

WIP

  • Klyshko (https://github.com/carbynestack/carbynestack/issues/58, https://github.com/carbynestack/klyshko/pull/1, R: Sven)

    • Build automation done ✔️
    • Helm chart available and tested ✔️
    • Integration and testing with SDK done ✔️
    • Open tasks: compliance, website update ✔️
    • Final review and merge into master ❌

  • Migration services to use gRPC instead of REST (R: Adel) ❌

    • 1st PoC is ready for Castor Upload
    • Will be "rolled out" to all Castor functionality and other services
    • Castor/Amphora clients will be consolidated

  • Upgraded vulnerable dependencies as classified by Snyk in Java-based services & modules (R: Sebastian) ✔️

    • Supports now also later versions of Java (above v8)

20/02/2023

General

  • CarbyneStackCon'22 recordings finally published on the CSC22 event page

WIP

  • Upgraded vulnerable dependencies as classified by Snyk in Java-based services & modules (R: Sebastian) ❌
    • PRs generated by Snyk have been closed for all repositories but cli-ng ✔️
  • Fixing bugs in ephemeral and castor that cause long running computation to fail (R: Sebastian)
    • Ephemeral #31: computatioons forcibly terminated with DeadlineExeeded ❌
    • Castor #46: Transaction Deadlock when fetching large amount of tuples in parallel ❌
  • Nettle PP-FL
    • PoC for E2E execution w/ MP-SPDZ-based secure aggregation implemented (R: Sven) ✔️

06/02/2023

General

WIP

  • End-user authentication (R: Veselin)
    • Respective CSEP-0049 has been published ✔️
    • Implementation started ❌
  • Klyshko (https://github.com/carbynestack/klyshko/pull/1, R: Sven)
    • Build automation done ✔️
    • Helm chart available and tested ✔️
    • Integration and testing with SDK done ✔️
    • Open tasks: compliance, website update ❌
  • Federated Learning based on Flower framework w/ CS-based secure aggregation PoC is in the making
  • Upgrading vulnerable dependencies as classified by Snyk in java based services & modules (R: Sebastian)

19/12/2022

General

  • Two new colleagues, Veselin and Adel, joined the Bosch team as developers

WIP

  • Update to latest K8s/Knative/Istio version (Veselin)
    • Update Ephemeral to latest Operator SDK version
    • Final touches: Going through the deployment process, updating Getting Started guide.
    • Organizing all the changes in PRs to the respected repos.
  • Klyshko (https://github.com/carbynestack/klyshko/pull/1, Sven)
    • PR mostly revised, some refactoring ongoing
    • Testing in 2 party setup done
    • Next steps: build automation, compliant images, SDK / website updates
  • IaC (Sebastian)
    • CDKTF based VC deployment is split into Platform Setup and Stack Deployment as reflected by the Getting Started Guides
    • Initial Platform Setup IaC configuration available (not yet shared)
      • ✔️ Supports up to 255 parties (VCPs)
      • ❌ Investigating a bug where the ingress gateway is regularly not assigned a public cluster IP

05/12/2022

General

  • Introduction of triaging process and CSEP processes
  • Update on joint demonstrator development

CSEPs

21/11/2022

General

  • Carbyne Stack won 3rd prize of the Deutscher IT Sicherheitspreis
  • 1st Workshop regarding cross-company joint demonstrator took place on 18/11/22. F/U workshop planned for 02/12/22.
  • PhD student started working on MPC-related topics at HRI Europe

Issues

  • Deployment on some clouds no longer possible to very old version of Kubernetes; will be worked on soon either by HRI or by Bosch.

07/11/2022

General

  • CSC22
    • CarybeStackCon'22 was a great success with almost 60 participants (20 on site, 40 remote)
    • Recordings to be published on the website
    • Results from interactive sessions are prepared for presentation
    • Invitation to discuss joint demonstrator will follow
    • Great feedback (3.6 / 4 overall rating)
  • Carbyne Stack is finalist of the Deutscher IT Sicherheitspreis; Officially announced on LinkedIn
  • Carbyne Stack is being featured by PD Dr. Hendrik Ballhausen in his nfdi InfraTalk on 07/11/2022 4pm - 5pm

WIP

  • WebAssembly based Amphora Client implementation using
    • Wasm doesn't support complex interfaces and cannot be used out of the box - more info to follow

17/10/2022

General

  • CSC22
    • Currently 57 registrations - 26 in person, 31 remotely
    • In-person registration still open until 23/10/22
  • Carbyne Stack is finalist of the Deutscher IT Sicherheitspreis; to be announced officially soon

WIP

  • Implementation of triage process to streaming issue / PR ingest process (see #44)

19/09/2022

General

  • CSC22
    • Currently 46 registrations - 22 in person, 24 remotely
    • Please
      • Spread the word in your networks
      • Register if not yet done
  • Several open student positions (internship, working student, thesies) at Bosch
    • Assessment ongoing

Outlook

  • Carbyne Stack will be featured as a lightning talk at the MPC Alliance's upcoming Data Privacy Summit on October 5, 2022.

WIP

  • Not much progress due to vacation time

6/09/2022

General

  • CSC22
    • All talks confirmed (see here)
      • To be announced on LinkedIn this week
    • Currently 33 registrations from 19 (!!) organizations
    • Please
      • Spread the word in your networks
      • Register if not yet done
  • Collaboration with Boston University will start soon
    • More information to be provided at CSC22
  • Several open student positions (internship, working student, thesies) at Bosch

WIP

  • Not much progress due to vacation time

15/08/2022

General

  • CSC22
    • 1st program draft published online
    • LinkedIn update with final program planned for early September

WIP

Tuple Streaming (@sbckr)

Deployment Scripts (@sbckr)

  • Evaluating tools for implementation

01/08/2022

General

  • CSC22
    • Registrations coming in, 18 registrations so far, many known participants not yet registered
    • 1st program draft to be online soon
    • LinkedIn update with final program planned for early September

WIP

Klyshko (@strieflin)

  • Implementation of tests progressing (currently ~ 80% coverage)
  • Next steps are additional tests, better documentation, GHA-based build, addition to CS deployment and revision of tutorials

Tuple Streaming (@sbckr)

  • Implementation finalized
    • 80% test coverage reached
    • Based on new, tuple-less, spdz base image
    • Test scenarios documented on issue, works as expected
  • Ready for final review

Next steps

Outlook

  • Boston University gift from Bosch has been approved, currently finalizing donation process
  • Goal for 2022 is fully and secure MVP including
    • Tuple Streaming
    • Secure offline phase
    • Authentication and authorization
    • Scalability testing

18/07/2022

General

WIP

Tuple Streaming (@sbckr)

General tuple streaming functionality available

  • Tested with gfp / gf2n consuming tuples
  • Only those tuples are fetched that are actually requested by the computation (MP-SPDZ does connect to those pipes only it wants to fetch tuples from)
  • Gfp and gf2n protocol MAC keys can be provided via configuration and are written to respective files
  • Several configuration parameters added
  • Supports programs compiled to use multiple threads
    • Number of threads extracted from program's schedule
  • Minor issue in castor discovered
    • query parameter tupletype for GET /tuples endpoint not camel case
      • Returns 400 if provided as camel case without further error details
    • Gf2n tuples can have different storage size depending on USE_GF2N_LONG being set to 1 or 0 when compiling MP-SPDZ. While ephemeral can support gf2n of different storage size when properly configured, castor does support short gf2n tuples only as size of 8 is hardcoded.

ToDo:

  • Findings from initial code review addressed, others to follow
  • Improve test coverage
  • Address Codacy findings
  • Further test functionality and provide more details test settings (results and programs used)

04/07/2022

General

  • Presentations at OSS-NA on 23/06/22 and Boston University on 21/06/22 took place
  • PR for open HRI position still not submitted?
  • Will start preparation of CarbyneStackCon (planned to take place end of late Q3 / early Q4)

WIP

Klyshko (@strieflin)

  • Implementation of tests has been started (currently ~ 10% coverage), will focus on this now
  • Most review comments from @kindlich have been addressed

Tuple Streaming (@sbckr)

  • Nothing new, currently investigating technical issues

Stale issue handling (@jaredweinfurtner)

20/06/2022

General

  • Carbyne Stack has been presented at TPMPC with > 150 attendees (single track) with positive feedback from mostly academic audience
  • Upcoming presentations at OSS-NA on 23/06/22 and Boston University on 21/06/22
  • There is (still) an open position at HRI to work on MPC

Issues

  • There are lot of mostly done but pending issues that should be finalized / closed ASAP to shorten issue turnaround time
  • Status of work on automated closing of stale issues?

WIP

Tag-based Filtering for Ephemeral

Klyshko (@strieflin)

  • Basic Klyshko implementation has been pushed using compliant base images; feedback welcome
  • Next steps: tests (including E2E)

Tuple Streaming (@sbckr and @kindlich)

  • Tuple streaming functionality implemented but to be tested

    • Supports all types of Tuples provided by Castor
    • Holds a specified number of tuples (default: 1000) and automatically refills them after consumption
    • Unconsumed tuples are discarded for the time being

  • TODO

    • Provide MAC key (write to file)
    • Integration tests (manual)
    • Improve test coverage where appropriate
    • Update guides and documentation

16/05/2022

General

  • HRI EU participation announced (Bosch Research post with 153 reactions, 2 comments, 9 shares)
  • Bosch Research made it into the 2nd round of Deutscher IT Sicherheitspreis
  • Carbyne Stack to be presented at Bosch Research to larger internal audience on 31/05/22

Issues

pre-commit

  • PRs often seemingly without having invoked pre-commit
  • Any problems with it?
    • Switching Java versions is annoying (migrating to Java 11/17 should have priority)
  • Codacy and pre-commit use different linters with different styles (remark seems to be working properly, do a pilot)

Changelog

New tuple reservation handling code (@sbckr)

  • First indications that new implementation works with the upcoming tuple streaming
  • Please test, feedback welcome.

WIP

Ephemeral on tagged objects (@grafjo)

  • Discussion of general approach
  • Inconsistencies can occur currently as there is no consensus mechanism implemented that ensures that Ephemeral at all VCPs selects the same objects as inputs. This is an accepted defect and will be documented in the README.

Support for new MP-SPDZ version (@grafjo)

  • CLI update pending, @sbckr will have a look into it

Tuple Streaming (@sbckr and @kindlich)

  • Tuple type support

    • Status quo

      • Castor GFp (arithmetic circuits) and GF2N (binary circuits)
      • Amphora GFp only
      • Ephemeral all tuple types (but only for baked-in tuples, no streaming)

    • Support should be consistent across services eventually

    • Optimized edabits to speedup non-linear operations (e.g. comparisons)

      MP-SPDZ allows to mix computation between arithmetic and binary secret sharing in the same security model. In the compiler, this is used to switch from arithmetic to binary computation for certain non-linear functions such as comparison, bit decomposition, truncation, and modulo power of two, which are use for fixed- and floating-point operations. There are several ways of achieving this as described below.

      To use that feature -Y must be used when compiling programs. Execution requires edaBits (random n-bit arithmetic numbers with their binary bit decomposition, all secret shared). For each n there is a edaBit dedicated tuple type.

    • Proposal

      • Do not support mixed circuits (i.e., edabits) yet.
      • Add GF2N support to Amphora in the future but not now.
      • Ephemeral will implement streaming for both GFp and GF2N.
      • Use e.g. aes.mpc (that does not require inputs) from MP-SPDZ to test ephemeral for now.

Klyshko (@strieflin)

  • Not pushed yet, due to compliance reasons
  • Fake tuple generation implemented
  • Simple threshold-based job scheduler available
  • Next steps: tests (including E2E), compliance, documentation
Clone this wiki locally