Refresh development docs and flow (#749)

* Refresh development docs and flow Signed-off-by: Pierangelo Di Pilato <pierangelodipilato@gmail.com> * Include hack scripts from eventing Signed-off-by: Pierangelo Di Pilato <pierangelodipilato@gmail.com> * Move to bach script from makefile Signed-off-by: Pierangelo Di Pilato <pierangelodipilato@gmail.com> * Update docs Signed-off-by: Pierangelo Di Pilato <pierangelodipilato@gmail.com> * Vendor script executable Signed-off-by: Pierangelo Di Pilato <pierangelodipilato@gmail.com> * One more make Signed-off-by: Pierangelo Di Pilato <pierangelodipilato@gmail.com> * Fix script location Signed-off-by: Pierangelo Di Pilato <pierangelodipilato@gmail.com> * Make script executable Signed-off-by: Pierangelo Di Pilato <pierangelodipilato@gmail.com> * Add newline (lint) Signed-off-by: Pierangelo Di Pilato <pierangelodipilato@gmail.com> * Remove Makefile Signed-off-by: Pierangelo Di Pilato <pierangelodipilato@gmail.com> * Fail when action not found Signed-off-by: Pierangelo Di Pilato <pierangelodipilato@gmail.com> * Delete create-kind-cluster.sh * Rename run to run.sh * Use run.sh * Use run.sh * Use run.sh * Print all actions (+ short docs for free) Signed-off-by: Pierangelo Di Pilato <pierangelodipilato@gmail.com> * Add protoc and protoc-gen-go docs Signed-off-by: Pierangelo Di Pilato <pierangelodipilato@gmail.com> * Add .sh ext Signed-off-by: Pierangelo Di Pilato <pierangelodipilato@gmail.com> * Move integration tests stuff where they belong Signed-off-by: Pierangelo Di Pilato <pierangelodipilato@gmail.com> * Use .sh in dev docs Signed-off-by: Pierangelo Di Pilato <pierangelodipilato@gmail.com> * Format Signed-off-by: Pierangelo Di Pilato <pierangelodipilato@gmail.com> * Apply suggestion Signed-off-by: Pierangelo Di Pilato <pierangelodipilato@gmail.com> * Update codegen Signed-off-by: Pierangelo Di Pilato <pierangelodipilato@gmail.com> * Make script executable Signed-off-by: Pierangelo Di Pilato <pierangelodipilato@gmail.com> * Skip sacura locally Signed-off-by: Pierangelo Di Pilato <pierangelodipilato@gmail.com> * Remove knative.dev/eventing/hack from vendor Signed-off-by: Pierangelo Di Pilato <pierangelodipilato@gmail.com> * Fail on errors Signed-off-by: Pierangelo Di Pilato <pierangelodipilato@gmail.com> * update codegen Signed-off-by: Pierangelo Di Pilato <pierangelodipilato@gmail.com> * Update codegen Signed-off-by: Pierangelo Di Pilato <pierangelodipilato@gmail.com> * Use fail_test function Signed-off-by: Pierangelo Di Pilato <pierangelodipilato@gmail.com> * Forgot to scale up controllers and rebooks
knative-extensions · May 24, 2021 · 9b122d5 · 9b122d5
1 parent 85a54f4
commit 9b122d5
Show file tree

Hide file tree

Showing 14 changed files with 388 additions and 189 deletions.
diff --git a/.github/workflows/kind-e2e.yaml b/.github/workflows/kind-e2e.yaml
@@ -65,6 +65,8 @@ jobs:
       ARTIFACTS: ${{ github.workspace }}/artifacts
 
       EVENTING_CONFIG: ${{ matrix.eventing-config }}
+      NODE_VERSION: ${{ matrix.k8s-version }}
+      NODE_SHA: ${{ matrix.kind-image-sha }}
 
     steps:
       - name: Install Dependencies
@@ -99,49 +101,14 @@ jobs:
           chmod +x ./kind
           sudo mv kind /usr/local/bin
 
-      - name: Configure KinD Cluster
-        run: |
-          set -x
-
-          # KinD configuration.
-          cat > kind.yaml <<EOF
-          apiVersion: kind.x-k8s.io/v1alpha4
-          kind: Cluster
-
-          # This is needed in order to support projected volumes with service account tokens.
-          # See: https://kubernetes.slack.com/archives/CEKK1KTN2/p1600268272383600
-          kubeadmConfigPatches:
-            - |
-              apiVersion: kubeadm.k8s.io/v1beta2
-              kind: ClusterConfiguration
-              metadata:
-                name: config
-              apiServer:
-                extraArgs:
-                  "service-account-issuer": "kubernetes.default.svc"
-                  "service-account-signing-key-file": "/etc/kubernetes/pki/sa.key"
-              networking:
-                dnsDomain: "${CLUSTER_SUFFIX}"
-          nodes:
-          - role: control-plane
-            image: kindest/node:${{ matrix.k8s-version }}@${{ matrix.kind-image-sha }}
-          - role: worker
-            image: kindest/node:${{ matrix.k8s-version }}@${{ matrix.kind-image-sha }}
-          EOF
-
       - name: Create KinD Cluster
-        run: |
-          set -x
+        run: ./hack/create-kind-cluster.sh
 
-          kind create cluster --config kind.yaml
+      - name: Install Dependencies (Eventing, Kafka, etc)
+        run: ./hack/run.sh deploy-infra
 
-      - name: Install Knative Eventing and Eventing Kafka Broker
-        run: |
-          source test/e2e-common.sh
-
-          knative_setup
-
-          test_setup
+      - name: Install Eventing Kafka Broker
+        run: ./hack/run.sh deploy
 
       - name: Wait for things to be up
         run: |
@@ -168,6 +135,9 @@ jobs:
           go test -race -count=1 -timeout=1h -short -tags=e2e \
              ${{ matrix.test-suite }} ${{ matrix.extra-test-flags }}
 
+      - name: Teardown Eventing Kafka Broker
+        run: ./hack/run.sh teardown
+
       - name: Collect system diagnostics
         if: ${{ failure() }}
         run: |

diff --git a/.github/workflows/knative-java-test.yaml b/.github/workflows/knative-java-test.yaml
@@ -12,10 +12,7 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-# This file is automagically synced here from github.com/knative-sandbox/.github
-# repo by knobots: https://github.com/mattmoor/knobots and will be overwritten.
-
-name: Test
+name: Java Test
 
 on:
 
@@ -58,9 +55,8 @@ jobs:
         with:
           files: .codecov.yaml
 
-      - if: steps.codecov-enabled.outputs.files_exists == 'true'
-        name: Java Test
-        run: mvn verify -B -U --file data-plane/pom.xml --no-transfer-progress
+      - name: Java Test
+        run: ./hack/run.sh unit-tests-data-plane
 
       - if: steps.codecov-enabled.outputs.files_exists == 'true'
         name: Codecov

diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -1,11 +1,57 @@
 # Contribution guidelines
 
-So you want to hack on Knative Eventing Kafka Broker? Yay! Please refer to
-Knative's overall
-[contribution guidelines](https://www.knative.dev/contributing/) to find out how
-you can help.
+So you want to hack on Knative Eventing Kafka Broker? Yay! Please refer to Knative's overall
+[contribution guidelines](https://www.knative.dev/contributing/) to find out how you can help.
 
-# Useful links to get started
+## Getting started
+
+1. [Create and checkout a repo fork](#checkout-your-fork)
+
+### Requirements
+
+You need to install:
+
+- [`ko`](https://github.com/google/ko) - (_required_)
+- [`docker`](https://www.docker.com/) - (_required_)
+- [`Go`](https://golang.org/) - (_required_)
+    - check
+      [go \<version\>](https://github.com/knative-sandbox/eventing-kafka-broker/blob/master/go.mod)
+      for the required Go version used in this project
+- [`Java`](https://www.java.com/en/) (we recommend an `openjdk` build) -
+  (_required_)
+    - check
+      [java.version](https://github.com/knative-sandbox/eventing-kafka-broker/blob/master/data-plane/pom.xml)
+      maven property for the required Java version used in this project
+- [`protoc`](https://github.com/protocolbuffers/protobuf)
+     - To install protoc, you can follow https://grpc.io/docs/protoc-installation/
+- `protoc-gen-go`
+     - To install `protoc-gen-go` run `go install google.golang.org/protobuf/cmd/protoc-gen-go@[version]`
+
+_Note: See [`protoc` and `protoc-gen-go` current used versions](https://github.com/knative-sandbox/eventing-kafka-broker/blob/main/control-plane/pkg/contract/contract.pb.go#L3)_
+
+Requirements signaled as "optional" are not required, but it's highly recommended having them installed. If a specific
+version of a requirement is not explicitly defined above, any version will work during development.
+
+### Checkout your fork
+
+To check out this repository:
+
+1. Create your own [fork of this repository](https://help.github.com/articles/fork-a-repo/):
+1. Clone it to your machine:
+
+```shell
+mkdir -p ${GOPATH}/src/knative.dev
+cd ${GOPATH}/src/knative.dev
+git clone git@github.com:knative/eventing.git # clone eventing repo
+git clone git@github.com:${YOUR_GITHUB_USERNAME}/eventing-kafka-broker.git
+cd eventing-kafka-broker
+git remote add upstream https://github.com/knative-sandbox/eventing-kafka-broker.git
+git remote set-url --push upstream no_push
+```
+
+_Adding the `upstream` remote sets you up nicely for regularly
+[syncing your fork](https://help.github.com/articles/syncing-a-fork/)._
+
+Once you reach this point you are ready to do a full build and deploy as follows.
 
-- [Internals](INTERNALS.md)
 - [Development](DEVELOPMENT.md)