Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

AIP-157 Partial response implementation #717

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

AIP-157 Partial response implementation #717

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
66 changes: 30 additions & 36 deletions cmd/api/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,42 +2,36 @@

## Variables

| Environment Variable | Description | Example |
|--------------------------|-----------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------|
| DB_USER | Postgres Database user | user |
| DB_PASSWORD | Postgres Database Password | hunter2 |
| DB_HOST | Postgres Database host | /cloudsql/my-project:us-east1:tekton-results |
| DB_NAME | Postgres Database name | tekton_results |
| DB_SSLMODE | Database SSL mode | verify-full |
| DB_SSLROOTCERT | Path to CA cert used to validate Database cert | /etc/tls/db/ca.crt |
| DB_ENABLE_AUTO_MIGRATION | Auto-migrate the database on startup (create/update schemas). For further details, refer to <https://gorm.io/docs/migration.html> | true (default) |
| PROFILING | Enable profiling server | false (default) |
| PROFILING_PORT | Profiling Server Port | 6060 (default) |
| DB_MAX_IDLE_CONNECTIONS | The number of idle database connections to keep open | 2 (default for golang, but specific database drivers may have settings for this too) |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You deleted all my tuning options here in the README @sayan-biswas .... we need to minially document them, even if they are not specified in the config/base/env/config file

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ohh.. Not sure why how this was missed. I'll rebase again to include the changes.

| DB_MAX_OPEN_CONNECTIONS | The maximum number of database connections, for best performance it should equal DB_MAX_IDLE_CONNECTIONS | unlimited (default for golang, but specific database drivers may have settings for this too) |
| GRPC_WORKER_POOL | The maximum number of goroutines pre-allocated for process GRPC requests. The GRPC server will also dynamically create threads. | 2 (default) |
| K8S_QPS | The QPS setting for the kubernetes client created. | 5 (default) |
| K8S_BURST | The burst setting for the kubernetes client created. | 10 (default) |
| SERVER_PORT | gRPC and REST Server Port | 8080 (default) |
| PROMETHEUS_PORT | Prometheus Port | 9090 (default) |
| PROMETHEUS_HISTOGRAM | Enable Prometheus histogram metrics to measure latency distributions of RPCs | false (default) |
| TLS_PATH | Path to TLS files | /etc/tls |
| AUTH_DISABLE | Disable RBAC check for resources | false (default) |
| AUTH_IMPERSONATE | Enable RBAC impersonation | true (default) |
| LOG_LEVEL | Log level for api server | info (default) |
| LOGS_API | Enable logs storage service | false (default) |
| LOGS_TYPE | Determine Logs storage backend type | File (default) |
| LOGS_BUFFER_SIZE | Buffer for streaming logs | 32768 (default) |
| LOGS_PATH | Logs storage path | logs (default) |
| S3_BUCKET_NAME | S3 Bucket name | <S3 Bucket Name> |
| S3_ENDPOINT | S3 Endpoint | https://s3.ap-south-1.amazonaws.com |
| S3_HOSTNAME_IMMUTABLE | S3 Hostname immutable | false (default) |
| S3_REGION | S3 Region | ap-south-1 |
| S3_ACCESS_KEY_ID | S3 Access Key ID | <S3 Acces Key> |
| S3_SECRET_ACCESS_KEY | S3 Secret Access Key | <S3 Access Secret> |
| S3_MULTI_PART_SIZE | S3 Multi part size | 5242880 (default) |
| GCS_BUCKET_NAME | GCS Bucket Name | <GCS Bucket Name> |
| STORAGE_EMULATOR_HOST | GCS Storage Emulator Server | http://localhost:9004 |
| Environment Variable | Description | Example |
|--------------------------|-----------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------|
| DB_USER | Postgres Database user | user |
| DB_PASSWORD | Postgres Database Password | hunter2 |
| DB_HOST | Postgres Database host | /cloudsql/my-project:us-east1:tekton-results |
| DB_NAME | Postgres Database name | tekton_results |
| DB_SSLMODE | Database SSL mode | verify-full |
| DB_SSLROOTCERT | Path to CA cert used to validate Database cert | /etc/tls/db/ca.crt |
| DB_ENABLE_AUTO_MIGRATION | Auto-migrate the database on startup (create/update schemas). For further details, refer to <https://gorm.io/docs/migration.html> | true (default) |
| SERVER_PORT | gRPC and REST Server Port | 8080 (default) |
| FEATURE_GATES | Enables server features | FeatureA=true,FeatureB=false | |
| PROMETHEUS_PORT | Prometheus Port | 9090 (default) |
| PROMETHEUS_HISTOGRAM | Enable Prometheus histogram metrics to measure latency distributions of RPCs | false (default) |
| TLS_PATH | Path to TLS files | /etc/tls |
| AUTH_DISABLE | Disable RBAC check for resources | false (default) |
| AUTH_IMPERSONATE | Enable RBAC impersonation | true (default) |
| LOG_LEVEL | Log level for api server | info (default) |
| LOGS_API | Enable logs storage service | false (default) |
| LOGS_TYPE | Determine Logs storage backend type | File (default) |
| LOGS_BUFFER_SIZE | Buffer for streaming logs | 32768 (default) |
| LOGS_PATH | Logs storage path | logs (default) |
| S3_BUCKET_NAME | S3 Bucket name | <S3 Bucket Name> |
| S3_ENDPOINT | S3 Endpoint | https://s3.ap-south-1.amazonaws.com |
| S3_HOSTNAME_IMMUTABLE | S3 Hostname immutable | false (default) |
| S3_REGION | S3 Region | ap-south-1 |
| S3_ACCESS_KEY_ID | S3 Access Key ID | <S3 Acces Key> |
| S3_SECRET_ACCESS_KEY | S3 Secret Access Key | <S3 Access Secret> |
| S3_MULTI_PART_SIZE | S3 Multi part size | 5242880 (default) |
| GCS_BUCKET_NAME | GCS Bucket Name | <GCS Bucket Name> |
| STORAGE_EMULATOR_HOST | GCS Storage Emulator Server | http://localhost:9004 |

These values can also be set in the config file located in the `config/env/config` directory.

Expand Down
16 changes: 15 additions & 1 deletion cmd/api/main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,10 @@ import (
"strings"
"time"

"github.com/tektoncd/results/pkg/api/server/features"

"github.com/tektoncd/results/internal/fieldmask"

"github.com/tektoncd/results/pkg/api/server/v1alpha2/auth/impersonation"
"golang.org/x/net/http2"
"golang.org/x/net/http2/h2c"
Expand Down Expand Up @@ -76,6 +80,12 @@ func main() {
// any data in the event of an unhandled error.
defer log.Sync() //nolint:errcheck

// Load server features
f := features.NewFeatureGate()
if err := f.Set(serverConfig.FEATURE_GATES); err != nil {
log.Errorf("Failed to load feature gates: %v", err)
}

// Load server TLS
certFile := path.Join(serverConfig.TLS_PATH, "tls.crt")
keyFile := path.Join(serverConfig.TLS_PATH, "tls.key")
Expand Down Expand Up @@ -208,6 +218,7 @@ func main() {
grpc_zap.UnaryServerInterceptor(grpcLogger, zapOpts...),
grpc_auth.UnaryServerInterceptor(determineAuth),
prometheus.UnaryServerInterceptor,
fieldmask.UnaryServerInterceptor(f.Get(features.PartialResponse)),
recovery.UnaryServerInterceptor(recovery.WithRecoveryHandler(recoveryHandler)),
),
grpc_middleware.WithStreamServerChain(
Expand Down Expand Up @@ -274,7 +285,10 @@ func main() {
if err != nil {
log.Fatalf("Error dialing gRPC endpoint: %v", err)
}
serverMuxOptions = append(serverMuxOptions, runtime.WithHealthzEndpoint(healthpb.NewHealthClient(clientConn)))
serverMuxOptions = append(serverMuxOptions,
runtime.WithHealthzEndpoint(healthpb.NewHealthClient(clientConn)),
runtime.WithMetadata(fieldmask.MetadataAnnotator),
)

// Create server for gRPC gateway
ctx := context.Background()
Expand Down
8 changes: 1 addition & 7 deletions config/base/env/config
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,7 @@ DB_NAME=tekton-results
DB_SSLMODE=disable
DB_SSLROOTCERT=
DB_ENABLE_AUTO_MIGRATION=true
DB_MAX_IDLE_CONNECTIONS=10
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm assuming you did this @sayan-biswas because I didn't need to set these defaults when I added the tuning options ? If so, thanks for the correction.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or was this accidental? ...If it was, in hindsight, I does probably make sense to not set these, even to the defaults. (though 10 does not line up with the default as described in the doc.

Users can always add these to their config map.

DB_MAX_OPEN_CONNECTIONS=10
GRPC_WORKER_POOL=2
K8S_QPS=5
K8S_BURST=10
PROFILING=false
PROFILING_PORT=6060
FEATURE_GATES=
SERVER_PORT=8080
PROMETHEUS_PORT=9090
PROMETHEUS_HISTOGRAM=false
Expand Down