-
Notifications
You must be signed in to change notification settings - Fork 128
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add GRPC interface #472
Merged
Merged
add GRPC interface #472
Changes from 27 commits
Commits
Show all changes
28 commits
Select commit
Hold shift + click to select a range
3a373cd
initial grpc impl
bobcallaway 89720ae
fix cmd line arg with viper alias
bobcallaway 37d7f4b
add debugging, install protoc via apt
bobcallaway 9ac8334
remove debugging and trailing newline
bobcallaway caf3d61
write header and value together
bobcallaway eb728b7
fix docker-compose, revert last fix
bobcallaway 44ae7d8
fix logging, add grpc mw
bobcallaway 0ec96a9
update protobuf defs, implement legacy interface
bobcallaway a945daa
fix spacing
bobcallaway 48fe1c7
use abstract unix domain socket
bobcallaway 28b18e6
send 201 and SCT with response
bobcallaway d877c0f
address code review comments
bobcallaway b64354c
debugging k8s failure
bobcallaway 310e054
trim space on chain before sending
bobcallaway 415bcd1
trim space on CSC call as well
bobcallaway 1b86826
fix prom and address review comments
bobcallaway 3c56ef7
s/@/:/
bobcallaway e1d15de
update version num, tweak cert chain nomenclature
bobcallaway cfdb7a4
modify api_test to use grpc server
bobcallaway 0778d60
remove unused code
bobcallaway 1ee57d3
Merge branch 'main' into grpc
bobcallaway f82e92b
remove ctl logger, fix newlines
bobcallaway 4eb71f9
fix test to not send nil challenge
bobcallaway 921c124
address review comments
bobcallaway 64ce8f1
commit generated file
bobcallaway 01a7c33
Merge remote-tracking branch 'upstream/main' into grpc
bobcallaway 2dcae0a
Merge remote-tracking branch 'upstream/main' into grpc
bobcallaway b57e30e
opt for message with 0 fields over empty for forward compatibility
bobcallaway File filter
Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| /pkg/generated/protobuf/** linguist-generated |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,151 @@ | ||
| // Copyright 2022 The Sigstore Authors. | ||
| // | ||
| // Licensed under the Apache License, Version 2.0 (the "License"); | ||
| // you may not use this file except in compliance with the License. | ||
| // You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, software | ||
| // distributed under the License is distributed on an "AS IS" BASIS, | ||
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
| // | ||
|
|
||
| package app | ||
|
|
||
| import ( | ||
| "context" | ||
| "fmt" | ||
| "net" | ||
|
|
||
| "github.com/goadesign/goa/grpc/middleware" | ||
| ctclient "github.com/google/certificate-transparency-go/client" | ||
| grpcmw "github.com/grpc-ecosystem/go-grpc-middleware" | ||
| grpc_zap "github.com/grpc-ecosystem/go-grpc-middleware/logging/zap" | ||
| grpc_recovery "github.com/grpc-ecosystem/go-grpc-middleware/recovery" | ||
| grpc_prometheus "github.com/grpc-ecosystem/go-grpc-prometheus" | ||
| "github.com/prometheus/client_golang/prometheus" | ||
| "github.com/sigstore/fulcio/pkg/api" | ||
| "github.com/sigstore/fulcio/pkg/ca" | ||
| "github.com/sigstore/fulcio/pkg/config" | ||
| gw "github.com/sigstore/fulcio/pkg/generated/protobuf" | ||
| gw_legacy "github.com/sigstore/fulcio/pkg/generated/protobuf/legacy" | ||
| "github.com/sigstore/fulcio/pkg/log" | ||
| "github.com/spf13/viper" | ||
| "google.golang.org/grpc" | ||
| ) | ||
|
|
||
| const ( | ||
| LegacyUnixDomainSocket = "@fulcio-legacy-grpc-socket" | ||
| ) | ||
|
|
||
| type grpcServer struct { | ||
| *grpc.Server | ||
| grpcServerEndpoint string | ||
| caService gw.CAServer | ||
| } | ||
|
|
||
| func passFulcioConfigThruContext(cfg *config.FulcioConfig) grpc.UnaryServerInterceptor { | ||
| return func(ctx context.Context, req interface{}, info *grpc.UnaryServerInfo, handler grpc.UnaryHandler) (interface{}, error) { | ||
| // For each request, infuse context with our snapshot of the FulcioConfig. | ||
| // TODO(mattmoor): Consider periodically (every minute?) refreshing the ConfigMap | ||
| // from disk, so that we don't need to cycle pods to pick up config updates. | ||
| // Alternately we could take advantage of Knative's configmap watcher. | ||
| ctx = config.With(ctx, cfg) | ||
| ctx, cancel := context.WithCancel(ctx) | ||
| defer cancel() | ||
|
|
||
| // Calls the inner handler | ||
| return handler(ctx, req) | ||
| } | ||
| } | ||
|
|
||
| func createGRPCServer(cfg *config.FulcioConfig, ctClient *ctclient.LogClient, baseca ca.CertificateAuthority) (*grpcServer, error) { | ||
| logger, opts := log.SetupGRPCLogging() | ||
|
|
||
| myServer := grpc.NewServer(grpc.UnaryInterceptor( | ||
| grpcmw.ChainUnaryServer( | ||
| grpc_recovery.UnaryServerInterceptor(grpc_recovery.WithRecoveryHandlerContext(panicRecoveryHandler)), // recovers from per-transaction panics elegantly, so put it first | ||
| middleware.UnaryRequestID(middleware.UseXRequestIDMetadataOption(true), middleware.XRequestMetadataLimitOption(128)), | ||
| grpc_zap.UnaryServerInterceptor(logger, opts...), | ||
| passFulcioConfigThruContext(cfg), | ||
| grpc_prometheus.UnaryServerInterceptor, | ||
| )), | ||
| grpc.MaxRecvMsgSize(int(maxMsgSize))) | ||
|
|
||
| grpcCAServer := api.NewGRPCCAServer(ctClient, baseca) | ||
| // Register your gRPC service implementations. | ||
| gw.RegisterCAServer(myServer, grpcCAServer) | ||
|
|
||
| grpcServerEndpoint := fmt.Sprintf("%s:%s", viper.GetString("grpc-host"), viper.GetString("grpc-port")) | ||
| return &grpcServer{myServer, grpcServerEndpoint, grpcCAServer}, nil | ||
| } | ||
|
|
||
| func (g *grpcServer) setupPrometheus(reg *prometheus.Registry) { | ||
| grpcMetrics := grpc_prometheus.DefaultServerMetrics | ||
| grpcMetrics.EnableHandlingTimeHistogram() | ||
| reg.MustRegister(grpcMetrics, api.MetricLatency, api.RequestsCount) | ||
| grpc_prometheus.Register(g.Server) | ||
| } | ||
|
|
||
| func (g *grpcServer) startTCPListener() { | ||
| go func() { | ||
| lis, err := net.Listen("tcp", g.grpcServerEndpoint) | ||
| if err != nil { | ||
| log.Logger.Fatal(err) | ||
| } | ||
| defer lis.Close() | ||
|
|
||
| tcpAddr := lis.Addr().(*net.TCPAddr) | ||
| g.grpcServerEndpoint = fmt.Sprintf("%v:%d", tcpAddr.IP, tcpAddr.Port) | ||
| log.Logger.Infof("listening on grpc at %s", g.grpcServerEndpoint) | ||
|
|
||
| log.Logger.Fatal(g.Server.Serve(lis)) | ||
| }() | ||
| } | ||
|
|
||
| func (g *grpcServer) startUnixListener() { | ||
| go func() { | ||
| unixAddr, err := net.ResolveUnixAddr("unix", LegacyUnixDomainSocket) | ||
| if err != nil { | ||
| log.Logger.Fatal(err) | ||
| } | ||
| lis, err := net.ListenUnix("unix", unixAddr) | ||
| if err != nil { | ||
| log.Logger.Fatal(err) | ||
| } | ||
| defer lis.Close() | ||
|
|
||
| log.Logger.Infof("listening on grpc at %s", unixAddr.String()) | ||
|
|
||
| log.Logger.Fatal(g.Server.Serve(lis)) | ||
| }() | ||
| } | ||
|
|
||
| func createLegacyGRPCServer(cfg *config.FulcioConfig, v2Server gw.CAServer) (*grpcServer, error) { | ||
| logger, opts := log.SetupGRPCLogging() | ||
|
|
||
| myServer := grpc.NewServer(grpc.UnaryInterceptor( | ||
| grpcmw.ChainUnaryServer( | ||
| grpc_recovery.UnaryServerInterceptor(grpc_recovery.WithRecoveryHandlerContext(panicRecoveryHandler)), // recovers from per-transaction panics elegantly, so put it first | ||
| middleware.UnaryRequestID(middleware.UseXRequestIDMetadataOption(true), middleware.XRequestMetadataLimitOption(128)), | ||
| grpc_zap.UnaryServerInterceptor(logger, opts...), | ||
| passFulcioConfigThruContext(cfg), | ||
| grpc_prometheus.UnaryServerInterceptor, | ||
| )), | ||
| grpc.MaxRecvMsgSize(int(maxMsgSize))) | ||
|
|
||
| legacyGRPCCAServer := api.NewLegacyGRPCCAServer(v2Server) | ||
|
|
||
| // Register your gRPC service implementations. | ||
| gw_legacy.RegisterCAServer(myServer, legacyGRPCCAServer) | ||
|
|
||
| return &grpcServer{myServer, LegacyUnixDomainSocket, v2Server}, nil | ||
| } | ||
|
|
||
| func panicRecoveryHandler(ctx context.Context, p interface{}) error { | ||
| log.ContextLogger(ctx).Error(p) | ||
| return fmt.Errorf("panic: %v", p) | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,118 @@ | ||
| // Copyright 2022 The Sigstore Authors. | ||
| // | ||
| // Licensed under the Apache License, Version 2.0 (the "License"); | ||
| // you may not use this file except in compliance with the License. | ||
| // You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, software | ||
| // distributed under the License is distributed on an "AS IS" BASIS, | ||
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
| // | ||
|
|
||
| package app | ||
|
|
||
| import ( | ||
| "context" | ||
| "fmt" | ||
| "net/http" | ||
| "strconv" | ||
| "strings" | ||
| "time" | ||
|
|
||
| "github.com/grpc-ecosystem/grpc-gateway/v2/runtime" | ||
| "github.com/pkg/errors" | ||
| "github.com/prometheus/client_golang/prometheus/promhttp" | ||
| "github.com/sigstore/fulcio/pkg/api" | ||
| gw "github.com/sigstore/fulcio/pkg/generated/protobuf" | ||
| legacy_gw "github.com/sigstore/fulcio/pkg/generated/protobuf/legacy" | ||
| "github.com/sigstore/fulcio/pkg/log" | ||
| "google.golang.org/grpc" | ||
| "google.golang.org/grpc/credentials/insecure" | ||
| "google.golang.org/grpc/metadata" | ||
| "google.golang.org/protobuf/proto" | ||
| ) | ||
|
|
||
| type httpServer struct { | ||
| *http.Server | ||
| httpServerEndpoint string | ||
| } | ||
|
|
||
| func extractOIDCTokenFromAuthHeader(ctx context.Context, req *http.Request) metadata.MD { | ||
| token := strings.Replace(req.Header.Get("Authorization"), "Bearer ", "", 1) | ||
| return metadata.Pairs(api.MetadataOIDCTokenKey, token) | ||
| } | ||
|
|
||
| func createHTTPServer(ctx context.Context, serverEndpoint string, grpcServer, legacyGRPCServer *grpcServer) httpServer { | ||
| mux := runtime.NewServeMux(runtime.WithMetadata(extractOIDCTokenFromAuthHeader), | ||
| runtime.WithForwardResponseOption(setResponseCodeModifier)) | ||
|
|
||
| opts := []grpc.DialOption{grpc.WithTransportCredentials(insecure.NewCredentials())} | ||
| if err := gw.RegisterCAHandlerFromEndpoint(ctx, mux, grpcServer.grpcServerEndpoint, opts); err != nil { | ||
| log.Logger.Fatal(err) | ||
| } | ||
|
|
||
| if legacyGRPCServer != nil { | ||
| endpoint := fmt.Sprintf("unix:%v", legacyGRPCServer.grpcServerEndpoint) | ||
| if err := legacy_gw.RegisterCAHandlerFromEndpoint(ctx, mux, endpoint, opts); err != nil { | ||
| log.Logger.Fatal(err) | ||
| } | ||
| } | ||
|
|
||
| // Limit request size | ||
| handler := api.WithMaxBytes(mux, maxMsgSize) | ||
| handler = promhttp.InstrumentHandlerDuration(api.MetricLatency, handler) | ||
| handler = promhttp.InstrumentHandlerCounter(api.RequestsCount, handler) | ||
|
|
||
| api := http.Server{ | ||
| Addr: serverEndpoint, | ||
| Handler: handler, | ||
|
|
||
| // Timeouts | ||
| ReadTimeout: 60 * time.Second, | ||
| ReadHeaderTimeout: 60 * time.Second, | ||
| WriteTimeout: 60 * time.Second, | ||
| IdleTimeout: 60 * time.Second, | ||
| } | ||
| return httpServer{&api, serverEndpoint} | ||
| } | ||
|
|
||
| func (h httpServer) startListener() { | ||
| log.Logger.Infof("listening on http at %s", h.httpServerEndpoint) | ||
| go func() { | ||
| if err := h.ListenAndServe(); err != nil && !errors.Is(err, http.ErrServerClosed) { | ||
| log.Logger.Fatal(err) | ||
| } | ||
| }() | ||
| } | ||
|
|
||
| func setResponseCodeModifier(ctx context.Context, w http.ResponseWriter, _ proto.Message) error { | ||
| md, ok := runtime.ServerMetadataFromContext(ctx) | ||
| if !ok { | ||
| return nil | ||
| } | ||
|
|
||
| // set SCT if present ahead of modifying response code | ||
| if vals := md.HeaderMD.Get(api.SCTMetadataKey); len(vals) > 0 { | ||
| delete(md.HeaderMD, api.SCTMetadataKey) | ||
| delete(w.Header(), "Grpc-Metadata-sct") | ||
| w.Header().Set("SCT", vals[0]) | ||
| } | ||
|
|
||
| // set http status code | ||
| if vals := md.HeaderMD.Get(api.HTTPResponseCodeMetadataKey); len(vals) > 0 { | ||
| code, err := strconv.Atoi(vals[0]) | ||
| if err != nil { | ||
| return err | ||
| } | ||
| // delete the headers to not expose any grpc-metadata in http response | ||
| delete(md.HeaderMD, api.HTTPResponseCodeMetadataKey) | ||
| delete(w.Header(), "Grpc-Metadata-X-Http-Code") | ||
| w.WriteHeader(code) | ||
| } | ||
|
|
||
| return nil | ||
| } | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What determines if
legacyGRPCServeris set , does this bubble up from config?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it doesn't bubble up from config (yet), but if we wanted to deprecate and turn that off we could just set it to nil
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
gotcha, I was thinking there might be a flag/config leveraged that would decide on what runs (restAPI or gPRC (or even both)).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
hmm, interesting thought. let me noodle on that for a bit.