Support embedded use-case #754

seeforschauer · 2022-06-21T12:05:04Z

Refs #3.

In this PR added support for embedded use-case for FerretDB.
That's when it's possible to use FerretDB as a library.

codecov · 2022-06-21T12:08:35Z

Codecov Report

Merging #754 (1284034) into main (fa6d2ee) will increase coverage by 0.12%.
The diff coverage is 63.63%.

@@            Coverage Diff             @@
##             main     #754      +/-   ##
==========================================
+ Coverage   59.27%   59.39%   +0.12%     
==========================================
  Files         212      214       +2     
  Lines        9348     9418      +70     
==========================================
+ Hits         5541     5594      +53     
- Misses       3163     3173      +10     
- Partials      644      651       +7

Impacted Files	Coverage Δ
internal/clientconn/conn.go	`45.34% <ø> (ø)`
internal/clientconn/listener.go	`73.25% <ø> (-0.61%)`	⬇️
internal/handlers/registry/registry.go	`36.66% <36.66%> (ø)`
ferretdb/ferretdb.go	`79.06% <79.06%> (ø)`
integration/setup.go	`83.84% <100.00%> (-0.13%)`	⬇️
internal/handlers/pg/pg.go	`80.00% <100.00%> (ø)`
internal/handlers/pg/msg_dbstats.go	`65.11% <0.00%> (-6.98%)`	⬇️
internal/util/logging/logging.go	`86.95% <0.00%> (+30.43%)`	⬆️

Flag	Coverage Δ
FerretDB	`62.62% <100.00%> (-0.06%)`	⬇️
MongoDB	`7.22% <0.00%> (+<0.01%)`	⬆️
integration	`62.72% <100.00%> (-0.06%)`	⬇️
unit	`27.04% <62.16%> (+4.05%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

ferretdb/ferretdb_test.go

ferretdb/handler_pg.go

ferretdb/os_unix.go

ferretdb/runner.go

rumyantseva

My biggest concern is about go.uber.org/zap being imported in our library with no ability for a user of our library to configure it.

In principle, if this (and other) design improvements are for a separate PR, let's write down everything we agreed to improve? (E.g. in a separate issue, or in the current issue). So, we have a clear scope for this "another PR".

ferretdb/ferretdb.go

seeforschauer · 2022-06-28T07:59:23Z

My biggest concern is about go.uber.org/zap being imported in our library with no ability for a user of our library to configure it.

In principle, if this (and other) design improvements are for a separate PR, let's write down everything we agreed to improve? (E.g. in a separate issue, or in the current issue). So, we have a clear scope for this "another PR".

@rumyantseva sorry we are not going to improve anything here, please see the task #3.

rumyantseva

I still have a couple of questions (they are quite minor though)

seeforschauer · 2022-06-28T11:30:47Z

I still have a couple of questions (they are quite minor though)

What another question is?

rumyantseva

I don't have questions left. I will propose improvements in a separate PR after this one is merged.

AlekSi

Before this PR, the default go build result of FerretDB does not include Tigris deps such as Tigris client, gRPC, etc:

~/C/F/FerretDB (main) -> go build -o ferretdb ./cmd/ferretdb

~/C/F/FerretDB (main) -> go version -m ferretdb
ferretdb: go1.18.3
	path	github.com/FerretDB/FerretDB/cmd/ferretdb
	mod	github.com/FerretDB/FerretDB	(devel)
	dep	github.com/AlekSi/pointer	v1.2.0	h1:glcy/gc4h8HnG2Z3ZECSzZ1IX1x2JxRVuDzaJwQE0+w=
	dep	github.com/beorn7/perks	v1.0.1	h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
	dep	github.com/cespare/xxhash/v2	v2.1.2	h1:YRXhKfTDauu4ajMg1TPgFO5jnlC2HCbmLXMcTG5cbYE=
	dep	github.com/golang/protobuf	v1.5.2	h1:ROPKBNFfQgOUMifHyP+KYbvpjbdoFNs+aK7DXlji0Tw=
	dep	github.com/jackc/chunkreader/v2	v2.0.1	h1:i+RDz65UE+mmpjTfyz0MoVTnzeYxroil2G82ki7MGG8=
	dep	github.com/jackc/pgconn	v1.12.1	h1:rsDFzIpRk7xT4B8FufgpCCeyjdNpKyghZeSefViE5W8=
	dep	github.com/jackc/pgerrcode	v0.0.0-20220416144525-469b46aa5efa	h1:s+4MhCQ6YrzisK6hFJUX53drDT4UsSW3DEhKn0ifuHw=
	dep	github.com/jackc/pgio	v1.0.0	h1:g12B9UwVnzGhueNavwioyEEpAmqMe1E/BN9ES+8ovkE=
	dep	github.com/jackc/pgpassfile	v1.0.0	h1:/6Hmqy13Ss2zCq62VdNG8tM1wchn8zjSGOBJ6icpsIM=
	dep	github.com/jackc/pgproto3/v2	v2.3.0	h1:brH0pCGBDkBW07HWlN/oSBXrmo3WB0UvZd1pIuDcL8Y=
	dep	github.com/jackc/pgservicefile	v0.0.0-20200714003250-2b9c44734f2b	h1:C8S2+VttkHFdOOCXJe+YGfa4vHYwlt4Zx+IVXQ97jYg=
	dep	github.com/jackc/pgtype	v1.11.0	h1:u4uiGPz/1hryuXzyaBhSk6dnIyyG2683olG2OV+UUgs=
	dep	github.com/jackc/pgx/v4	v4.16.1	h1:JzTglcal01DrghUqt+PmzWsZx/Yh7SC/CTQmSBMTd0Y=
	dep	github.com/jackc/puddle	v1.2.1	h1:gI8os0wpRXFd4FiAY2dWiqRK037tjj3t7rKFeO4X5iw=
	dep	github.com/matttproud/golang_protobuf_extensions	v1.0.1	h1:4hp9jkHxhMHkqkrB3Ix0jegS5sx/RkqARlsWZ6pIwiU=
	dep	github.com/pmezard/go-difflib	v1.0.0	h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
	dep	github.com/prometheus/client_golang	v1.12.2	h1:51L9cDoUHVrXx4zWYlcLQIZ+d+VXHgqnYKkIuq4g/34=
	dep	github.com/prometheus/client_model	v0.2.0	h1:uq5h0d+GuxiXLJLNABMgp2qUWDPiLvgCzz2dUR+/W/M=
	dep	github.com/prometheus/common	v0.35.0	h1:Eyr+Pw2VymWejHqCugNaQXkAi6KayVNxaHeu6khmFBE=
	dep	github.com/prometheus/procfs	v0.7.3	h1:4jVXhlkAyzOScmCkXBTOLRLTz8EeU+eyjrwB/EPq0VU=
	dep	go.uber.org/atomic	v1.7.0	h1:ADUqmZGgLDDfbSL9ZmPxKTybcoEYHgpYfELNoN+7hsw=
	dep	go.uber.org/multierr	v1.6.0	h1:y6IPFStTAIT5Ytl7/XYmHvzXQ7S3g/IeZW9hyZ5thw4=
	dep	go.uber.org/zap	v1.21.0	h1:WefMeulhovoZ2sYXz7st6K0sLj7bBhpiFaud4r4zST8=
	dep	golang.org/x/crypto	v0.0.0-20220525230936-793ad666bf5e	h1:T8NU3HyQ8ClP4SEE+KbFlg6n0NhuTsN4MyznaarGsZM=
	dep	golang.org/x/exp	v0.0.0-20220613132600-b0d781184e0d	h1:vtUKgx8dahOomfFzLREU8nSv25YHnTgLBn4rDnWZdU0=
	dep	golang.org/x/sys	v0.0.0-20220615213510-4f61da869c0c	h1:aFV+BgZ4svzjfabn8ERpuB4JI4N6/rdy1iusx77G3oU=
	dep	golang.org/x/text	v0.3.7	h1:olpwvP2KacW1ZWvsR7uQhoyTYvKAupfQrRGBFM352Gk=
	dep	google.golang.org/protobuf	v1.28.0	h1:w43yiav+6bVFTBQFZX0r7ipe9JQ1QsbMgHwbBziscLw=
	build	-compiler=gc
	build	CGO_ENABLED=1
	build	CGO_CFLAGS=
	build	CGO_CPPFLAGS=
	build	CGO_CXXFLAGS=
	build	CGO_LDFLAGS=
	build	GOARCH=arm64
	build	GOOS=darwin
	build	vcs=git
	build	vcs.revision=ddaa0d1ad65cc7b879643194e25b7ee39c8e9de5
	build	vcs.time=2022-06-27T12:31:38Z
	build	vcs.modified=false

We should preserve this property.

Currently, it includes them:

~/C/F/FerretDB (pr/seeforschauer/754) -> go build -o ferretdb ./cmd/ferretdb

~/C/F/FerretDB (pr/seeforschauer/754) -> go version -m ferretdb
ferretdb/ferretdb: go1.18.3
        path    github.com/FerretDB/FerretDB/cmd/ferretdb
        mod     github.com/FerretDB/FerretDB    (devel)
        dep     cloud.google.com/go/compute     v1.6.1  h1:2sMmt8prCn7DPaG4Pmh0N3Inmc8cT8ae5k1M6VJ9Wqc=
        dep     github.com/AlekSi/pointer       v1.2.0  h1:glcy/gc4h8HnG2Z3ZECSzZ1IX1x2JxRVuDzaJwQE0+w=
        dep     github.com/beorn7/perks v1.0.1  h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
        dep     github.com/cespare/xxhash/v2    v2.1.2  h1:YRXhKfTDauu4ajMg1TPgFO5jnlC2HCbmLXMcTG5cbYE=
        dep     github.com/deepmap/oapi-codegen v1.11.0 h1:f/X2NdIkaBKsSdpeuwLnY/vDI0AtPUrmB5LMgc7YD+A=
        dep     github.com/gertd/go-pluralize   v0.2.1  h1:M3uASbVjMnTsPb0PNqg+E/24Vwigyo/tvyMTtAlLgiA=
        dep     github.com/getkin/kin-openapi   v0.94.0 h1:bAxg2vxgnHHHoeefVdmGbR+oxtJlcv5HsJJa3qmAHuo=
        dep     github.com/ghodss/yaml  v1.0.0  h1:wQHKEahhL6wmXdzwWG11gIVCkOv05bNOh+Rxn0yngAk=
        dep     github.com/go-openapi/jsonpointer       v0.19.5 h1:gZr+CIYByUqjcgeLXnQu2gHYQC9o73G2XUeOFYEICuY=
        dep     github.com/go-openapi/swag      v0.21.1 h1:wm0rhTb5z7qpJRHBdPOMuY4QjVUMbF6/kwoYeRAOrKU=
        dep     github.com/golang/protobuf      v1.5.2  h1:ROPKBNFfQgOUMifHyP+KYbvpjbdoFNs+aK7DXlji0Tw=
        dep     github.com/google/gnostic       v0.6.9  h1:ZK/5VhkoX835RikCHpSUJV9a+S3e1zLh59YnyWeBW+0=
        dep     github.com/google/uuid  v1.3.0  h1:t6JiXgmwXMjEs8VusXIJk2BXHsn+wx8BZdTaoZ5fu7I=
        dep     github.com/grpc-ecosystem/go-grpc-middleware    v1.3.0  h1:+9834+KizmvFV7pXQGSXQTsaWhq2GjuNUt0aUU0YBYw=
        dep     github.com/grpc-ecosystem/grpc-gateway/v2       v2.10.2 h1:ERKrevVTnCw3Wu4I3mtR15QU3gtWy86cBo6De0jEohg=
        dep     github.com/iancoleman/strcase   v0.2.0  h1:05I4QRnGpI0m37iZQRuskXh+w77mr6Z41lwQzuHLwW0=
        dep     github.com/jackc/chunkreader/v2 v2.0.1  h1:i+RDz65UE+mmpjTfyz0MoVTnzeYxroil2G82ki7MGG8=
        dep     github.com/jackc/pgconn v1.12.1 h1:rsDFzIpRk7xT4B8FufgpCCeyjdNpKyghZeSefViE5W8=
        dep     github.com/jackc/pgerrcode      v0.0.0-20220416144525-469b46aa5efa      h1:s+4MhCQ6YrzisK6hFJUX53drDT4UsSW3DEhKn0ifuHw=
        dep     github.com/jackc/pgio   v1.0.0  h1:g12B9UwVnzGhueNavwioyEEpAmqMe1E/BN9ES+8ovkE=
        dep     github.com/jackc/pgpassfile     v1.0.0  h1:/6Hmqy13Ss2zCq62VdNG8tM1wchn8zjSGOBJ6icpsIM=
        dep     github.com/jackc/pgproto3/v2    v2.3.0  h1:brH0pCGBDkBW07HWlN/oSBXrmo3WB0UvZd1pIuDcL8Y=
        dep     github.com/jackc/pgservicefile  v0.0.0-20200714003250-2b9c44734f2b      h1:C8S2+VttkHFdOOCXJe+YGfa4vHYwlt4Zx+IVXQ97jYg=
        dep     github.com/jackc/pgtype v1.11.0 h1:u4uiGPz/1hryuXzyaBhSk6dnIyyG2683olG2OV+UUgs=
        dep     github.com/jackc/pgx/v4 v4.16.1 h1:JzTglcal01DrghUqt+PmzWsZx/Yh7SC/CTQmSBMTd0Y=
        dep     github.com/jackc/puddle v1.2.1  h1:gI8os0wpRXFd4FiAY2dWiqRK037tjj3t7rKFeO4X5iw=
        dep     github.com/josharian/intern     v1.0.0  h1:vlS4z54oSdjm0bgjRigI+G1HpF+tI+9rE5LLzOg8HmY=
        dep     github.com/json-iterator/go     v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM=
        dep     github.com/mailru/easyjson      v0.7.7  h1:UGYAvKxe3sBsEDzO8ZeWOSlIQfWFlxbzLZe7hwFURr0=
        dep     github.com/matttproud/golang_protobuf_extensions        v1.0.1  h1:4hp9jkHxhMHkqkrB3Ix0jegS5sx/RkqARlsWZ6pIwiU=
        dep     github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd      h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg=
        dep     github.com/modern-go/reflect2   v1.0.2  h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9Gz0M=
        dep     github.com/pmezard/go-difflib   v1.0.0  h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
        dep     github.com/prometheus/client_golang     v1.12.2 h1:51L9cDoUHVrXx4zWYlcLQIZ+d+VXHgqnYKkIuq4g/34=
        dep     github.com/prometheus/client_model      v0.2.0  h1:uq5h0d+GuxiXLJLNABMgp2qUWDPiLvgCzz2dUR+/W/M=
        dep     github.com/prometheus/common    v0.35.0 h1:Eyr+Pw2VymWejHqCugNaQXkAi6KayVNxaHeu6khmFBE=
        dep     github.com/prometheus/procfs    v0.7.3  h1:4jVXhlkAyzOScmCkXBTOLRLTz8EeU+eyjrwB/EPq0VU=
        dep     github.com/tigrisdata/tigris-client-go  v1.0.0-alpha.18 h1:opvqzYQARvdWLoiZOW02+TcTF0ZujkTwJYpUNYFaqC8=
        dep     go.uber.org/atomic      v1.7.0  h1:ADUqmZGgLDDfbSL9ZmPxKTybcoEYHgpYfELNoN+7hsw=
        dep     go.uber.org/multierr    v1.6.0  h1:y6IPFStTAIT5Ytl7/XYmHvzXQ7S3g/IeZW9hyZ5thw4=
        dep     go.uber.org/zap v1.21.0 h1:WefMeulhovoZ2sYXz7st6K0sLj7bBhpiFaud4r4zST8=
        dep     golang.org/x/crypto     v0.0.0-20220525230936-793ad666bf5e      h1:T8NU3HyQ8ClP4SEE+KbFlg6n0NhuTsN4MyznaarGsZM=
        dep     golang.org/x/exp        v0.0.0-20220613132600-b0d781184e0d      h1:vtUKgx8dahOomfFzLREU8nSv25YHnTgLBn4rDnWZdU0=
        dep     golang.org/x/net        v0.0.0-20220526153639-5463443f8c37      h1:lUkvobShwKsOesNfWWlCS5q7fnbG1MEliIzwu886fn8=
        dep     golang.org/x/oauth2     v0.0.0-20220524215830-622c5d57e401      h1:zwrSfklXn0gxyLRX/aR+q6cgHbV/ItVyzbPlbA+dkAw=
        dep     golang.org/x/sys        v0.0.0-20220615213510-4f61da869c0c      h1:aFV+BgZ4svzjfabn8ERpuB4JI4N6/rdy1iusx77G3oU=
        dep     golang.org/x/text       v0.3.7  h1:olpwvP2KacW1ZWvsR7uQhoyTYvKAupfQrRGBFM352Gk=
        dep     google.golang.org/genproto      v0.0.0-20220526192754-51939a95c655      h1:56rmjc5LUAanErbiNrY+s/Nd47wDQEJkpqS7i43M1I0=
        dep     google.golang.org/grpc  v1.46.2 h1:u+MLGgVf7vRdjEYZ8wDFhAVNmhkbJ5hmrA1LMWK1CAQ=
        dep     google.golang.org/protobuf      v1.28.0 h1:w43yiav+6bVFTBQFZX0r7ipe9JQ1QsbMgHwbBziscLw=
        dep     gopkg.in/yaml.v2        v2.4.0  h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=
        dep     gopkg.in/yaml.v3        v3.0.1  h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
        build   -compiler=gc
        build   CGO_ENABLED=1
        build   CGO_CFLAGS=
        build   CGO_CPPFLAGS=
        build   CGO_CXXFLAGS=
        build   CGO_LDFLAGS=
        build   GOARCH=arm64
        build   GOOS=darwin
        build   vcs=git
        build   vcs.revision=53cbaa27022fb55a5e92701ac9706de381682268
        build   vcs.time=2022-06-28T10:38:59Z
        build   vcs.modified=false

seeforschauer · 2022-06-28T14:38:25Z

Before this PR, the default go build result of FerretDB does not include Tigris deps such as Tigris client, gRPC, etc:

well, that surprises me a bit, I didn't expect it since there is no change in build tags.. Thank you and let me check.

seeforschauer · 2022-06-28T15:13:56Z

Before this PR, the default go build result of FerretDB does not include Tigris deps such as Tigris client, gRPC, etc:

@AlekSi I solved it and it's not beautiful. But now it doesn't include Tigris deps.

AlekSi · 2022-06-29T06:24:59Z

I solved it and it's not beautiful.

It wasn't good, yeah. So why was it submitted for review?

I thought I could simplify it a bit and ended up cleaning up the whole PR. @noisersup, @rumyantseva, @w84thesun, @seeforschauer PTAL. We are almost done.

seeforschauer · 2022-06-29T07:12:41Z

I solved it and it's not beautiful.

It wasn't good, yeah. So why was it submitted for review?

didn't find a more beautiful solution.

seeforschauer added 2 commits June 21, 2022 15:04

wip

db37e83

Merge branch 'main' into issue-3-embedded

8f4ac17

seeforschauer linked an issue Jun 21, 2022 that may be closed by this pull request

Support embedded use-case #3

Closed

seeforschauer self-assigned this Jun 21, 2022

seeforschauer added 9 commits June 21, 2022 15:45

Merge branch 'main' into issue-3-embedded

ad0f272

wip

4cd48a8

example run

3c6ec9f

example run

3cceb7e

Merge branch 'main' into issue-3-embedded

7e5fb80

lint

d02178d

Merge branch 'main' into issue-3-embedded

1b2235f

comments

bc8dbd4

comments

8cff125

seeforschauer marked this pull request as ready for review June 21, 2022 15:13

seeforschauer requested a review from AlekSi as a code owner June 21, 2022 15:13

seeforschauer requested review from noisersup, rumyantseva and w84thesun and removed request for w84thesun and noisersup June 21, 2022 15:13

AlekSi requested changes Jun 21, 2022

View reviewed changes

seeforschauer added 5 commits June 21, 2022 19:03

extract

dde09fc

Merge branch 'main' into issue-3-embedded

4228269

change ExampleRun to mongodb conn string

8daa497

change ExampleRun to mongodb conn string

05920ab

remove run

5ebd026

seeforschauer requested review from noisersup and w84thesun June 22, 2022 09:25

improve comment

9b8712e

Merge branch 'main' into issue-3-embedded

879b959

w84thesun previously approved these changes Jun 27, 2022

View reviewed changes

Merge branch 'main' into issue-3-embedded

d827463

rumyantseva requested changes Jun 28, 2022

View reviewed changes

ferretdb/ferretdb.go Outdated Show resolved Hide resolved

ferretdb/ferretdb.go Outdated Show resolved Hide resolved

ferretdb/ferretdb.go Outdated Show resolved Hide resolved

seeforschauer requested a review from rumyantseva June 28, 2022 07:59

log level fatal

53cbaa2

seeforschauer dismissed w84thesun’s stale review via 53cbaa2 June 28, 2022 10:39

seeforschauer requested a review from w84thesun June 28, 2022 10:40

rumyantseva reviewed Jun 28, 2022

View reviewed changes

w84thesun previously approved these changes Jun 28, 2022

View reviewed changes

rumyantseva reviewed Jun 28, 2022

View reviewed changes

rumyantseva self-requested a review June 28, 2022 12:44

AlekSi requested changes Jun 28, 2022

View reviewed changes

build configuration

a3717b5

seeforschauer dismissed w84thesun’s stale review via a3717b5 June 28, 2022 15:12

seeforschauer requested review from w84thesun and AlekSi June 28, 2022 15:24

Simplify

1284034

w84thesun approved these changes Jun 29, 2022

View reviewed changes

rumyantseva approved these changes Jun 29, 2022 •

edited

Loading

View reviewed changes

AlekSi disabled auto-merge June 29, 2022 11:01

AlekSi approved these changes Jun 29, 2022

View reviewed changes

AlekSi merged commit 59d9a5d into FerretDB:main Jun 29, 2022

AlekSi added the code/feature Some user-visible feature is not implemented yet label Jul 11, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support embedded use-case #754

Support embedded use-case #754

seeforschauer commented Jun 21, 2022 •

edited

Loading

codecov bot commented Jun 21, 2022 •

edited

Loading

rumyantseva left a comment •

edited

Loading

seeforschauer commented Jun 28, 2022

rumyantseva left a comment

seeforschauer commented Jun 28, 2022

rumyantseva left a comment

AlekSi left a comment

seeforschauer commented Jun 28, 2022 •

edited

Loading

seeforschauer commented Jun 28, 2022 •

edited

Loading

AlekSi commented Jun 29, 2022

seeforschauer commented Jun 29, 2022

Support embedded use-case #754

Support embedded use-case #754

Conversation

seeforschauer commented Jun 21, 2022 • edited Loading

codecov bot commented Jun 21, 2022 • edited Loading

Codecov Report

rumyantseva left a comment • edited Loading

Choose a reason for hiding this comment

seeforschauer commented Jun 28, 2022

rumyantseva left a comment

Choose a reason for hiding this comment

seeforschauer commented Jun 28, 2022

rumyantseva left a comment

Choose a reason for hiding this comment

AlekSi left a comment

Choose a reason for hiding this comment

seeforschauer commented Jun 28, 2022 • edited Loading

seeforschauer commented Jun 28, 2022 • edited Loading

AlekSi commented Jun 29, 2022

seeforschauer commented Jun 29, 2022

seeforschauer commented Jun 21, 2022 •

edited

Loading

codecov bot commented Jun 21, 2022 •

edited

Loading

rumyantseva left a comment •

edited

Loading

seeforschauer commented Jun 28, 2022 •

edited

Loading

seeforschauer commented Jun 28, 2022 •

edited

Loading