Add documentation, prepare release (#1732)

Closes #1694.
FerretDB · Jan 2, 2023 · c323091 · c323091 · vercel · Jan 2, 2023
1 parent e6fac0f
commit c323091
Show file tree

Hide file tree

Showing 17 changed files with 643 additions and 555 deletions.
diff --git a/.textlintignore b/.textlintignore
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,26 +1,76 @@
 # Changelog
 
-## v0.8.0 (to be released)
+## [v0.8.0](https://github.com/FerretDB/FerretDB/releases/tag/v0.8.0) (2023-01-02)
 
 ### What's Changed
 
-In this release, we made two big changes in the way FerretDB stores data in PostgreSQL.
+We are pleased to announce our first Beta release!
 
-The first change is about the format we use to store documents.
+#### Storage changes for PostgreSQL
 
-Previously, we were storing information about data types in the fields themselves.
-Starting from this release, we store information about data types (document's schema) in a special field.
+We made a few _backward-incompatible_ changes in the way we store data in PostgreSQL to improve FerretDB performance.
+In the future, those changes will allow us to use indexes and query collections faster.
 
-This will allow us to implement more query push downs in the future.
+To keep your data:
 
-The second change is about the way FerretDB stores metadata about the collections.
+* backup FerretDB databases using `mongodump` or `mongoexport`;
+* backup PostgreSQL database using `pg_dump` or other tool (just in case);
+* stop FerretDB;
+* drop PostgreSQL views for FerretDB databases;
+* start FerretDB 0.8;
+* restore databases using `mongorestore` or `mongoimport`.
 
-Starting from this release, we store metadata about collections in separate rows of the settings table.
+#### Authentication
 
-This will allow us to have fewer locks on the settings table and improve performance.
+It is now possible to use the backend's authentication mechanisms in FerretDB.
+See [documentation](https://docs.ferretdb.io/security/).
 
-There are no changes in the API, but since the data is stored in a different way, **this change is not backward compatible**.
-Please make a dump of your database before upgrading, delete the databases, upgrade FerretDB, and restore the dump afterwards.
+### New Features 🎉
+* Support `$min` field update operator by @chilagrow in https://github.com/FerretDB/FerretDB/pull/1652
+* Support `ordered` argument for `insert` command by @noisersup in https://github.com/FerretDB/FerretDB/pull/1673
+* Implement authentication for PostgreSQL by @AlekSi in https://github.com/FerretDB/FerretDB/pull/1725
+
+### Fixed Bugs 🐛
+* Fix unset document being updated by invalid value of `$inc` by @chilagrow in https://github.com/FerretDB/FerretDB/pull/1685
+
+### Enhancements 🛠
+* Update building documentation by @AlekSi in https://github.com/FerretDB/FerretDB/pull/1713
+
+### Documentation 📄
+* Add section for comparison and logical query operators by @Fashander in https://github.com/FerretDB/FerretDB/pull/1647
+* Add documentation for element query operators by @Fashander in https://github.com/FerretDB/FerretDB/pull/1675
+* Add documentation for array query operator by @Fashander in https://github.com/FerretDB/FerretDB/pull/1695
+* Enable blog post section by @Fashander in https://github.com/FerretDB/FerretDB/pull/1700
+
+### Other Changes 🤖
+* Simplify release procedure by @AlekSi in https://github.com/FerretDB/FerretDB/pull/1657
+* Modify `pjson` format by @rumyantseva in https://github.com/FerretDB/FerretDB/pull/1620
+* Bump deps by @AlekSi in https://github.com/FerretDB/FerretDB/pull/1664
+* Remove leading space from `SELECT` queries by @noisersup in https://github.com/FerretDB/FerretDB/pull/1665
+* Add `InTransactionRetry` helper by @rumyantseva in https://github.com/FerretDB/FerretDB/pull/1670
+* Add `mongo` test script example by @AlekSi in https://github.com/FerretDB/FerretDB/pull/1600
+* Use faster runner instances by @AlekSi in https://github.com/FerretDB/FerretDB/pull/1678
+* Update issue templates by @w84thesun in https://github.com/FerretDB/FerretDB/pull/1671
+* Bump Tigris version by @AlekSi in https://github.com/FerretDB/FerretDB/pull/1680
+* Move update tests to compat tests by @chilagrow in https://github.com/FerretDB/FerretDB/pull/1659
+* Add TODO comments by @AlekSi in https://github.com/FerretDB/FerretDB/pull/1687
+* Add `saslStart` stub by @w84thesun in https://github.com/FerretDB/FerretDB/pull/1649
+* Improve the way of storing data about collections by @rumyantseva in https://github.com/FerretDB/FerretDB/pull/1650
+* Implement `iterator.Interface` for `types.Document` and `types.Array` by @w84thesun in https://github.com/FerretDB/FerretDB/pull/1683
+* Improve issue template by @AlekSi in https://github.com/FerretDB/FerretDB/pull/1692
+* Remove `$elemMatch` and `$slice` projection operators by @chilagrow in https://github.com/FerretDB/FerretDB/pull/1698
+* Add `currentOp` stub by @chilagrow in https://github.com/FerretDB/FerretDB/pull/1708
+* Add basic benchmark for query pushdowns by @noisersup in https://github.com/FerretDB/FerretDB/pull/1689
+* Enable authentication in PostgreSQL by @AlekSi in https://github.com/FerretDB/FerretDB/pull/1716
+* Fix Docker build by @AlekSi in https://github.com/FerretDB/FerretDB/pull/1715
+* Minor refactorings of iterators by @AlekSi in https://github.com/FerretDB/FerretDB/pull/1718
+* Test `ordered` argument validation by @rumyantseva in https://github.com/FerretDB/FerretDB/pull/1719
+* Add stub for getting client-specific connection by @AlekSi in https://github.com/FerretDB/FerretDB/pull/1723
+* Add compat tests for `InsertOne` in addition to `InsertMany` by @rumyantseva in https://github.com/FerretDB/FerretDB/pull/1726
+* Run `govulncheck` on CI by @noisersup in https://github.com/FerretDB/FerretDB/pull/1729
+
+[All closed issues and pull requests](https://github.com/FerretDB/FerretDB/milestone/28?closed=1).
+[All commits](https://github.com/FerretDB/FerretDB/compare/v0.7.1...v0.8.0).
 
 
 ## [v0.7.1](https://github.com/FerretDB/FerretDB/releases/tag/v0.7.1) (2022-12-19)
@@ -551,7 +601,8 @@ We are pleased to announce our first Alpha release!
 ## [v0.5.0](https://github.com/FerretDB/FerretDB/releases/tag/v0.5.0) (2022-07-11)
 
 ### What's Changed
-This release enables the usage of FerretDB as a Go library. See [this blog post](https://www.ferretdb.io/0-5-0-release-is-out-embedding-ferretdb-into-go-programs/).
+This release enables the usage of FerretDB as a Go library.
+See [this blog post](https://www.ferretdb.io/0-5-0-release-is-out-embedding-ferretdb-into-go-programs/).
 
 ### New Features 🎉
 * Support embedded use-case by @seeforschauer in https://github.com/FerretDB/FerretDB/pull/754
@@ -1064,7 +1115,7 @@ In short, we want FerretDB to be compatible with MongoDB first and fast second,
 * Support basic `hostInfo` command by @ekalinin in https://github.com/FerretDB/FerretDB/pull/188
 * Support `collStats` command by @ekalinin in https://github.com/FerretDB/FerretDB/pull/206
 ### Fixed Bugs 🐛
-* Accept $ and . in object field names by @AlekSi in https://github.com/FerretDB/FerretDB/pull/127
+* Accept $ and `.` in object field names by @AlekSi in https://github.com/FerretDB/FerretDB/pull/127
 * Make `checkConnection` less strict for common UTF8 localizations by @klokar in https://github.com/FerretDB/FerretDB/pull/135
 * Wait for PostgreSQL on `make env-up` by @agneum in https://github.com/FerretDB/FerretDB/pull/149
 * Fix build info parsing by @AlekSi in https://github.com/FerretDB/FerretDB/pull/205
@@ -1082,7 +1133,7 @@ In short, we want FerretDB to be compatible with MongoDB first and fast second,
 * Build multi-arch Docker images by @AlekSi in https://github.com/FerretDB/FerretDB/pull/107
 * Verify modules on `make init` by @AlekSi in https://github.com/FerretDB/FerretDB/pull/123
 * Enable go-consistent linter by @AlekSi in https://github.com/FerretDB/FerretDB/pull/124
-* Use composite GitHub Action for Go setup. (#122) by @klokar in https://github.com/FerretDB/FerretDB/pull/126
+* Use composite GitHub Action for Go setup in https://github.com/FerretDB/FerretDB/pull/126
 * Use shared setup-go action by @AlekSi in https://github.com/FerretDB/FerretDB/pull/131
 * Add an option to use read-only user in tests by @AlekSi in https://github.com/FerretDB/FerretDB/pull/132
 * Refactor handler tests by @AlekSi in https://github.com/FerretDB/FerretDB/pull/136
@@ -1120,7 +1171,7 @@ In short, we want FerretDB to be compatible with MongoDB first and fast second,
 
 ## [v0.0.4](https://github.com/FerretDB/FerretDB/releases/tag/v0.0.4) (2021-12-01)
 
-* A new name! ([see here](https://github.com/FerretDB/FerretDB/discussions/100))
+* A new name in https://github.com/FerretDB/FerretDB/discussions/100
 * Added support for databases sizes in `listDatabases` command ([#61](https://github.com/FerretDB/FerretDB/issues/61), thanks to [Leigh](https://github.com/OpenSauce)).
 
 [All closed issues and pull requests](https://github.com/FerretDB/FerretDB/milestone/3?closed=1).
@@ -1129,8 +1180,8 @@ In short, we want FerretDB to be compatible with MongoDB first and fast second,
 
 ## [v0.0.3](https://github.com/FerretDB/FerretDB/releases/tag/v0.0.3) (2021-11-19)
 
-* Added support for `$regex` evaluation query operator. ([#28](https://github.com/FerretDB/FerretDB/issues/28))
-* Fixed handling of Infinity, -Infinity, NaN BSON Double values. ([#29](https://github.com/FerretDB/FerretDB/issues/29))
+* Added support for `$regex` evaluation query operator in https://github.com/FerretDB/FerretDB/issues/28
+* Fixed handling of Infinity, -Infinity, NaN BSON Double values in https://github.com/FerretDB/FerretDB/issues/29
 * Improved documentation for contributors (thanks to [Leigh](https://github.com/OpenSauce)).
 
 [All closed issues and pull requests](https://github.com/FerretDB/FerretDB/milestone/2?closed=1).
@@ -1139,8 +1190,8 @@ In short, we want FerretDB to be compatible with MongoDB first and fast second,
 
 ## [v0.0.2](https://github.com/FerretDB/FerretDB/releases/tag/v0.0.2) (2021-11-13)
 
-* Added support for comparison query operators: `$eq`,`$gt`,`$gte`,`$in`,`$lt`,`$lte`,`$ne`,`$nin`. ([#26](https://github.com/FerretDB/FerretDB/issues/26))
-* Added support for logical query operators: `$and`, `$not`, `$nor`, `$or`. ([#27](https://github.com/FerretDB/FerretDB/issues/27))
+* Added support for comparison query operators: `$eq`,`$gt`,`$gte`,`$in`,`$lt`,`$lte`,`$ne`,`$nin` in https://github.com/FerretDB/FerretDB/issues/26
+* Added support for logical query operators: `$and`, `$not`, `$nor`, `$or` in https://github.com/FerretDB/FerretDB/issues/27
 
 [All closed issues and pull requests](https://github.com/FerretDB/FerretDB/milestone/1?closed=1).
 [All commits](https://github.com/FerretDB/FerretDB/compare/v0.0.1...v0.0.2).

diff --git a/build/version/version.go b/build/version/version.go
@@ -34,8 +34,9 @@
 // # Debug builds
 //
 // Debug builds of FerretDB behave differently in a few aspects:
-//   - The default logging level is set to debug.
 //   - Some internal errors cause crashes instead of being handled more gracefully.
+//   - Metrics are written to stderr on exit.
+//   - The default logging level is set to debug.
 package version
 
 import (

diff --git a/cmd/ferretdb/main.go b/cmd/ferretdb/main.go
@@ -215,6 +215,20 @@ func runTelemetryReporter(ctx context.Context, opts *telemetry.NewReporterOpts)
 	r.Run(ctx)
 }
 
+// dumpMetrics dumps all Prometheus metrics to stderr.
+func dumpMetrics() {
+	mfs, err := prometheus.DefaultGatherer.Gather()
+	if err != nil {
+		panic(err)
+	}
+
+	for _, mf := range mfs {
+		if _, err := expfmt.MetricFamilyToText(os.Stderr, mf); err != nil {
+			panic(err)
+		}
+	}
+}
+
 // run sets up environment based on provided flags and runs FerretDB.
 func run() {
 	if cli.Version {
@@ -318,15 +332,11 @@ func run() {
 		logger.Error("Listener stopped", zap.Error(err))
 	}
 
+	stop()
+
 	wg.Wait()
 
-	mfs, err := prometheus.DefaultGatherer.Gather()
-	if err != nil {
-		panic(err)
-	}
-	for _, mf := range mfs {
-		if _, err := expfmt.MetricFamilyToText(os.Stderr, mf); err != nil {
-			panic(err)
-		}
+	if version.Get().DebugBuild {
+		dumpMetrics()
 	}
 }
diff --git a/integration/query_compat_test.go b/integration/query_compat_test.go
@@ -28,10 +28,11 @@ import (
 
 // queryCompatTestCase describes query compatibility test case.
 type queryCompatTestCase struct {
-	filter     bson.D                   // required
-	sort       bson.D                   // defaults to `bson.D{{"_id", 1}}`
-	projection bson.D                   // nil for leaving projection unset
-	resultType compatTestCaseResultType // defaults to nonEmptyResult
+	filter         bson.D                   // required
+	sort           bson.D                   // defaults to `bson.D{{"_id", 1}}`
+	projection     bson.D                   // nil for leaving projection unset
+	resultType     compatTestCaseResultType // defaults to nonEmptyResult
+	resultPushdown bool                     // TODO https://github.com/FerretDB/FerretDB/issues/1279
 }
 
 // testQueryCompat tests query compatibility test cases.
@@ -69,6 +70,10 @@ func testQueryCompat(t *testing.T, testCases map[string]queryCompatTestCase) {
 				t.Run(targetCollection.Name(), func(t *testing.T) {
 					t.Helper()
 
+					// Run `explain` on `targetCollection` only, check response's `pushdown` with tc.resultPushdown
+					// https://github.com/FerretDB/FerretDB/issues/1279
+					_ = tc.resultPushdown
+
 					targetCursor, targetErr := targetCollection.Find(ctx, filter, opts)
 					compatCursor, compatErr := compatCollection.Find(ctx, filter, opts)
 

diff --git a/internal/clientconn/listener.go b/internal/clientconn/listener.go
@@ -86,7 +86,7 @@ func (l *Listener) Run(ctx context.Context) error {
 	if l.Listener.Addr != "" {
 		var err error
 		if l.tcpListener, err = net.Listen("tcp", l.Listener.Addr); err != nil {
-			return lazyerrors.Error(err)
+			return err
 		}
 
 		close(l.tcpListenerReady)
@@ -97,7 +97,7 @@ func (l *Listener) Run(ctx context.Context) error {
 	if l.Listener.Unix != "" {
 		var err error
 		if l.unixListener, err = net.Listen("unix", l.Listener.Unix); err != nil {
-			return lazyerrors.Error(err)
+			return err
 		}
 
 		close(l.unixListenerReady)
@@ -108,7 +108,7 @@ func (l *Listener) Run(ctx context.Context) error {
 	if l.Listener.TLS != "" {
 		var err error
 		if l.tlsListener, err = setupTLSListener(l.Listener.TLS, l.Listener.TLSCertFile, l.Listener.TLSKeyFile); err != nil {
-			return lazyerrors.Error(err)
+			return err
 		}
 
 		close(l.tlsListenerReady)
@@ -183,31 +183,23 @@ func (l *Listener) Run(ctx context.Context) error {
 // setupTLSListener returns a new TLS listener or and error.
 func setupTLSListener(addr, certFile, keyFile string) (net.Listener, error) {
 	if _, err := os.Stat(certFile); err != nil {
-		if os.IsNotExist(err) {
-			return nil, fmt.Errorf("certificate file %q does not exist", certFile)
-		}
-
-		return nil, lazyerrors.Error(err)
+		return nil, fmt.Errorf("TLS certificate file: %w", err)
 	}
 
 	if _, err := os.Stat(keyFile); err != nil {
-		if os.IsNotExist(err) {
-			return nil, lazyerrors.Errorf("TLS key file %q does not exist", keyFile)
-		}
-
-		return nil, lazyerrors.Error(err)
+		return nil, fmt.Errorf("TLS key file: %w", err)
 	}
 
-	cer, err := tls.LoadX509KeyPair(certFile, keyFile)
+	cert, err := tls.LoadX509KeyPair(certFile, keyFile)
 	if err != nil {
-		return nil, lazyerrors.Error(err)
+		return nil, err
 	}
 
 	config := tls.Config{
 		// TODO ClientAuth, ClientCAs, maybe something else
 		// https://github.com/FerretDB/FerretDB/issues/1707
 
-		Certificates: []tls.Certificate{cer},
+		Certificates: []tls.Certificate{cert},
 	}
 
 	listener, err := tls.Listen("tcp", addr, &config)

diff --git a/internal/handlers/common/limit.go b/internal/handlers/common/limit.go
@@ -27,7 +27,6 @@ func LimitDocuments(docs []*types.Document, limit int64) ([]*types.Document, err
 		}
 		return docs[:limit], nil
 	default:
-		// TODO https://github.com/FerretDB/FerretDB/issues/79
 		return nil, NewCommandErrorMsg(ErrNotImplemented, "LimitDocuments: negative limit values are not supported")
 	}
 }
diff --git a/internal/handlers/pg/msg_explain.go b/internal/handlers/pg/msg_explain.go
@@ -100,6 +100,7 @@ func (h *Handler) MsgExplain(ctx context.Context, msg *wire.OpMsg) (*wire.OpMsg,
 			"queryPlanner", queryPlanner,
 			"explainVersion", "1",
 			"command", cmd,
+			"pushdown", false, // TODO https://github.com/FerretDB/FerretDB/issues/1279
 			"serverInfo", serverInfo,
 			"ok", float64(1),
 		))},

diff --git a/internal/handlers/tigris/msg_explain.go b/internal/handlers/tigris/msg_explain.go
@@ -79,6 +79,7 @@ func (h *Handler) MsgExplain(ctx context.Context, msg *wire.OpMsg) (*wire.OpMsg,
 			"queryPlanner", queryPlanner,
 			"explainVersion", "1",
 			"command", cmd,
+			"pushdown", false, // TODO https://github.com/FerretDB/FerretDB/issues/1279
 			"serverInfo", serverInfo,
 			"ok", float64(1),
 		))},

diff --git a/website/blog/_guide/template.md b/website/blog/_guide/template.md
@@ -1,15 +1,15 @@
 ---
 title: FerretDB Blog post
-slug: 
-author: 
+slug:
+author:
 author_title:
-author_url: 
-author_image_url: 
+author_url:
+author_image_url:
 description: This is a decription of a FerretDB blog post.
 keywords:
   - keyword1
   - keyword2
-image: 
+image:
 tags: [tag1, tag2]
 ---
 

diff --git a/website/docs/basic_operations/update.md b/website/docs/basic_operations/update.md
@@ -9,12 +9,12 @@ FerretDB supports update operators, such as `$set` and `$setOnInsert` to update
 
 At present, FerretDB currently supports the following update operators:
 
-| Operator name | Description |
-| --- | --- |
-| `$set` | Assigns the value for an updated field to the document. |
+| Operator name  | Description                                                                                                                                                        |
+| -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
+| `$set`         | Assigns the value for an updated field to the document.                                                                                                            |
 | `$setOnInsert` | Specifies the value of a field when an update operation results in the addition of a document.  However, there is no effect when it modifies an existing document. |
-| `$unset` | Removes a specific field from a document. |
-| `$pop` | In an array, this operator removes the first or last item. |
+| `$unset`       | Removes a specific field from a document.                                                                                                                          |
+| `$pop`         | In an array, this operator removes the first or last item.                                                                                                         |
 
 ## Update a single document
 
@@ -163,7 +163,7 @@ The command will query the second field of the array in every document for `"pen
 ```js
 db.employees.updateMany({
     "catalog.1": "pencils"
-}, 
+},
 {
     $set: {
         "catalog.0": "ruler"
@@ -191,7 +191,7 @@ The following operation updates any embedded document that matches the specified
 ```js
 db.employees.updateMany({
     "name.first": "Clarke"
-}, 
+},
 {
     $set: {
         "name.last": "Elliot"

diff --git a/website/docs/reference/operators/array-operators.md b/website/docs/reference/operators/array-operators.md
@@ -6,11 +6,11 @@ sidebar_position: 3
 
 Array query operators allow you to search for specific elements within an array field in a document.
 
-|Operator|Description|
-|-------|-----------|
-|[`$all`](#all)|Matches an array that contains all the elements in the specified query|
-|[`$elemMatch`](#elemmatch)|Matches a document that contains an array field with at least one element that matches all the specified query criteria|
-|[`$size`](#size)|Matches an array with a specified number of elements|
+| Operator                   | Description                                                                                                             |
+| -------------------------- | ----------------------------------------------------------------------------------------------------------------------- |
+| [`$all`](#all)             | Matches an array that contains all the elements in the specified query                                                  |
+| [`$elemMatch`](#elemmatch) | Matches a document that contains an array field with at least one element that matches all the specified query criteria |
+| [`$size`](#size)           | Matches an array with a specified number of elements                                                                    |
 
 For the examples in this section, insert the following documents into the `team` collection: