mongo Query is a library that makes it easy to build MongoDB queries in Golang
Formulating requests with the MongoDB API is sometimes very difficult and error-prone. You have to build and nest untyped objects (bson.M, bson.D, ...
). This will quickly become confusing and therefore difficult to read and maintain. mongo Query solves this problem with an easier to understand (DSL like) API.
For example, the following 'mongo Query' expression finds all documents in a MongoDB database where the array of 'amenities' has a size of 15 and the 'ListingUrl' equals '' and the 'pictureUrl' of the 'image' equals '':
Listing.Amenities.ArraySize(15).
And(Listing.ListingUrl.Equals("<value>"),
Listing.Images.PictureUrl.Equals("<value>"))
The (DSL-) queries are created with 'Filter structs' which can be generated using a commandline tool, which is also part of this project (see Generating filter types).
The API documentation for the filter structs can be found here. It is based on the 'native MongoDB query API'.
The following example shows how the queries differ between the MongoDB API and mongo Query. First of all, the struct with which the data is mapped in the MongoDB database:
type ListingAndReview struct {
ListingUrl string `bson:"listing_url"`
Name string `bson:"name"`
Bathrooms primitive.Decimal128 `bson:"bathrooms"`
Amenities []string `bson:"amenities"`
Images struct {
ThumbnailUrl string `bson:"thumbnail_url"`
MediumUrl string `bson:"medium_url"`
PictureUrl string `bson:"picture_url"`
XlPictureUrl string `bson:"xl_picture_url"`
} `bson:"images"`
}
The data and data structures used in the examples come from a freely available Airbnb example database. You will find the complete ListingAndReview struct in this repo (it's only a subset in the sample above).
If you want to query the MongoDB collection and search for values with a specific ListingUrl
and Name
with the MongoDB-API, you have to write the following:
filter := bson.D{{"$and", []bson.D{
{
{"listing_url", "https://www.airbnb.com/rooms/10009999"},
},
{
{"name", "Horto flat with small garden"},
},
}}}
In this example you have to:
- build the
bson.D
structure right (see the typ, slice, ...) - know about the MongoDB query operators (
"$and"
) - know the BSON struct tags for all your fields (
"listing_url"
,"name"
) - cross your fingers that no one will change the BSON mapping
In contrast, the same query with mongo Query looks like this:
filter := Listing.ListingUrl.Equals("https://www.airbnb.com/rooms/10009999").
And(Listing.Name.Equals("Horto flat with small garden"))
It is:
- readable
- expressive
- easy to write
- independent of the
bson.D
structure - resistant to renaming BSON field names
mongo Query uses its own simple API with Expression
, Field
and Operator
types.
You can simply define a filter type for each struct you want to query. Instances of this filter types can than be used as parameter to the MongoDB API. They will automatically be marshalled as MongoDB bson.D
objects.
You can generate this filter types with a generator which is also part of this project! See below.
type ListingFilter struct {
ListingUrl Field
Name Field
Bedrooms Field
Amenities ArrayField
Images ImagesFilter
}
type ImagesFilter struct {
ThumbnailUrl Field
MediumUrl Field
PictureUrl Field
XlPictureUrl Field
}
var Listing = ListingFilter{
ListingUrl: Field("listing_url"),
Name: Field("name"),
Bedrooms: Field("bedrooms"),
Amenities: ArrayField("amenities"),
Images: ImagesFilter{
ThumbnailUrl: Field("images.thumbnail_url"),
MediumUrl: Field("images.medium_url"),
PictureUrl: Field("images.picture_url"),
XlPictureUrl: Field("images.xl_picture_url"),
},
}
And then you can use the filter and query data via the MongoDB API:
...
filter := Listing.ListingUrl.Equals("https://www.airbnb.com/rooms/10009999")
collection := client.Database("airbnb").Collection("listingsAndReviews")
cursor, err := collection.Find(ctx, filter)
...
or with embedded documents...
...
filter := Listing.Amenities.ArraySize(15).
And(Listing.ListingUrl.Equals("<value>"),
Listing.Images.PictureUrl.Equals("<value>"))
collection := client.Database("airbnb").Collection("listingsAndReviews")
cursor, err := collection.Find(ctx, filter)
...
You will find a complete sample within the unit tests of this project.
Defining filter types is easy. Just use the generator which is also included in the project. Install it via go install
and use it (see an example here):
go install github.com/sourcefellows/mongo-query/cmd/mongo-query-gen@latest
kkoehler@project$ mongo-query-gen
no input file given
-in string
path to file with Golang structs
-only string
list of struct names - only given struct names will be used for code generation
-outDir string
path to output directory - a subdirectory "filter" will be generated automatically
For example:
mongo-query-gen -in Types.go -outDir .
- Query embedded documents (Specify Equality Match on a Nested Field) (see here or local impl)
//MongoDB API
err = findWithFilter(ctx, collection, bson.D{{"size.uom", "in"}})
//mongo Query
err = findWithFilter(ctx, collection, InventoryFilter.Size.Uom.Equals("in"))
- Query embedded documents (Specify AND Condition) (see here or local impl)
//MongoDB API
err = findWithFilter(ctx, collection, bson.D{
{"size.h", bson.D{
{"$lt", 15},
}},
{"size.uom", "in"},
{"status", "D"},
})
//mongo Query
err = findWithFilter(ctx, collection,
InventoryFilter.Size.H.Lt(15).
And(InventoryFilter.Size.Uom.Equals("in"),
InventoryFilter.Status.Equals("D")))
- More samples can be found in the unit tests of the project.