-
Notifications
You must be signed in to change notification settings - Fork 5
MongoDB
Дар'я Харлан edited this page Dec 28, 2022
·
7 revisions
Supported versions: MongoDB 3.0+
-
Connection string - database connection string e.g.
mongodb://mongo:27017
- Username
- Password
- Authentication Source - for delegated authentication, specify an alternate database name. Leave blank for normal authentication
- Database
- Collection
-
Offset type -
OBJECTID
,STRING
orDATE
. Default -OBJECTID
-
Initial offset - Number of days ago or specific date (format
YYYY-MM-DD HH:mm:ss
) when to start pulling data from (default 3 days ago). -
Offset field - default
_id
- Batch size - how many records to send to further pipeline stages. Default - 1000 records
- Max batch wait time (seconds) - how much time to wait until the batch reaches its size. Default - 5 seconds
> agent source create
Choose source (mongo, kafka, influx): mongo
Enter unique name for this source config: mongo_test
Connection string: mongodb://mongo:27017
Username []:
Password []:
Authentication Source []:
Database: test
Collection: transactions
Is collection capped [False]:
Initial offset: 0
Offset type (OBJECTID, STRING, DATE) [OBJECTID]:
Offset field [_id]:
Batch size [1000]:
Max batch wait time (seconds) [5]:
Source config created
Property | Type | Description |
---|---|---|
type |
String | Specify source type. Value - influx
|
name |
String | Unique source name - also the config filename |
config |
Object | Source configuration |
All properties are required
Property | Type | Required | Description |
---|---|---|---|
configBean.mongoConfig.connectionString | String | yes | database connection string e.g. mongodb://mongo:27017
|
configBean.mongoConfig.username | String | no | username |
configBean.mongoConfig.password | String | no | password |
configBean.mongoConfig.authSource | String | no | for delegated authentication, specify alternate database name. Leave blank for normal authentication |
configBean.mongoConfig.database | String | yes | database |
configBean.mongoConfig.collection | String | yes | Collection |
configBean.offsetType | String | no |
OBJECTID , STRING or DATE . Default - OBJECTID
|
configBean.initialOffset | String | no | Amount of days ago or specific date (format YYYY-MM-DD HH:mm:ss) when to start pulling data from (default 3 days ago) |
configBean.offsetField | String | no | default _id
|
configBean.batchSize | String | no | how many records to send to further pipeline stages. Default - 1000 records |
configBean.maxBatchWaitTime | String | no | how many time to wait until batch will reach it's size. Default - 5 seconds |
Example
[
{
"type": "mongo",
"name": "test_mongo_1",
"config": {
"configBean.mongoConfig.connectionString": "mongodb://mongo:27017",
"configBean.mongoConfig.database": "test",
"configBean.mongoConfig.collection": "adtech",
"configBean.mongoConfig.username": "root",
"configBean.mongoConfig.password": "root",
"configBean.mongoConfig.authSource": "admin"
}
}
]
- Home
- CLI reference
- API
- Kubernetes setup using Helm
- Podman setup
- Creating pipelines
- Test sources
- Data formats (JSON, CSV, AVRO, LOG)
- How to parse logs with grok patterns
- How to store sensitive information
- Automated pipelines creation
- Filtering
- Transformation files
- Fields
- DVP Configuration
- Integrations
- Sending events to Anodot