Node-RED DocumentDB Connector

A comprehensive Node-RED package for connecting to DocumentDB and MongoDB databases with support for all common operations and a visual aggregation pipeline builder.

Features

DocumentDB Connector Node

• Easy Configuration: Simple setup with DocumentDB/MongoDB connection parameters
• Multiple Operations: Support for common DocumentDB operations (find, insert, update, delete, aggregate, etc.)
• Flexible Input: Operations can be configured in the node or dynamically via message properties
• Error Handling: Comprehensive error handling with meaningful status indicators
• SSL Support: Optional SSL/TLS connection support with custom CA certificates
• Authentication: Support for username/password authentication

DocumentDB Aggregation Builder Node

• Visual Pipeline Builder: Interactive editor for creating aggregation pipelines
• Syntax Highlighting: JSON syntax highlighting powered by Monaco Editor
• Auto-completion: Intelligent completion for MongoDB aggregation stages and operators
• Real-time Validation: Instant JSON validation with error reporting
• Sample Pipelines: Pre-built examples for common aggregation patterns
• Saved Aggregations: Save, manage, and reuse frequently used pipelines with database storage
• Import/Export: Share aggregation libraries between projects and teams
• Database Persistence: Aggregations stored in the connected database for cross-instance access
• Advanced Editor Features: Code folding, bracket matching, and formatting

Installation

Option 1: Local Installation (Development)

1. Clone or download this repository to your local machine
2. Navigate to your Node-RED user directory (usually ~/.node-red)
3. Install the node using npm:

cd ~/.node-red
npm install /path/to/this/documentdb-connector-directory

Option 2: From npm (if published)

cd ~/.node-red
npm install node-red-contrib-documentdb-connector

Option 3: Manual Installation

1. Copy the files to your Node-RED user directory:

   • documentdb-connector.js
   • documentdb-connector.html
   • package.json

2. Install dependencies:

cd ~/.node-red
npm install mongodb

3. Restart Node-RED

Quick Start

DocumentDB Connector Node

1. Add DocumentDB Configuration Node:

   • Drag a DocumentDB Connector node to your flow
   • Double-click to edit
   • Click the pencil icon next to "DocumentDB" to create a new configuration
   • Enter your DocumentDB/MongoDB connection details (host, port, database, credentials)

2. Configure the Operation:

   • Select the desired operation (find, insert, update, etc.)
   • Optionally specify a collection name

3. Send Messages:

   • Wire an inject node or other input to send messages to the DocumentDB node
   • Structure your msg.payload according to the operation type

DocumentDB Aggregation Builder Node

1. Add Aggregation Builder Node:

   • Drag a DocumentDB Aggregation node to your flow
   • Connect it to a DocumentDB Connector node (set to aggregate operation)

2. Build Your Pipeline:

   • Use the Monaco editor with syntax highlighting
   • Try the sample pipelines as starting points
   • Use Ctrl+Space for auto-completion

3. Execute the Aggregation:

   • The builder outputs a formatted message for the MongoDB connector
   • Results flow through your pipeline as usual

Configuration

DocumentDB Configuration Node

Property	Description	Required	Default
Name	Friendly name for the configuration	No	-
Host	DocumentDB/MongoDB server hostname or IP	Yes	localhost
Port	DocumentDB/MongoDB server port	Yes	27017
Database	Database name	Yes	-
Username	Authentication username	No	-
Password	Authentication password	No	-
Auth Source	Authentication database	No	-
Use SSL	Enable SSL/TLS connection	No	false
CA File	Path to CA certificate file for SSL verification	No	-

DocumentDB Connector Node

Property	Description	Required	Default
Name	Node name	No	-
DocumentDB	DocumentDB configuration	Yes	-
Collection	Collection name	No	-
Operation	Default operation type	No	find

DocumentDB Aggregation Builder Node

Property	Description	Required	Default
Name	Node name	No	-
DocumentDB	DocumentDB configuration	Yes	-
Collection	Collection name	No	-
Output to	Where to place the aggregation request	No	msg.payload

SSL/TLS Configuration

When connecting to MongoDB with SSL/TLS enabled, you can:

1. Use system CA certificates (default): Simply check "Use SSL" and leave the CA File field empty
2. Use custom CA certificate: Check "Use SSL" and provide the absolute path to your CA certificate file

CA Certificate File Requirements:

• Must be in PEM format
• Must be accessible to the Node-RED process
• Use absolute paths (e.g., /path/to/your/ca-certificate.pem)
• Ensure proper file permissions for Node-RED to read the file

Example SSL configurations:

DocumentDB Atlas (cloud):

Host: cluster0.example.documentdb.net
Port: 27017
Use SSL: ✓ (checked)
CA File: (leave empty - uses system certificates)

Self-signed certificate:

Host: your-documentdb-server.com
Port: 27017
Use SSL: ✓ (checked)
CA File: /etc/ssl/certs/documentdb-ca.pem

DocumentDB Aggregation Builder

MongoDB Aggregation Builder

The aggregation builder node provides a visual interface for creating complex DocumentDB/MongoDB aggregation pipelines with syntax highlighting and auto-completion.

Features

• Monaco Editor Integration: Professional code editor with syntax highlighting
• Auto-completion: Intelligent suggestions for DocumentDB/MongoDB aggregation stages and operators
• Real-time Validation: Instant JSON validation with error reporting
• Sample Pipelines: Pre-built examples for common aggregation patterns
• Keyboard Shortcuts: Full editor features including Ctrl+Space for completion

Building Aggregation Pipelines

The aggregation builder accepts pipelines as JSON arrays. Each element in the array represents an aggregation stage.

Basic Pipeline Structure:

[
  {"$match": {"status": "active"}},
  {"$group": {"_id": "$category", "count": {"$sum": 1}}},
  {"$sort": {"count": -1}}
]

Common Aggregation Stages

$match - Filter documents:

{"$match": {"age": {"$gte": 18}}}

$group - Group and aggregate:

{"$group": {
  "_id": "$department",
  "avgSalary": {"$avg": "$salary"},
  "count": {"$sum": 1}
}}

$project - Shape output:

{"$project": {
  "name": 1,
  "email": 1,
  "fullName": {"$concat": ["$firstName", " ", "$lastName"]}
}}

$lookup - Join collections:

{"$lookup": {
  "from": "orders",
  "localField": "_id",
  "foreignField": "customerId",
  "as": "customerOrders"
}}

$unwind - Deconstruct arrays:

{"$unwind": "$tags"}

Sample Pipelines

The node includes several sample pipelines:

1. Basic Match & Project: Filter and reshape documents
2. Group & Count: Group by field and count documents
3. Lookup Join: Join with another collection
4. Date Aggregation: Group by date components

Saved Aggregations

The aggregation builder includes a powerful database-backed saved aggregations system:

Database Storage:

• Aggregations are automatically saved to the _node_red_aggregations collection in your configured database
• Persistent storage ensures aggregations survive Node-RED restarts and are accessible across different instances
• Uses the same database connection as your DocumentDB connector configuration

Saving Aggregations:

1. Ensure a DocumentDB configuration node is selected
2. Create your aggregation pipeline in the editor
3. Enter a descriptive name in the "Save" field
4. Click "Save Current" to store the pipeline in the database

Managing Saved Aggregations:

• Load: Select from the dropdown and click "Load" to restore a saved pipeline from the database
• Delete: Select an aggregation and click "Delete" to remove it from the database
• Export: Download all saved aggregations as a JSON file for backup or sharing
• Import: Upload a previously exported JSON file and save aggregations to the database

Troubleshooting:

• If you see "No DocumentDB config selected", ensure you've selected a configuration node first
• Check that your database user has read/write permissions for the aggregations collection
• Aggregations are stored in the database specified in your DocumentDB configuration

Use Cases:

• Team Collaboration: Export aggregations and share with team members
• Project Templates: Create reusable aggregation patterns
• Backup & Restore: Export aggregations before major changes
• Cross-Environment: Move aggregations between development/production

Using with MongoDB Connector

Connect the aggregation builder output to a DocumentDB connector node:

1. Set the DocumentDB connector operation to "aggregate"
2. The aggregation builder will format the pipeline correctly
3. Results will be returned through the connector node

Example Flow:

[Inject] → [Aggregation Builder] → [DocumentDB Connector] → [Debug]

DocumentDB Connector Operations

Find Documents

Find multiple documents matching a query.

Input:

msg.payload = {
    query: { status: "active" },     // MongoDB query object
    options: {                       // Query options (optional)
        limit: 10,
        sort: { createdAt: -1 },
        projection: { name: 1, email: 1 }
    }
};

Output:

msg.payload = [
    { _id: "...", name: "John", email: "john@example.com" },
    { _id: "...", name: "Jane", email: "jane@example.com" }
];

Find One Document

Find a single document.

Input:

msg.payload = {
    query: { _id: ObjectId("507f1f77bcf86cd799439011") },
    options: { projection: { password: 0 } }  // Exclude password field
};

Insert One Document

Insert a single document.

Input:

msg.payload = {
    document: {
        name: "John Doe",
        email: "john@example.com",
        createdAt: new Date()
    }
};

Output:

msg.payload = {
    acknowledged: true,
    insertedId: ObjectId("...")
};

Insert Many Documents

Insert multiple documents.

Input:

msg.payload = {
    documents: [
        { name: "John", email: "john@example.com" },
        { name: "Jane", email: "jane@example.com" }
    ]
};

Update One Document

Update a single document.

Input:

msg.payload = {
    filter: { name: "John" },
    update: { 
        $set: { 
            email: "newemail@example.com",
            updatedAt: new Date()
        }
    },
    options: { upsert: false }  // Optional
};

Update Many Documents

Update multiple documents.

Input:

msg.payload = {
    filter: { status: "pending" },
    update: { $set: { status: "processed" } }
};

Delete One Document

Delete a single document.

Input:

msg.payload = {
    filter: { _id: ObjectId("507f1f77bcf86cd799439011") }
};

Delete Many Documents

Delete multiple documents.

Input:

msg.payload = {
    filter: { status: "inactive" }
};

Count Documents

Count documents matching a query.

Input:

msg.payload = {
    query: { status: "active" },
    options: { limit: 1000 }  // Optional
};

Output:

msg.payload = 42;  // Number of matching documents

Aggregate

Run an aggregation pipeline.

Input:

msg.payload = {
    pipeline: [
        { $match: { status: "active" } },
        { $group: { 
            _id: "$category", 
            count: { $sum: 1 },
            avgPrice: { $avg: "$price" }
        }},
        { $sort: { count: -1 } }
    ],
    options: { allowDiskUse: true }  // Optional
};

Dynamic Configuration

You can override the node configuration using message properties:

msg.collection = "users";           // Override collection name
msg.operation = "findone";          // Override operation type
msg.payload = { query: { _id: "..." } };

Error Handling

The node provides comprehensive error handling:

• Connection Errors: Displayed in node status and logged
• Operation Errors: Passed to catch nodes in your flow
• Validation Errors: Missing required parameters are reported

Node status indicators:

• 🟢 Green dot: Connected and operation completed successfully
• 🔵 Blue dot: Operation in progress
• 🔴 Red ring: Connection failed
• 🔴 Red dot: Operation error

Example Flows

Simple User Lookup

[
    {
        "id": "inject1",
        "type": "inject",
        "name": "Find User",
        "props": [
            {
                "p": "payload",
                "v": "{\"query\":{\"email\":\"john@example.com\"}}",
                "vt": "json"
            }
        ],
        "wires": [["mongodb1"]]
    },
    {
        "id": "mongodb1",
        "type": "mongodb-connector",
        "mongodb": "mongoconfig1",
        "collection": "users",
        "operation": "findone",
        "wires": [["debug1"]]
    }
]

Insert with Timestamp

[
    {
        "id": "inject2",
        "type": "inject",
        "name": "Add User",
        "wires": [["function1"]]
    },
    {
        "id": "function1",
        "type": "function",
        "func": "msg.payload = {\n    document: {\n        name: \"Jane Doe\",\n        email: \"jane@example.com\",\n        createdAt: new Date()\n    }\n};\nreturn msg;",
        "wires": [["mongodb2"]]
    },
    {
        "id": "mongodb2",
        "type": "mongodb-connector",
        "mongodb": "mongoconfig1",
        "collection": "users",
        "operation": "insertone",
        "wires": [["debug2"]]
    }
]

Requirements

• Node.js 14.0.0 or higher
• Node-RED 3.0.0 or higher
• MongoDB server (local or remote)

Dependencies

• mongodb ^6.3.0 - Official MongoDB Node.js driver

Troubleshooting

Connection Issues

1. Cannot connect to DocumentDB:

   • Verify DocumentDB/MongoDB server is running
   • Check host, port, and network connectivity
   • Verify credentials if authentication is enabled

2. Authentication failed:

   • Check username and password
   • Verify the authentication database (authSource)
   • Ensure user has appropriate permissions

3. SSL/TLS errors:

   • Verify SSL configuration on DocumentDB/MongoDB server
   • Check certificate validity
   • For custom CA certificates, ensure the file path is correct and accessible
   • Verify the CA certificate is in PEM format
   • Check file permissions on the CA certificate file

Operation Issues

1. Collection not found:

   • Verify collection name spelling
   • Ensure collection exists in the specified database

2. Invalid query format:

   • Check DocumentDB/MongoDB query syntax
   • Ensure proper JSON formatting in payloads

Saved Aggregations Issues

1. "No DocumentDB config selected" error:

   • Ensure you have selected a DocumentDB configuration node in the aggregation builder
   • The configuration dropdown should not be empty

2. "Configuration node not found" error:

   • The selected DocumentDB configuration node may have been deleted
   • Create a new DocumentDB configuration node and select it

3. Cannot save/load aggregations:

   • Check database connectivity by trying to save a simple aggregation
   • Check that your DocumentDB/MongoDB user has read/write permissions
   • Ensure the database specified in your configuration exists

4. Aggregations not appearing across Node-RED instances:

   • Verify all instances are connecting to the same database
   • Check that the DocumentDB configuration uses the same database name
   • Aggregations are stored in the _node_red_aggregations collection

5. Import/Export not working:

   • For imports, aggregations are saved to the database, not local storage
   • Exported files include metadata and can be shared between systems
   • Check file format is valid JSON when importing

License

MIT License

Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests if applicable
5. Submit a pull request

Support

For issues and questions:

1. Check the troubleshooting section
2. Review DocumentDB/MongoDB documentation
3. Open an issue on the project repository

Version History

1.4.0

• Renamed all nodes from "mongodb" to "documentdb" for better compatibility
• Updated package name to node-red-contrib-documentdb-connector
• Enhanced documentation for DocumentDB and MongoDB compatibility
• Maintained full backward compatibility with MongoDB connections

1.3.0

• Added saved aggregations feature to aggregation builder
• Implemented save, load, delete functionality for custom pipelines
• Added import/export capabilities for aggregation libraries
• Enhanced aggregation management with timestamps and descriptions
• Improved team collaboration with shareable aggregation files

1.2.1

• Fixed text input issue in aggregation builder editor
• Added fallback textarea editor when Monaco Editor fails to load
• Improved error handling for CDN loading issues
• Enhanced editor initialization with proper fallback mechanism

1.2.0

• Added MongoDB Aggregation Builder node with visual pipeline editor
• Integrated Monaco Editor for syntax highlighting and auto-completion
• Added real-time JSON validation and error reporting
• Included sample aggregation pipelines for common use cases
• Enhanced editor features: code folding, bracket matching, formatting

1.1.0

• Added support for custom CA certificate files for SSL connections
• Enhanced SSL configuration with CA file path option
• Improved SSL error handling and logging
• Added dynamic UI to show/hide CA file field when SSL is enabled

1.0.0

• Initial release
• Support for basic MongoDB operations
• Configuration node for connection management
• Comprehensive error handling and status reporting