A comprehensive Node.js Express API for RSS feed aggregation and content management. Udgibil enables users to discover, subscribe to, and manage RSS feeds while providing advanced search capabilities and content processing features.

Frontend Repository: udgibil-front

UD.GILIB is an ancient Sumerian word that translates to "news" in English.

The word is made up of two parts:

• UD = day, time, daily
• GILIB = (combined with UD to mean news/information)

Together, UD.GILIB literally means something like "daily information" or "news of the day" - which is essentially what we call "news" today.

• RSS Feed Management: Automatic feed discovery, parsing, and content extraction
• User Management: Google OAuth authentication with personalized collections
• Content Processing: Article extraction, sanitization, and processing
• Search & Discovery: Full-text search across feeds and articles
• Bookmark System: Save and organize favorite articles
• YouTube Integration: Support for YouTube RSS feeds
• Image Processing: Integration with ImageKit for image optimization
• Email Notifications: Postmark integration for transactional emails
• Admin Dashboard: Feed and user management capabilities
• API Documentation: RESTful API with comprehensive endpoints

• Runtime: Node.js (ES Modules)
• Framework: Express.js
• Database: MongoDB with connection pooling
• Authentication: Google OAuth 2.0 + JWT
• Feed Processing: Custom feed parser with fallback mechanisms
• Content Extraction: Cheerio, DOMPurify, HTML sanitization
• Image Service: ImageKit integration
• Email Service: SendGrid
• Date/Time: Luxon
• Validation: Custom validators and input sanitization
• Deployment: Docker support

• Node.js (v16 or higher)
• MongoDB (local or cloud instance)
• Google OAuth credentials
• ImageKit account (for image processing)
• SendGrid account (for emails)

1. Clone the repository

   git clone <repository-url>
   cd udgibil-back

2. Install dependencies

   npm install

3. Set up environment configuration

   Copy the example environment files and configure them with your values:

   # Copy example files
   cp bin/development.env.example bin/development.env
   cp bin/production.env.example bin/production.env

   Then edit the files with your actual configuration values:

   • bin/development.env - Development environment settings
   • bin/production.env - Production environment settings

   Required Configuration:

   • MONGO_URI - MongoDB connection string
   • MONGO_DB - Database name
   • PORT - Server port (defaults to 5188)
   • Google OAuth credentials (Client ID, Secret, API key)
   • JWT secret key (use a strong, random string)
   • UDGIBIL_HOST - Main application host URL
   • UDGIBIL_HOST_MOBILE - Mobile application host URL
   • External service API keys (SendGrid, ImageKit, etc.)

4. Set up MongoDB

   Ensure MongoDB is running and create the required collections:

   • users - User accounts and preferences
   • feeds - RSS feed metadata
   • articles - Processed articles
   • bookmarks - User bookmarks
   • views - Article view tracking
   • support - Support tickets
   • logs - Application logs

npm start

• Runs with NODE_ENV=development
• Uses nodemon for auto-restarting
• Enables CORS for local development
• Serves on port 5188 (default) or PORT environment variable

npm run prod

• Runs with NODE_ENV=production
• Optimized for production deployment
• Serves on configured PORT

All protected endpoints require authentication via JWT token obtained through Google OAuth.

Public Endpoints (No Auth Required):

• /auth/* - Authentication routes
• /cron/* - Scheduled task endpoints
• /shared/* - Shared content
• /land/* - Landing page APIs
• /sofeco/* - Sofeco integration

Protected Endpoints (Auth Required):

• /feed/* - Feed management
• /articles/* - Article operations
• /user/* - User management
• /bookmarks/* - Bookmark operations
• /search/* - Search functionality
• /info/* - System information

GET /feed/url?url={rss-url}     # Fetch and process RSS feed
GET /feed/discover              # Discover feeds by category
POST /feed/subscribe            # Subscribe to a feed
DELETE /feed/unsubscribe        # Unsubscribe from feed

GET /articles                   # Get user's articles
GET /articles/feed/{feedId}     # Get articles from specific feed
POST /articles/bookmark         # Bookmark an article
GET /articles/{id}              # Get specific article

GET /search/feeds?q={query}     # Search feeds
GET /search/articles?q={query}  # Search articles
GET /search/own?q={query}       # Search user's subscribed feeds

GET /user/profile               # Get user profile
PUT /user/profile               # Update user profile
GET /user/collections           # Get user's feed collections
POST /user/collections          # Create new collection
GET /onboard                    # User onboarding endpoints

├── app.js                 # Main application setup
├── bin/
│   ├── www.mjs           # Server entry point
│   ├── development.env   # Development environment config
│   └── production.env    # Production environment config
├── controllers/          # Business logic controllers
│   ├── articles.js
│   ├── feeds.js
│   ├── users.js
│   └── ...
├── libs/                 # Utility libraries
│   ├── mongo.js         # MongoDB handler
│   ├── feedfetcher.js   # RSS feed fetching
│   ├── feedhandler.js   # Feed processing
│   ├── utils.js         # Common utilities
│   └── ...
├── middlewares/          # Express middlewares
│   └── authenticate.js   # JWT authentication
├── models/              # Data models
│   ├── users.js
│   ├── feed.js
│   ├── articles.js
│   └── ...
├── routes/              # API route definitions
│   ├── auth.js
│   ├── feeds.js
│   ├── articles.js
│   └── ...
└── public/              # Static files and data
    ├── uploads/         # File uploads
    └── json/            # JSON data files

MongoDB Handler (libs/mongo.js)

• Singleton connection pattern
• Connection pooling with configurable limits
• Comprehensive query methods for all collections
• Aggregation pipeline support for complex searches

Feed Processing Pipeline

1. URL Validation - Normalize and validate RSS URLs
2. Feed Fetching - Primary axios request with ZenRows fallback
3. Content Parsing - FeedParser processes RSS/Atom feeds
4. Article Extraction - Content sanitization and processing
5. Database Storage - Duplicate detection via MD5 hashing
6. Metadata Enhancement - Automatic favicon and categorization

Authentication Flow

1. Google OAuth 2.0 integration
2. JWT token generation and validation
3. User profile creation and management
4. Session management with activity tracking

The project includes Docker configuration for containerized deployment:

# Build and run with docker-compose
docker-compose up -d

# Or build custom image
docker build -t udgibil-api .
docker run -p 5003:5003 udgibil-api

• Service Name: udgibil-api
• Image: booper/udgibil:api
• Default Port: 5188
• Environment Configuration: Loads from bin/{NODE_ENV}.env

• ES Modules (import/export syntax)
• Async/await for asynchronous operations
• Consistent error handling with try/catch blocks
• Input validation and sanitization
• Security best practices (no secrets in code)

Users Collection

{
  _id: ObjectId,
  name: String,
  email: String,
  googleId: String,
  collections: Array,    // Feed organization
  bookmarks: Array,      // Saved articles
  preferences: Object,   // User settings
  createdAt: Date,
  lastActivity: Date
}

Feeds Collection

{
  _id: ObjectId,
  title: String,
  description: String,
  xmlUrl: String,
  link: String,
  image: Object,
  language: String,
  rating: Number,
  followed: Number,      // Follower count
  available: Boolean,
  lastUpdate: Date,
  type: String          // 'news', 'blog', 'udgibil'
}

Articles Collection

{
  _id: ObjectId,
  feedId: ObjectId,
  title: String,
  content: String,
  link: String,
  pubDate: Date,
  available: Boolean,
  titleHash: String     // For duplicate detection
}

• Application Logs: Morgan HTTP request logging
• Error Tracking: Comprehensive error handling with stack traces
• Performance Monitoring: Connection pooling metrics
• Health Checks: Built-in health endpoint for monitoring

• Authentication: Google OAuth 2.0 + JWT tokens
• Input Validation: Request parameter validation and sanitization
• Content Security: HTML sanitization for user-generated content
• Rate Limiting: Configurable request timeouts
• Environment Security: Sensitive data in environment variables
• CORS: Configurable cross-origin resource sharing

1. Fork the repository
2. Create a feature branch: git checkout -b feature-name
3. Make your changes following the existing code style
4. Test your changes thoroughly
5. Commit with descriptive messages
6. Push to your fork and submit a pull request

This project is licensed under the MIT License - see the LICENSE file for details.

For support and questions:

• Create an issue in the GitHub repository
• Follow on Twitter: @Enkiboo

If this RSS reader has made your news consumption a little brighter (or saved you from doomscrolling), consider buying me a coffee! ☕ Your support helps keep the servers running and the features flowing.

Every cup counts and keeps the ancient Sumerian news spirits happy! 📰✨

• GraphQL API implementation
• Real-time notifications via WebSockets
• Advanced analytics and reporting
• Mobile app API enhancements
• Content recommendation engine
• Multi-language support expansion