API Collector Service

This service collects data from DailyHot API endpoints and stores it in a PostgreSQL database.

Features

• Collects data from 56+ API endpoints
• Stores data in PostgreSQL database
• Runs every 30 minutes automatically
• Configurable via environment variables
• Dockerized for easy deployment

Deployment

Using Docker Compose

1. Create a docker-compose.yml file:

version: '3.8'

services:
  api-collector:
    build:
      context: https://github.com/ryan32382/api-collector.git
      dockerfile: Dockerfile
    container_name: api-collector
    restart: unless-stopped
    environment:
      - TZ=Asia/Shanghai
      - API_BASE_URL=http://192.168.1.2:6688
      - DB_HOST=192.168.1.2
      - DB_PORT=9918
      - DB_NAME=test_db
      - DB_USER=noney
      - DB_PASSWORD=jiegede3
    volumes:
      - ./logs:/app/logs
    networks:
      - default

networks:
  default:
    driver: bridge

2. Start the service:

docker-compose up -d

Configuration

All configuration is done via environment variables:

• API_BASE_URL: Base URL of the DailyHot API
• DB_HOST: PostgreSQL database host
• DB_PORT: PostgreSQL database port
• DB_NAME: PostgreSQL database name
• DB_USER: PostgreSQL database username
• DB_PASSWORD: PostgreSQL database password
• TZ: Timezone (e.g., Asia/Shanghai)

Database Schema

The service creates a table api_collections with the following schema:

CREATE TABLE api_collections (
    id SERIAL PRIMARY KEY,
    platform VARCHAR(50) NOT NULL,
    collection_time TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
    data JSONB NOT NULL,
    status INTEGER DEFAULT 200,
    response_time INTEGER DEFAULT 0
);

Logs

Logs are stored in the logs/collector.log file and also printed to the console.

License

MIT