Release v1.0.0

• ✅ refactoring codebase (crawler and server)
• ✅ implementing testing with Pytest (30% code coverage)
• ✅ formatting all scripts by Black
• ✅ updating README

Sprint 5 (Fix bugs + Process null result + Log & Use Cloud Run and Load Balancing to deploy)

• ✅ receiving logs of Airflow running on docker and setting alarm strategy on Cloud Logging
• ✅ dealing with null searched results and showing "No Data Found" text on dashboard
• ✅ adding descriptions on concurrency analysis of commenters
• ✅ providing comment frequency on dashboard
• ✅ dockernizing server (APIs and Dashboard)
• ✅ deploying server with Cloud Run for autoscaling and rapid deployment
• ✅ ensuring Redis security level and TTL
• ✅ creating logs between Cloud Run and Redis to deal with disconnection for a period of time
• ✅ using Cloud Scheduler, Pub/Sub, and Cloud Functions to keep connection between Cloud Run container and Redis
• ✅ implementing simple stress test with k6

Sprint 4 (Modify dashboards + Log & Monitor servers and programs)

• ✅ modifying dashboards (unifying language, adding about page, making description concise)
• ✅ collecting two servers' hardware logs (Ops Agent)
• ✅ collecting Redis logs about updating information used in dashboard (Client Library)
• ✅ collecting Airflow logs about DAGs execution status (Ops Agent + Remote Logging)
• ✅ building docker image (dashboard and API) for future use (e.g., Cloud Run

Sprint 3 (Update main function + Log programs)

• ✅ elaborating commenters' connection by computing concurrency
• ✅ creating and separating dashboards (/dashboard, /overview, /keyword, /commenter)
• ✅ beautifying dashboards and making user-friendly (buttons to link each other, loading animation, and hint text)
• ✅ using Redis as cache to accelerate dashboard
• ✅ creating background server to update data in stored in Redis
• ✅ releasing three APIs (authors, keywords, ip address)
• ✅ setting mongo DB read preference to reduce burden on primary node
• ✅ deploying server (API + dashboard & background) to GCP
• ✅ logging airflow crawler and background server

Sprint 2 (Complete main function + determine main tools)

• ✅ crawling script (PTT HatePolitics)
• ✅ completing crawling data pipelines by adding data examination and cleaning
• ✅ presenting top 10 breaking (爆卦) & 10 popular (爆文) on dashboard
• ✅ extracting data directly from MongoDB for data analysis
• ✅ drawing 2D network graph with plotly
• ✅ analyzing comments and accounts with keywords user provided
• ✅ analyzing commenter's activity time
• ✅ building and segmenting dashboard (overview + detail)

Sprint 1 (POC: prototype of main functions + researching techniques)

• ✅ crawling script (PTT Gossiping)
• ✅ deploying MongoDB on Atlas for storing crawling data
• ✅ deploying Airflow with Docker on EC2 for constantly crawling (the newest 4 pages and historical pages)
• ✅ building FastAPI framework (releasing 1 API for searching all articles which has commented by a commenter)
• ✅ generating word cloud graph
• ✅ building dashboard template for showing some info of a commenter, for instance, word cloud of comments (integrating with FastAPI)
• ✅ deploying Kafka with Docker on EC2 and connecting MongoDB as source
• ✅ deploying Neo4j database on EC2 for storing relationships embedded in data crawled
• ✅ inserting and retrieving dome data from Neo4J for presenting relationships and groups on dashboard