Skip to content

Sprint 1 (POC: prototype of main functions + researching techniques)

Choose a tag to compare

View all tags
@CuteChuanChuan CuteChuanChuan released this 20 Sep 05:18
· 88 commits to main since this release
v0.1.0
b513938
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Verified
Learn about vigilant mode.
  • ✅ crawling script (PTT Gossiping)
  • ✅ deploying MongoDB on Atlas for storing crawling data
  • ✅ deploying Airflow with Docker on EC2 for constantly crawling (the newest 4 pages and historical pages)
  • ✅ building FastAPI framework (releasing 1 API for searching all articles which has commented by a commenter)
  • ✅ generating word cloud graph
  • ✅ building dashboard template for showing some info of a commenter, for instance, word cloud of comments (integrating with FastAPI)
  • ✅ deploying Kafka with Docker on EC2 and connecting MongoDB as source
  • ✅ deploying Neo4j database on EC2 for storing relationships embedded in data crawled
  • ✅ inserting and retrieving dome data from Neo4J for presenting relationships and groups on dashboard
Assets 2
Loading