Addition to Cleanup, Fix Bugs and New Features

.github/workflows/build-publish-deploy.yaml

@@ -14,7 +14,7 @@ on:
 env:
   ECR_REPOSITORY_URL: 673156464838.dkr.ecr.us-west-2.amazonaws.com
   ECR_REPOSITORY_NAME: gpt-survey-summarizer
-  APPLICATION_NAME: gpt-survey-summarizer 
+  APPLICATION_NAME: gpt-survey-summarizer
   DOCKER_TAG_PREFIX: ${{ github.event.inputs.docker_tag_prefix }}
   PLATFORM: staging
   NAMESPACE: governance
@@ -24,7 +24,6 @@ jobs:
     name: Build Publish and Deploy Docker Image
     runs-on: minafoundation-default-runners
     steps:
-
       - name: ⚙️  Set environment variables
         run: |
           echo "AWS_REGION=$AWS_REGION" >> $GITHUB_ENV

.gitignore

@@ -4,6 +4,8 @@ node_modules/
 # Environment variable files
 .env
 .env.*
+.env.local
+.env.local*
 
 # Logs
 logs

Dockerfile

@@ -1,4 +1,4 @@
-ARG NODE_VERSION=21.7.1
+ARG NODE_VERSION=22.0.0
 
 FROM node:${NODE_VERSION}-alpine
 
@@ -9,15 +9,16 @@ WORKDIR /app
 # Copy package root
 COPY ./package.json ./.
 COPY ./package-lock.json ./.
+COPY ./tsconfig.json ./.
+COPY ./settings.json ./.
 
 # Npm clean-install
 RUN npm ci
 
 # Copy source code
-COPY ./src/* ./src/
+COPY ./src ./src/
 
 ENTRYPOINT ["npm", "run"]
 
 # Possible modes: [ bot (default), summarizer ]
-CMD ["bot"]
-
+CMD ["bot"]

README.md

@@ -6,21 +6,27 @@ Join the discord server for discussing and developing this tool [here](https://d
 
 ## Usage
 
-1. Create a .env file with the following variables: 
-
-    ```shell
-    DISCORD_TOKEN=
-    OPENAI_API_KEY=
-    REDIS_URL=
-    CLIENT_ID=
-    GUILD_ID=
-    SUMMARIZE_FREQUENCY_SECONDS=3600
-    ```
+1. Create a .env.local file with the following variables:
+
+   ```shell
+   DISCORD_TOKEN=XXXXXXXXX
+   OPENAI_API_KEY=XXXXXXXXX
+   REDIS_HOST=XXXXXX without port
+   REDIS_PORT=XXXXX
+   REDIS_PASSWORD=XXXXXXX
+   CLIENT_ID=XXXXXXXXX
+   GUILD_ID=XXXXXXXXXX
+   SUMMARIZE_FREQUENCY_SECONDS=3600
+   ```
 
 1. Run `npm install`
-2. Start redis with `redis-server`
-3. Start the main bot script with `npm run bot`
-4. Start the gpt summarization script with `npm run summarizer`
+1. Fill the REDIS_HOST, REDIS_PORT and REDIS_PASSWORD for connecting your Redis DB
+   - Go to [Redis](https://redis.io/) and create an account.
+   - Create a db and click `connect` copy and paste your password, socket and port into .env.local file
+   - You can see the redis configuration in `config.js`
+1. Start the main bot script with `npm run bot`
+1. Or Start the main bot script with `npm run dev` to use nodemon in development phase.
+1. Start the gpt summarization script with `npm run summarizer`
 
 ## Docker image
 

SETUP.md

@@ -1,88 +1,102 @@
 # Step 1: Environment Setup
 
 ## A. Discord Developer Portal
+
 - Create a Discord Account: If you haven’t already, sign up for a Discord account at [Discord](https://discord.com/).
 - Create a New Application:
- - Visit the Discord Developer Portal.
- - Click on the "New Application" button.
- - Name your application and create it.
+- Visit the Discord Developer Portal.
+- Click on the "New Application" button.
+- Name your application and create it.
 - Create a Bot User:
- - In your application settings, navigate to the "Bot" tab and click "Add Bot".
- - Customize your bot as needed (username, icon).
+- In your application settings, navigate to the "Bot" tab and click "Add Bot".
+- Customize your bot as needed (username, icon).
 - Copy Your Bot Token:
- - Still under the "Bot" tab, find your Token and click "Copy". Keep this safe; you'll need it later.
+- Still under the "Bot" tab, find your Token and click "Copy". Keep this safe; you'll need it later.
 
 ## B. OpenAI Account
+
 - Sign Up with OpenAI: Create an account at OpenAI if you don't have one.
 - Access Your API Key:
- - Navigate to the API section and copy your API key for later use.
+- Navigate to the API section and copy your API key for later use.
 
 ## C. Redis Setup
-- Install Redis:
- - Ensure Redis is installed on your local system or use a cloud provider like Redis Labs.
- - Follow the installation guide for your operating system or cloud provider's instructions.
+
+- Go to [Redis](https://redis.io/) and create an account.
+- Create a db and click `connect` copy and paste your password, socket and port into .env.local file
 
 # Step 2: Project Initialization
 
 ## A. Node.js and NPM
+
 - Install Node.js: Download and install Node.js (including NPM) from Node.js official website.
 
 ## B. Initialize Your Project
+
 - Create a Project Directory: Choose a location for your project and create a directory for it.
 - Open a Terminal/Command Prompt: Navigate to your project directory.
 - Initialize NPM:
- - Run `npm init` and follow the prompts to create a package.json file.
+- Run `npm init` and follow the prompts to create a package.json file.
 - Install Dependencies:
- - Run `npm install discord.js @discordjs/rest @discordjs/builders redis openai dotenv`.
+- Run `npm install discord.js @discordjs/rest @discordjs/builders redis openai dotenv`.
 
 # Step 3: Environment Variables:
+
 - Create a .env file in your project root.
 - Add your Discord Bot Token, OpenAI API Key, and Redis connection details (if applicable). Example:
-DISCORD_TOKEN=<Your Discord Bot Token>
-OPENAI_API_KEY=<Your OpenAI API Key>
-CLIENT_ID=<Your Discord Application Client ID> (It's available in the [Discord Developer Portal](https://discord.com/developers/applications/)
-GUILD_ID=<Your Discord Server (Guild) ID> (To get the Guild ID, you must ensure that "Developer Mode" is enabled in your Discord client settings)
-REDIS_URL=redis://localhost:6379
+  DISCORD_TOKEN=<Your Discord Bot Token>
+  OPENAI_API_KEY=<Your OpenAI API Key>
+  CLIENT_ID=<Your Discord Application Client ID> (It's available in the [Discord Developer Portal](https://discord.com/developers/applications/)
+  GUILD_ID=<Your Discord Server (Guild) ID> (To get the Guild ID, you must ensure that "Developer Mode" is enabled in your Discord client settings)
+  REDIS_URL=redis://localhost:6379
 
 ## Finding the Client ID
+
 - Navigate to the [Discord Developer Portal](https://discord.com/developers/applications/)
 - Select Your Application: Click on the application you created for your bot.
 - Locate the Client ID:
- - Under the "Application" section (usually the first page you land on after selecting your application), you will find your Client ID. It's a long string of numbers.
- - Click the "Copy" button next to the Client ID to copy it to your clipboard.
+- Under the "Application" section (usually the first page you land on after selecting your application), you will find your Client ID. It's a long string of numbers.
+- Click the "Copy" button next to the Client ID to copy it to your clipboard.
 
 ## Finding the Guild ID (Server ID)
+
 To get the Guild ID, you must ensure that "Developer Mode" is enabled in your Discord client settings.
 
 ### Enabling Developer Mode
+
 - Open Discord Settings: Click on the gear icon next to your username at the bottom left corner of the Discord interface.
 - Access Advanced Settings: In the left sidebar, scroll down and select "Advanced" under the "App Settings" section.
 - Enable Developer Mode: Toggle the "Developer Mode" switch to the on position.
 
 ### Copying the Guild ID
+
 - Right-Click Your Server Icon:
- - After enabling Developer Mode, go back to the server list on the left side of the Discord interface.
- - Right-click the server (guild) where you intend to use the bot.
+- After enabling Developer Mode, go back to the server list on the left side of the Discord interface.
+- Right-click the server (guild) where you intend to use the bot.
 - Select 'Copy ID': At the bottom of the context menu, click "Copy ID" to copy the Guild ID to your clipboard.
 
 # Step 4: Registering Slash Commands
+
 - Use the Discord.js guide to register your slash commands either globally or to a specific guild for testing. The command registration can be part of your bot startup process or a separate script.
 
 # Step 5: Running Your Bot
+
 - Start Your Bot:
- - In your terminal/command prompt, navigate to your project directory.
- - Run `node bot.js` to start your bot.
+- In your terminal/command prompt, navigate to your project directory.
+- Run `node bot.js` to start your bot.
 - Invite Your Bot to Your Server:
- - In the Discord Developer Portal, under your application's "OAuth2" settings, generate an invite link with the necessary bot permissions.
- - Use the generated link to invite your bot to your server.
+- In the Discord Developer Portal, under your application's "OAuth2" settings, generate an invite link with the necessary bot permissions.
+- Use the generated link to invite your bot to your server.
 
 # Step 6: Set Up Redis
+
 - Ensure Redis is running and accessible. If you installed Redis locally, it should be available at `redis://localhost:6379`. For remote instances, configure according to your provider's instructions.
 
 # Step 7: Testing Your Setup
+
 - Test your bot on your Discord server by using the registered slash commands to create, list, and respond to surveys.
 - Monitor the Redis database to ensure data is being saved and retrieved correctly.
 
 # Step 8: Final Checks
+
 - Ensure all environment variables are correctly set.
 - Confirm that your bot has the necessary permissions on your Discord server to read messages, send messages, and manage interactions.

docker-compose.yaml

@@ -4,18 +4,22 @@ services:
     build: .
     command: bot
     environment:
-      CLIENT_ID: ""
       DISCORD_TOKEN: ""
+      REDIS_HOST: redis
+      REDIS_PORT: 6379
+      REDIS_PASSWORD: ""
+      CLIENT_ID: ""
       GUILD_ID: ""
-      REDIS_URL: "redis://redis:6379"
       SUMMARIZE_FREQUENCY_SECONDS: ""
   summarizer:
     image: gpt-survey-summarizer
     build: .
     command: summarizer
     environment:
       OPENAI_API_KEY: ""
-      REDIS_URL: "redis://redis:6379"
+      REDIS_HOST: redis
+      REDIS_PORT: 6379
+      REDIS_PASSWORD: ""
       SUMMARIZE_FREQUENCY_SECONDS: ""
   redis:
     image: redis

initial_notes.md

@@ -5,32 +5,36 @@ The following were initial notes used for drafting the implementation. It is sli
 # notes
 
 Product Requirements:
-  - Users can make new surveys, which each have a list of questions
-  - Users can answer surveys by providing answers to the list of questions
-  - Users can see what surveys they can answer
-  - Users can see the survey summary
-  - Users can see which comments have not been added to the survey summary yet
+
+- Users can make new surveys, which each have a list of questions
+- Users can answer surveys by providing answers to the list of questions
+- Users can see what surveys they can answer
+- Users can see the survey summary
+- Users can see which comments have not been added to the survey summary yet
 
 Discord bot commands
-  - `/gptsurvey new [surveyName]`
-  - `/gptsurvey summary [surveyname]`
-  - `/gptsurvey respond [surveyname]`
-  - `/gptsurvey list`
+
+- `/gptsurvey new [surveyName]`
+- `/gptsurvey summary [surveyname]`
+- `/gptsurvey respond [surveyname]`
+- `/gptsurvey list`
 
 Backend Architecture
-  - a nodejs module that runs the discord bot code
-  - a redis server that holds survey data
-  - a nodejs module that monitors the redis server and updates the gpt computed survey data
-      - The gpt computed survey data would be a similar process as talk to the city, just with comments instead of claims
 
+- a nodejs module that runs the discord bot code
+- a redis server that holds survey data
+- a nodejs module that monitors the redis server and updates the gpt computed survey data
+  - The gpt computed survey data would be a similar process as talk to the city, just with comments instead of claims
 
 Data format of a survey form
+
 ```
 - title: text
 - questions: List[text]
 ```
 
 Data format of a survey summary
+
 ```
 - title: text,
 - topics: [
@@ -41,7 +45,7 @@ Data format of a survey summary
                 {
                   subtopicName: text,
                   subtopicDescription: text,
-                  comments: [ 
+                  comments: [
                     {
                       submitter: text,
                       answers: List[text]
@@ -62,22 +66,29 @@ Data format of a survey summary
 What the discord bot response to `/gptsurvey summary [surveyname]` could look like:
 
 ### [surveyname]
+
 #### [topic1Name]
+
 [Description]
 [# of comments in the topic]
 [% of comments in the topic out of the total]
+
 ##### [subtopic1Name]
+
 [description]
 [Thread with all comments in the subtopic]
 [# of comments in the subtopic]
 [% of comments in the subtopic out of the # of comments in the topic]
+
 ##### [subtopic2Name]
+
 ...
 
 #### [topic2Name]
+
 ...
 
 #### Comments not yet categorized
+
 These comments will be categorized when the survey summary is recomputed in [x] minutes
 [Thread with all the comments not yet included in the survey summary]
-