Athena

Athena is a file processor service, that consumes files stored in the files.com API and runs a series of reports over the downloaded artifacts and subsequently it talks with the Salesforce API for performing actions (currently, only comments are supported)

Basics

There are 3 software components in athena:

1. Athena-monitor: Monitor changes in several directories across a file.com account and if new files are found those are sent to the processor for background processing.

2. Nats: Nats is a light messaging daemon that allows a pubsub system to be implemented on top, its used to dispatch messages from athena-monitor to a athena-processor

3. Athena-processor: Subscribes to messages from monitor and routes the reports that have to be run over a given detected file, subsequently it will perform an action on salesforce (such as posting a comment, etc)

The basic flowchart of interaction is as follows

Hacking

In order to stand up a development environment, you will need

• make
• docker
• docker-compose
• golang >= 1.19

For running a docker based installation locally you will need a sandbox account on Salesforce and a sandbox directory on files.com. Supply

1. A list of the corresponding credentials in creds.yaml,

   db:
     dialect: mysql
     dsn: "athena:athena@tcp(db:3306)/athena?charset=utf8&parseTime=true"
   
   filescom:
     key : "***"
     endpoint: "https://..."
   
   salesforce:
     endpoint: "https://..."
     username: "***"
     password: "***"
     security-token: "***"

2. A list of directories to monitor in athena-monitor-directories.yaml,

   monitor:
     directories:
       - "/sandbox/..."
       - "/sandbox/..."

3. A path for where the report uploads will go in athena-processor-upload.yaml,

   processor:
     reports-upload-dir: "/sandbox/..."

And finally run

make devel

In case the docker-build step fails you can try to re-run the make command without using the cache,

NOCACHE=1 make devel

The devel deployment includes a debug container which can be used to inspect the database.

$ docker exec --interactive --tty debug bash
# mysql -h db -u athena -pathena athena
mysql> describe files;
+------------+---------------------+------+-----+---------+----------------+
| Field      | Type                | Null | Key | Default | Extra          |
+------------+---------------------+------+-----+---------+----------------+
| id         | bigint(20) unsigned | NO   | PRI | NULL    | auto_increment |
| created_at | datetime(3)         | YES  |     | NULL    |                |
| updated_at | datetime(3)         | YES  |     | NULL    |                |
| deleted_at | datetime(3)         | YES  | MUL | NULL    |                |
| created    | datetime(3)         | YES  |     | NULL    |                |
| dispatched | tinyint(1)          | YES  |     | 0       |                |
| path       | longtext            | YES  |     | NULL    |                |
+------------+---------------------+------+-----+---------+----------------+
7 rows in set (0.01 sec)