Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Error loading bot queues information: #293

Open
kodebrain opened this issue Jan 4, 2023 · 3 comments
Open

Error loading bot queues information: #293

kodebrain opened this issue Jan 4, 2023 · 3 comments

Comments

@kodebrain
Copy link

Installation completed using native package - Ubuntu 20.04
Followed - Installation

 Error loading bot queues information: <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>500 Internal Server Error</title>
</head><body>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error or
misconfiguration and was unable to complete
your request.</p>
<p>Please contact the server administrator at
webmaster@localhost to inform them of the time this error occurred,
and the actions you performed just before this error.</p>
<p>More information about this error may be available
in the server error log.</p>
<hr>
<address>Apache/2.4.41 (Ubuntu) Server at 3<> Port 80</address>
</body></html>
Internal Server Error
Get an error error, Internal Server Error when trying to obtain config file properly

The default paths created during the installation are as below
:/etc/intelmq$ ls -la
total 68
drwxrwxr-x   3 intelmq intelmq  4096 Jan  4 10:47 .
drwxr-xr-x 101 root    root     4096 Jan  4 10:47 ..
-rw-rw-r--   1 intelmq intelmq   506 Sep  2  2021 api-apache.conf
-rw-rw-r--   1 intelmq intelmq   241 Sep  2  2021 api-config.json
-rw-rw-r--   1 intelmq intelmq   264 Sep  2  2021 api-sudoers.conf
-rw-rw-r--   1 intelmq intelmq 21097 Sep 10  2021 harmonization.conf
drwxrwxr-x   2 intelmq intelmq  4096 Jan  4 10:47 manager
-rw-rw-r--   1 intelmq intelmq   385 Sep  2  2021 manager-apache.conf
-rw-rw-r--   1 intelmq intelmq  6324 Jan  4 10:46 runtime.yaml
-rw-rw-r--   1 intelmq intelmq  6313 Jan  4 10:46 runtime.yaml.bak

:/var/log/intelmq$ ls -al
total 12
drwxr-xr-x  2 intelmq intelmq 4096 Jan  4 10:46 .
drwxrwxr-x 13 root    syslog  4096 Jan  4 10:47 ..
-rw-r--r--  1 intelmq intelmq 1045 Jan  4 11:04 intelmqctl.log

/var/lib/intelmq$ ls -al
total 24
drwxr-xr-x  3 intelmq intelmq 4096 Jan  4 11:11 .
drwxr-xr-x 42 root    root    4096 Jan  4 10:47 ..
-rw-------  1 intelmq intelmq  915 Jan  4 11:11 .viminfo
drwxrwxr-x  4 intelmq intelmq 4096 Jan  4 10:46 bots
-rw-rw-r--  1 intelmq intelmq 7125 Sep 10  2021 state.json

/usr/share/dbconfig-common/data/intelmq-api/install$ ls -la
total 12
drwxr-xr-x 2 root root 4096 Jan  4 10:47 .
drwxr-xr-x 3 root root 4096 Jan  4 10:47 ..
-rw-r--r-- 1 root root  429 Sep  2  2021 sqlite3
cat /etc/intelmq/api-config.json 
{
        "intelmq_ctl_cmd": ["sudo", "-u", "intelmq", "intelmqctl"],
        "allowed_path": "/opt/intelmq/var/lib/bots/",
        "session_store": "/var/lib/dbconfig-common/sqlite3/intelmq-api/intelmqapi",
        "session_duration": 86400,
        "allow_origins": ["*"]
}

~$ intelmqctl check
Reading configuration files.
Checking runtime and pipeline configuration.
Checking harmonization configuration.
Checking for bots.
No issues found.

this is the name of your application's database.

dbc_dbname='/var/lib/dbconfig-common/sqlite3/intelmq-api'
The above path doesn't exist on my installation

"/var/lib/dbconfig-common/sqlite3/intelmq-api/intelmqapi"
:/etc/intelmq$ jq .session_store ${INTELMQ_API_CONFIG:-/etc/intelmq/api-config.json} | tr -d '"' | xargs ls -l
ls: cannot access '/var/lib/dbconfig-common/sqlite3/intelmq-api/intelmqapi': No such file or directory```

changed the path of session_store to /usr/share/dbconfig-common/data/intelmq-api/install

$ intelmq-api-adduser --user <> --password <>
Loading config from /etc/intelmq/api-config.json
Traceback (most recent call last):
  File "/usr/bin/intelmq-api-adduser", line 27, in <module>
    session_store = intelmq_api.session.SessionStore(str(api_config.session_store), api_config.session_duration)
  File "/usr/lib/python3/dist-packages/intelmq_api/session.py", line 86, in __init__
    self.init_sqlite_db()
  File "/usr/lib/python3/dist-packages/intelmq_api/session.py", line 100, in init_sqlite_db
    with self.connect() as con:
  File "/usr/lib/python3/dist-packages/intelmq_api/session.py", line 91, in connect
    return sqlite3.connect(self.dbname, check_same_thread=False,
sqlite3.OperationalError: unable to open database file

How to create the api-session.sqlite file?
@sebix
Copy link
Member

sebix commented Jan 30, 2023

The user needs read and write access to the database and the directory containing it. Is this the case?

@kodebrain
Copy link
Author

I had to create the db manually using sudo api-session.sqlite based on the config in /etc/intelmq/api-config.json
/usr/share/dbconfig-common/data/intelmq-api/install/sqlite3
write permission set for user and group for api-session.sqlite

After logging back into the portal, getting more errors
9:07 AM 341× Error loading bot queues information: �[31mNot logging to file: [Errno 13] Permission denied: '/var/log/intelmq/intelmqctl.log'�[0m
sudo -u www-data intelmqctl --type json list queues-and-status

8:59 AM Saved positions are not valid or not complete. The configuration has possibly been modified outside of the IntelMQ-Manager.

-rw-r--r-- 1 intelmq intelmq 1245 Feb 1 08:58 intelmqctl.log

all errors disappeared after setting the permission to 774
-rwxrwxr-- 1 intelmq intelmq 1245 Feb 1 08:58 intelmqctl.log

To import custom bots, is the below the correct location to install it?

/usr/lib/python3/dist-packages/intelmq/bots$ ls -al
total 28
drwxr-xr-x 7 root root 4096 Jan 4 10:46 .
drwxr-xr-x 8 root root 4096 Jan 4 10:46 ..
-rw-r--r-- 1 root root 0 Sep 10 2021 init.py
drwxr-xr-x 2 root root 4096 Jan 4 10:46 pycache
drwxr-xr-x 24 root root 4096 Jan 4 10:46 collectors
drwxr-xr-x 40 root root 4096 Jan 4 10:46 experts
drwxr-xr-x 22 root root 4096 Jan 4 10:46 outputs
drwxr-xr-x 52 root root 4096 Jan 4 10:46 parsers

@sebix
Copy link
Member

sebix commented Feb 7, 2023

To import custom bots, is the below the correct location to install it?

Yes, but if you want a development installation, I recommend reading the dev guide: https://intelmq.readthedocs.io/en/maintenance/dev/guide.html#installation
Using a local repository makes development easier

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@sebix @kodebrain