EMBArk login fails

[torabi12]

Hi,

I am using a Kali 2021.04 linuxin VMware Workstation.
I am doing a partly manual installation of emba + EMBArk like this:

change shell to /bin/bash
docker installation:
$ sudo apt install docker-ce -y

$ git clone https://github.com/e-m-b-a/embark.git
$ cd embark
$ git clone https://github.com/e-m-b-a/emba.git
$ git clone https://github.com/cve-search/cve-search.git

Install system requirements:
$ cd cve-search
$ sudo xargs apt-get install -y < requirements.system
$ sudo pip3 install -r requirements.txt

MongoDB installation
$ sudo apt install -y mongodb-org

and I install EMBArk with the default -d mode. The install looks fine, I can start the server with:
$ sudo ./run-server.sh
and I can register a user in the browser but after login I get 404 error.

If I start with developer mode:
sudo ./dev-tools/debug-server-start.sh
I can log in.

I checked also the -F installation which is also looks fine, but if I start the server with the default command:
$ sudo ./run-server.sh

I get this error:
Finished setup mysql and redis docker images
mkdir: cannot create directory ‘/app/www/logs’: No such file or directory
mkdir: cannot create directory ‘/app/www/conf’: No such file or directory

[ JOB] Redis logs are copied to ./embark/logs/redis_dev.log

[ JOB] DB logs are copied to ./embark/logs/mysql_dev.log
./run-server.sh: line 98: /app/www/logs/redis.log: No such file or directory
./run-server.sh: line 100: /app/www/logs/mysql.log: No such file or directory
cp: cannot create directory '/app/www/embark/': No such file or directory
./run-server.sh: line 113: /app/www/conf/embark.conf: No such file or directory
./run-server.sh: line 116: cd: /app/www/embark/: No such file or directory

The output of docker-compose ps:

Name Command State Ports

embark_db docker-entrypoint.sh mysqld Up 0.0.0.0:3306->3306/tcp,:::3306->3306/tcp, 33060/tcp
embark_redis docker-entrypoint.sh --por ... Up 6379/tcp, 0.0.0.0:7777->7777/tcp,:::7777->7777/tcp

---

I can use only the
$ sudo ./dev-tools/debug-server-start.sh

mode even if I install -d or with -F but the normal way does not work.
What could be the problem?

If I want to open a report after a full emba test in EMBArk I get this message:

TemplateDoesNotExist at /emba_logs/1/html-report/index.html

/app/emba/emba_logs/emba_logs/1/html-report/index.html

Request Method: GET
Request URL: http://127.0.0.1:8000/emba_logs/1/html-report/index.html
Django Version: 4.0.1
Exception Type: TemplateDoesNotExist
Exception Value:

/app/emba/emba_logs/emba_logs/1/html-report/index.html

Exception Location: /home/kali/embark/.venv/lib/python3.9/site-packages/django/template/loader.py, line 19, in get_template
Python Executable: /home/kali/embark/.venv/bin/python
Python Version: 3.9.9
Python Path:

['/home/kali/embark/embark',
'/home/kali/embark',
'/home/kali/embark/embark',
'/usr/lib/python39.zip',
'/usr/lib/python3.9',
'/usr/lib/python3.9/lib-dynload',
'/home/kali/embark/.venv/lib/python3.9/site-packages']

Server time: Sun, 30 Jan 2022 10:14:42 +0000
Template-loader postmortem

Django tried loading these templates, in this order:

Using engine django:

django.template.loaders.filesystem.Loader: /app/emba/emba_logs/emba_logs/1/html-report/index.html (Source does not exist)

But other pages are working and I am able to download the logs in HTML with "Download Logs" button.

Regards,
Torabi

[torabi12]

I am going to try a new clean install, without preinstalling Docker and MongoDB, but I would like to use the EMBArk tool behind proxy and Docker and other components require custom proxy settings, which can be made only if the tool is preinstalled. emba can not set up my proxy settings, which is not a problem:)

I will add the details of 404 error if I get it again.

[torabi12]

I did a brand new installation, Kali linux 2021.04, VMware Workstation Pro.

I just followed the official instructions:
$ git clone https://github.com/e-m-b-a/embark.git
$ cd embark
$ sudo ./installer.sh -d
$ sudo ./run-server.sh

Registration works but I can not login, I got this error in the browser:

Page not found (404)
Request Method: GET
Request URL: http://127.0.0.1:8001/app/www/embark/app/www/home/upload/1/

Using the URLconf defined in embark.urls, Django tried these URL patterns, in this order:

admin/
[name='embark-login']
register/ [name='embark-register']
home/ [name='embark-home']
home/upload/<int:refreshed>/ [name='embark-start-analysis']
home/delete/ [name='embark-delete']
home/upload/<int:refreshed>/save_file [name='embark-FileSave']
home/serviceDashboard/ [name='embark-ServiceDashboard']
mainDashboard/ [name='embark-MainDashboard-unauth']
home/mainDashboard/ [name='embark-MainDashboard']
home/reportDashboard/ [name='embark-ReportDashboard']
home/individualReportDashboard/<int:analyze_id> [name='embark-IndividualReportDashboard']
download_zipped/<int:analyze_id>/ [name='embark-download']
home/log/<int:log_type>/<int:lines>/ [name='log']
emba_logs/<int:analyze_id>/html-report/style/<str:img_file> [name='embark-html-report-resource']
emba_logs/<int:analyze_id>/html-report/<str:html_file> [name='embark-html-report']
emba_logs/<int:analyze_id>/html-report/<str:html_path>/<str:html_file> [name='embark-html-report-path']
emba_logs/<int:analyze_id>/html-report/<path:html_path>/<str:download_file> [name='embark-html-report-download']
get_load/ [name='embark-get-load']
get_individual_report/<int:analyze_id>/ [name='embark-get-individual-report']
get_accumulated_reports/ [name='embark-get-accumulated-reports']
check_login/ [name='embark-check-login']
signin [name='embark-signin']
signup [name='embark-signup']
signout [name='embark-signout']
password_change [name='embark-password']
acc_delete [name='embark-delete']
^static/(?P<path>.*)$

The current path, app/www/home/upload/1/, didn’t match any of these.

Please help, because I have no idea.

[BenediktMKuehne]

Hi Torabi,
Sorry for the inconvenience. We recently added the Apache as WSGI and I assume that’s what causes the issue here.

[BenediktMKuehne]

I did a brand new installation, Kali linux 2021.04, VMware Workstation Pro.

I just followed the official instructions: $ git clone https://github.com/e-m-b-a/embark.git $ cd embark $ sudo ./installer.sh -d $ sudo ./run-server.sh

Registration works but I can not login, I got this error in the browser:

Page not found (404) Request Method: GET Request URL: http://127.0.0.1:8001/app/www/embark/app/www/home/upload/1/

Using the URLconf defined in embark.urls, Django tried these URL patterns, in this order:

admin/
[name='embark-login']
register/ [name='embark-register']
home/ [name='embark-home']
home/upload/<int:refreshed>/ [name='embark-start-analysis']
home/delete/ [name='embark-delete']
home/upload/<int:refreshed>/save_file [name='embark-FileSave']
home/serviceDashboard/ [name='embark-ServiceDashboard']
mainDashboard/ [name='embark-MainDashboard-unauth']
home/mainDashboard/ [name='embark-MainDashboard']
home/reportDashboard/ [name='embark-ReportDashboard']
home/individualReportDashboard/<int:analyze_id> [name='embark-IndividualReportDashboard']
download_zipped/<int:analyze_id>/ [name='embark-download']
home/log/<int:log_type>/<int:lines>/ [name='log']
emba_logs/<int:analyze_id>/html-report/style/<str:img_file> [name='embark-html-report-resource']
emba_logs/<int:analyze_id>/html-report/<str:html_file> [name='embark-html-report']
emba_logs/<int:analyze_id>/html-report/<str:html_path>/<str:html_file> [name='embark-html-report-path']
emba_logs/<int:analyze_id>/html-report/<path:html_path>/<str:download_file> [name='embark-html-report-download']
get_load/ [name='embark-get-load']
get_individual_report/<int:analyze_id>/ [name='embark-get-individual-report']
get_accumulated_reports/ [name='embark-get-accumulated-reports']
check_login/ [name='embark-check-login']
signin [name='embark-signin']
signup [name='embark-signup']
signout [name='embark-signout']
password_change [name='embark-password']
acc_delete [name='embark-delete']
^static/(?P<path>.*)$

The current path, app/www/home/upload/1/, didn’t match any of these.

Please help, because I have no idea.

My first thought here, would be that you might have used the wrong port. (The normal WSGI should be on port 80.)
Do you use the browser for the sign-up?
If that is not the case, could you provide me with a complete console-log of the ./run-server.sh or debug-server-start ?

[BenediktMKuehne]

Hi,

I am using a Kali 2021.04 linuxin VMware Workstation. I am doing a partly manual installation of emba + EMBArk like this:

change shell to /bin/bash docker installation: $ sudo apt install docker-ce -y

$ git clone https://github.com/e-m-b-a/embark.git $ cd embark $ git clone https://github.com/e-m-b-a/emba.git $ git clone https://github.com/cve-search/cve-search.git

Install system requirements: $ cd cve-search $ sudo xargs apt-get install -y < requirements.system $ sudo pip3 install -r requirements.txt

MongoDB installation $ sudo apt install -y mongodb-org

and I install EMBArk with the default -d mode. The install looks fine, I can start the server with: $ sudo ./run-server.sh and I can register a user in the browser but after login I get 404 error.

If I start with developer mode: sudo ./dev-tools/debug-server-start.sh I can log in.

I checked also the -F installation which is also looks fine, but if I start the server with the default command: $ sudo ./run-server.sh

I get this error: Finished setup mysql and redis docker images mkdir: cannot create directory ‘/app/www/logs’: No such file or directory mkdir: cannot create directory ‘/app/www/conf’: No such file or directory

[ JOB] Redis logs are copied to ./embark/logs/redis_dev.log

[ JOB] DB logs are copied to ./embark/logs/mysql_dev.log ./run-server.sh: line 98: /app/www/logs/redis.log: No such file or directory ./run-server.sh: line 100: /app/www/logs/mysql.log: No such file or directory cp: cannot create directory '/app/www/embark/': No such file or directory ./run-server.sh: line 113: /app/www/conf/embark.conf: No such file or directory ./run-server.sh: line 116: cd: /app/www/embark/: No such file or directory

The output of docker-compose ps:

Name Command State Ports

embark_db docker-entrypoint.sh mysqld Up 0.0.0.0:3306->3306/tcp,:::3306->3306/tcp, 33060/tcp embark_redis docker-entrypoint.sh --por ... Up 6379/tcp, 0.0.0.0:7777->7777/tcp,:::7777->7777/tcp

I can use only the $ sudo ./dev-tools/debug-server-start.sh

mode even if I install -d or with -F but the normal way does not work. What could be the problem?

If I want to open a report after a full emba test in EMBArk I get this message:

TemplateDoesNotExist at /emba_logs/1/html-report/index.html

/app/emba/emba_logs/emba_logs/1/html-report/index.html

Request Method: GET Request URL: http://127.0.0.1:8000/emba_logs/1/html-report/index.html Django Version: 4.0.1 Exception Type: TemplateDoesNotExist Exception Value:

/app/emba/emba_logs/emba_logs/1/html-report/index.html

Exception Location: /home/kali/embark/.venv/lib/python3.9/site-packages/django/template/loader.py, line 19, in get_template Python Executable: /home/kali/embark/.venv/bin/python Python Version: 3.9.9 Python Path:

['/home/kali/embark/embark', '/home/kali/embark', '/home/kali/embark/embark', '/usr/lib/python39.zip', '/usr/lib/python3.9', '/usr/lib/python3.9/lib-dynload', '/home/kali/embark/.venv/lib/python3.9/site-packages']

Server time: Sun, 30 Jan 2022 10:14:42 +0000 Template-loader postmortem

Django tried loading these templates, in this order:

Using engine django:

django.template.loaders.filesystem.Loader: /app/emba/emba_logs/emba_logs/1/html-report/index.html (Source does not exist)

But other pages are working and I am able to download the logs in HTML with "Download Logs" button.

Regards, Torabi

A quick overview of what the -d installation does:

1. Pipenv installation
2. User www-embark gets created
3. Creates directories for the Apache-server (./www)

Also /emab_logs/ location are different between the 2 installation options which would explain this error (if there actually was a log html-report at this location)

[BenediktMKuehne]

Since the EMBArk Server itself isn’t a docker-container, proxy-settings would have to be done on the host-VM, no? (of course, I might be wrong there)
Maybe you could describe the network-setup you would like to deploy EMBArk on in more detail.

take care
-Benedikt

[torabi12]

Hello Benedikt,

Sorry for late reply, I had a lot of tasks, but I tried again EMBArk installation.
Without proxy I was able to install and register a user and use it well. What is worked for me is:
Docker install:
$ sudo apt install -y containerd docker.io docker-compose

$ git clone embark
$ cd embark
$ git clone emba
$ git clone cve-search
$ cd cve-search
$ sudo xargs requirements.system
$ sudo pip3 install requirements.txt

Then MongoDB installation:
$ sudo apt install mongodb-org

Then:
$ sudo ./installer.sh -d

finally:
$ sudo ./run-server

and I could open 127.0.0.1:80 and upload a FW file and test is running.

If I use proxy I just install only emba, but it works only if I execute all commands as "sudo su -". If is use default kali user the cve search DB can not be reached by emba and I can not execute tests.

This is my actual progress of the installation.

[BenediktMKuehne]

The recommended installation process would be:

1. git clone https://github.com/e-m-b-a/embark.git
2. cd embark
3. sudo ./installer.sh -d

Which would accomplish all installations and configures EMBArk (and EMBA) to work correctly with its dependencies
to analyze firmware and generate reports.

cve-search for example gets automatically installed by EMBA (https://github.com/e-m-b-a/emba.git)
AND doesn't connect/function correctly when installed on its own.
(because of docker-network configs, ip etc.)

Yes EMBA and almost all of EMBArk requires root privileges on the host.

Would you be able to elaborate on what exactly you mean by "use of proxy"?
Do you want to make EMBArk accessible to a outside network through a proxy-server via routing?

[torabi12]

Hello Benedikt,

Use of proxy means that I created this script to setup proxy. I am executing this as root user, but if I start the ./installer.sh -d as root it will drop the process in the middle and I can't install the full EMBArk environment. If I use direct internet access on my private workstation it work fine. Could you give me any hint what to modify in this proxy settings?

#/bin/bash

HTTP_PROXY=A.B.C.D:1234
HTTPS_PROXY=A.B.C.D:1234
NO_PROXY=localhost,127.0.,127.36.,192.168.*
GR='\033[1;32m' # Bold Green
RD='\033[1;31m' # Bold Red
NC='\033[0m' # Color Off

echo -e "\n${GR}Users with Bash SHELL:${NC}"
grep bash /etc/passwd | cut -f1,7 -d: | sed '/postgres/d' | sort

echo -e "\n${GR}Proxy Setup in /etc/environment${NC}"
echo "http_proxy=http://${HTTP_PROXY}/" >>/etc/environment &&
echo "https_proxy=http://${HTTPS_PROXY}/" >>/etc/environment &&
echo "no_proxy=${NO_PROXY}" >>/etc/environment
tail -3 /etc/environment

echo -e "\n${GR}APT Proxy Setup${NC}"
touch /etc/apt/apt.conf.d/proxy.conf &&
echo "Acquire::http::Proxy "http://${HTTP_PROXY}";" >>/etc/apt/apt.conf.d/proxy.conf &&
echo "Acquire::https::Proxy "http://${HTTPS_PROXY}";" >>/etc/apt/apt.conf.d/proxy.conf
cat /etc/apt/apt.conf.d/proxy.conf

echo -e "\n${GR}Exporting http_proxy & https_proxy & no_proxy Environment Variables${NC}"
export http_proxy=${HTTP_PROXY}
export https_proxy=${HTTPS_PROXY}
export no_proxy=${NO_PROXY}
export | grep -i proxy | cut -f3 -d" "

echo -e "\n${GR}wget Proxy Setup for root user${NC}"
cp /etc/wgetrc /root/.wgetrc
echo -e "use_proxy = on\nhttp_proxy = http://${HTTP_PROXY}\nhttps_proxy = http://${HTTPS_PROXY}\nftp_proxy = http://${HTTP_PROXY}" >>/root/.wgetrc
tail -4 /root/.wgetrc

echo -e "\n${GR}wget Proxy Setup for kali user${NC}"
cp /etc/wgetrc /home/kali/.wgetrc
echo -e "use_proxy = on\nhttp_proxy = http://${HTTP_PROXY}\nhttps_proxy = http://${HTTPS_PROXY}\nftp_proxy = http://${HTTP_PROXY}" >>/home/kali/.wgetrc
tail -4 /home/kali/.wgetrc

echo -e "\n${GR}Git Proxy Setup${NC}"
git config --global http.proxy http://${HTTP_PROXY}
git config --global https.proxy http://${HTTPS_PROXY}
git config --list --show-origin | cut -f2,3,4,5,6 -d.
echo -e "\n\t${RD}Please log out and log back in!${NC}\n"

Thank you very much!

[torabi12]

I followed the official installation as kali user with the above proxy setup and the installer stopped at the database download:

The installer is going to populate the database.
Starting redis-server (via systemctl): redis-server.service.
Traceback (most recent call last):
File "/home/kali/emba/external/cve-search/./sbin/db_mgmt_cpe_dictionary.py", line 27, in
from lib.Sources_process import CPEDownloads
File "/home/kali/emba/external/cve-search/sbin/../lib/Sources_process.py", line 13, in
from pymongo import TEXT, ASCENDING
ModuleNotFoundError: No module named 'pymongo'
Traceback (most recent call last):
File "/home/kali/emba/external/cve-search/./sbin/db_mgmt_json.py", line 22, in
from lib.Sources_process import CVEDownloads
File "/home/kali/emba/external/cve-search/sbin/../lib/Sources_process.py", line 13, in
from pymongo import TEXT, ASCENDING
ModuleNotFoundError: No module named 'pymongo'
Traceback (most recent call last):
File "/home/kali/emba/external/cve-search/./sbin/db_updater.py", line 21, in
from lib.DatabaseSchemaChecker import SchemaChecker
File "/home/kali/emba/external/cve-search/sbin/../lib/DatabaseSchemaChecker.py", line 7, in
from lib.DatabaseHandler import DatabaseHandler
File "/home/kali/emba/external/cve-search/sbin/../lib/DatabaseHandler.py", line 1, in
from lib.ApiRequests import JSONApiRequest
File "/home/kali/emba/external/cve-search/sbin/../lib/ApiRequests.py", line 5, in
from nested_lookup import nested_lookup, nested_update
ModuleNotFoundError: No module named 'nested_lookup'

I wanted to fix it with this command, but I failed:

$ sudo ./sbin/db_mgmt_cpe_dictionary.py -p
Traceback (most recent call last):
File "/home/kali/emba/external/cve-search/./sbin/db_mgmt_cpe_dictionary.py", line 27, in
from lib.Sources_process import CPEDownloads
File "/home/kali/emba/external/cve-search/sbin/../lib/Sources_process.py", line 13, in
from pymongo import TEXT, ASCENDING
ModuleNotFoundError: No module named 'pymongo'

I know it was just an emba installation, but the same happens with EMBArk if I use kali and not root user.

Maybe the proxy settings are incomplete.

[BenediktMKuehne]

Ok, since my understanding of what and how cve_search does its downloading is minimal, I would refer you to https://github.com/cve-search/cve-search for specifics.

But

from pymongo import TEXT, ASCENDING
ModuleNotFoundError: No module named 'pymongo'

would suggest to me that the module was not installed correctly.
Maybe pip needs custom proxy-settings too?

try running the installer with
https_proxy=https://[username:password@]proxyserver:port sudo ./installer.sh -d
...but thats just a guess

Is the original issue resolved?

Please feel free to open a new one for the Proxy issue.

[torabi12]

Hi Benedikt,

Yes the original issue has been solved, I am able to install EMBArk on Kali 2021.4 without proxy settings and the tool works fine. I am going to start a new conversation about proxy settings.