API Issue? #103

kvedder-amplex · 2019-10-29T15:20:50Z

This looks to be a promising alternative to salt enterprise. I am testing it out in docker.

Describe the bug
Getting 500 Error when refreshing key/minions. API socket returns OK. Not seeing any other data.

API Config:

 rest:
    ^url: http://docker:8001/api/token/verify/
    fred:
      - .*
      - '@jobs'
      - '@wheel'
      - '@runner'

Am I missing something in the API config?

Here is the docker env's.

The text was updated successfully, but these errors were encountered:

mattLLVW · 2019-10-29T17:32:52Z

Thanks for giving Alcali a try!

the docs should definitely be clearer.

There's currently two different ways to authenticate Alcali against the Salt-Api.

Using a custom auth module. In that case, Alcali just need to be able to reach the Salt-Api.
Using the Api Endpoint. In that case, the Salt-Master also need to be able to reach Alcali.

Is your docker container reachable from the Salt-Master? If not, you should consider using the first option.

kvedder-amplex · 2019-10-29T18:33:06Z

I am going with the API endpoint, and the servers live on the same subnet and should have full communications enabled.

mattLLVW · 2019-10-29T19:15:30Z

Ok, here is the master's log in debug mode during a rest call:

master_1  | 20:59:41 master.1 | [DEBUG   ] Requesting URL http://web:8000/api/token/verify/ using POST method
master_1  | 20:59:41 master.1 | [DEBUG   ] Using backend: tornado
master_1  | 20:59:41 master.1 | [DEBUG   ] Response Status Code: 200
master_1  | 20:59:41 master.1 | [DEBUG   ] eauth REST call returned 200: {'body': '{"admin":null}', 'dict': {'admin': None}, 'status': 200}

And alcali's log:

web_1     | [29/Oct/2019 15:59:41] "POST /api/token/verify/ HTTP/1.1" 200 14

Could you provide the same logs?

Here is the curl command to reproduce the Rest Call:

curl -sSk -H 'Accept: application/json' -d username=admin -d password=b811a13c582fe24f287a1c13ba308f634f04159c http://web:8000/api/token/verify/

Where password is the user's token

kvedder-amplex · 2019-10-30T13:09:03Z

Response from the Salt Master when querying keys:

[DEBUG   ] Reading configuration from /etc/salt/minion
[DEBUG   ] Including configuration from '/etc/salt/minion.d/_schedule.conf'
[DEBUG   ] Reading configuration from /etc/salt/minion.d/_schedule.conf
[DEBUG   ] Using cached minion ID from /etc/salt/minion_id: salt-stack.amplex.net
[DEBUG   ] Requesting URL http://dock1.amplex.net:8001/api/token/verify/ using POST method
[DEBUG   ] Using backend: tornado
[DEBUG   ] Response Status Code: 200
[DEBUG   ] eauth REST call returned 200: {'body': '{"admin":null}', 'status': 200, 'dict': {'admin': None}}
[DEBUG   ] Could not LazyLoad rest.groups: 'rest.groups' is not available.
[DEBUG   ] LazyLoaded localfs.mk_token

Response from the curl command issued from the salt-master to the docker server running ALCALI.
curl -sSk -H 'Accept: application/json' -d username=admin -d password=xxxxxxx http://dock1:8001/api/token/verify/

{"admin":null}

So couple of oddities, I am running the docker instance with the GUI on port 8001 instead of 8000. Could this cause an issue?

mattLLVW · 2019-10-30T17:27:54Z

Unfortunately, according to your logs, it looks like the authentication is successful.

That mean that the problem is after that, i''m guessing a database issue.

an easy way to find out is:

Stop Alcali
Add: DJANGO_DEBUG=true to your env var.
Restart Alcali

It should show you the traceback.

/!\ Be advise that anyone who can access the service while debug mode is on can probably retrieve all environment variable!

Possible database issue:

Is the db schema properly inserted ? (Salt would scream a lot if not but you never know)

Have you run the database migration?

What does alcali check return?

kvedder-amplex · 2019-10-31T14:50:28Z

So after enabling debug logging.....

Watching for file changes with StatReloader,
Performing system checks...,
,
System check identified no issues (0 silenced).,
October 31, 2019 - 09:42:37,
Django version 2.2.6, using settings 'config.settings',
Starting development server at http://0.0.0.0:8001/,
Quit the server with CONTROL-C.,
[31/Oct/2019 09:42:52] "GET / HTTP/1.1" 200 1184,
[31/Oct/2019 09:42:52] "GET /static/css/chunk-vendors.47ff193e.css HTTP/1.1" 304 0,
[31/Oct/2019 09:42:52] "GET /static/css/app.321866b5.css HTTP/1.1" 304 0,
[31/Oct/2019 09:42:52] "GET /static/js/chunk-vendors.439a09fa.js HTTP/1.1" 304 0,
[31/Oct/2019 09:42:52] "GET /static/js/app.b0501687.js HTTP/1.1" 304 0,
[31/Oct/2019 09:42:53] "POST /api/token/verify/ HTTP/1.1" 200 14,
Unauthorized: /api/jobs/graph,
Unauthorized: /api/keys/keys_status/,
[31/Oct/2019 09:42:53] "GET /api/keys/keys_status/ HTTP/1.1" 401 58,
Unauthorized: /api/stats/,
[31/Oct/2019 09:42:53] "GET /api/jobs/graph?fun=all&period=7 HTTP/1.1" 401 58,
[31/Oct/2019 09:42:53] "GET /api/stats/ HTTP/1.1" 401 58,
Unauthorized: /api/jobs/filters/,
[31/Oct/2019 09:42:53] "GET /api/jobs/filters/ HTTP/1.1" 401 58,
Unauthorized: /api/jobs/,
[31/Oct/2019 09:42:53] "GET /api/jobs/?limit=10 HTTP/1.1" 401 58,
Unauthorized: /api/userssettings/1/,
[31/Oct/2019 09:42:53] "GET /api/userssettings/1/ HTTP/1.1" 401 58,
Bad Request: /api/token/refresh/,
[31/Oct/2019 09:42:53] "POST /api/token/refresh/ HTTP/1.1" 400 43,
Not Found: /favicon.ico,
[31/Oct/2019 09:42:53] "GET /favicon.ico HTTP/1.1" 404 4894,
Unauthorized: /api/token/,
[31/Oct/2019 09:42:59] "POST /api/token/ HTTP/1.1" 401 63,
Bad Request: /api/token/refresh/,
[31/Oct/2019 09:42:59] "POST /api/token/refresh/ HTTP/1.1" 400 43,
[31/Oct/2019 09:43:35] "POST /api/token/ HTTP/1.1" 200 492,
[31/Oct/2019 09:43:36] "POST /api/token/verify/ HTTP/1.1" 200 14,
[31/Oct/2019 09:43:36] "GET /api/keys/keys_status/ HTTP/1.1" 200 53,
[31/Oct/2019 09:43:36] "GET /api/jobs/graph?fun=all&period=7 HTTP/1.1" 200 146,
[31/Oct/2019 09:43:36] "GET /api/minions/conformity/ HTTP/1.1" 200 70,
[31/Oct/2019 09:43:36] "GET /api/stats/ HTTP/1.1" 200 35,
[31/Oct/2019 09:43:36] "GET /api/jobs/filters/ HTTP/1.1" 200 25,
[31/Oct/2019 09:43:36] "GET /static/fonts/materialdesignicons-webfont.0aee1bb1.woff2 HTTP/1.1" 304 0,
[31/Oct/2019 09:43:36] "GET /api/userssettings/1/ HTTP/1.1" 200 232,
[31/Oct/2019 09:43:36] "GET /api/jobs/?limit=10 HTTP/1.1" 200 2,
[31/Oct/2019 09:43:39] "GET /api/minions/ HTTP/1.1" 200 2,
[31/Oct/2019 09:43:41] "POST /api/minions/refresh_minions/ HTTP/1.1" 200 16,
[31/Oct/2019 09:43:42] "GET /api/minions/ HTTP/1.1" 200 2,
[31/Oct/2019 09:43:58] "GET /api/keys/ HTTP/1.1" 200 2,
[31/Oct/2019 09:44:00] "POST /api/token/verify/ HTTP/1.1" 200 14,
Internal Server Error: /api/keys/refresh/,
Traceback (most recent call last):,
  File "/opt/alcali/.local/lib/python3.7/site-packages/pepper/libpepper.py", line 227, in req,
    f = urlopen(req, context=con),
  File "/usr/local/lib/python3.7/urllib/request.py", line 222, in urlopen,
    return opener.open(url, data, timeout),
  File "/usr/local/lib/python3.7/urllib/request.py", line 531, in open,
    response = meth(req, response),
  File "/usr/local/lib/python3.7/urllib/request.py", line 641, in http_response,
    'http', request, response, code, msg, hdrs),
  File "/usr/local/lib/python3.7/urllib/request.py", line 569, in error,
    return self._call_chain(*args),
  File "/usr/local/lib/python3.7/urllib/request.py", line 503, in _call_chain,
    result = func(*args),
  File "/usr/local/lib/python3.7/urllib/request.py", line 649, in http_error_default,
    raise HTTPError(req.full_url, code, msg, hdrs, fp),
urllib.error.HTTPError: HTTP Error 401: Unauthorized,
,
During handling of the above exception, another exception occurred:,
,
Traceback (most recent call last):,
  File "/opt/alcali/.local/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner,
    response = get_response(request),
  File "/opt/alcali/.local/lib/python3.7/site-packages/django/core/handlers/base.py", line 115, in _get_response,
    response = self.process_exception_by_middleware(e, request),
  File "/opt/alcali/.local/lib/python3.7/site-packages/django/core/handlers/base.py", line 113, in _get_response,
    response = wrapped_callback(request, *callback_args, **callback_kwargs),
  File "/opt/alcali/.local/lib/python3.7/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view,
    return view_func(*args, **kwargs),
  File "/opt/alcali/.local/lib/python3.7/site-packages/rest_framework/viewsets.py", line 114, in view,
    return self.dispatch(request, *args, **kwargs),
  File "/opt/alcali/.local/lib/python3.7/site-packages/rest_framework/views.py", line 505, in dispatch,
    response = self.handle_exception(exc),
  File "/opt/alcali/.local/lib/python3.7/site-packages/rest_framework/views.py", line 465, in handle_exception,
    self.raise_uncaught_exception(exc),
  File "/opt/alcali/.local/lib/python3.7/site-packages/rest_framework/views.py", line 476, in raise_uncaught_exception,
    raise exc,
  File "/opt/alcali/.local/lib/python3.7/site-packages/rest_framework/views.py", line 502, in dispatch,
    response = handler(request, *args, **kwargs),
  File "/opt/alcali/code/api/views/alcali.py", line 66, in refresh,
    get_keys(refresh=True),
  File "/opt/alcali/code/api/backend/netapi.py", line 45, in get_keys,
    api_ret = api.wheel("key.list_all")["return"][0]["data"]["return"],
  File "/opt/alcali/.local/lib/python3.7/site-packages/pepper/libpepper.py", line 436, in wheel,
    return self.low([low]),
  File "/opt/alcali/.local/lib/python3.7/site-packages/pepper/libpepper.py", line 298, in low,
    return self.req(path, lowstate),
  File "/opt/alcali/.local/lib/python3.7/site-packages/pepper/libpepper.py", line 239, in req,
    raise PepperException('Authentication denied'),
pepper.exceptions.PepperException: Authentication denied,
[31/Oct/2019 09:44:00] "POST /api/keys/refresh/ HTTP/1.1" 500 16018,

Alcali check returns...

  File "/opt/alcali/.local/bin/alcali", line 6, in <module>
    from pkg_resources import load_entry_point
  File "/opt/alcali/.local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3251, in <module>
    @_call_aside
  File "/opt/alcali/.local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3235, in _call_aside
    f(*args, **kwargs)
  File "/opt/alcali/.local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 3264, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/opt/alcali/.local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 583, in _build_master
    ws.require(__requires__)
  File "/opt/alcali/.local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 900, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/opt/alcali/.local/lib/python3.7/site-packages/pkg_resources/__init__.py", line 786, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'alcali' distribution was not found and is required by the application

Looks like my docker build is hosed. Perhaps I should start over.

kvedder-amplex · 2019-10-31T14:55:24Z

When running ./manage.py check:
db: ok env: ['MASTER_MINION_ID']

when I ran ./manage.py migrate:

./manage.py migrate Operations to perform: Apply all migrations: admin, api, auth, contenttypes, sessions Running migrations: No migrations to apply.

./manage.py current_version alcali version 2019.2.2

mattLLVW · 2019-11-01T04:25:20Z

Ok, i got it:

Do you have a user named admin defined in external_auth with sufficient rights?

external_auth:
  rest:
    ^url: http://web:8000/api/token/verify/
    admin:
      - .*
      - '@runner'
      - '@wheel'

from the traceback, the auth to Alcali is ok, it's salt who's not happy.

And i can reproduce the same behaviour.

kvedder-amplex · 2019-11-01T13:33:02Z

---
rest_cherrypy:
  port: 8080
  host: 0.0.0.0
  log_access_file: '/var/log/salt/api_access.log'
  log_error_file: '/var/log/salt/api_error.log'
  ssl_crt:  **
  ssl_key: **

external_auth:
  pam:
    amplexadmin:
      - .*
      - '@wheel'   # to allow access to all wheel modules
      - '@runner'  # to allow access to all runner modules
      - '@jobs'    # to allow access to the jobs runner and/or wheel module
    silica_api:
      - .*
      - '@wheel'   # to allow access to all wheel modules
      - '@runner'  # to allow access to all runner modules
      - '@jobs'    # to allow access to the jobs runner and/or wheel module
  rest:
    ^url: http://dock1.amplex.net:8001/api/token/verify/
    admin:
      - .*
      - '@runner'
      - '@wheel'

The PAM auth works correctly. I just ran some queries to the API. ALCALI is still not happy.

kvedder-amplex · 2019-11-01T14:23:04Z

And it just started working after a couple more restarts. Very Odd. I need to add the job returner for postgres next. Thanks for taking the time to assist! I will plan on using this tool, it may save me from getting SSE.

kvedder-amplex added the bug Something isn't working label Oct 29, 2019

mattLLVW added question Further information is requested and removed bug Something isn't working labels Oct 29, 2019

mattLLVW mentioned this issue Nov 1, 2019

Catch all Authentication denied exception from Pepper and trigger Toast #104

Closed

kvedder-amplex closed this as completed Nov 1, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

API Issue? #103

API Issue? #103

kvedder-amplex commented Oct 29, 2019 •

edited

Loading

mattLLVW commented Oct 29, 2019

kvedder-amplex commented Oct 29, 2019

mattLLVW commented Oct 29, 2019

kvedder-amplex commented Oct 30, 2019

mattLLVW commented Oct 30, 2019

kvedder-amplex commented Oct 31, 2019

kvedder-amplex commented Oct 31, 2019 •

edited

Loading

mattLLVW commented Nov 1, 2019

kvedder-amplex commented Nov 1, 2019 •

edited

Loading

kvedder-amplex commented Nov 1, 2019

API Issue? #103

API Issue? #103

Comments

kvedder-amplex commented Oct 29, 2019 • edited Loading

mattLLVW commented Oct 29, 2019

kvedder-amplex commented Oct 29, 2019

mattLLVW commented Oct 29, 2019

kvedder-amplex commented Oct 30, 2019

mattLLVW commented Oct 30, 2019

kvedder-amplex commented Oct 31, 2019

kvedder-amplex commented Oct 31, 2019 • edited Loading

mattLLVW commented Nov 1, 2019

kvedder-amplex commented Nov 1, 2019 • edited Loading

kvedder-amplex commented Nov 1, 2019

kvedder-amplex commented Oct 29, 2019 •

edited

Loading

kvedder-amplex commented Oct 31, 2019 •

edited

Loading

kvedder-amplex commented Nov 1, 2019 •

edited

Loading