401 Unauthenticated with valid api tokens

[animalillo]

Support guidelines

• I've read the support guidelines

I've found a bug and checked that ...

• ... the documentation does not mention anything about my problem
• ... there are no open or closed issues that are related to my problem
• ... it's definitely me, not you

Description

I have followed the install guides and setup a new vm and deployed firefly iii via ansible
Then I went on to try and play around with it on my local network, but didn't get the importer nor the api to work properly.

I tried to get the importer to run using both a token and the oauth as specified on the guide, with oauth it asks for me to authorize the importer but then both converge on the same problem.

The api always ends up responding:

Configuration / connection error :(

The importer could not connect to Firefly III. Please remedy the error below first, and check out the documentation if necessary.

Error accessing "http://192.168.1.112/api/v1/about". Status code is 401. Body is: >{"message":"Unauthenticated","exception":"AuthenticationException"}

Probably related to: #5490

Expected behaviour

Importer / api authetication should work

Debug information

Debug information generated at 2022-01-21 00:15:37 Europe/Madrid for Firefly III version 5.6.10.

Scope	Version
Firefly III	5.6.10
Firefly III API	1.5.5
Build	(unknown), (unknown)
PHP	8.1.1
Host	Linux

System info	Value
Installation ID	fa410f47-944d-4df9-868a-42de04395504
Using docker?	false
Layout	v1
System TZ	Europe/Madrid
Browser TZ	Europe/Madrid
App environment	local
App debug mode	true
App cache driver	file
App logging	debug, stack
Display errors	Off
Error reporting	ALL errors
Interface	fpm-fcgi
Default language	es_ES
Default locale	equal
BCscale	24
DB drivers	mysql, sqlite
Current driver	mysql
DB version	18 (exp. 18)
Login provider	eloquent
Trusted proxies (.env)

User info	Value
Session start
Session end
Session first
User ID	1
User language	es_ES
User locale	es_ES
Attempt at "es_ES.utf8"	false
Attempt at "es_ES.UTF-8"	false
User agent	Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:96.0) Gecko/20100101 Firefox/96.0

Steps to reproduce

1. Set up an ubuntu 20.04 lxc container
2. Deploy with ansible.
3. Set up and just do the most very basic stuff

---
- name: Setup server requirements.
  hosts: firefly
  become: true

  vars:
    mysql_user_home: /root
    mysql_user_name: root
    mysql_user_password: root
    php_enable_php_fpm: true
    php_version: "8.1"
    php_packages_extra:
      - "php{{ php_version }}-bcmath"
      - "php{{ php_version }}-curl"
      - "php{{ php_version }}-zip"
      - "php{{ php_version }}-gd"
      - "php{{ php_version }}-xml"
      - "php{{ php_version }}-mbstring"
      - "php{{ php_version }}-ldap"
      - "php{{ php_version }}-mysql"
      - "php{{ php_version }}-intl"
      - "php{{ php_version }}-opcache"
      # - "php{{ php_version }}-"
    mysql_databases:
      - name: firefly
        encoding: utf8
        collation: utf8_general_ci
        state: present
    mysql_users:
      - name: firefly
        host: localhost
        password: "replaceme"
        priv: "firefly.*:ALL"
    apache_vhosts:
      - servername: "192.168.1.112"
        documentroot: "/var/www/firefly/public/"
        extra_parameters: |
              ProxyPassMatch ^/(.*\.php(/.*)?)$ "fcgi://127.0.0.1:9000/var/www/firefly/public"

  roles:
    - geerlingguy.apache
    - geerlingguy.php-versions
    - geerlingguy.php
    - geerlingguy.apache
    - geerlingguy.apache-php-fpm
    # - geerlingguy.apache-versions
    - geerlingguy.mysql
    - geerlingguy.composer

  tasks:
    - name: Ensure directory exist and permissions are correct for the firefly folder.
      file:
        path: "/var/www/firefly"
        state: directory
        owner: www-data
        group: www-data
        mode: "0770"

    - stat:
        path: /var/www/firefly/composer.json
      register: composerfile

    - name: Composer create Firefly project if composer.json is not present.
      composer:
        command: create-project
        arguments: "-n grumpydictator/firefly-iii . 5.6.10"
        working_dir: /var/www/firefly
        prefer_dist: true
      when: composerfile.stat.exists == false
      register: firefly_project

    - name: Ensure directory exist and permissions are correct for the firefly folder files.
      file:
        path: "/var/www/firefly/"
        # state: directory
        owner: www-data
        group: www-data
        # mode: "0660"
        recurse: true

    - name: Copy firefly .env
      copy:
        src: templates/firefly/.env
        dest: /var/www/firefly/.env
        owner: www-data
        group: www-data
        mode: "0660"

    - name: Run firefly migrations.
      shell: |
        php /var/www/firefly/artisan migrate:refresh --seed && \
        php /var/www/firefly/artisan firefly-iii:upgrade-database && \
        php /var/www/firefly/artisan passport:install
      when: firefly_project.changed

Additional info

No response

[JC5]

So, it looks like you can reach Firefly III, the network's working, it just tells you the token is invalid.

Why are you running PHP 8.1 instead of 8.0? It shouldn't matter, but still.

[JC5]

This command will drop your database very time Firefly III starts:

        php /var/www/firefly/artisan migrate:refresh --seed && \

Change it to:

        php /var/www/firefly/artisan migrate --seed && \

I'm not sure how often it runs of course, but combined with php /var/www/firefly/artisan passport:install it means a new keypair is generated, invalidating previous tokens.

[animalillo]

Oh! that would just run when the app gets installed! but thanks for the tip! I will change that.

[JC5]

Mmmm, it does mean I'm running out of ideas here. Is the firefly/storage directory writeable?

[animalillo]

firefly root dir:

root@firefly:/var/www/firefly# ll
total 1068
drwxrwx--- 14 www-data www-data   4096 Jan 21 18:21 ./
drwxr-xr-x  4 root     root       4096 Jan 18 21:49 ../
drwxr-xr-x  3 www-data www-data   4096 Jan 18 21:49 .deploy/
-rw-rw----  1 www-data www-data  11349 Jan 21 18:21 .env
-rw-r--r--  1 www-data www-data  11294 Jan  8 23:30 .env.example
-rw-r--r--  1 www-data www-data    184 Jan  8 23:30 .gitattributes
drwxr-xr-x  5 www-data www-data   4096 Jan 18 21:49 .github/
-rw-r--r--  1 www-data www-data    264 Jan  8 23:30 .gitignore
-rw-r--r--  1 www-data www-data    305 Jan  8 23:30 .htaccess
-rw-r--r--  1 www-data www-data  34523 Jan  8 23:30 COPYING
-rw-r--r--  1 www-data www-data  34523 Jan  8 23:30 LICENSE
-rw-r--r--  1 www-data www-data     59 Jan  8 23:30 Procfile
drwxr-xr-x 26 www-data www-data   4096 Jan 18 21:49 app/
-rw-r--r--  1 www-data www-data   1411 Jan  8 23:30 app.json
-rw-r--r--  1 www-data www-data   1686 Jan  8 23:30 artisan
drwxr-xr-x  3 www-data www-data   4096 Jan 18 21:49 bootstrap/
-rw-r--r--  1 www-data www-data 195724 Jan  8 23:30 changelog.md
-rw-r--r--  1 www-data www-data   6836 Jan  8 23:30 composer.json
-rw-r--r--  1 www-data www-data 406520 Jan  8 23:30 composer.lock
drwxr-xr-x  2 www-data www-data   4096 Jan 18 21:49 config/
-rw-r--r--  1 www-data www-data    276 Jan  8 23:30 crowdin.yml
drwxr-xr-x  4 www-data www-data   4096 Jan 18 21:49 database/
drwxr-xr-x  3 www-data www-data   4096 Jan 18 21:49 frontend/
-rw-r--r--  1 www-data www-data   1763 Jan  8 23:30 index.php
-rw-r--r--  1 www-data www-data    355 Jan  8 23:30 nginx_app.conf
-rw-r--r--  1 www-data www-data    742 Jan  8 23:30 package.json
drwxr-xr-x  6 www-data www-data   4096 Jan 19 19:09 public/
-rw-r--r--  1 www-data www-data  11406 Jan  8 23:30 readme.md
-rw-r--r--  1 www-data www-data   2384 Jan  8 23:30 releases.md
drwxr-xr-x  6 www-data www-data   4096 Jan 18 21:49 resources/
drwxr-xr-x  2 www-data www-data   4096 Jan 18 21:49 routes/
-rw-r--r--  1 www-data www-data   1399 Jan  8 23:30 server.php
-rw-r--r--  1 www-data www-data    576 Jan  8 23:30 sonar-project.properties
drwxr-xr-x 10 www-data www-data   4096 Jan 19 18:55 storage/
drwxr-xr-x 47 www-data www-data   4096 Jan 18 21:49 vendor/
-rw-r--r--  1 www-data www-data   1461 Jan  8 23:30 webpack.mix.js
-rw-r--r--  1 www-data www-data 251233 Jan  8 23:30 yarn.lock
root@firefly:/var/www/firefly# 

storage dir

drwxr-xr-x 10 www-data www-data 4096 Jan 19 18:55 ./
drwxrwx--- 14 www-data www-data 4096 Jan 21 18:21 ../
drwxr-xr-x  3 www-data www-data 4096 Jan 18 21:49 app/
drwxr-xr-x  2 www-data www-data 4096 Jan 18 21:49 build/
drwxr-xr-x  2 www-data www-data 4096 Jan 18 21:49 database/
drwxr-xr-x  2 www-data www-data 4096 Jan 18 21:49 debugbar/
drwxr-xr-x  2 www-data www-data 4096 Jan 18 21:49 export/
drwxr-xr-x  6 www-data www-data 4096 Jan 18 21:49 framework/
drwxr-xr-x  2 www-data www-data 4096 Jan 20 23:08 logs/
-rw-r--r--  1 www-data www-data 3322 Jan 18 23:24 oauth-private.key
-rw-r--r--  1 www-data www-data  812 Jan 18 23:24 oauth-public.key
drwxr-xr-x  2 www-data www-data 4096 Jan 18 21:49 upload/

looks permissions are ok

Also, just tried by using php 8.0 and i still get the same 401 response

[nottheend]

I have a docker image setup via cloudron and since the latest release few days ago I face the issue that I can login successfully, but when I edit/modify/create a transaction, I get an error 'Unauthenticated' within the Transaction Edit window itself. Seems to be related to this one here

[zunami]

i have the same problem when i try to log in with the firefly iii importer with the token i get this error.

Yunohost 11 under Debian 11

[teidova]

Same thing happens to me for some reason. That's strange because it was working at some point I even imported data.

[speedyflipper]

I’m having this issue too. Every couple days, I get the error and create a token and it works. Then repeat

[SanderVocke]

(TLDR: for me, generating a new token and then restarting the firefly container worked)

I am also having this issue. I didn't touch my firefly docker-compose install (using personal token for importer) since May. Now I fired it up again and the importer can't connect. Have since tried:

• upgrade container versions
• create new personal token and apply via env in docker-compose file
• create OAuth token

result is the same: 401 error. I am stuck for now, any suggestion would be much appreciated!

Edit: I found that a restart of the firefly container (not just the importer container) was needed for the new token to work.

[cskujawa]

I was similarly getting the 401, but I believe it was as a result of an improperly formatted request. Sort of like a fallback error. The server I have assembling the payload to POST to firefly was running into an exception when creating the data part of the request, but still firing it off.

In short, the request had the correct headers, URL, and could reach the firefly server, but was getting 401 back strictly based off the data portion of the request being either missing or invalid.

After fixing the server handling for the data the exact same requests work fine now.

[th0rgall]

I got this issue when booting up an old docker-compose Firefly III project (last used in May 2022), that I'm pretty sure worked with the Data Importer back then. My versions were 5.7.6 (Firefly III) & v0.9.9 (data importer).

I tried creating a new Personal API token, clearing cookies, restarting containers, to no avail.

Finally I upgraded the containers to the latest version (v6.1.6 & v1.4.1), which seems to have gone well out-of-the box, and also fixed the authentication issue.

[cortex3]

I have the same issue but a new token and restarting the importer and the firefly pod as @SanderVocke suggested did not work for me.

I cannot even use a new Token with a Curl request but it did work for me in the past. I switched over to Oauth and set the vanity URL and it's working for me now. This is of course more a workaround than a fix.