Netbird Self-Hosted Turns Off Single-Account Mode

[trbutler]

Describe the problem

I'm using a self-hosted instance setup by the Quickstart script. It is on single account mode, but somehow decided to switch on its own to multiple account moment. There seems to be no way to assign new users to the original account.

To Reproduce

1. Go to the server and click "Register" on the Zitadel login screen.
2. The first new user ends up being a new administrator of a separate account.
3. A subsequent new user ends up being assigned as a user under that new administrator.
4. Those accounts cannot see the original user's peers nor can the original administrator see them.

Expected behavior

I'd expect that all users would be assigned to the original administrator (as they did previously with the same Quickstart script). Barring that, there'd be some way to determine which account users get assigned to or to change their assignment.

Are you using NetBird Cloud?

We're running self-hosted 0.30.0.

NetBird version

I'm running 0.30.2 on clients.

[mlsmaycon]

by any chance the netbird domain was changed during the configuration?

[xan-it]

same here. All I have done was a restart of the VM wich hosts the netbird instance.
I'm using the quickstart script with version 0.29.4.
I have not disabled the single account mode.

[trbutler]

by any chance the netbird domain was changed during the configuration?

Yes, @mlsmaycon, I configured it to a custom domain. Could that be the cause? Is there a fix? Thank you!

[trbutler]

So, I've been doing some exploring: I found the store.db SQLite database in the Docker mount and tried to manually edit it, because deleting the accounts that had been malformed via Zitadel did absolutely nothing to remove them from Netbird. So, I deleted them out of the accounts and users. I also noted that the original account still had netbird.selfhosted as its domain, despite the system running (and working just fine!) with my custom domain, so I modified that as well.

Restarting Netbird had it switch back to single user mode. However, I then tried to create another user account and it immediately went back to multi-account mode and created the new user as an administrator again.

Notably, I did just go back into SQLite, modify the user to belong to the original account and assigning its peer to the original account too. But this seem to highlight three problems:

1. Netbird seems to be ignoring the single user mode even when I corrected the account in the SQLite accounts database to show my custom domain.
2. Netbird is not keeping in sync with Zitadel which is a problem because of point 3.
3. Netbird doesn't expose to the main/original administrator other users in other accounts, so there is absolutely no way from within Netbird to administer these erroneous extra users (and because of point 2, they can't be handled in Zitadel either).

[rihards-simanovics]

Same issue here but on 0.33.0. We have moved from netbird.selfhosted to our own orgs nodes.domain.com, at some point in the past. Prior to migration adding a new user say one end with @gmail.com was easy enough but now each time new user get's created, they just become their own admin??? There is little to no documentation on a proper procedure for migrating from netbird.selfhosted to another one...

@trbutler can you explain what you did exactly to get it working again?

[rihards-simanovics]

Ok I figured it out. if you go inside the management container and run /go/bin/netbird-mgmt management --help it quite clearly states that

--single-account-mode-domain string   Enables single account mode. This means that all the users will be under the same account grouped by the specified domain. If the installation has more than one account, the property is ineffective. Enabled by default with the default domain netbird.selfhosted (default "netbird.selfhosted")

If the installation has more than one account, the property is ineffective ignored.

@mlsmaycon perhaps a better wording would be ignored since this is effectively ignored due to multiple domain accounts. That said I figured out the issue either way.

Once I removed all erroneous users from accounts table in store.db leaving only one (i.e. current admin which I matched with account_id from Zitadel), and then just re-signing in with a new user, the said user is automatically populated as a "user" and not the "owner" in store.db : users table.

Now I just need to clean up the orphan peers that have been added during the testing and I'm done 😅

[trbutler]

@rihards-simanovics You must have fixed something I haven't as of yet. You did exactly what I did -- but even after I did that, cleared out the erroneous users from both accounts and users and reassigned the remaining peers, signing into a new account at the login screen created the mess over again.

When you cleared it out did you manually run /go/bin/netbird-mgmt management --single-account-mode-domain domain.com so that it realized that the domain is no longer netbird.selfhosted? I've changed the Docker configuration, but never directly tried to run netbird-mgmt.

[rihards-simanovics]

Hmm so here are some facts about my setup. the docker file under management has this:

  # Management
  management:
    image: netbirdio/management:latest
    restart: unless-stopped
    networks: [netbird]
    volumes:
      - netbird_management:/var/lib/netbird
      - ./management.json:/etc/netbird/management.json
    command: [
      "--port", "80",
      "--log-file", "console",
      "--log-level", "info",
      "--disable-anonymous-metrics=false",
      "--single-account-mode-domain=node.domain.com",
      "--dns-domain=node.domain.com",
      "--idp-sign-key-refresh-enabled",
    ]
    logging:
      driver: "json-file"
      options:
        max-size: "500m"
        max-file: "2"

once you force reconfigured the container with a docker command, stop the management container completely so it is not writing to the store.db - this could be the problem in your case as DB is being locked for edits so when you update something it just gets reverted, od docker is doing something weird, just make sure to make a copy of the file before any changes.

Then make sure that only one entry exists under accounts table with the domain name you specified in the --single-account-mode-domain

once done that head to users table and make sure that the user with the same account id as in accounts table has the "owner" role field and that all other entries with the "user" role have the account_id equal to the "owner" user account_id field:

Start the container, then try logging in with the new user, no need to mess with Zitadel or any other IdP as Netbird only needs the users account ID.

[nazarewk]

Hello @trbutler,

We're currently reviewing our open issues and would like to verify if this problem still exists in the latest NetBird version.

Could you please confirm if the issue is still there?

We may close this issue temporarily if we don't hear back from you within 2 weeks, but feel free to reopen it with updated information.

Thanks for your contribution to improving the project!

[trbutler]

Yes, I can confirm it is still happening. In fact, I just had someone sign in to create a new user tonight and it immediately created a separate account for the user. I had to manually edit the sqlite db once again (setting the user to "user" rather than "owner" and setting the user's account to the main account), as there was no other way to correct the issue. I'm running 0.43.0.