Restore username registration functionality for production apps

[markmhendrickson]

We had to disable username registration functionality for production apps upon the release of Stacks 2.0 since the underlying subdomain registrar powering both app.blockstack.org and browser.blockstack.org was not updated in time.

Several app developers have disabled their onboarding flows entirely while they wait for us to restore this functionality.

This issue is meant to track the entirety of this work in one place, linking to other issues and PRs as relevant.

Steps towards restoration

• Update subdomain registrar to Stacks 2.0 (Tintash)
• Update ENV variable handling in support of hosting needs (UserX @ Hiro PBC)
• Resolve issues with Atlas for testnet and mainnet (Blockchain @ Hiro PBC)
• Host registrar against testnet (Devops @ Hiro PBC)
• Remove core.blockstack.org dependency in favor of Stacks Blockchain API for Stacks Wallet, Blockstack Connect and Stacks.js (UserX @ Hiro PBC)
• Deploy Atlas fixes to mainnet (Blockchain @ Hiro PBC)
• Host registrar against mainnet (Devops @ Hiro PBC for Stacks Foundation)
• Deploy new version of Blockstack Connect (app.blockstack.org) with username registration restored using registrar newly hosted by Stacks Foundation (UserX @ Hiro PBC)
• Reenable onboarding flows for apps (App developers)

Contributing parties

• Tintash team: @asimm241
• Stacks Foundation: @blocks8 @jcnelson
• Hiro PBC: @andresgalante @diwakergupta @gina @hstove @wileyj @markmhx
• App developers: @pradel @raj015 @sdsantos @wilsonbright

[markmhendrickson]

@asimm241 @hstove I have a few outstanding questions for you two here regarding the registrar updates: stacks-network/subdomain-registrar#42 (comment)

@blocks8 @jcnelson Could one of you indicate the Foundation's availability to host this registrar soon once the updates are finalized? I believe @wileyj may be available to help out where relevant, but I'm not entirely sure what your hosting setup looks like for this.

[markmhendrickson]

I'm working with devops at Hiro PBC to set up a testnet instance of the newly updated registrar (per https://github.com/blockstackpbc/devops/issues/593, internal to PBC), now that the registrar PR has been approved with all relevant updates (per stacks-network/subdomain-registrar#42).

[markmhendrickson]

An update with some timing goals from the Hiro PBC side:

• Get the new registrar running against testnet internally by Thursday, Feb 11th so we can start wiring up username registration against it with the Stacks Wallet for web, Blockstack Connect (as maintained on app.blockstack.org), and Blockstack Browser.
• Work with the Stacks Foundation to help them deploy the registrar against mainnet by Monday, Feb 15th.
• Deploy new versions of the Stacks Wallet for web, Blockstack Connect and Blockstack Browser with username registration supported by the Foundation's registrar by Tues, Feb 16th.

[markmhendrickson]

Since publishing my update a couple days ago, we've realized that newly discovered issues with Atlas on both mainnet and testnet have introduced a blocker for getting this registrar up and running on both networks.

Blockchain engineers are currently working to resolve these Atlas issues as a top priority and we don't yet have a clear estimate of when they will be resolved, though we hope imminently.

I will update this thread with new timing goals once they are resolved and we can proceed to run the registrar on testnet then mainnet as supported by these fixes. It currently appears as though the above dates will get shifted back by at least a few days.

Thanks for your patience as always. 🙏

[briantegridy]

Thanks for the update and transparency.

[markmhendrickson]

I've been informed that these are the three issues being tackled on the Stacks blockchain side for Atlas that need resolution before we can proceed with hosting the registrar for testnet or mainnet:

• Atlas: attachment downloader starves the P2P state-machine of event IDs stacks-network/stacks-core#2428
• Atlas: get_attachments_missing_at_page_index() can create an unbound-length SQL query stacks-network/stacks-core#2427
• Atlas: do not use chainstate transactions on the network path if it can be avoided stacks-network/stacks-core#2425

Much thanks to @lgalabru in particular for diving into these. 🙏 💯

In the meantime, I've created a separate issue that tracks configuration needs for the Stack Foundation's instance of this updated registrar, which will power authentication for apps as supported by the Hiro PBC products listed above by default.

[markmhendrickson]

Hi all, we don't have a precise ETA yet, but work is moving along to resolve the above Atlas issues on the following PR: stacks-network/stacks-core#2453

[sdsantos]

Any update on this @markmhx? This checklist looks untouched since it was created: stacks-network/subdomain-registrar#43

[markmhendrickson]

It appears as though @lgalabru has entered the testing phase for "Resolve issues with Atlas for testnet and mainnet " per stacks-network/stacks-core#2453.

I just checked off "Update ENV variable handling in support of hosting needs" in the list since @hstove resolved that one recently and we merged all the registrar's upgrades into main.

Once the Atlas fixes are merged, we can proceed to hosting the new registrar against testnet and preparing client-side changes with it.

[lgalabru]

hey @markmhx! The atlas fixes are being reviewed, but I don't think that testing has to be sequential. we did produce an image (https://github.com/blockstack/stacks-blockchain/runs/1983724616?check_suite_focus=true), used internally by Tintash, that can be deployed. So if you're at a point where the registrar is ready and can be tested on staging / testnet, I think we should proceed.

[markmhendrickson]

@lgalabru Fantastic! So this means @CharlieC3 and @wileyj can proceed directly to deploying that image to testnet?

[lgalabru]

@markmhx yeah that would be great for end to end testing.

[CharlieC3]

@lgalabru Does this image need to be deployed for both the testnet miner and API's follower? Or would just one of them be sufficient?

[lgalabru]

Multiple would be great, so we can test the replication.

[markmhendrickson]

Update: We're still working to get the Atlas fixes running on testnet so we can then proceed to hosting the registrar against it in support of integration and testing work.

[pradel]

@markmhx do you have an update regarding the dates?

[markmhendrickson]

I don't have any specific an update regarding dates unfortunately, though we're still working to restore this functionality ASAP.

The Atlas fixes are now running on testnet and we've proceeded to host the registrar against it. I expect we'll have it functioning sometime this week and can start re-integration of username registration into the Stacks Wallet and Blockstack Connect.

[markmhendrickson]

Update: We've gotten the Atlas fixes running on testnet and successfully deployed the new subdomain registrar as configured against testnet. We're proceeding to re-integrate username registration back into Blockstack Connect for testing.

[markmhendrickson]

We're now testing re-integration of username registration in Blockstack Connect on this PR: #1103

We expect to push it live sometime early next week assuming Atlas fixes go live on mainnet then (cc @lgalabru). 🚀

[agraebe]

quick update: yesterday, Hiro deployed the latest node changes non-live mainnet followers. once these are caught up, we can switch API traffic to them. we also seem to be ready to redeploy the subdomain registrar changes. Our assumption is that we should have the Atlas a registrar updates rolled out to mainnet today EOD

[markmhendrickson]

Note that the ETA for these deployments has been pushed to EOD today instead of yesterday. We're getting close!

[joaogmatos]

Any updates when the re-integration of username registration would be complete?
Thank you.

[markmhendrickson]

We're finalizing code review and QA for the above Blockstack Connect PR: #1103

We expect to push these changes for restoring username registration live very soon! 🤞

The mainnet registrar is also already live at https://registrar.stacks.co and we have a related API fix going live soon for BNS here: hirosystems/stacks-blockchain-api#560

[markmhendrickson]

Username registration has been restored for Blockstack Connect as hosted at https://app.blockstack.org/ 🥳

Developers will need to upgrade their apps to the latest version of @stacks/connect (version 4.3.17), and pass registerSubdomain: true as part of authOptions, to take advantage of this restoration. See example.

Please let us know if you have any questions! And thanks for your patience as we've pulled together all the necessary pieces to restore this functionality. 🙏

[pradel]

@markmhx thanks for the news, I just tested it locally and the username flow registration is working 🎉.
But I am having an issue fetching the profile of the user using the username. It looks like I am not able to get the information about this new account.

import { lookupProfile } from '@stacks/auth';

await lookupProfile({ username: 'testusenameregistration.id.stx' });

Throw an error with the following message "Error: Name not found".

[hstove]

Hey @pradel , good callout. Essentially you'll need to update @stacks/auth to the latest version as well - connect already links to it but you probably have multiple version dependencies.

There is a beta already published, but it'll be in a production release (hopefully today) once this PR is merged: hirosystems/stacks.js#984 (review)

I would try using 1.3.0-beta-1 for @stacks/network and 1.3.0-beta-3 for @stacks/auth if you want to debug ASAP.

[pradel]

@hstove I tried to upgrade the SDK but got the following issue hirosystems/stacks.js#1016