feat: unify import/export private key format #15415

joyqvq · 2023-12-18T20:56:48Z

Description

CLI changes

sui keytool convert: this converts hex, base64 to bech32
sui keytool import: can only import as bech32
sui keytoo export: new, export as bech32

SDK changes

/**
 * This returns a Bech32 encoded string starting with `suiprivkey`,
 * encoding 33-byte `flag || bytes` for the given the 32-byte private
 * key and its signature scheme.
 */
export function encodeSuiPrivateKey(bytes: Uint8Array, scheme: SignatureScheme): string

/**
 * This returns an ParsedKeypair object based by validating the
 * 33-byte Bech32 encoded string starting with `suiprivkey`, and
 * parse out the signature scheme and the private key in bytes.
 */
 export function decodeSuiPrivateKey(value: string): ParsedKeypair

Wallet changes

Export UI supports Bech32 only.
Import UI supports both Hex and Bech32. Hex importing shows warning for deprecation for March 1, 2024.

Test Plan

unit test
CLI scenario

# import bech32 works 
target/debug/sui keytool import suiprivkey1q8typ5p96jtw3s3cqlnxulcg6me26cswjcpa8984z6k4ey4dwefcwlssmpg ed25519
Keys saved as Base64 with 33 bytes `flag || privkey` ($BASE64_STR). 
        To see Bech32 format encoding, use `sui keytool export --address $ADDR` where 
        $ADDR can be found with `sui keytool list`. Or use `sui keytool convert $BASE64_STR`.
╭─────────────────┬──────────────────────────────────────────────────────────────────────╮
│ alias           │                                                                      │
│ suiAddress      │  0x1b87a727f58830d9ba2bfe6ecdc8fb49aa96fa2a2bbe175e128bfee13f6895ff  │
│ publicBase64Key │  AQMSxGmd92VJuD30t5sAOVMO//UWqx85kbuOMDRrUCzWfw==                    │
│ keyScheme       │  secp256k1                                                           │
│ flag            │  1                                                                   │
│ peerId          │                                                                      │
╰─────────────────┴──────────────────────────────────────────────────────────────────────╯

# import with alias and bech32 works
 target/debug/sui keytool import suiprivkey1q8typ5p96jtw3s3cqlnxulcg6me26cswjcpa8984z6k4ey4dwefcwlssmpg --alias test-pr ed25519
Keys saved as Base64 with 33 bytes `flag || privkey` ($BASE64_STR). 
        To see Bech32 format encoding, use `sui keytool export --address $ADDR` where 
        $ADDR can be found with `sui keytool list`. Or use `sui keytool convert $BASE64_STR`.
╭─────────────────┬──────────────────────────────────────────────────────────────────────╮
│ alias           │                                                                      │
│ suiAddress      │  0x1b87a727f58830d9ba2bfe6ecdc8fb49aa96fa2a2bbe175e128bfee13f6895ff  │
│ publicBase64Key │  AQMSxGmd92VJuD30t5sAOVMO//UWqx85kbuOMDRrUCzWfw==                    │
│ keyScheme       │  secp256k1                                                           │
│ flag            │  1                                                                   │
│ peerId          │                                                                      │
╰─────────────────┴──────────────────────────────────────────────────────────────────────╯

# import hex does not work
target/debug/sui keytool import 0x1b87a727f58830d9ba2bfe6ecdc8fb49aa96fa2a2bbe175e128bfee13f6895ff ed25519
Sui Keystore and Sui Wallet no longer support importing 
                    private key as Hex, if you are sure your private key is encoded in Hex, use 
                    `sui keytool convert $HEX` to convert first then import the Bech32 encoded 
                    private key starting with `suiprivkey`.

# export works
target/debug/sui keytool export 0x1b87a727f58830d9ba2bfe6ecdc8fb49aa96fa2a2bbe175e128bfee13f6895ff
╭────────────────────┬────────────────────────────────────────────────────────────────────────────────────────────╮
│ exportedPrivateKey │  suiprivkey1q8typ5p96jtw3s3cqlnxulcg6me26cswjcpa8984z6k4ey4dwefcwlssmpg                    │
│ key                │ ╭─────────────────┬──────────────────────────────────────────────────────────────────────╮ │
│                    │ │ alias           │                                                                      │ │
│                    │ │ suiAddress      │  0x1b87a727f58830d9ba2bfe6ecdc8fb49aa96fa2a2bbe175e128bfee13f6895ff  │ │
│                    │ │ publicBase64Key │  AQMSxGmd92VJuD30t5sAOVMO//UWqx85kbuOMDRrUCzWfw==                    │ │
│                    │ │ keyScheme       │  secp256k1                                                           │ │
│                    │ │ flag            │  1                                                                   │ │
│                    │ │ peerId          │                                                                      │ │
│                    │ ╰─────────────────┴──────────────────────────────────────────────────────────────────────╯ │
╰────────────────────┴────────────────────────────────────────────────────────────────────────────────────────────╯

# export alias works
target/debug/sui keytool export --alias nostalgic-hiddenite                                ✔  10119  10:40:40
╭────────────────────┬────────────────────────────────────────────────────────────────────────────────────────────╮
│ exportedPrivateKey │  suiprivkey1q8typ5p96jtw3s3cqlnxulcg6me26cswjcpa8984z6k4ey4dwefcwlssmpg                    │
│ key                │ ╭─────────────────┬──────────────────────────────────────────────────────────────────────╮ │
│                    │ │ alias           │                                                                      │ │
│                    │ │ suiAddress      │  0x1b87a727f58830d9ba2bfe6ecdc8fb49aa96fa2a2bbe175e128bfee13f6895ff  │ │
│                    │ │ publicBase64Key │  AQMSxGmd92VJuD30t5sAOVMO//UWqx85kbuOMDRrUCzWfw==                    │ │
│                    │ │ keyScheme       │  secp256k1                                                           │ │
│                    │ │ flag            │  1                                                                   │ │
│                    │ │ peerId          │                                                                      │ │
│                    │ ╰─────────────────┴──────────────────────────────────────────────────────────────────────╯ │
╰────────────────────┴────────────────────────────────────────────────────────────────────────────────────────────╯

# convert works

target/debug/sui keytool convert 0x1b87a727f58830d9ba2bfe6ecdc8fb49aa96fa2a2bbe175e128bfee13f6895ff
╭────────────────┬──────────────────────────────────────────────────────────────────────────╮
│ bech32WithFlag │  suiprivkey1qqdc0fe87kyrpkd690lxanwgldy649h69g4mu967z29lacfldz2l766z9q2  │
│ base64WithFlag │  ABuHpyf1iDDZuiv+bs3I+0mqlvoqK74XXhKL/uE/aJX/                            │
│ hexWithoutFlag │  1b87a727f58830d9ba2bfe6ecdc8fb49aa96fa2a2bbe175e128bfee13f6895ff        │
│ scheme         │  ed25519                                                                 │
╰────────────────┴──────────────────────────────────────────────────────────────────────────╯

If your changes are not user-facing and not a breaking change, you can skip the following section. Otherwise, please indicate what changed, and then add to the Release Notes section as highlighted during the release process.

Type of Change (Check all that apply)

protocol change
user-visible impact
breaking change for a client SDKs
breaking change for FNs (FN binary must upgrade)
breaking change for validators or node operators (must upgrade binaries)
breaking change for on-chain data layout
necessitate either a data wipe or data migration

Release notes

Sui Keystore CLI no longer supports an import of a private key as a 32-byte hex string. It now only supports import or export Bech32-string encoded 33-byte flag || private_key starting with suiprivkey. See usage for sui keytool convert -h if you want to see all formatted private keys. See sui keytool export -h if you need to export a private key in bech32 format. This matches the import and export private key format in Sui Wallet and SDK. See SIP for more standard details.

vercel · 2023-12-18T20:56:57Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
sui-core	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jan 29, 2024 11:25pm
sui-typescript-docs	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jan 29, 2024 11:25pm

4 Ignored Deployments

Name	Status	Preview	Updated (UTC)
explorer	⬜️ Ignored (Inspect)	Visit Preview	Jan 29, 2024 11:25pm
multisig-toolkit	⬜️ Ignored (Inspect)	Visit Preview	Jan 29, 2024 11:25pm
mysten-ui	⬜️ Ignored (Inspect)	Visit Preview	Jan 29, 2024 11:25pm
sui-kiosk	⬜️ Ignored (Inspect)	Visit Preview	Jan 29, 2024 11:25pm

sdk/typescript/src/cryptography/keypair.ts

sdk/typescript/src/keypairs/ed25519/keypair.ts

pchrysochoidis

After having another look, it seems ExportedKeyPair and Keypair.export are not necessary anymore, so I opened this PR to deprecate them.

pchrysochoidis · 2024-01-22T12:30:45Z

.changeset/two-olives-enjoy.md

@@ -0,0 +1,5 @@
+---
+'@mysten/sui.js': major


not sure if we do major updates for the sdk - noting for @hayes-mysten or @Jordan-Mysten to confirm

yes everything should be a minor until we get to 1.0

Jordan-Mysten · 2024-01-23T00:45:49Z

apps/wallet/package.json

@@ -135,6 +135,7 @@
 		"@sentry/browser": "^7.61.0",
 		"@tanstack/react-query": "^5.0.0",
 		"@tanstack/react-query-persist-client": "^4.29.25",
+		"bech32": "^2.0.0",


Looks like this might not be needed anymore?

Jordan-Mysten · 2024-01-23T00:46:14Z

apps/wallet/src/shared/utils/from-exported-keypair.ts

+	return keypair;
+}
+
+export function fromExportedKeypair(keypair: ExportedKeypair, legacySupport = false): Keypair {


When will we ever drop legacy support? I'm not sure it makes sense for this to be configurable?

This was added for wallet storage backwards compatibility, cc @pchrysochoidis

We will need to run a storage migration to get rid of this. Although I've this PR #15833 that changes it a bit. We should review/merge that one first.

Jordan-Mysten · 2024-01-23T00:46:30Z

apps/wallet/src/shared/utils/from-exported-keypair.ts

-export function fromExportedKeypair(keypair: ExportedKeypair): Keypair {
-	const secretKey = fromB64(keypair.privateKey);
+export function validateExportedKeypair(keypair: ExportedKeypair): ExportedKeypair {
+	const _kp = decodeSuiPrivateKey(keypair.privateKey);


You can just call the function and not grab a return value.

I removed it in #15833

Jordan-Mysten · 2024-01-23T00:47:34Z

apps/wallet/src/ui/app/helpers/validation/privateKeyValidation.ts

 		try {
-			privateKeyBytes = hexToBytes(hexValue);
+			decodeSuiPrivateKey(privateKey);


Given it's going to take a while for wallets to update, we probably don't want to ship with this, and should be able to detect legacy formats and upgrade them seamlessly.

allowing both hex and bech32 in the current impl

Co-authored-by: stefan-mysten <135084671+stefan-mysten@users.noreply.github.com>

* this allows supporting keys that are stored in base64 format already in the wallet storage

## Description followup to #15415 ## Test Plan How did you test the new or updated feature? --- If your changes are not user-facing and do not break anything, you can skip the following section. Otherwise, please briefly describe what has changed under the Release Notes section. ### Type of Change (Check all that apply) - [ ] protocol change - [ ] user-visible impact - [ ] breaking change for a client SDKs - [ ] breaking change for FNs (FN binary must upgrade) - [ ] breaking change for validators or node operators (must upgrade binaries) - [ ] breaking change for on-chain data layout - [ ] necessitate either a data wipe or data migration ### Release notes

joyqvq requested a review from stefan-mysten December 18, 2023 20:56

joyqvq requested review from a team, patrickkuo, kchalkias and benr-ml as code owners December 18, 2023 20:56

joyqvq requested review from plam-ml and Nikhil-Mysten and removed request for a team December 18, 2023 20:56

joyqvq force-pushed the privkey branch from 8ee2468 to 8dcd647 Compare December 19, 2023 20:22

joyqvq requested review from oxade, amnn and wlmyng as code owners December 19, 2023 20:22

vercel bot deployed to Preview – sui-typescript-docs December 19, 2023 20:22 View deployment

vercel bot deployed to Preview – sui-core December 19, 2023 20:23 View deployment

vercel bot deployed to Preview – mysten-ui December 19, 2023 20:23 View deployment

joyqvq force-pushed the privkey branch from 8dcd647 to 5603275 Compare December 19, 2023 23:05

joyqvq requested review from emmazzz and suiwombat as code owners December 19, 2023 23:05

vercel bot deployed to Preview – sui-typescript-docs December 19, 2023 23:05 View deployment

vercel bot deployed to Preview – sui-core December 19, 2023 23:06 View deployment

vercel bot deployed to Preview – mysten-ui December 19, 2023 23:06 View deployment

hayes-mysten reviewed Dec 20, 2023

View reviewed changes

sdk/typescript/src/cryptography/keypair.ts Outdated Show resolved Hide resolved

sdk/typescript/src/keypairs/ed25519/keypair.ts Outdated Show resolved Hide resolved

sdk/typescript/src/keypairs/ed25519/keypair.ts Outdated Show resolved Hide resolved

joyqvq force-pushed the privkey branch from 5603275 to ddd7c26 Compare December 22, 2023 16:32

vercel bot deployed to Preview – sui-core December 22, 2023 16:33 View deployment

vercel bot deployed to Preview – mysten-ui December 22, 2023 16:34 View deployment

vercel bot deployed to Preview – sui-typescript-docs December 22, 2023 16:34 View deployment

vercel bot deployed to Preview – sui-core December 22, 2023 18:40 View deployment

vercel bot deployed to Preview – sui-typescript-docs December 22, 2023 18:41 View deployment

vercel bot deployed to Preview – sui-core January 18, 2024 15:51 View deployment

joyqvq requested a review from pchrysochoidis January 19, 2024 14:38

pchrysochoidis reviewed Jan 22, 2024

View reviewed changes

hayes-mysten approved these changes Jan 22, 2024

View reviewed changes

Jordan-Mysten reviewed Jan 23, 2024

View reviewed changes

github-actions bot added the Type: Documentation Improvements or additions to documentation label Jan 29, 2024

vercel bot deployed to Preview – sui-typescript-docs January 29, 2024 23:16 View deployment

vercel bot deployed to Preview – sui-core January 29, 2024 23:16 View deployment

joyqvq and others added 6 commits January 29, 2024 18:16

feat: use bech32 encoding for private key

d3a6a5d

address keytool comments

9dd806a

Co-authored-by: stefan-mysten <135084671+stefan-mysten@users.noreply.github.com>

lint and rebase

b6885b0

fix rust test

ae61db5

update pnpm lockfile

7f8888c

support base64 exported keypairs

6bb1bce

* this allows supporting keys that are stored in base64 format already in the wallet storage

joyqvq force-pushed the privkey branch from 43a9b6d to 6bb1bce Compare January 29, 2024 23:17

vercel bot deployed to Preview – mysten-ui January 29, 2024 23:19 View deployment

vercel bot deployed to Preview – sui-typescript-docs January 29, 2024 23:19 View deployment

pchrysochoidis added 5 commits January 29, 2024 18:20

deprecate ExportedKeyPair and Keypair.export

ac94e11

lint

3514945

changeset

b777e21

stop using ExportedKeyPair in wallet

56fa875

support importing hex private keys

705ab63

vercel bot deployed to Preview – sui-typescript-docs January 29, 2024 23:22 View deployment

vercel bot deployed to Preview – sui-core January 29, 2024 23:25 View deployment

joyqvq requested review from Jordan-Mysten and pchrysochoidis January 29, 2024 23:34

joyqvq merged commit a34f1cb into main Jan 30, 2024
41 checks passed

joyqvq deleted the privkey branch January 30, 2024 15:37

joyqvq mentioned this pull request Jan 30, 2024

fix: update sui wallet changeset #16005

Merged

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: unify import/export private key format #15415

feat: unify import/export private key format #15415

joyqvq commented Dec 18, 2023 •

edited by ronny-mysten

vercel bot commented Dec 18, 2023 •

edited

pchrysochoidis left a comment

pchrysochoidis Jan 22, 2024

hayes-mysten Jan 22, 2024

Jordan-Mysten Jan 23, 2024

Jordan-Mysten Jan 23, 2024

hayes-mysten Jan 23, 2024 •

edited

pchrysochoidis Jan 23, 2024

Jordan-Mysten Jan 23, 2024

pchrysochoidis Jan 23, 2024

Jordan-Mysten Jan 23, 2024

joyqvq Jan 30, 2024

feat: unify import/export private key format #15415

feat: unify import/export private key format #15415

Conversation

joyqvq commented Dec 18, 2023 • edited by ronny-mysten

Description

CLI changes

SDK changes

Wallet changes

Test Plan

Type of Change (Check all that apply)

Release notes

vercel bot commented Dec 18, 2023 • edited

pchrysochoidis left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hayes-mysten Jan 23, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

joyqvq commented Dec 18, 2023 •

edited by ronny-mysten

vercel bot commented Dec 18, 2023 •

edited

hayes-mysten Jan 23, 2024 •

edited