Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

restoreKeystore extensions #2445

Merged
merged 5 commits into from May 25, 2021
Merged

restoreKeystore extensions #2445

merged 5 commits into from May 25, 2021

Conversation

KtorZ
Copy link
Contributor

@KtorZ KtorZ commented Mar 9, 2021

  • cf8eb65
    馃搷 extend restoreKeystore's util to also look into top-level ESK
    From the cardano-sl's format, there are two fields which may contains an encrypted secret key: '_usKeys' and '_usWalletSet'. It isn't really clear when and why the '_usWalletSet' was used in the old code base, but the old wallet API offered an endpoint to recover such keystore.

    It is therefore plausible that some users have keystore of that shape in their possession.

    As a disclaimer, this is still untested and based on manual 'reverse engineering' of the old cardano-sl code. This is tedious for multiple reasons:

    • Old versions of cardano-sl do not compile anymore because the build tools have evolved through major versions.
    • Some dependencies of the old cardano-sl refers to git repositories which do not exist anymore.
    • The old code isn't straightforward and make use of template-haskell which is meant to generate haskell code at compile-time. This obfuscate quite a bit the sources.

    I will give a try at getting sources compiled to get my hand on some default keystores from an old installation in order to test this addition.

    Besides, I've noticed that default "empty" keystores have a size of 203 bytes, which is oddly identical to what a few users have been reporting lately. So it could be that users with such keystores are simply holding on default placeholder keystore generated by the application when starting.

Todos

  • Test this addition on a keystore from an old installation (< 2018)

Screenshots

N/A

Testing Checklist

Review Checklist

Basics

  • PR has been assigned and has appropriate labels (feature/bug/chore, release-x.x.x)
  • PR is updated to the most recent version of the target branch (and there are no conflicts)
  • PR has a good description that summarizes all changes
  • PR has default-sized Daedalus window screenshots or animated GIFs of important UI changes:
    • In English
    • In Japanese
  • CHANGELOG entry has been added to the top of the appropriate section (Features, Fixes, Chores) and is linked to the correct PR on GitHub
  • Automated tests: All acceptance and unit tests are passing (yarn test)
  • Manual tests (minimum tests should cover newly added feature/fix): App works correctly in development build (yarn dev)
  • Manual tests (minimum tests should cover newly added feature/fix): App works correctly in production build (yarn package / CI builds)
  • There are no flow errors or warnings (yarn flow:test)
  • There are no lint errors or warnings (yarn lint)
  • There are no prettier errors or warnings (yarn prettier:check)
  • There are no missing translations (running yarn manage:translations produces no changes)
  • Text changes are proofread and approved (Jane Wild / Amy Reeve)
  • Japanese text changes are proofread and approved (Junko Oda)
  • UI changes look good in all themes (Alexander Rukin)
  • Storybook works and no stories are broken (yarn storybook)
  • In case of dependency changes yarn.lock file is updated

Code Quality

  • Important parts of the code are properly commented and documented
  • Code is properly typed with flow
  • React components are split-up enough to avoid unnecessary re-renderings
  • Any code that only works in main process is neatly separated from components

Testing

  • New feature/change is covered by acceptance tests
  • New feature/change is manually tested and approved by QA team
  • All existing acceptance tests are still up-to-date
  • New feature/change is covered by Daedalus Testing scenario
  • All existing Daedalus Testing scenarios are still up-to-date

After Review

  • Merge the PR
  • Delete the source branch
  • Move the ticket to done column on the YouTrack board
  • Update Slack QA thread by marking it with a green checkmark

@KtorZ KtorZ requested a review from nikolaglumac March 9, 2021 08:56
@KtorZ KtorZ self-assigned this Mar 9, 2021
@KtorZ
extend restoreKeystore's util to also look into top-level ESK
538dda5 
  From the cardano-sl's format, there are two fields which may contains an encrypted secret key: '_usKeys' and '_usWalletSet'. It isn't really clear when and why the '_usWalletSet' was used in the old code base, but the old wallet API offered an endpoint to recover such keystore.

  It is therefore _plausible_ that some users have keystore of that shape in their possession.

  As a disclaimer, **this is still untested** and based on manual 'reverse engineering' of the old cardano-sl code. This is tedious for multiple reasons:

  - Old versions of `cardano-sl` do not compile anymore because the build tools have evolved through major versions.
  - Some dependencies of the old cardano-sl refers to git repositories which do not exist anymore.
  - The old code isn't straightforward and make use of template-haskell which is meant to generate haskell code at compile-time. This obfuscate quite a bit the sources.

  I will give a try at getting sources compiled to get my hand on some default keystores from an old installation in order to test this addition.

  Besides, I've noticed that default "empty" keystores have a size of 203 bytes, which is oddly identical to what a few users have been reporting lately. So it could be that users with such keystores are simply holding on default placeholder keystore generated by the application when starting.
@KtorZ KtorZ force-pushed the KtorZ/restore-keystore-extension branch from cf8eb65 to 538dda5 Compare March 9, 2021 08:57
@nikolaglumac nikolaglumac mentioned this pull request Mar 30, 2021
Closed
nikolaglumac
nikolaglumac approved these changes May 25, 2021
View reviewed changes
Copy link
Contributor

@nikolaglumac nikolaglumac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

馃挴

@nikolaglumac nikolaglumac merged commit b64bcad into develop May 25, 2021
@iohk-bors iohk-bors bot deleted the KtorZ/restore-keystore-extension branch May 25, 2021 13:28
cleverca22 pushed a commit that referenced this pull request Jun 1, 2021
@KtorZ @nikolaglumac @cleverca22
restoreKeystore extensions (#2445)
dcdcb3b 
* extend restoreKeystore's util to also look into top-level ESK

  From the cardano-sl's format, there are two fields which may contains an encrypted secret key: '_usKeys' and '_usWalletSet'. It isn't really clear when and why the '_usWalletSet' was used in the old code base, but the old wallet API offered an endpoint to recover such keystore.

  It is therefore _plausible_ that some users have keystore of that shape in their possession.

  As a disclaimer, **this is still untested** and based on manual 'reverse engineering' of the old cardano-sl code. This is tedious for multiple reasons:

  - Old versions of `cardano-sl` do not compile anymore because the build tools have evolved through major versions.
  - Some dependencies of the old cardano-sl refers to git repositories which do not exist anymore.
  - The old code isn't straightforward and make use of template-haskell which is meant to generate haskell code at compile-time. This obfuscate quite a bit the sources.

  I will give a try at getting sources compiled to get my hand on some default keystores from an old installation in order to test this addition.

  Besides, I've noticed that default "empty" keystores have a size of 203 bytes, which is oddly identical to what a few users have been reporting lately. So it could be that users with such keystores are simply holding on default placeholder keystore generated by the application when starting.

* Adds CHANGELOG, Runs linters

Co-authored-by: Nikola Glumac <niglumac@gmail.com>
@nikolaglumac nikolaglumac added release-4.1.0 Daedalus Mainnet and removed 鈴硆elease-vNext labels Jun 2, 2021
@nikolaglumac nikolaglumac mentioned this pull request Jun 4, 2021
Merged
nikolaglumac added a commit that referenced this pull request Sep 14, 2021
@topseniors @nikolaglumac
@danielmain @KtorZ @cleverca22 @tomislavhoracek
[DDW-414] Bump electron and related packages (#2206)
e94535d 
* [DDW-414]: Bump electron and related package versions

* [DDW-414]: Fix yarn.lock

* [DDW-414]: Update changelog

* [DDW-414] Fix builds

* [DDW-414]: Update directory open command due to bumped up electron version

* [DDW-414]: Fix yarn.lock

* [DDW-414]: Fix yarn.lock

* [DDW-414]: Update state snapshot log params for logging

* Electron upgrade to v12, added necessary nix libraries as dependencies

* [DDW-414] Electron 12 set contextIsolation default to true, so global variables are not available in the renderer, that is why its need to be disabled. A better solution is planned.

* [DDW-414] Added sha256 for other distributions.

* [DDW-414] Fixed eslint import

* [DDW-414] commit nix-shell

* [DDW-414] reverted arm64 support, upgrade node to current LTS version

* [DDW-414] yarn lockfile:fix

* [DDW-414] Upgrade to cardano-js 0.4.7, updated nix packages to use node v14

* [DDW-414] fixed windowsElectronVersion

* [DDW-414] upgrade of electron to 12.0.9
[DDW-414] fixed sha256 for electron-gyp
[DDW-414] wrong variable name fixed

* [DDW-414] updated yarn.lock

* [DDW-414] upgraded lodash due a security issue
[DDW-414] fixed lodash version in resolutions section
[DDW-414] removing resolutions
[DDW-414] restored acorn resolution
[DDW-414] reverted resolutions minimatch
[DDW-414] reverted resolutions
[DDW-414] resolution fix version for minimatch

* restoreKeystore extensions (#2445)

* extend restoreKeystore's util to also look into top-level ESK

  From the cardano-sl's format, there are two fields which may contains an encrypted secret key: '_usKeys' and '_usWalletSet'. It isn't really clear when and why the '_usWalletSet' was used in the old code base, but the old wallet API offered an endpoint to recover such keystore.

  It is therefore _plausible_ that some users have keystore of that shape in their possession.

  As a disclaimer, **this is still untested** and based on manual 'reverse engineering' of the old cardano-sl code. This is tedious for multiple reasons:

  - Old versions of `cardano-sl` do not compile anymore because the build tools have evolved through major versions.
  - Some dependencies of the old cardano-sl refers to git repositories which do not exist anymore.
  - The old code isn't straightforward and make use of template-haskell which is meant to generate haskell code at compile-time. This obfuscate quite a bit the sources.

  I will give a try at getting sources compiled to get my hand on some default keystores from an old installation in order to test this addition.

  Besides, I've noticed that default "empty" keystores have a size of 203 bytes, which is oddly identical to what a few users have been reporting lately. So it could be that users with such keystores are simply holding on default placeholder keystore generated by the application when starting.

* Adds CHANGELOG, Runs linters

Co-authored-by: Nikola Glumac <niglumac@gmail.com>

* [DDW-414] AppKit added in nix dependencies

[DDW-414] AppKit added in nix dependencies

* patch usb_bindings

* [DDW-414] TEST: testing with electron 13

* [DDW-414] TEST: testing with electron 13, fixed yarn.lock

* [DDW-414] TEST: updated hashes

* [DDW-414] TEST: updated hashes

* fix mac signing for the new electron

* also ship base64-js and ieee754 on windows and linux

* switch over to using a bash function, and fix the wildcard in signing keccak

* fix windows builds

* [DDW-681] Fixed mouse cursor on high DPI Linux

* [DDW-414] Upgraded to electron 13.1.0

* [DDW-414] Fixed ipc communication due electron breaking change

* [DDW-414] Fixed ipc communication due electron breaking change

* [DDW-414] TEST: Added aarm64

* [DDW-414] TEST: Reverted aarm64 (to be continued in the future)

* generate the correct hash for the directory

* [DDW-414] Converting to plain JS Objects any argument passed via ipc

* [DDW-414] Replaced any with proper types

* [DDW-414] Simplified properties and added more types

* [DDW-414] Added more types

* [DDW-414] Added more types

* [DDW-414] All usages of LocalStorageApi.set that contains type 'any' are converted into plain JS Objects

* [DDW-414] changed to shell.openPath

* [DDW-414] compiled new libraries with node14

* [DDW-414] Issues with ledger nano fixed

* [DDW-414] Issues with ledger nano fixed

* [DDW-414] merded from develop

* [DDW-414] Added ledger bindings for linux

* [DDW-414] added blake2b-wasm

* remove redundant and unsupported git reference

* update HID-hidraw path

* [DDW-414] Simplified the conversion of complex objects

* fix merge conflict gone wrong

* [DDW-414] Fixed flow issues

* [DDW-414] Fixed own merge conflicts

* [DDW-414] Rolled back improvement of SIGN_CMD

* [DDW-414] Fixed BlankScreenFix

* [DDW-414] Fixed wrong fix

* [DDW-414] Added console.logs for debug purposes

* [DDW-414] Added console.logs for debug purposes

* [DDW-414] Reverted logs

* [DDW-414] Logs for getHardwareWalletTransportChannel

* [DDW-414] removed mobx trace

* [DDW-414] More logging in init ledger

* [DDW-414] Added logs in listener

* [DDW-414] Fixed issue with Blank screen fix

* [DDW-414] Refactoring, trying new versions

* [DDW-414] Fixed merge conflicts

* [DDW-414] Changed console.log to logger.debug

* [DDW-414] Fix yarn.lock

* [DDW-414] Fix yarn.lock

* [DDW-414] Fix wrong dependency on yarn.lock

* remove unused typescript

* [DDW-414] Using original yarn.lock from develop + new packages introduced in this PR

* [DDW-414] Fix dependency wor windows

* [DDW-414] Fix dependency wor windows

* [DDW-414] Added logs for buildkite

* [DDW-414] Added logs for buildkite

* [DDW-414] Fixed dependencies for Windows

* [DDW-414] Using original yarn.lock

* [DDW-414] Added a tracer for missing dependencies

* [DDW-414] Fixed yarn.lock

* [DDW-414] Added @types/ws

* [DDW-414] Removed @types/ws

* [DDW-414] Fixed @szmarczak/http-timer dependency

* [DDW-414] Added more tracing

* [DDW-414] Nodejs in debug mode

* [DDW-414] Rollback change

* re-add the old electron-packager support

* [DDW-414] Cleaned up console.logs and tracer

* [DDW-414] Fix HID issue

* [DDW-414] CHANGELOG update

* [DDW-414] Run prettier

* [DDW-414] Fix minimatch resolution

* [DDW-414] Removed aarch64-darwin

* [DDW-414] Enable Catalyst voting on Testnet

* [DDW-414] Addtional debugging log

* [DDW-414] Debugging - store specific error

* [DDW-414] Downgrade node-hid package to last stable non-ts version

* [DDW-414] Improve MacInstaller

* [DDW-414] plist modification

* [DDW-414] plist modification

* [DDW-414] fixed dependency

* [DDW-414] Reverted libraries

* [DDW-414] Removed additional plist

* [DDW-414] Update electron 13.1.7

* [DDW-414] Remove debugging logs

* [DDW-414] Rolled back electron version

* [DDW-414] Rollback Electron version. Added patch for node-usb

* [DDW-414] Patched usb-1.7.1

* [DDW-414] Using newer ledgerhq version

* [DDW-414] Added more options to compile usb, fixed usb version in yarn.lock

* [DDW-414] Modified patch to only contain changes in binding.gyp

* [DDW-414] Fixed wrong diff in binding.gyp

* [DDW-414] Applied full patch

* [DDW-414] Patched the wrong usb version before. Now using 1.7.0

* fix yarn.lock ordering

* fix blake2b urls

* fix yarn.lock

* [DDW-414] SuppressDestruct patch on node-usb

* [DDW-414] Unsubscribing on quit

* [DDW-414] Patch package removed

* [DDW-414] Patching via nix script

* [DDW-414] Unsubscribing on quit

* [DDW-414] Fix patch in nix script

* [DDW-414] Fix yarn.lock and nix script

* [DDW-414] Added postInstall

* [DDW-414] Added postInstall, wrong position

* [DDW-414] Using cheap patch command now

* [DDW-414] Using cheap patch command now, fix path

* [DDW-414] Added logs

* [DDW-414] Changing path, added pwd as log

* [DDW-414] Testing with buildPhase

* [DDW-414] Testing with ${src}

* [DDW-414] Using workaround for node-gyp

* saving current changes

* [DDW-414] Fixed isBlankScreenFixActive issue

* [DDW-414] Fixed electron version (13.1.0)

* [DDW-414] Using 13.1.0

* downgrade electron-rebuild to fix linux builds

* [DDW-414] The release version is patched not the debug one

* [DDW-414] electron rebuild without usb

* [DDW-414] Added force option to the patch

* [DDW-414] Patching in Haskell

* [DDW-414] Fix wrong path by patching in Haskell

* [DDW-414] Added ignore white space

* [DDW-414] Patch compatible with mac

* [DDW-414] Patch using p1 parameter

* [DDW-414] Patching without options

* [DDW-414] Cleaned up patching in Haskell- and nix-scripts

* try using patch -f

* [DDW-414] Fixed local build

* add some debug

* add patch -i

* add patch -p1

* use patch -N

* sh time

* [DDW-414] Removed unused debug commands

* [DDW-414] Rollback isBlankScreenFixActive

* [DDW-414] Disabled flow line. To be reenabled when trezor flow is not ignored anymore

* [DDW-414] Fix GPU status logging

* [DDW-414] Fixes electron-store reset

* [DDW-414] Yarn lockfile fix

* [DDW-414] Fix Yarn lock file fix script

* [DDW-414] Unused script that caused problems in Netlify

* [DDW-414] Fix when not enough fees

* [DDW-414] Using fix versions everywhere

* [DDW-414] Updated static usb libs for windows

* [DDW-414] Removed patch. Using now the fixed usb-node version 1.7.2

* [DDW-414] Using older node-hid version. Building libraries in debug mode

* [DDW-414] Updated static usb libs for windows (including debug libs)

* [DDW-414] Using python3 for node-gyp

* [DDW-414] Documentation why we ship usb debug version

* [DDW-414] Adding logs, downgrade of node-hid

* [DDW-414] Fixed wrong status

* [DDW-414] Wrong toJS conversion

* [DDW-414] Fix lockfile

* [DDW-414] Removes unused package

* [DDW-414] Decrease ws version

* [DDW-414] Remove console logs, Revert voting-testnet-change

Co-authored-by: Nikola Glumac <niglumac@gmail.com>
Co-authored-by: Daniel Main <daniel.main.cernhoff@icloud.com>
Co-authored-by: Matthias Benkort <5680256+KtorZ@users.noreply.github.com>
Co-authored-by: Michael Bishop <michael.bishop@iohk.io>
Co-authored-by: Daniel Main <daniel.main@iohk.io>
Co-authored-by: Daniel Main <daniel@funktional.dev>
Co-authored-by: Tomislav Hora膷ek <tomislav@tt-media.hr>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nikolaglumac nikolaglumac nikolaglumac approved these changes

Assignees

@KtorZ KtorZ

Labels
chore release-4.1.0 Daedalus Mainnet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@KtorZ @nikolaglumac