Skip to content
This repository has been archived by the owner on Jan 19, 2021. It is now read-only.

Migrate to Typescript #96

Merged
merged 21 commits into from Aug 13, 2019
Merged

Migrate to Typescript #96

merged 21 commits into from Aug 13, 2019

Conversation

s1na
Copy link
Contributor

@s1na s1na commented Jul 29, 2019

Breaking changes are how classes are exported (using named exports). Users can import one of CheckpointTrie, SecureTrie, BaseTrie via `import { SecureTrie } from 'merkle-patricia-tree'.

I don't consider this migration complete. The library in its current is difficult to type and I had to use a lot of type casting (e.g. key as Buffer) and ignoring TS errors via // @ts-ignore for now to get it to compile, and to avoid making this PR bigger than it is. Some parts really need refactoring to have proper types (specially TrieNode), which I plan to do in next PRs. Promisification is also left for future PRs.

I've tested this branch in ethereumjs-vm and tests are passing

@lgtm-com
Copy link

lgtm-com bot commented Jul 29, 2019

This pull request fixes 1 alert when merging b6247ff into 1c4c539 - view on LGTM.com

fixed alerts:

  • 1 for Superfluous trailing arguments

@lgtm-com
Copy link

lgtm-com bot commented Jul 30, 2019

This pull request fixes 1 alert when merging 9a9a033 into 5a5b1a1 - view on LGTM.com

fixed alerts:

  • 1 for Superfluous trailing arguments

@holgerd77
Copy link
Member

@s1na Have updated this branch with the Travis xvfb fix, now the main tests generally pass but there seems to be some babel version related problem left preventing the coveralls and browser test runs to pass, can you please fix?

@alcuadrado
Copy link
Member

I haven't reviewed the code but got curious about the xvfb thing, so I tried running the tests locally and I'm getting a different set of errors. I think there's no need for babel here, so I copied the karma.conf.js from the -block TS migration PR, and it works with some minor changes.

It's here:

module.exports = function(config) {
  config.set({
    browserNoActivityTimeout: 60000,
    frameworks: ['browserify', 'detectBrowsers', 'tap'],
    files: ['./test/*.js'],
    preprocessors: {
      './dist/**/*.js': ['browserify'],
      './test/**/*.js': ['browserify']
    },
    singleRun: true,
    detectBrowsers: {
      enabled: true,
      usePhantomJS: false,
      postDetection: function(availableBrowsers) {
        return ['Firefox']
      },
    }
  })
}

This changes also let us remove the file .babelrc and these dependencies: babelify, karma-chrome-launcher, and karma-env-preprocessor.

@lgtm-com
Copy link

lgtm-com bot commented Jul 31, 2019

This pull request fixes 1 alert when merging 1a2400a into 5a5b1a1 - view on LGTM.com

fixed alerts:

  • 1 for Superfluous trailing arguments

@lgtm-com
Copy link

lgtm-com bot commented Jul 31, 2019

This pull request fixes 1 alert when merging eebf74b into 5a5b1a1 - view on LGTM.com

fixed alerts:

  • 1 for Superfluous trailing arguments

@lgtm-com
Copy link

lgtm-com bot commented Jul 31, 2019

This pull request fixes 1 alert when merging e326642 into 5a5b1a1 - view on LGTM.com

fixed alerts:

  • 1 for Superfluous trailing arguments

@holgerd77
Copy link
Member

Ok, tests working, seems coverage run needs some additional fix?

@s1na
Copy link
Contributor Author

s1na commented Jul 31, 2019

I think it might have something with the fact that we run tests on dist and not on src. But even then, coveralls should have run and gave weird results.

Another point is that when I run npm run coverage locally, it doesn't produce .nyc_outpu/lcov.info, but when I run npx nyc npm run test && npx nyc report --reporter=text-lcov > .nyc_output/lcov.info explicitly it does. Not sure yet what's the cause.

@s1na
Copy link
Contributor Author

s1na commented Jul 31, 2019

By the way, thanks @alcuadrado for the fixed karma config :)

@lgtm-com
Copy link

lgtm-com bot commented Aug 1, 2019

This pull request fixes 1 alert when merging 910cd20 into 5a5b1a1 - view on LGTM.com

fixed alerts:

  • 1 for Superfluous trailing arguments

@s1na
Copy link
Contributor Author

s1na commented Aug 1, 2019

Modified coverage. It now does generate coverage for dist files, and does produce a lcov.info. But the coveralls task still seems to be stuck. Out of ideas, if anyone else has any idea I could try them out.

@alcuadrado
Copy link
Member

(closing and reopening this PR, as that supposedly fixes coveralls)

@alcuadrado alcuadrado closed this Aug 1, 2019
@alcuadrado alcuadrado reopened this Aug 1, 2019
@alcuadrado
Copy link
Member

That didn't work. We should try this, but I don't have enough permissions. @holgerd77 maybe?

@lgtm-com
Copy link

lgtm-com bot commented Aug 2, 2019

This pull request fixes 1 alert when merging 467a457 into 5a5b1a1 - view on LGTM.com

fixed alerts:

  • 1 for Superfluous trailing arguments

@s1na
Copy link
Contributor Author

s1na commented Aug 2, 2019

Finally! Either updating the coveralls package or using npm run coverage && coveralls < .nyc_output/lcov.info in the script instead of the ethereumjs-config coveralls script seems to have worked (I'm guessing the first one).

Coverage went down 2%, probably because it's measuring coverage over the compiled files and not the sources. Not sure what to do about it

Before this, I also added CircleCI config files (to test if coveralls works there). Do we want to switch to circle or remain with travis?

@holgerd77
Copy link
Member

Great! 😄

I would have a tendency to not side-introduce CI tool switches here and keep on Travis at least along with this PR.

@holgerd77
Copy link
Member

@s1na Have increased the failure decrease threshold, for the TS transitions I think it's ok to make an exception with all these heavy structural changes likely shifting things a bit.

There is still a 90% absolute threshold which will cover us here.

@lgtm-com
Copy link

lgtm-com bot commented Aug 5, 2019

This pull request fixes 1 alert when merging c693c65 into 5a5b1a1 - view on LGTM.com

fixed alerts:

  • 1 for Superfluous trailing arguments

@s1na
Copy link
Contributor Author

s1na commented Aug 5, 2019

Ok, removed the circleci config, but only afterwards removed this repo from the circleci app, that's why it tried to build but failed.

If necessary I can do a rebase after the review to "fix" the circleci error.

@holgerd77
Copy link
Member

Yes, maybe a rebase would be good here, especially for these large/central PRs it looks a bit ugly if CI is displayed as "failing". Can you actually do before the review, otherwise the approval will be again dismissed on rebase?

@holgerd77
Copy link
Member

Update: or maybe wait, I'll likely have some small comments to update.

Copy link
Member

@holgerd77 holgerd77 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First round on non-source files, looks good so far, to be continued...

.gitignore Show resolved Hide resolved
.nycrc Show resolved Hide resolved
package.json Show resolved Hide resolved
README.md Show resolved Hide resolved
@s1na
Copy link
Contributor Author

s1na commented Aug 8, 2019

My main concern for rebasing was that Github might not detect modifications to the files and doesn't show a good diff, which would make the review process harder.

Copy link
Member

@holgerd77 holgerd77 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sometimes a review is already worth to be done to be able to fully appreciate other ones work, wasn't aware that the update on this library was so massive and extensive, thanks for giving this such a direct shot. 👍 😄

Some comments, some questions, some change requests, but generally looks pretty good and complete already.

src/baseTrie.ts Outdated
const ReadStream = require('./readStream')
const PrioritizedTaskExecutor = require('./prioritizedTaskExecutor')
const { callTogether } = require('./util/async')
const { stringToNibbles, matchingNibbleLength, doKeysMatch } = require('./util/nibbles')

/**
* Use `require('merkel-patricia-tree')` for the base interface. In Ethereum applications
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you update this introductory comment text here as well with the accurate require instructions?

src/baseTrie.ts Show resolved Hide resolved
src/baseTrie.ts Show resolved Hide resolved
src/baseTrie.ts Show resolved Hide resolved
src/baseTrie.ts Show resolved Hide resolved
src/trieNode.ts Show resolved Hide resolved
src/util/async.ts Show resolved Hide resolved
src/util/hex.ts Show resolved Hide resolved
src/util/nibbles.ts Show resolved Hide resolved
test/streams.js Show resolved Hide resolved
@s1na
Copy link
Contributor Author

s1na commented Aug 12, 2019

coveralls error: Bad response: 503 <h2>This website is under heavy load (queue full)</h2><p>We're sorry, too many people are accessing this website at the same time. We're working on this problem. Please try again later.</p> 🤦‍♂️

@lgtm-com
Copy link

lgtm-com bot commented Aug 12, 2019

This pull request fixes 1 alert when merging a07c39f into 5a5b1a1 - view on LGTM.com

fixed alerts:

  • 1 for Superfluous trailing arguments

@s1na
Copy link
Contributor Author

s1na commented Aug 12, 2019

@holgerd77 Thanks a lot for doing this extensive review. For several comments I had to double-check to see if I had made a mistake. Glad to know if I sometime do make a mistake, you'll likely catch it!

A note on the new errors and asserts: In those places errors were possible, it's just that before the code would have failed a bit further with an unrelated error such as undefined has no attribute X... The typescript compiler pushed me to make the errors more explicit.

Copy link
Member

@holgerd77 holgerd77 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good now! 🙂 Let's see if we can re-trigger the Travis run...

@holgerd77
Copy link
Member

Ah, CI is not working, re-trigger didn't work.

@holgerd77
Copy link
Member

Hmm, coverage looks good on Travis. This seems to be some (hopefully) temporary problem with Coveralls. Have temporarily disabled the check as being required and will merge now.

@holgerd77 holgerd77 merged commit 882799e into master Aug 13, 2019
@holgerd77 holgerd77 deleted the ts branch August 13, 2019 08:15
@s1na
Copy link
Contributor Author

s1na commented Aug 13, 2019

Yeah coveralls is down since yesterday (even their homepage). Hopefully it'll be fixed soon. Thanks for merging.

@holgerd77
Copy link
Member

@s1na Just read your comment about the TS release on the other issue thread. Would it be worth to do an in-between major release here? This is hanging around for quite some time and then we would have a new work basis, eventually this would also ease the transition for people if they can do this in-between hop. Or are you planning to integrate the refactoring changes from #100 and subsequent PRs on a very short-term basis?

@s1na
Copy link
Contributor Author

s1na commented Nov 4, 2019

Or are you planning to integrate the refactoring changes from #100 and subsequent PRs on a very short-term basis?

If we decide it's better to release all changes together I'd be okay with making the changes in a relatively short time frame.

Would it be worth to do an in-between major release here?

Now whether it makes sense to do a major release before making additional changes:

there have been a lot of changes that have not yet been released. Considering that doing a release is sensible. The good thing about doing a release now is that as you said the breaking changes are not that much and it'd be easier to migrate.

On the other hand, in it's current form the codebase is not very polished. I had to have a look at the recent PRs and here I wrote I don't consider this migration complete.. So there's definitely a lot possible changes we could do to improve things. But these improvements will likely include a few breaking changes (one major being Promises).

So in short, the current state is not so bad not to be releasable, but there is a lot of room for improvement. What do you think @holgerd77?

@holgerd77
Copy link
Member

@s1na There not too much time pressure regarding a major release, apart from the fact that it would be nice that others can benefit of the improvements made. If you can do the simplifications/refactorings within the next 2-3 weeks I would be fine with waiting for a larger major release. We just shouldn't draw this release beyond the Christmas holidays.

@s1na
Copy link
Contributor Author

s1na commented Nov 5, 2019

Okay, let's target end of November as a target for release.

@ryanio ryanio mentioned this pull request Apr 17, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants