Mercurial Resolver for Bower #1448

Closed
wants to merge 3 commits into
from

Projects

None yet
@phenomnomnominal

Seems to be working! Resolver for ssh/http/https Mercurial repositories + tests.

@rayshan
Member
rayshan commented Aug 16, 2014

Very cool! I'm not familiar w/ Mercurial, but just want to say thanks for the PR and the hard work!

@sheerun
Contributor
sheerun commented Sep 6, 2014

@sindresorhus Can this go to 1.4.x?

@sheerun sheerun added the enhancement label Sep 6, 2014
@sindresorhus
Member

Yes. Up to you :)

@sindresorhus sindresorhus added this to the 1.4.x milestone Sep 6, 2014
@sheerun sheerun was assigned by sindresorhus Sep 6, 2014
@codingtony

Any update on this PR?

@sheerun
Contributor
sheerun commented Jan 16, 2015

Someone needs to review this. Currently I'm working on other bower stuff.

@sheerun sheerun removed their assignment Jan 16, 2015
@RobbinHabermehl

@sheerun; I'd be willing to test this PR in an actual environment, would that be of any help?

@sheerun
Contributor
sheerun commented Jan 19, 2015

yes :) just confirmation everything works as expected at least on short run

@RobbinHabermehl

Last week I've tested this version of Bower, specifically TradeMe/bower@b64b365. I've elaborated my findings in the following table:

command scenario outcome
bower install fresh install working properly
bower update update dependency working properly
bower update resolve specific commit not working, seems intended
bower update resolve specific branch working properly
bower update resolve specific tag working properly

Overall, Bower supporting Mercurial in general is worth the inability to resolve to a specific commit in my opinion.

@sheerun
Contributor
sheerun commented Jan 26, 2015

Any clue why you cannot resolve specific commit? Is there a reason?

@RobbinHabermehl

When specifying a commit like this:

{
  "dependencies": {
    "repo-name": "hg+ssh://user@domain.ext/repo-name#da39a3ee5e6b4b0d3255bfef95601890afd80709"
  }
}

Bower reports the following:

bower repo-name#da39a3ee5e6b4b0d3255bfef95601890afd80709
  ENORESTARGET Tag/branch da39a3ee5e6b4b0d3255bfef95601890afd80709 does not exist

Considering the fact that "Tag/branch" is mentioned specifically implicates the lack of commit resolving is intended. Also, when looking at the source code and unit tests the commit resolving mechanism just seems to be absent.

@sheerun
Contributor
sheerun commented Jan 26, 2015

There seems to be commit detection: https://github.com/TradeMe/bower/blob/master/lib/core/resolvers/HgResolver.js#L158-L165

It works incorrectly then. @phenomnomnominal Could you fix it?

@sheerun
Contributor
sheerun commented Jan 26, 2015

I think bower shoudn't support revision number, but only changeset ids.

AFAIK using revision numbers is not deterministic?

@RobbinHabermehl

It should only support changeset ID's indeed.

Apparently I've missed that feature.. Looking at the regular expression (/^r[a-f0-9]{4,40}/) the changeset ID should be prefixed with an r to mark the target as a revision. I've done so in my Bower configuration and now it works as expected!

The prefixing feels off though, especially since it's not the way to go for Git repositories on both Bower and NPM.

@phenomnomnominal

Happy to make any changes needed, what would the expected behaviour be?

@sheerun
Contributor
sheerun commented Jan 26, 2015

Recognize commits as /[a-f0-9]{40}/

@sheerun
Contributor
sheerun commented Jan 26, 2015

Plus maybe some tests

@ghost
ghost commented Feb 18, 2015

We are about to start a private bower registry and need to have Mercurial behind it. Any chance I can help getting this pull request into the master in the near future? Thx.

@RobbinHabermehl

@sheerun, the tests seem to be pretty complete actually, so I went ahead with adjusting the commit resolving. As the pull request has gotten pretty outdated I tried to merge the most recent version of Bower into it. Unfortunately, this caused the following unit tests to fail:

  • HgResolver ._resolve should call the necessary functions by the correct order;
  • HgResolver ._savePkgMeta should save the release in the package meta;
  • HgResolver .resolve should export correctly if resolution is a tag;
  • HgResolver .resolve should export correctly if resolution is a commit;
  • HgResolver .resolve should export correctly if resolution is a branch.

@phenomnomnominal, could you please fix this? You can check out my fork to see the issues directly.

@phenomnomnominal phenomnomnominal Merged latest, fixed UTs, fixed revision ID RegExp
* Merged in latest bower
* Update unit tests with changes
* Fixed revision ID matching RegExps to not require a leading ‘r’
0c27039
@phenomnomnominal

@RobbinHabermehl think I've got this all sorted if someone wants to have a look.

@RobbinHabermehl

@phenomnomnominal, I just checked out your updates and ran the tests. Unfortunately four unit tests are still failing, the good news is that it's one less than last time. Also, the cause of the failures has changed, there appear to be some difficulties running hg commands.

I've posted the stack traces below:

1) HgResolver ._savePkgMeta should save the release in the package meta:
     Uncaught Error: expected '0.0.1' to equal '0.0.1+build.5'
      ..
      at ~/bower/test/core/resolvers/hgResolver.js:726:42
      ...
  From previous event:
      ...
      at Context.<anonymous> (~/bower/test/core/resolvers/hgResolver.js:761:14)
      ...

  2) HgResolver .resolve should export correctly if resolution is a tag:
     Uncaught Error: Failed to execute "hg branches", exit code of #255
      at createError (~/bower/lib/util/createError.js:4:15)
      at ChildProcess.<anonymous> (~/bower/lib/util/cmd.js:102:21)
      at ChildProcess.emit (events.js:110:17)
      at maybeClose (child_process.js:1008:16)
      at Process.ChildProcess._handle.onexit (child_process.js:1080:5)
  From previous event:
      at Function.defer (~/bower/node_modules/q/q.js:542:19)
      at executeCmd (~/bower/lib/util/cmd.js:61:22)
      at PThrottler._processEntry (~/bower/node_modules/p-throttler/index.js:89:25)
      at Promise.apply (~/bower/node_modules/p-throttler/node_modules/q/q.js:1122:26)
      at Promise.promise.promiseDispatch (~/bower/node_modules/p-throttler/node_modules/q/q.js:752:41)
      at ~/bower/node_modules/p-throttler/node_modules/q/q.js:1337:14
      at flush (~/bower/node_modules/p-throttler/node_modules/q/q.js:108:17)

  3) HgResolver .resolve should export correctly if resolution is a commit:
     Uncaught Error: Failed to execute "hg clone ~/bower/test/assets/package-hg -r 0 /var/folders/t3/36f8vf4510791gy9sfjs__qm0000gn/T/Robbin/bower/package-hg-94711-ftC4Wb", exit code of #255
      ... Stack trace is identical to 2 ...

  4) HgResolver .resolve should export correctly if resolution is a branch:
     Uncaught Error: Failed to execute "hg branches", exit code of #255
      at createError (~/bower/lib/util/createError.js:4:15)
      at ChildProcess.<anonymous> (~/bower/lib/util/cmd.js:102:21)
      at ChildProcess.emit (events.js:110:17)
      at maybeClose (child_process.js:1008:16)
      at Process.ChildProcess._handle.onexit (child_process.js:1080:5)
  From previous event:
      at Function.defer (~/bower/node_modules/q/q.js:542:19)
      at executeCmd (~/bower/lib/util/cmd.js:61:22)
      at PThrottler._processEntry (~/bower/node_modules/p-throttler/index.js:89:25)
      at Promise.apply (~/bower/node_modules/p-throttler/node_modules/q/q.js:1122:26)
      at Promise.promise.promiseDispatch (~/bower/node_modules/p-throttler/node_modules/q/q.js:752:41)
      at ~/bower/node_modules/p-throttler/node_modules/q/q.js:1337:14
      at flush (~/bower/node_modules/p-throttler/node_modules/q/q.js:108:17)
@phenomnomnominal

Hmm that's bizarre, they all pass on my machine. I'll investigate further!

@RobbinHabermehl

Perhaps it has to do with the OS, Node.js or Mercurial? I'm running Mac OS X 10.10, Node.js is running version 0.12.0 and my Mercurial version is 3.1.1. The Node.js dependencies are up to date as I performed a fresh install.

@phenomnomnominal

@RobbinHabermehl Yeah it might be something, although I just ran them on my Windows box at work and they all passed (apart from some breaking due to different new line characters). I'll try updating my mercurial at home and having another go.

@RobbinHabermehl

I just updated Mercurial to version 3.3 and the number of failing tests has been reduced a little further, now only three tests are failing:

 1) HgResolver .resolve should export correctly if resolution is a tag:
     Uncaught Error: Failed to execute "hg tags", exit code of #255
      at createError (~/bower/lib/util/createError.js:4:15)
      at ChildProcess.<anonymous> (~/bower/lib/util/cmd.js:102:21)
      at ChildProcess.emit (events.js:110:17)
      at maybeClose (child_process.js:1008:16)
      at Process.ChildProcess._handle.onexit (child_process.js:1080:5)
  From previous event:
      at Function.defer (~/bower/node_modules/q/q.js:542:19)
      at executeCmd (~/bower/lib/util/cmd.js:61:22)
      at PThrottler._processEntry (~/bower/node_modules/p-throttler/index.js:89:25)
      at Promise.apply (~/bower/node_modules/p-throttler/node_modules/q/q.js:1122:26)
      at Promise.promise.promiseDispatch (~/bower/node_modules/p-throttler/node_modules/q/q.js:752:41)
      at ~/bower/node_modules/p-throttler/node_modules/q/q.js:1337:14
      at flush (~/bower/node_modules/p-throttler/node_modules/q/q.js:108:17)

  2) HgResolver .resolve should export correctly if resolution is a commit:
     Uncaught Error: Failed to execute "hg clone ~/bower/test/assets/package-hg -r 0 /var/folders/t3/36f8vf4510791gy9sfjs__qm0000gn/T/Robbin/bower/package-hg-99099-fWA8ts", exit code of #255
      ... Identical to 1 ...

  3) HgResolver .resolve should export correctly if resolution is a branch:
     Uncaught Error: Failed to execute "hg branches", exit code of #255
      ... Identical to 1 and 2 ...

Apparently something causes Mercurial to exit with status code 255.

@phenomnomnominal

Still can't repro this, tried a few machines...

@RobbinHabermehl

I tried removing the repository locally and cloning it again, but then I'm still encountering these three failing tests. In order to determine whether the problems are caused by my MacBook I ran the tests on another machine (OS X 10.10.2, Node.js v0.12.0, NPM v2.5.1 and Mercurial v3.3) as well, unfortunately resulting in the same errors.

As this didn't help I started to dig deeper and started to analyze the stdout and stderr of the failing hg commands. Per failing test I logged both and the output is as follows:

1) HgResolver .resolve should export correctly if resolution is a tag

hg clone /Users/Robbin/Development/bower/test/assets/package-hg /var/folders/t3/36f8vf4510791gy9sfjs__qm0000gn/T/Robbin/bower/package-hg-4530-XZEzT9
abort: repository /Users/Robbin/Development/bower/test/assets/package-hg not found!

hg clone /Users/Robbin/Development/bower/test/assets/package-hg /var/folders/t3/36f8vf4510791gy9sfjs__qm0000gn/T/Robbin/bower/package-hg-4530-rsZ3Ey
abort: repository /Users/Robbin/Development/bower/test/assets/package-hg not found!

hg branches
abort: no repository found in '/private/var/folders/t3/36f8vf4510791gy9sfjs__qm0000gn/T/Robbin/bower/package-hg-4530-XZEzT9' (.hg not found)!

2) HgResolver .resolve should export correctly if resolution is a commit

hg tags
abort: no repository found in '/private/var/folders/t3/36f8vf4510791gy9sfjs__qm0000gn/T/Robbin/bower/package-hg-4530-rsZ3Ey' (.hg not found)!

hg clone /Users/Robbin/Development/bower/test/assets/package-hg -r 0 /var/folders/t3/36f8vf4510791gy9sfjs__qm0000gn/T/Robbin/bower/package-hg-4530-FIrLd8
abort: repository /Users/Robbin/Development/bower/test/assets/package-hg not found!

3) HgResolver .resolve should export correctly if resolution is a branch

Due to the order in the console it seems that no errors are related to the third failing test. Mocha's stack trace shows the cause is Failed to execute "hg branches", exit code of #255 though. Therefore the order is probably just a timing thing making my grouping incorrect.

Two of the logs clearly state the .hg file is missing, perhaps this could lead to something?

@phenomnomnominal

Awesome, will look into this tonight!

@phenomnomnominal

I have no idea what this is. Can't reproduce, have cloned it on every computer I have access to. Stumped.

@RobbinHabermehl

Which OS's have you tried @phenomnomnominal? And which version of Node.js was installed? I just noticed Bower is optimized for Node.js v0.10.0, while I've been running v0.12.0 on both devices. Perhaps that's the cause.

I guess that appveyor.yml should be extended with a choco install hg in order to support the Mercurial tests on the CI server. Can you confirm whether the absence of this is the cause of the failed build @sheerun? I can't seem to find any details about the PR's build.

@phenomnomnominal

Tried it on Windows 7 and OS X 10.10, both with 0.10.36. I'm not sure if the node version would be the problem, because node is basically just running Mercurial. It must be an issue with how Mercurial is running. Frustrating.

@RobbinHabermehl

Done, without errors. I reverted Node.js to version 0.10.32 and now it works like a charm!

Later today I'll run the integration tests again to determine whether this PR can finally be merged.

@RobbinHabermehl

As promised, hereby the results of my integration tests:

command scenario outcome
bower install fresh install working properly
bower update update dependency working properly
bower update resolve specific commit working properly (!)
bower update resolve specific branch working properly
bower update resolve specific tag working properly

As you can see all of these scenarios have passed now.

@sheerun, could you take a look at my previous question and let us know how to handle that? Once that's been taken care of, this pull request seems ready to be merged!

@sheerun
Contributor
sheerun commented Apr 7, 2015

@phenomnomnominal I have time to look at it. Could you rebase it on master? There were many changes there since this PR (including many more tests to possibly fail).

We need passing tests on all platforms (Linux, OSX on Travis and Windows on Appveyor).

@sheerun
Contributor
sheerun commented Apr 7, 2015

Also, please fix indentation and whitespacing (tabs instead of 4 spaces etc.) :)

@renegat4

Any news on this? I would realy like to see Mercurial support in bower.

@sheerun
Contributor
sheerun commented Jul 15, 2015

Yes, pluggable revolvers I'm working on will allow it.

On Wednesday, July 15, 2015, renegat4 notifications@github.com wrote:

Any news on this? I would realy like to see Mercurial support in bower.


Reply to this email directly or view it on GitHub
#1448 (comment).

@phenomnomnominal

Sorry, this thing has fallen off my radar a bit as my current project is no longer using bower. I will hopefully have some free time to get this PR sorted and up to date it if is still even required?

@alvaromb

Yes, that would be awesome @phenomnomnominal. We use mercurial in our company and we're struggling to find a reasonable option to handle common, non-public front-end libraries. Let me know if I can help you with anything.

@sheerun
Contributor
sheerun commented Sep 7, 2015

@phenomnomnominal I managed to implement Pluggable Resolvers interface for Bower. It has different (simpler) API than native resolvers, so some refactor is needed to implement Mercurial resolver, but now you can publish it by yourself :) No need to wait for anyone to review.

I really appreciate the work you've done! I hope you'd find time to re-write it as Pluggable Resolver.

The plan is to re-write all native resolvers as plugin resolvers, and switch to them in Bower 2.0. I'm looking forward to to include Mercurial as one of the built-in pluggable resolvers.

@sheerun sheerun closed this Sep 7, 2015
@tetor
tetor commented Sep 8, 2015

It's good news!
@phenomnomnominal @sheerun Nice work 😄

@phenomnomnominal

I'm having a look at this now, will see how I get on in the next couple of days. Repo will be here: https://github.com/phenomnomnominal/mercurial-bower-resolver

@phenomnomnominal

https://github.com/phenomnomnominal/mercurial-bower-resolver is pretty much working now, please try it and open issues!

@ormico
ormico commented Nov 16, 2016

is the intention still to ship the mercurial resolver plug-in as one of the build-in resolvers?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment