Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: initial Azure Pipelines support #7556

Merged
merged 6 commits into from
Jan 14, 2019
Merged

Conversation

kaylangan
Copy link
Contributor

@kaylangan kaylangan commented Dec 27, 2018

Summary

As discussed in #7211, this PR is to add initial support for CI on Azure Pipelines across Mac, Windows, and Linux. Currently, Travis doesn't run master on Windows so there's a test gap.

This PR adds the azure-pipelines.yaml file to run test-ci-partial on the 3 OS's with Node 10. This PR can also be updated to run multiple versions of Node if that's desired. Some of the tests were updated as well.

The tests on Windows are currently failing. One appears to be path separator related and another fs.watch related (which can be finicky on Windows).

You can see the output of the fork @willsmythe set up. He also set up test reporting:

image

If this change looks good, here's how to proceed to set up Azure Pipelines:

  1. Merge this PR.
  2. Set up the Azure Pipelines app with this repo. You'll need to have the appropriate repo permissions for this. You can find more info in our docs. If granting write access to Azure Pipelines is a non-starter, you can instead create an Azure Pipelines org and set up the connection using OAuth.
  3. Follow the set up flow that immediately follows the app set up to create a Pipeline.

I'll update the CHANGELOG.md once the PR is created.

Test plan

@willsmythe did most of the leg work here and can comment more on the testing he did.

@facebook-github-bot
Copy link
Contributor

Thank you for your pull request and welcome to our community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. In order for us to review and merge your code, please sign up at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need the corporate CLA signed.

If you have received this in error or have any questions, please contact us at cla@fb.com. Thanks!

@facebook-github-bot
Copy link
Contributor

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Facebook open source project. Thanks!

@SimenB
Copy link
Member

SimenB commented Dec 27, 2018

@kaylangan thanks!

@rickhanlonii, do you know who's got the correct permissions? @JoelMarcey perhaps?

@SimenB
Copy link
Member

SimenB commented Dec 30, 2018

The failing tests seems to mostly be a case of not being able to replace rootDir, thus getting long paths.

@kaylangan what's the easiest way to debug? I've got access to a windows box through a virtualbox image, and those tests pass there (at least the few I tested now).

I'd guess the issue is somehow with this: https://github.com/facebook/jest/blob/7b7fd013508de475e2151e50847373713fc97c7e/e2e/Utils.js#L73-L90

No real need for us to roll our own here either, should be possible to find some module which can do this for us. E.g. npm uses this: https://www.npmjs.com/package/tacks

If not that, maybe cwd is wrong when running from os.tmpdir()

@JoelMarcey
Copy link
Contributor

I just gave Azure Pipelines access to the Jest repo.

@kaylangan
Copy link
Contributor Author

@SimenB Sorry for the delayed reply. It looks like the issue with the Windows tests was that TEMP uses the short path name (e.g. ../VSSADM~1/AppData/Local/Temp) whereas jest computes and writes a relative path that uses the long name (e.g. ../VssAdministrator/AppData/Local/Temp). I've updated azure-pipelines.yml to account for this, though it may be worth considering updating jest itself to handle this case. You can find the latest run here.

@SimenB
Copy link
Member

SimenB commented Jan 9, 2019

Thanks!

though it may be worth considering updating jest itself to handle this case.

Not sure I get this. We just use os.tmpdir(), I think? We might call realpath on it, not sure as I'm not sure where the issue is. Is it Jest's cacheDirectory or
when we do writeFiles in the tests? Or in the reporter when we calculate the relative path for printing?

Also, what's the next step now that the app has access to the repo? Is it possible to log in to Azure Pipelines using GH?

@kaylangan
Copy link
Contributor Author

Not sure I get this. We just use os.tmpdir(), I think? We might call realpath on it, not sure as I'm not sure where the issue is. Is it Jest's cacheDirectory or
when we do writeFiles in the tests? Or in the reporter when we calculate the relative path for printing?

The reason the tests failed were partly due to an inconsistency on the Azure Pipelines end (which we're working on). On our Windows machines, TMP and TEMP use the shortened path (VSSADM~1) which is what is returned by os.tmpdir(). Other environment variables like HOME and USERPROFILE use the full name (VssAdministrator).

I'd have to do more digging but it appears that either in writeFiles or the reporter, the full path name VssAdministrator is getting written. Perhaps when using path.resolve or path.relative? It only affected 30 or so tests.

Also, what's the next step now that the app has access to the repo? Is it possible to log in to Azure Pipelines using GH?

Unfortunately, we don't yet have log in with GitHub. But we're working on it!

@JoelMarcey when you gave access to the Azure Pipelines app, did you also create an Azure Pipelines organization? If you already have an Azure Pipelines org, you can merge this PR, then in the Azure Pipelines org click "New Pipeline" and it will detect the .azure-pipelines.yml file. Otherwise, you'll want to merge the PR then go through the flow that follows giving the Azure Pipelines app access to the repo (https://github.com/apps/azure-pipelines).

@willsmythe
Copy link
Contributor

@SimenB - some, if not all, of the tests were failing because of how Jest was reporting the test paths.

When Jest is run from the short/8.3 form of a directory on Windows, "cwd" will stay the short/8.3 name, but "rootDir" (and other paths in the config) get expanded to their long form. When Jest then calculates the relative path for a test (for reporting), the "from" argument is in the short/8.3 form, but the "to" argument is in the normal/expanded form. This results in Jest reporting a path that is technically correct, but not expected ;)

This problem isn't unique to Azure Pipelines. You can see it on Windows by running Jest from the short and normal forms of the same directory:

Running Jest from normal/expanded directory:

C:\work\willsmythe\simple-jest-test>yarn test
yarn run v1.12.3
$ jest
 PASS  ./sum.test.js
  √ adds 1 + 2 to equal 3 (3ms)

(./sum.test.js lives in this directory and its path is correctly reported)

Running Jest from the short/8.3 form of the same directory:

C:\work\WILLSM~1\SIMPLE~1>yarn test
yarn run v1.12.3
$ jest
 PASS  ../../willsmythe/simple-jest-test/sum.test.js
  √ adds 1 + 2 to equal 3 (3ms)

(the reported path is technically correct, but different from above)

I think the right solution requires changes in Jest itself --- either changes to how paths like rootDir are (or are not) normalized, or how relative paths are being calculated for reporting.

We "fixed" the problem in Azure Pipelines by setting a variable that causes TEMP to point to a normal/expanded path. Alternatively, we could have changed e2e/runJest.js to expand the working directory that Jest is run under (i.e. call reapath-native), but this didn't feel like the right fix and would have caused a lot of extra file system calls.

Make sense?

@SimenB
Copy link
Member

SimenB commented Jan 10, 2019

Yeah, that makes sense, thanks! I was able to reproduce in a windows image I've got. Will take a look 🙂

@SimenB
Copy link
Member

SimenB commented Jan 10, 2019

@willsmythe I opened up #7598 which calls realpath on cwd (and rootDir for consistency) during our initial config normalization at startup. Hopefully it passes all tests and doesn't mess anything up (thinking network drives or some symlinked setup).

Do you know of any way to normalize the short form to the long form on windows that does not use realpath? I tried googling, found nothing

@SimenB
Copy link
Member

SimenB commented Jan 10, 2019

@kaylangan @willsmythe Merged that PR, mind rebasing and seeing if things work without the workaround you added?

Also, there's currently a linting error (see https://circleci.com/gh/facebook/jest/42898), a quick yarn eslint e2e/Utils.js --fix should resolve it 🙂

@willsmythe
Copy link
Contributor

@SimenB - thanks for the quick fix. We removed the VSTS_OVERWRITE_TEMP workaround, re-ran the build, but still hit some path-related problems on Windows (much fewer than before).

The failing tests were findRelatedFiles.test.js, jestRequireActual.test.js, and jestRequireMock.test.js (see build 369 for more details). The root cause for these was code in jest-cli (SearchSource.js) and jest-config (getCacheDirectory.js) that gets the current working directory and temp directory by calling process.cwd and os.tmpdir directly . To get these tests working, I made a targeted fix to both. I am certainly cool with backing these changes out, but wanted to get the tests working so we could have the conversation.

In general, it seems like you might need (or need to enforce) a common way to get the current working and temp directories. This would avoid sprinkling realpath(process.cwd()) / os.tmpdir all over (there are 32 places were process.cwd is called directly in non-test code still).

There was also a new test failure on macOS that surfaced after rebasing. It was in e2e\__tests__\hasteMapSize.test.js (introduced in #7580). This test constructs a new HasteMap and sets the root directory to a directory under temp. Apparently there are issues with "watch" on macOS when watching a non-real path (this caused the file change event to never trigger, which caused the test to eventually time out). I made a targeted fix to get around this, but it's worth discussing whether calls to realpath should happen directly in HasteMap instead (probably should, but I wasn't sure about the lifecycle for a HasteMap object).

To answer your previous question, I'm not aware of an alternate way to get the normal/expanded path for a short directory on Windows.

@willsmythe
Copy link
Contributor

See latest build results in Azure Pipelines:

image

There is one intermittent test failure on Linux ("can press "u" to update snapshots"). It appears an extra line is occasionally getting picked up in stdout for this test, which causes a mismatch with the snapshot. We will investigate ...

@SimenB
Copy link
Member

SimenB commented Jan 11, 2019

We moved cwd assignment in #7146, so making sure to access that variable makes sense to me instead of figuring it out other places makes sense to me. Probably just an oversight

it's worth discussing whether calls to realpath should happen directly in HasteMap instead (probably should, but I wasn't sure about the lifecycle for a HasteMap object).

@rubennorte thoughts on that?


Thank you so much for sticking with me through this btw. Very much appreciated! 🎉

@SimenB
Copy link
Member

SimenB commented Jan 14, 2019

Looks like everything is green, that's awesome 😀 I'll see if I can ping some FB peeps to do the necessary configuration so we can land this

@willsmythe
Copy link
Contributor

willsmythe commented Jan 14, 2019 via email

@Daniel15
Copy link
Contributor

Daniel15 commented Jan 14, 2019

I'll see if I can ping some FB peeps to do the necessary configuration so we can land this

@SimenB - What changes do you need? Feel free to ping me if you have trouble getting anything done.

@Daniel15
Copy link
Contributor

As a side note, there's still some tests that are skipped on Windows (search for skipSuiteOnWindows in the repo)... It'd be good to clean those up one day.

@SimenB
Copy link
Member

SimenB commented Jan 14, 2019

Thanks @Daniel15! See this from @kaylangan:

@JoelMarcey when you gave access to the Azure Pipelines app, did you also create an Azure Pipelines organization? If you already have an Azure Pipelines org, you can merge this PR, then in the Azure Pipelines org click "New Pipeline" and it will detect the .azure-pipelines.yml file. Otherwise, you'll want to merge the PR then go through the flow that follows giving the Azure Pipelines app access to the repo (github.com/apps/azure-pipelines).

Also, I'm not sure if FB wants an organization, or if Jest should have its own?

As a side note, there's still some tests that are skipped on Windows (search for skipSuiteOnWindows in the repo)... It'd be good to clean those up one day.

Yeah, for sure. I had one passthrough in May (#6310), might be worth having another one

@SimenB
Copy link
Member

SimenB commented Jan 14, 2019

you can merge this PR, then in the Azure Pipelines org click "New Pipeline" and it will detect the .azure-pipelines.yml file.

Ergh, I guess that means we can merge this, so it's detected correctly

@SimenB SimenB merged commit 34d0222 into jestjs:master Jan 14, 2019
@Daniel15
Copy link
Contributor

Also, I'm not sure if FB wants an organization, or if Jest should have its own?

It's probably fine to have a Jest org. The jest org name is taken, but jestjs is available if you want to create that.

@SimenB
Copy link
Member

SimenB commented Jan 14, 2019

Ah, I can do that, cool! Yeah, jestjs works, that matches the website domain.

Did that, however I got an error when trying to confirm.

image

https://dev.azure.com/jestjs/jest

@Daniel15
Copy link
Contributor

Hmm, we may need an admin of the repo to do it. I don't have admin access to this repo, unfortunately. @cpojer Do you know who has admin access to this repo?

@SimenB
Copy link
Member

SimenB commented Jan 14, 2019

@JoelMarcey has (IIRC, @cpojer does not).

@willsmythe
Copy link
Contributor

@SimenB / @Daniel15 - we (Microsoft) can help you get the Azure Pipelines (aka Azure DevOps) organization name you want. In general, we recommend that your Azure DevOps org name matches your GitHub org name (and the project names within this org match your GitHub repo names).

If you want to go this route, let us know and we can help. In the meantime, feel free to continue down the path of setting up a "jestjs" organization (it can be renamed later).

@JoelMarcey
Copy link
Contributor

Jest should be approved -- it was before, I just approved a second time. Hopefully that doesn't mess anything up.

@Daniel15
Copy link
Contributor

@JoelMarcey I think someone with admin access to the repo needs to actually configure it on the Azure Pipelines end, as it has to add a webhook to the repo.

@SimenB
Copy link
Member

SimenB commented Jan 14, 2019

I'll send an invite to the org to your fb email. If I understand things correctly, go to https://dev.azure.com/jestjs/jest/_build, hit "new" and follow the prompts

@Daniel15
Copy link
Contributor

@SimenB Could you please send me an invite to the org? I usually use daaniel@live.com.au as my Windows Live account, but you could use my FB account danlo@fb.com. Joel can give me temporary admin access to the Jest org to get everything configured.

@SimenB
Copy link
Member

SimenB commented Jan 14, 2019

Done! The live one 🙂 you should be able to invite the fb one if you want

@Daniel15
Copy link
Contributor

Looks like I could add it and it's running successfully: https://dev.azure.com/jestjs/jest/_build/results?buildId=1

@SimenB
Copy link
Member

SimenB commented Jan 14, 2019

Yeah, seems like it works!

#6650

image

Need a badge for the readme, and give it a week or something then turn off appveyor?

@kaylangan
Copy link
Contributor Author

@SimenB I just submitted #7630 for the build badge. 😄

@github-actions
Copy link

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Please note this issue tracker is not a help forum. We recommend using StackOverflow or our discord channel for questions.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators May 12, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants