Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error: Cannot find module 'azure-pipelines-task-lib/task' #485

Closed
hoffiz opened this issue Dec 21, 2018 · 15 comments
Closed

Error: Cannot find module 'azure-pipelines-task-lib/task' #485

hoffiz opened this issue Dec 21, 2018 · 15 comments
Labels

Comments

@hoffiz
Copy link

@hoffiz hoffiz commented Dec 21, 2018

I have an extension with multiple Azure pipeline tasks in it. Some of the task properly install the dependencies listed in the package.json but others do not. Looking at only 2 tasks, one that downloads the necessary modules into \node_modules and one that does not shows no possible suspect.

I can manually, for the tasks that don't download it, run npm install and my pipeline build will work properly and will no longer get the:

Error: Cannot find module 'azure-pipelines-task-lib/task'

Any suggestions of what to try or review?

Looking at similar submitted issues I could not find any clues of what is causing this bug on my side.

@bryanmacfarlane

This comment has been minimized.

Copy link
Contributor

@bryanmacfarlane bryanmacfarlane commented Dec 21, 2018

Tasks need to carry their dependencies. In other words, node_modules and npm install is a task build time operation (not runtime).

Some of the task properly install the dependencies listed in the package.json but others do not

A task shouldn't install at runtime. Also, every task needs to carry it's full dependency tree (every task is self contained) even if there's multiple in one extension.

Does that help?

@hoffiz

This comment has been minimized.

Copy link
Author

@hoffiz hoffiz commented Jan 4, 2019

It does thanks!

@damccorm damccorm added the question label Jan 4, 2019
@damccorm damccorm closed this Jan 4, 2019
@Farzad-Jalali

This comment has been minimized.

Copy link

@Farzad-Jalali Farzad-Jalali commented Jun 6, 2019

Tasks need to carry their dependencies. In other words, node_modules and npm install is a task build time operation (not runtime).

Some of the task properly install the dependencies listed in the package.json but others do not

A task shouldn't install at runtime. Also, every task needs to carry it's full dependency tree (every task is self contained) even if there's multiple in one extension.

Does that help?

Could you be more specific about how to do it?
thanks in advance

@damccorm

This comment has been minimized.

Copy link
Member

@damccorm damccorm commented Jun 6, 2019

Basically, before you publish your task you should run npm install and then you should include the node modules in the package that you are publishing. Does that make sense? If not, could you be more specific about where the issue is?

@shudv

This comment has been minimized.

Copy link

@shudv shudv commented Jun 23, 2019

For anyone who hits this going forward, you can use this starter template for quickly get up and running with a custom pipelines task.

@Mykyta

This comment has been minimized.

Copy link

@Mykyta Mykyta commented Jul 26, 2019

Hi guys, I've faced with the same error. But to reduce the extension package size, I point the path to lib in manifest file, such as

 {"path": "myApp/node_modules/azure-pipelines-task-lib"},
 {"path": "myApp/node_modules/balanced-match"},
{"path": "myApp/node_modules/brace-expansion"},
{"path": "myApp/node_modules/concat-map"},
 {"path": "myApp/node_modules/minimatch"},
 {"path": "myApp/node_modules/mockery"},
{"path": "myApp/node_modules/q"},
{"path": "myApp/node_modules/semver"},
 {"path": "myApp/node_modules/shelljs"},
{"path": "myApp/node_modules/uuid"}

with whole dependencies. Maybe some resolve that issue?
I've un-package .vsix file and see my main.js contains
const tl = __importStar(require("azure-pipelines-task-lib"));
where main.js file is located in the same directory as azure-pipelines-task-lib. That is really strange why I see Cannot find module 'azure-pipelines-task-lib'

@CodeTroopers

This comment has been minimized.

Copy link

@CodeTroopers CodeTroopers commented Dec 6, 2019

I have the same error. I tried with glob path expression in the manifest file with :
"path": "./node_modules/azure-pipelines-task-lib/**/*.js"
but creating package fails with Error: ENOENT: no such file or directory, lstat 'C:\Dev\Extension\Task\node_modules\azure-pipelines-task-lib\**\*.js'

I open an issue here

What is the correct way to include dependencies in the extension? I also tried to compile TypeScript in a single file but commonjs module is not supported!

@mansellan

This comment has been minimized.

Copy link

@mansellan mansellan commented Feb 12, 2020

I had this too - in my case it was because the node_modules directory was not in the same directory as the task script.

@tillig

This comment has been minimized.

Copy link

@tillig tillig commented Feb 24, 2020

This makes it really hard to do any sort of shared code across extensions.

/Common
  common.ts
/Extension1
  task.ts # references common.ts
  task.json
/Extension2
  task.ts # references common.ts
  task.json
/node_modules # has modules used by all three

I see in the official azure-pipelines-tasks repo that any shared code has a really crazy thing where the common code gets packed into a .tgz and then referenced as a .tgz from each task. I'm guessing that's why.

It means any shared code you might have needs to be published as a module outside the task repo and then it'll end up being included multiple times if you have multiple extensions (or a multiple-version extension) in one package.

Is there really no workaround for having more shared code in a single package?

@tillig

This comment has been minimized.

Copy link

@tillig tillig commented Feb 24, 2020

Related: I noticed that on a local dev box doing node task.js on a task can correctly invoke it and the standard Node module search works; it very much doesn't work on the hosted build agents. That makes it really hard to debug or find these issues out. I think I've buried two days in trying to get a shared library to work, getting it totally working on my local machine, and then having it totally not work in the actual agent.

@jessehouwing

This comment has been minimized.

Copy link
Contributor

@jessehouwing jessehouwing commented Feb 24, 2020

I used a slightly different approach here.

https://github.com/microsoft/azure-devops-extension-tasks

But every take must be self contained. Has to do with the fact that tasks pre-date the marketplace extensions and are managed individually internally.

@tillig

This comment has been minimized.

Copy link

@tillig tillig commented Feb 24, 2020

@jessehouwing I was about to file a whole "can we improve the packaging situation" issue but I'm guessing there's nothing to be done there. Like, you can't use webpack because the loc() localization function is hard-tied to the filesystem and if it can't find the files you end up in a stack overflow error. (Yeah, I tried). Which means you may also get pretty limited by the 50MB VSIX max package size if you have build tasks of sufficient complexity - just a Hello World level build task packs in a VSIX at about 500KB - 1MB.

Thanks for the example, I see how you've added things to the files in tsconfig.json instead of using references, which I'm guessing is part of the magic. I also see package.json in every folder. I'll see if I can update my extension to be a little closer to that. I had that, then had a lot of challenge getting the common code references to work. Finally got it, then ended up here. The docs are definitely not sufficient when it comes to putting multiple build tasks in a single extension.

@jessehouwing

This comment has been minimized.

Copy link
Contributor

@jessehouwing jessehouwing commented Feb 24, 2020

With the direction GitHub actions are taking, only a single task per repo... Since the underlying infra is the same... I'm working to make new tasks with just a single task that has an option to select what functionality to trigger.

For really large tasks, create a bootstrapper task that runs npm install on first run. Or that downloads a payload from a binary blob in azure storage.

@tillig

This comment has been minimized.

Copy link

@tillig tillig commented Feb 24, 2020

Back up a few comments it appears we shouldn't be running npm install on first run, but I see what you're saying. The GitHub Actions note is interesting and makes me think perhaps I should go that one-repo-per-task route for our internal actions, too, just so we don't have to refactor it all out later. It'd be more repos/VSIX packages right now, but we'd be ready for the switch when it's time.

@jessehouwing

This comment has been minimized.

Copy link
Contributor

@jessehouwing jessehouwing commented Feb 24, 2020

Yeah, so you can do a fully self-contained task with its own node_modules, but have it run npm install on a subfolder. Like how the Extension Tasks run npm install to install tfx-cli.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
10 participants
You can’t perform that action at this time.