Cross-project dependencies

[scriby]

I have two typescript projects (app and util). The app project references files in the util project.

I set up my gulp-tsc with sources from the app project, but during compilation, the typescript compiler also compiles files in the util project. This is mostly ok, but because the util files don't get added to the gulp output files, there's a couple caveats:

• The files under util don't get moved to the destination. This is actually a good thing because I wouldn't want them moved under app.
• Further "pipes" in the gulp stream don't get applied to the util files.

I'm not even sure there's anything worthwhile to do about this, but just thought I'd bring it up to see if you had any ideas.

Thanks,

Chris

[scriby]

For my current use case, if it moved files under both app and util to the specified location (relative to each project), that would be correct. And it would be nice if it piped the util files through the rest of the gulp stream as well.

I'm not sure if that's possible, or good default behavior.

[kotas]

@scriby Thanks for report!

Yep, it is not designed behavior that files referenced by compile targets leave the compiled files in the source directory...

If I understand correctly, the structure you mentioned is like this:

app/
    main.ts    # references sub.ts and util.ts
    sub.ts
util/
    util.ts

And what you want as the result is like:

app/
    main.ts
    sub.ts
    build/
        main.js
        sub.js
util/
    util.ts
    build/
        util.js  

Is that correct?

IMO it is not "natural" that one gulp task put genereated files into multiple destinations (app/build and util/build) by default.

Instead, I think gulp-tsc should simulate --outDir option of tsc command, and if you write a gulp task like this:

gulp.task('compile', function () {
  return gulp.src('app/main.ts')
    .pipe(typescript())
    .pipe(gulp.dest('build/'))
});

Then, the result should be:

build/
    app/
        main.js
        sub.js
    util/
        util.js
app/
    main.ts
    sub.ts
util/
    util.ts

This is the same result as running tsc --outDir build app/main.ts. I think it is natural and suitable for default behavior as gulp plugin.

Do you agree with this?

---

By the way, just an idea: The result structure you desire would be achieved by using another gulp plugin "mapping piped files to multiple destination" that allows you to write a gulp task like this:

gulp.task('compile', function () {
  return gulp.src('app/main.ts')
    .pipe(typescript())
    .pipe(multidest({
      'app/**/*.js': 'app/build/',
      'util/**/*.js': 'util/build/'
    }))
});

But I can't find out an ideal plugin does things like that. So I think I will write up the plugin by myself when I have time. Would it be great?

[kotas]

As you know, the current version of gulp-tsc tries to "calculate" output files from source files like .ts -> .js or .ts -> .d.ts, etc. But this is unstable and gulp-tsc cannot follow files compiled implicitly as you mentioned in this issue.

On PR above, I have completely changed how gulp-tsc finds the output files. By passing --outDir option to tsc command with a temporary directory, all output files go into that directory so that gulp-tsc can easily find out all output files and clean up them by just removing the temporary directory.

If you have some comment on this change, I'd like to hear that! :)

[scriby]

I tested out the branch and it mostly works. The biggest issue is this:

app
  build
    app
      source
        code.js
    util
      source
        code.js
  node_modules
  source
    code.ts

util
  node_modules
  source
    code.ts

Once the util files get compiled into app's build directory, they can no longer load the node_modules from util. I was playing around with this structure and you can make it work if you symlink the node_modules into the build folder. This seemed overly complicated though, and I just decided to compile the source in-place. It's a little "less clean", but all the references and things work out correctly.

I also noticed a bug in the new branch where if I compile the source in place, the .ts files at the top level don't get compiled (or their output gets lost).

I also tried this structure to fix the node_modules problem:

app
  node_modules
  build
    code.js
  source
    code.ts

util
  node_modules
  build
    code.js
  source
    code.ts

But it doesn't work either. The problem there is that code in app that references util won't work at runtime. The references will be pointing to code living under source, but at runtime it needs to be pointing at code under build.

But just compiling in place solves all the issues. And my IDE supports folding the .js and .map files under the .ts file, so it's not so bad.

Thanks,

Chris

[kotas]

@scriby Big thanks to your cooperation!

Once the util files get compiled into app's build directory, they can no longer load the node_modules from util.

I see... I didn't realize that you have node_modules for both app and util. But yes, it is cross-"project".

In-place compile is obvious way to solve it, but how about using main option in package.json?

Example project is here: https://github.com/kotas/gulp-tsc-cross-project

app/
    node_modules/
    build/
       app.js
    source/
       app.ts
    gulpfile.js

util/
    node_modules/
    build/
        utils.js
    source/
        utils.ts
    package.json

In util/package.json, set main to build/utils.js so that you can refer to util/build/utils.js from app/build/app.js by require("../../util").

package.json:
{
  "name": "util",
  "main": "build/utils.js",
}

However, some trick is needed for gulp-tsc to output files to build directory under each project.

// app/gulpfile.js
var gulp = require('gulp');
var typescript = require('gulp-tsc');
var map = require('map-stream');

gulp.task('default', function () {
  return gulp.src('source/**/*.ts')
    .pipe(typescript())
    .pipe(map(function (file, callback) {
      // Fix path from (outdir)/app/source/ to (outdir)/app/build/
      file.path = file.path.replace(/\/(app|util)\/source\//, '/$1/build/');
      callback(null, file);
    }))
    .pipe(gulp.dest('../'));
});

Or, I think it is much cleaner and more consistent to split your util project into other repository so that app project can install it via the repository as dependency.

---

I also noticed a bug in the new branch where if I compile the source in place, the .ts files at the top level don't get compiled (or their output gets lost).

Sorry, I can't reproduce that. What the structure and gulp task is like?

When the structure is like:

app/
  source/
    top1.ts
    top2.ts
    sub/
        sub1.ts
        sub2.ts
  gulpfile.js

And gulp task is something like:

gulp.task('default', function () {
    return gulp.src('source/**/*.ts')
        .pipe(typescript())
        .pipe(gulp.dest('source'))
});

the .ts files at the top level don't get compiled sounds like source/top1.js and source/top2.js are not compiled, but actually they are compiled on my env.

[scriby]

I do have the util project split into a separate repository (you just have to check both out into the same parent directory).

I think your structure with the package.json "main" looks interesting. I'm happy enough w/ the in-place compile and will probably go with it for now.

I really appreciate you spending all this time to help me out! Thanks!!!

[kotas]

All right! I'll close the issue and ship the branch as a next version.

Thank you so much for using my plugin! 😃

[alex6o]

hi! first of all awesome plugin and support kotas :)
we have a very similar project structure and we have noticed the same problem when compiling the source files with cross project references (angular app). so far we get the same results as described in this issue

app
  build
    app
      source
        code.js
    util
      source
        code.js

the util project is in our case a common project which should be compiled independently from the app. is there a way to resolve this issue so that we get the following file structure (including working sourcemaps) ??

app
  build
     code.js

thanks in advance :)

[kotas]

@MrMuh Hi, thank you for using gulp-tsc! :)

I have published gulp-tsc v0.8.0 with new pathFilter option.

You can modify paths of compiled files by using the option. See document for details.

For your case, you will need a gulp task like following:

gulp.task('compile', function () {
    return gulp.src(['app/source/**/*.ts'], { read: false })
        .pipe(typescript({
            sourcemap: true,
            outDir: '.',
            pathFilter: { 'app/source': 'app/build', 'util/source': 'util/build' }
        }))
        .pipe(gulp.dest('.'));
});

By pathFilter, app/source/xxx.js will go to app/build/xxx.js and util/source/ to util/build/. And changing destination directory to the root of projects will make your compiled files go to (root)/app/build/ or (root)/util/build/ respectively.

Sourcemaps are supported!

Tell me if it doen't meet your requirements. Thanks!

[alex6o]

@kotas awesome! works like a charm :) thx for your great support!