Skip to content

Conversation

@lina128
Copy link
Collaborator

@lina128 lina128 commented Mar 9, 2020

Add necessary commands for linking and building dependency packages.

p.s. it explicitly build dependencies including the nested dependencies. This is because, if we rely on transitive dependency building, which means the union package, tfjs's, build-ci has to include build-layers-ci, build-data-ci, etc, which will result in these packages built twice, one from cloudbuild.yml, and one from build. This adds too much hidden processes and is error-prone. To make things simple, packages that have nested dependencies, currently only tfjs-node, should take care of building all the dependencies it needs. This is the same behavior if we use yalc link/publish, which doesn't take care of nested dependency either.

p.p.s. Need to remove 'scripts' from tfjs-converter's tsconfig file, and also change the main entry and type in the package.json file. Otherwise, typescript can't find modules symlinked. This issue is reported in typescript's repo, solution is to change from dist/src/index.js to dist/index.js: microsoft/TypeScript#23502
We shouldn't be too worried, because after the change, tfjs-converter's tsconfig and main/type entry will be exactly same as data and layers, which should have been the same in the first place.

This change is Reviewable

Copy link
Contributor

@dsmilkov dsmilkov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 10 of 10 files at r1.
Reviewable status: 0 of 1 approvals obtained (waiting on @annxingyuan, @dsmilkov, @lina128, and @nsthorat)


tfjs-converter/tsconfig.json, line 26 at r1 (raw file):

  },
  "include": [
    "src/"

this means that scripts/**.ts are not going to be compiled anymore (you can introduce a compile error and it won't be caught). If not possible to keep here, can you introduce another tsconfig in scripts and make sure we build them on ci

@dsmilkov
Copy link
Contributor

One comment about scripts. Everything else looks great

@lina128
Copy link
Collaborator Author

lina128 commented Mar 10, 2020

Reviewed 10 of 10 files at r1.
Reviewable status: 0 of 1 approvals obtained (waiting on @annxingyuan, @dsmilkov, @lina128, and @nsthorat)

tfjs-converter/tsconfig.json, line 26 at r1 (raw file):

  },
  "include": [
    "src/"

this means that scripts/**.ts are not going to be compiled anymore (you can introduce a compile error and it won't be caught). If not possible to keep here, can you introduce another tsconfig in scripts and make sure we build them on ci

Hi Daniel,
I think it will still get compiled, the tsconfig is inferred in this case, see this reply for example: microsoft/vscode#17480 (comment)
Another observation, under scripts folder, gen-json.ts is called in CI: https://github.com/tensorflow/tfjs/blob/master/tfjs-converter/cloudbuild.yml#L38 and it works fine for this PR, as shown here: https://pantheon.corp.google.com/cloud-build/builds/87bee5cc-452e-4d70-b629-9d7db9a58a0d;step=4?project=learnjs-174218

Besides gen-json, other scripts are exactly same as tfjs-layers and tfjs-data, which don't have scripts in the their tsconfig, and have been working fine.

Given that scripts are not public APIs and not part of package(?), as long as they are compiled and executed, I think we can live without tsconfig for simplicity. WDYT?

Copy link
Contributor

@dsmilkov dsmilkov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 1 of 1 approvals obtained (waiting on @annxingyuan, @lina128, and @nsthorat)


tfjs-converter/tsconfig.json, line 26 at r1 (raw file):

Previously, lina128 (Na Li) wrote…

Hi Daniel, I think it will still get compiled, the tsconfig is inferred in this case, see this reply for example: microsoft/vscode#17480 (comment) Another observation, under scripts folder, gen-json.ts is called in CI: https://github.com/tensorflow/tfjs/blob/master/tfjs-converter/cloudbuild.yml#L38 and it works fine for this PR, as shown here: https://pantheon.corp.google.com/cloud-build/builds/87bee5cc-452e-4d70-b629-9d7db9a58a0d;step=4?project=learnjs-174218

Besides gen-json, other scripts are exactly same as tfjs-layers and tfjs-data, which don't have scripts in the their tsconfig, and have been working fine.

Given that scripts are not public APIs and not part of package(?), as long as they are compiled and executed, I think we can live without tsconfig for simplicity. WDYT?

That makes sense. Thanks for checking that they get compiled and executed on CI. LGTM!

@lina128 lina128 merged commit b429581 into tensorflow:master Mar 10, 2020
@lina128 lina128 deleted the node branch April 14, 2020 23:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants