Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] absolute->relative module path transformation #15479

Closed
Kitanotori opened this issue Apr 30, 2017 · 71 comments
Closed

[FEATURE] absolute->relative module path transformation #15479

Kitanotori opened this issue Apr 30, 2017 · 71 comments

Comments

@Kitanotori
Copy link

@Kitanotori Kitanotori commented Apr 30, 2017

Problem

tsc does not support transforming absolute module paths into relative paths for modules located outside node_modules. In client side apps, this is often not an issue because people tend to use Webpack or similar tool for transformations and bundling, but for TypeScript apps targeting Node.js, this is an issue because there is usually no need for complex transformations and bundling, and adding additional build steps and tools on top of tsc only for path transformation is cumbersome.

Example input (es2015 style):

import { myModule } from 'myModuleRoot/a/b/my_module';

Example output (CommonJS style):

const myModule = require('./a/b/my_module');

My personal opinion is that relative module paths (import { ... } from '../../xxx/yyy';) are an abomination and make it difficult to figure out and change application structure. The possibility of using absolute paths would also be a major benefit of using TypeScript for Node.js apps.

Solution

Compiler options for tsc similar to Webpack's resolve.modules.

Could this be achieved for example with existing baseUrl and paths options and adding a new --rewriteAbsolute option?

Related

#5039
#12954

@aluanhaddad

This comment has been minimized.

Copy link
Contributor

@aluanhaddad aluanhaddad commented May 5, 2017

My personal opinion is that relative module paths (import { ... } from '../../xxx/yyy';) are an abomination and make it difficult to figure out and change application structure.

While I heartily agree with your sentiment, I think rewriting such imports would open up a can of worms that would ultimately break or complicate an insanely large number of tools and workflows that rely on the emitted JavaScript using the same module specifiers. Even under a flag, I think it will introduce a lot of complexity.

I hate relative paths that go up, it is just awful for maintainability, but my two cents is that this needs to be done on the NodeJS side, not the transpiler side. Of course it's extremely unlikely that will ever happen...

@ikokostya

This comment has been minimized.

Copy link

@ikokostya ikokostya commented May 5, 2017

This issue can be solved if add node_modules directory with symlink to src directory:

project_root/
  node_modules/  <-- external modules here
  src/
     node_modules/  <-- keep this folder in git
        src -> ../src  <-- symlink to src
     a/
        b/
           c.ts
     d.ts
  tsconfig.json

In this case you can use the following import in c.ts:

import * as d from 'src/d';

instead

import * as d from '../../d';

All will work in typescript and commonjs. You just need to add exclude field in tsconfig.json:

{
    "exclude": [
        "src/node_modules"
    ]
}
@Kitanotori

This comment has been minimized.

Copy link
Author

@Kitanotori Kitanotori commented May 6, 2017

@aluanhaddad I've been using path rewrite (Webpack) on client-side since I began writing React apps. How would it cause problems on the server side if it doesn't cause problems on the client side? Editors already support setting module roots, and linters (ESLint, TSLint) seem to be fine also.

To the maintainers of TypeScript: Please add support for custom module roots so that we can get rid of the awful and unmaintainable import paths.

@Kitanotori

This comment has been minimized.

Copy link
Author

@Kitanotori Kitanotori commented May 6, 2017

@ikokostya Many tools have special treatment for node_modules, and it is always treated as a folder for external code. Putting app code into node_modules may solve the import path problem, but introduces potential other problems. It is also ugly solution that shouldn't, in my opinion, be recommended to anyone.

@ikokostya

This comment has been minimized.

Copy link

@ikokostya ikokostya commented May 6, 2017

Many tools have special treatment for node_modules, and it is always treated as a folder for external code.

Could you provide example of such tools? In my example all external code are placed in node_modules in project_root.

Putting app code into node_modules may solve the import path problem, but introduces potential other problems.

Which problems?

It is also ugly solution that shouldn't, in my opinion, be recommended to anyone.

Maybe you should read this

And why it's ugly? It uses standard way for loading from node_modules.

@aluanhaddad

This comment has been minimized.

Copy link
Contributor

@aluanhaddad aluanhaddad commented May 7, 2017

@Kitanotori perhaps I misunderstood your suggestion, TypeScript supports this feature with the --baseUrl flag.
My point was that NodeJS doesn't support it and that TypeScript should not attempt to provide the future on top of NodeJS by rewriting paths in output code.

@ikokostya

Could you provide example of such tools?

There are too many to count but TypeScript is definitely an example of such a tool.

I think putting application code in a node_modules folder is a very ill-advised hack.

@Kitanotori

This comment has been minimized.

Copy link
Author

@Kitanotori Kitanotori commented May 7, 2017

@aluanhaddad --baseUrl setting enables to transpile the app, but what's the point of being able to transpile successfully if you can't execute it? ts-node does not seem to support --baseUrl, so I think it is inaccurate to say that TypeScript supports custom absolute paths.

@ikokostya Thanks for the links. It seems that setting NODE_PATH in the startup script is the best way currently. I think I will go with that approach.

@aluanhaddad

This comment has been minimized.

Copy link
Contributor

@aluanhaddad aluanhaddad commented May 9, 2017

@Kitanotori

@aluanhaddad --baseUrl setting enables to transpile the app, but what's the point of being able to transpile successfully if you can't execute it? ts-node does not seem to support --baseUrl, so I think it is inaccurate to say that TypeScript supports custom absolute paths.

The point is that it does execute perfectly in environments that support that. RequireJS, SystemJS, and even Webpack support setting a base URL.

What I'm trying to say is that the issue is on the NodeJS side. TypeScript provides base URL configuration to integrate with and take advantage of those other tools.

@RyanCavanaugh

This comment has been minimized.

Copy link
Member

@RyanCavanaugh RyanCavanaugh commented May 9, 2017

Our general take on this is that you should write the import path that works at runtime, and set your TS flags to satisfy the compiler's module resolution step, rather than writing the import that works out-of-the-box for TS and then trying to have some other step "fix" the paths to what works at runtime.

We have about a billion flags that affect module resolutions already (baseUrl, path mapping, rootDir, outDir, etc). Trying to rewrite import paths "correctly" under all these schemes would be an endless nightmare.

@Kitanotori

This comment has been minimized.

Copy link
Author

@Kitanotori Kitanotori commented May 9, 2017

@RyanCavanaugh Sorry to hear that. If Webpack team was able to deliver such feature, I thought TypeScript team could also - considering that TypeScript even has major corporate backing. It's a shame that people are forced to add another build step on top of tsc for such a widely needed feature.

@morlay

This comment has been minimized.

Copy link

@morlay morlay commented May 10, 2017

Still hope TypeScript could support this.
Or Just make it easy to create a plugin, so we can build something like babel-plugin-module-resolver to make it work. (ref: #11441)

@azarus

This comment has been minimized.

Copy link

@azarus azarus commented Jun 4, 2017

So anyone got any solution or a really nice workaround without babel to make this happen ?

Edit
I've ended up with a custom transform script using gulp & tsify & gulp-typescript
https://gist.github.com/azarus/f369ee2ab0283ba0793b0ccf0e9ec590
Browserify & Gulp samples included.
So anyone got any solution or a really nice workaround without babel to make this happen ?

@Kitanotori

This comment has been minimized.

Copy link
Author

@Kitanotori Kitanotori commented Jun 9, 2017

@azarus I went with the solution of adding this to my top level file:

import * as AppModulePath from 'app-module-path';
AppModulePath.addPath(__dirname);

The downside is that if the loading order differs from expected, the app crashes. Better solution would be to have a compiler plugin for converting absolute paths to relative paths. However, I'm not sure if TypeScript has any kind of plugin feature.

@Kitanotori

This comment has been minimized.

Copy link
Author

@Kitanotori Kitanotori commented Jun 9, 2017

I created a post-build script for converting absolute paths to relative (doesn't yet have way to set the module root paths, but one can easily implement such): https://gist.github.com/Kitanotori/86c906b2c4032d4426b750d232a4133b

I was thinking of having the module roots being set in package.json via moduleRoots array containing path strings relative to package.json. I wonder what kind of risks there are in this kind of approach?

@azarus

This comment has been minimized.

Copy link

@azarus azarus commented Jun 9, 2017

I've ended up with my own post build script too,
https://gist.github.com/azarus/f369ee2ab0283ba0793b0ccf0e9ec590
Browserify & Gulp samples included.

It acutally uses the tsconfig.json paths and baseUrl setup :)
I am gonna make a npm package from this during the weekend, i just haven't had time to properly test the script.

@kayjtea

This comment has been minimized.

Copy link

@kayjtea kayjtea commented Aug 3, 2017

I have this in my tsconfig:

{
  "compilerOptions": {
    ...
    "module": "commonjs",
    "moduleResolution": "node",
    "rootDir": "src",
    "baseUrl": "src",
    "paths": {
      "~/*": ["*"]
    }
  },

I can't remember if the other options above are necessary, but "paths" will allow this:

import {fundManagersApi} from "~/core/api/fund-managers.api";

The core directory is at src/core.

And then I use the npm package "module-alias" and do:

require('module-alias').addAlias("~", __dirname + "/src");

At the top of my top level file.

Webstorm understands the paths option and will auto-import correctly even.

@vakhtang

This comment has been minimized.

Copy link

@vakhtang vakhtang commented Sep 6, 2017

I use tsconfig-paths to handle this during runtime. I'm not aware of any performance implications or issues otherwise. The one objection I could see is monkey-patching Node's module resolution strategy.

@0x80

This comment has been minimized.

Copy link

@0x80 0x80 commented Jan 6, 2018

@kayjtea I don't think module and moduleResolution are related. Here's a slightly simpeler version that seems to work:

    "baseUrl": "./",
    "paths": {
      "~/*": [
        "src/*"
      ]
    },

Personally, I prefer writing @src, and for this you can use:

 "baseUrl": "./",
    "paths": {
      "@src/*": [
        "src/*"
      ]
    },

In VSCode I don't seem to have to do anything with my top-level files or npm module-alias.

--- edit ---
Ah damn it. I see now that the paths don't work at runtime work if you don't do more then make the compiler and VSCode happy.

@azarus

This comment has been minimized.

Copy link

@azarus azarus commented Jan 6, 2018

@0x80

This comment has been minimized.

Copy link

@0x80 0x80 commented Jan 6, 2018

@azarus I think I might have to go that route too.

I have two repo's that are also depending on roughly the same set of helpers and services. And I am getting tired of copying code between the two.

@azarus

This comment has been minimized.

Copy link

@azarus azarus commented Jan 6, 2018

@0x80 i ran into a similar problem when had to share model definitions between 5 different services. I solved this by creating a shared folder that everyone were able to access if needed.
Setting up an NPM link is also easy and less hassle than you would think.
But i am sad that proper solution is still not in place.

@duffman

This comment has been minimized.

Copy link

@duffman duffman commented Jan 14, 2018

Use tspath (https://www.npmjs.com/package/tspath) run it in the project folder after compile, no config needed, it uses the tsconfig to figure things out (a recent version of node is needed)

@0x80

This comment has been minimized.

Copy link

@0x80 0x80 commented Jan 14, 2018

@duffman Thanks for the tip! I'll check it out 👍

@stereobooster

This comment has been minimized.

Copy link

@stereobooster stereobooster commented Feb 20, 2018

In case you want to configure create-react-app project add:

{
  "compilerOptions": {
    //... other react-scripts-ts options
    "paths": {
      "src/*": ["*"]
    },
    "baseUrl": "src"
  },

Is equivalent to NODE_PATH=./ in .env in standard c-r-a project.

@xiGUAwanOU

This comment has been minimized.

Copy link

@xiGUAwanOU xiGUAwanOU commented May 15, 2019

Are there any working solution to make path aliases work for type declaration files? I was trying to use TypeScript to write a reusable library, however the path won't be resolved in the generated *.d.ts file. I could use 3rd party tools to resolve the generated *.js code, but they don't work with type declaration files.

@longlho

This comment has been minimized.

Copy link

@longlho longlho commented May 15, 2019

We wrote one for that use case in Dropbox https://github.com/dropbox/ts-transform-import-path-rewrite

@craigsumner

This comment has been minimized.

Copy link

@craigsumner craigsumner commented May 15, 2019

This solution works for output javascript and type definition files:
https://www.npmjs.com/package/@zerollup/ts-transform-paths

@stavalfi

This comment has been minimized.

Copy link

@stavalfi stavalfi commented May 17, 2019

@longlho @craigsumner none of them is working... at least I couldn't configure them to work. any chance you link to a small repo for use example with ttypescript?

@codeaid

This comment has been minimized.

Copy link

@codeaid codeaid commented May 17, 2019

@stavalfi I can confirm that @craigsumner's suggestion works, we've been successfully using it in production for months.

It's been a while since I set it up but looking at my configuration files it looks like this is what you need to do:

First install @zerollup/ts-transform-paths and ttypescript as your NPM/Yarn dependencies. Next update your tsconfig.json file to include the plugins section. This is how mine looks like:

{
    "extends": "./tsconfig.json",
    "include": ["src/**/*"],
    "compilerOptions": {
        "baseUrl": ".",
        "removeComments": true,
        "paths": {
            "lib": ["src/lib"],
            "lib/*": ["src/lib/*"],
            "types": ["src/types"],
            "types/*": ["src/types/*"]
        },
        "plugins": [{ "transform": "@zerollup/ts-transform-paths" }]
    }
}

You probably don't need the paths but in case it doesn't work without them, notice that I had to include each path twice - once for the root directory and once for all its contents.

Finally, update your package.json to use ttsc. I've improved with additional steps and it looks like this but you can obviously just point build at the ttsc command:

"scripts": {
    "build": "yarn build:clean && yarn build:compile && yarn build:assets",
    "build:assets": "copyfiles \"./src/**/assets/**/*\" ./dist --up 1",
    "build:clean": "rimraf dist",
    "build:compile": "ttsc -p tsconfig.prod.json"
}

We have separate configuration files for development and production hence I'm referencing tsconfig.prod.json. In your case it would probably be just ttsc without any parameters.

@stavalfi

This comment has been minimized.

Copy link

@stavalfi stavalfi commented May 17, 2019

If any typescript library creator still has this problem, I created a Webpack loader for babel-plugin-module-resolver, which converts absolute paths to relative paths:
https://github.com/stavalfi/babel-plugin-module-resolver-loader

It also solved the problem of .d.ts files with absolute paths by converting them to relative paths.


What do you think?

@ford04

This comment has been minimized.

Copy link

@ford04 ford04 commented Jun 20, 2019

It gets even more shaky, when you have .d.ts files in your source directory that contain absolute path aliases and are exposed to client projects (in my case they were needed by the main types file for the package). 👀

I found tscpaths (this one) to be useful for the task, as it converts paths of a) emitted js files b) emitted declaration files c) your source declaration files, if you copy them over to your build dir first. It is also just a compile time dependency. Good job!


PS: I randomly came across an example, where tsc compiler itself actually resolves path aliases to relative paths in the emitted d.ts files.
Take this repo for example: https://github.com/ford04/tsc-declaration-files-abs-path-sample
After running npm i && npm run build, you can see that the code from src/index.ts

import { MyType } from "my-lib/types";
...
const doSomething = (persist: MyType) => {
  return 42;
};

is emitted in dist/index.d.ts in form of

declare const doSomething: (persist: import("./types").Persistence<import("./other-types").DateType, import("./other-types").DateType>) => number;

So there is apparently already some functionality out there for the compiler that actually does path conversions? It seems to be emitted like this, when there is a transitive dependency that spans over multiple files, but I am not sure.
Could you explain, why this resolve mechanism cannot be used for all ts and d.ts files?

Greetings
ford04

@jayarjo

This comment has been minimized.

Copy link

@jayarjo jayarjo commented Oct 9, 2019

How is this still a thing? Obviously someone should do something about in the TypeScript tooling itself.

@dko-slapdash

This comment has been minimized.

Copy link

@dko-slapdash dko-slapdash commented Oct 14, 2019

@RyanCavanaugh, please give us some hope, you’ve been always so kind in other discussions. You’ve been keeping silence about this particular topic for 2 years since your last answer. Hundreds (if not thousands or even tens of thousands) people spend hours and days on hacky work-arounds and experimenTS. I feel that by October 2019 nothing in TS missing features resonates more than the topic about rewriting the absolute imports.

Please!

@jshado1

This comment has been minimized.

Copy link

@jshado1 jshado1 commented Oct 14, 2019

Honestly, this issue is the main reason I've moved my department off of TypeScript, and mostly due to the TypeScript team's arrogant "eff you, community" attitude. Just more classic Microsoft 😞

@Kitanotori

This comment has been minimized.

Copy link
Author

@Kitanotori Kitanotori commented Oct 17, 2019

@jshado1 I have also moved away from TypeScript - and JavaScript - on backend. There are good alternatives, such as Java/Kotlin (*have come to like Spring Boot in particular), Go, and even Rust. With the current module import syntax and the quirks related to it (CommonJS vs ES2015, etc.), I just can't consider TypeScript/Node.js as a serious candidate for large-scale backend apps (had particularly high hopes for LoopBack..)

@RyanCavanaugh

This comment has been minimized.

Copy link
Member

@RyanCavanaugh RyanCavanaugh commented Oct 17, 2019

@dko-slapdash we don't do things that disagree with our core design goals, which this does. It doesn't matter how many people ask us to do something that we think will make TypeScript actively worse.

@jshado1 honestly, that hurts. I try to respond to everyone here as a real human, and we have to have the ability to make technical decisions, of which this is one.

@kvendrik

This comment has been minimized.

Copy link

@kvendrik kvendrik commented Oct 17, 2019

@RyanCavanaugh makes total sense! As a suggestion maybe you could point out out what design goals this violates and why so everyone is clear exactly on why this is not being implemented? Then, I'm also thinking it might help to also post a list of methods people can use to fix this issue so it's not a blocker for consumers. Or, if this is not something Typescript is supposed to do itself, if there's value in it and there's enough demand, maybe there should be an officially supported adapter/middleware that addresses this issue? What do you think?

@dko-slapdash

This comment has been minimized.

Copy link

@dko-slapdash dko-slapdash commented Oct 17, 2019

@RyanCavanaugh, thanks for the explanation. We wonder though, how adding an optional flag to compiler options (which would immediately be utilized by thousands of engineers) could make TypeScript worse, especially actively worse. Considering lots of other flags changing the compiler output (--reactNamespace, --build, --jsx) and features (like “define class properties in constructor parameters”) already exist. Where is the boundary?

@RyanCavanaugh

This comment has been minimized.

Copy link
Member

@RyanCavanaugh RyanCavanaugh commented Oct 17, 2019

Design goal number 7 says

Preserve runtime behavior of all JavaScript code.

When you write the JavaScript line

var n = 10;

TypeScript says "This is JavaScript code" and will always emit

var n = 10;

When you write JavaScript code, TypeScript says that you wrote JavaScript code, and the behavior of this is preserved.

When you write JavaScript code, TypeScript doesn't change the behavior of it.

This is critical, because it means you can take existing JS code, move it to a TS file, and be guaranteed that the code will, at runtime, behave the same way, no matter what your config file says, no matter what the code does.

This allows you to make confident predictions about how your code will run, which is the most important thing!

This is why we don't have any of these extremely-requested features, even under a commandline flag:

  • Partial classes
  • Extension methods
  • Type-based overloading dispatch
  • Runtime exceptions on divide-by-zero
  • Operator overloading
  • Automatic resolution of class members that should be accessed via this.
  • Typed 'catch' clauses

In the 7 years I've been doing this, we've always said that we don't do features that change the behavior of JS code, and people have been telling me that TS will fail unless we add a commandline flag that sometimes changes the behavior of their JS code. Maybe we have failed, I don't know, but we're going to keep going on this path either way.

import x from "./y";

This is a line of JavaScript code. When you write JavaScript code, TypeScript doesn't change the behavior of it. Even under a commandline flag!

For this feature in particular, our message has consistently been: Just like for all other things, you should write the JS the way it should be at runtime, and tell TS how to interpret that. That's why we have eleventyhundred different commandline flags for controlling module resolution and zero for controlling module path rewriting. It's a straightforward principle that aligns with every other way we address JS code.

@jayarjo

This comment has been minimized.

Copy link

@jayarjo jayarjo commented Oct 17, 2019

@RyanCavanaugh why is this: import x from "./y"; a JavaScript code? What would be proper TypeScript version of this I wonder?

@RyanCavanaugh

This comment has been minimized.

Copy link
Member

@RyanCavanaugh RyanCavanaugh commented Oct 17, 2019

why is this: import x from "./y"; a JavaScript code?

It's every bit as much JavaScript code as console.log("hello, world"); is

@dko-slapdash

This comment has been minimized.

Copy link

@dko-slapdash dko-slapdash commented Oct 17, 2019

@RyanCavanaugh, that explains a lot! Ok, so there will be no command-line flag which changes the behavior of a valid (!) JS code, that’s clear. If some code is a clear JS code, it’s never changed at compile-time; if some code is not (like tsx or properties defined in constructor arguments), it may be changed at compile-time and translated to some other JS code.

What about plugins which are able to change the generated code though? People currently struggle with ttypescript to support them (but it’s pretty ugly, since everyone has to run ttsc instead of tsc and tts-node instead of ts-node). I mean - is it in a roadmap, to allow people create native TS plugins (without ttypescript crutches) which affect code generation?

@RyanCavanaugh

This comment has been minimized.

Copy link
Member

@RyanCavanaugh RyanCavanaugh commented Oct 17, 2019

Plugins are under discussion; see #16607

@kvendrik

This comment has been minimized.

Copy link

@kvendrik kvendrik commented Oct 18, 2019

This is great @RyanCavanaugh thank you for writing that up! For anyone who needs a absolute->relative module path transformation right now, what would you recommend? Should they keep using what they're using now and then if Compiler Plugins become a thing there is a chance for someone to write a possibly cleaner alternative? Do you ever see a scenario where Microsoft will start officially supporting some of these plugins to provide people with those highly-requested features or will you likely leave that to the community?

@marcandrews

This comment has been minimized.

Copy link

@marcandrews marcandrews commented Nov 17, 2019

I am encountering this issue and, like others have suggested, using a post tsc build script to correct the import statements in the compiled JavaScript and TypeScript definition files.

According to the Module Resolution section of the documentation (as of time of writing):

The TypeScript compiler supports the declaration of such mappings using "paths" property in tsconfig.json files

Indeed, given the following project layout ...

projectRoot
├── src
│   ├── file1.ts (import * as file2 from 'file2')
│   └── file2.ts
└── tsconfig.json

... and the following tsconfig.json ...

{
  "compilerOptions": {
    "baseUrl": ".",
    "paths": {
      "*": [
        "src/*",
      ]
    }
  }
}

... tsc --noEmit will succeed. Additionally, there will be no errors in VSCode.

However, if you compile the project and attempt to run your app, or if this is a library and you attempt to consume it, you will encounter a runtime error.

I understand the design goals as explained by @RyanCavanaugh, but is the documentation wrong? If supporting absolute paths is against the design goals, why support it during type checking, but not during compilation? If absolute path resolution is really against the design goals, the "paths" feature should be removed completely. It would save a lot of headaches and be less confusing to those migrating their codebases to TypeScript.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.