Suggestion: multi-file external modules

[RyanCavanaugh]

Support compiling multiple input .ts files into one external module.

Need to determine exactly how the module boundaries are defined when doing this.

[basarat]

👍

[vvakame]

👍

[reverofevil]

I'm highly interested in this issue getting closed. I don't like an idea of concatenating *.ts files in my source directory with a build script.

What do you mean by "module boundaries"?

[basarat]

What do you mean by "module boundaries"?

Given:

a.ts:

export class A{}

b.ts:

export class B{}

---

What will the .d.ts will be?

---

This 1:

declare module 'foo'{
     class A{};
     class B{};
}

or 2

declare module 'foo/a'{
     class A{};
}
declare module 'foo/b'{
     class B{};
}

---

My vote : 1

[reverofevil]

Why should it create a module for each class at all, if we're explicitly making one module file?

My vote goes for the first option.

[basarat]

Support compiling multiple input .ts files into one external module.

@RyanCavanaugh can you append the following as well

Support compiling multiple input .ts files into one external module.
Support generating a definition file .d.ts for such an external module.

If you want I can create a separate issue for that.

[charlessolar]

To add my 2 cents, I agree I would like to see 1, but I know that can be a challenge if for example 2 different files contain the same class or variable name.
I am hoping, that like the CommonJS namespace merging you already do we would simply get a 'multiple declaration' exception.

But either 1 or 2 I would like to see this implemented someway because my application uses requirejs extensively to deliver SPA functionality with durandal.

[MrJul]

I'd like to see 1) too since our current desired use case is to use one file per class/interface/enum and one folder per module.

While I think it shouldn't be too hard too implement inside the compiler, the tooling is harder, notably in VS: how do you determine which files compose a module?

Personally, what I would really like to see to solve this problem is a mix between internal and external module syntax at the language level. Something along the lines of:

// A.ts
external module "library" {
  class A { }
}

// B.ts
external module "library" {
  class B { }
}

Both files would be compiled into one module file library.js. Each file can only declare at most one external module, and cannot have any other top-level declarations if there's an external module declared.

Another benefit of this approach is that it would no longer be magic that a file becomes an external module (with its contents no longer in global scope) as soon as there's an import or export since there's the possibility of explicitly defining that a module is external. That's what most people I've seen working with TS and external modules are struggling with when introduced with the concept. (Still, the old way would still work, for compatibility reasons and implicit single-file external module).

[kevinbarabash]

My vote is for option 1 as well.

In terms of determining which files are part of a module, maybe we could use reference paths... something like this:

// library.ts
/// <reference path="A.ts"/>
/// <reference path="B.ts"/>

export = library;

If you compile this with tsc --out library.js library.ts --module commonjs --declaration I would expect there to be line at the end of library.js with module.exports = library which there isn't. Adding this line isn't a huge deal, but it would be nice if it were done automatically.

[KeithWoods]

I say my vote would be for 1 too.

If you're working in a large org or team and want to develop reusable packages as part of a large system (i.e. any large enterprise app) then you'll hit this problem.

You'd be be using external modules as you'll be relying on an external loader (unlike internal modules that assume it's all loaded already). Each module will defer to the module loader to find the correct script not by an assumption it's already loaded (this just referring to it's module object) .

The separate modules need a single .d.ts so it can be consumed by dependencies at compile time. Currently having (potentially hundreds) of single .d.ts for an external module package is useless.

We've got our system running using dts-bundle which scrapes all the single files into a single .d.ts (same approach as outlined in #17), but there are other teams here stumped by the complexity required to get this running, there off building monolithic internal module applications that can't be split out into separate packages.

related issue #1236

[xealot]

Everyone in this thread is voting for 1, so I imagine it's just me being slow to understand how it would work. With option 1 you're changing how things are actually defined if when using external modules aren't you?

If my definition for class A lives in foo/a.ts I would need to import foo/a to get access to the scope that A is defined in. If the d.ts module rewrites it to foo/A wouldn't it break the imports? Additionally, if I made the example a little more silly but still completely valid:

a.ts:

export class A{}

b.ts:

export class A{}

Would the output be:

declare module 'foo'{
     class A{};
     class A{};
}

Additionally, if 2 was produced:

declare module 'foo/a'{
     class A{};
}
declare module 'foo/b'{
     class A{};
}

How would the compiler know where the base path was? When you import this project into other projects the module names should be from and inclusive of the project root correct? So in another project the modules would really need to be defined as:

declare module 'project/foo/b'{
     class A{};
}

I imagine this is going to boil down to the addition of a --include or --library option perhaps? I'm not sure what the solution is, but our biggest stumbling block right now is trying to use typescript modules in other code effectively without shipping the entire raw source. Internal modules work for a while, but if you ever want your project to be isomorphic (node & browser) or traceable using something like browserify external modules are a must.

[kevinbarabash]

@xealot Are you suggesting that the folder structure defines modules? I'd rather module definitions be more explicit, that's why I was suggesting having a main library.ts file that contains references to the things you want in your module. The nice thing about having a separate file is that you could potential have multiple version of the library.ts which have different features.

[charlessolar]

Well currently typescript is using folder structure to import modules, at least for AMD modules it does. @xealot and @MrJul makes a good point that merging definitions like 1 would make it hard on tooling (intelisense, etc)

I agree with @MrJul 's solution by adding a new 'external module' keyword would make the most sense to me and be nice on tooling.

[xealot]

@KevinB7 I'm not suggesting that folder structure defines modules so much as pointing out that's already how things work. With the single exception of Typescript's internal module system, external TS modules, CommonJS, AMD and the upcoming ES6 standard base modules at files and their locations in the path if I'm not mistaken.

[kevinbarabash]

Time for me to go re-read the spec. 😓

[park9140]

I've been working with a large project for a while now, we are attempting to break down our project into multiple smaller libraries now and have run into problems around this topic.

We have come up with two possible paths this could go.

First, make our smaller libraries into internal modules, then wrap them manually into an external module.
This seems like a perfectly valid use case but requires some level of mapping file to show what portions of the internal module should be exported.

Second, keep our existing external module setup, one file for each class/interface/function, each with a single default export ('export = itemName'). The issue with this path ends up being the same as the internal module path. We still need to create a 'library' file that imports the public types and exports them.

In both of these cases, one simple problem blocks implementation. The compiler does not let you export a type declared in a different external module as part of your module.

To that end I would suggest a third option and or extension to option 1

As per the es6 exports specification
http://people.mozilla.org/~jorendorff/es6-draft.html#sec-exports export ExportClause FromClause

We would then do
a.ts

export default interface A{} //current form export = class A{}

b.ts

import default as A from 'a'
export default class B implements A{} 

library.ts

export {default as A} from 'a'
export {default as B} from 'b'

this should then compile a library.d.ts file like this

declare interface A {}
declare class B implements A {}

And should throw a compilation error if you don't export A from your module since B implements A

This probably means #1215 has to be finished first.

I disagree with making the use of folder structure, and automatic exports, since most of classes, functions, and interfaces used internally to a library should not be exposed unless the library explicitly needs them to be shared.

[jbondc]

Like the suggestion of @park9140. Similarly, I've been using the following pattern:

Add a comment on internal modules:
a.ts

module MyLib {
   export class A {}
}
// export default MyLib.A

b.ts

module MyLib {
   export class B {}
}
// export default MyLib.B

lib.ts

module "mylib" {
   // import default as A from "./a";
   // import default as B from "./b";
}

With some hackery in node, internal modules are evaluated in an 'internal sandbox' and we return the default export in the sandbox:
e.g.

vm.runInContext(code, sandBoxInternal, "a.ts")
return sandBoxInternal[sandBoxInternal.export.default] || null

'external' modules still use the regular require(). At build time (tsc lib.ts), you can concatenate the 'internal' modules (a.ts, b.ts) inside your external module.

[nycdotnet]

I hate being in a position where I am feeling pain but I don't have a suggestion for how to fix it - I just want it to go away. When I use TypeScript external modules for something non-trivial, I'm in pain.

Some smart people on this thread have proposed fixes - if @park9140 's aligns with ES6, that sounds awesome to me.

[kevinbarabash]

One library I created I wanted to have things in separate files and then export each modules from a common namespace... I ended up doing this in lib.ts:

export import A = require("a");
export import B = require("b");

[comdiv]

My suggestion is that this scenario is available if you build your solution in single file:

1. Pure typescript modules as source (they allow simple merging)
2. Using --out MODULENAME.js --module AMD
   For now it generates something like this:
   var MODULENAME;
   (function (MODULENAME) {
   //BODY OF FILE 1
   })(newthe || (newthe = {}));
   var MODULENAME;
   (function (MODULENAME) {
   //BODY OF FILE 2
   })(newthe || (newthe = {}));

So as u see - we got valid single JS module that can be used with AMD with shim:{MODULENAME:{exports:"MODULENAME"}}

My suggestion to allow TS generate fully complicated AMD/UMD modules:

file A:
///<reference path="lib1.d.ts"/>
module X {
  export class A{}
}
file B:
///<reference path="lib2.d.ts"/>
module X {
   export class B{}
}
file C:
///<amd-dependency path="lib1"/>
///<amd-dependency path="lib2"/>
export module X;

1 Compiler should fail if TWO export module with same name exist
2 Compiler should add export module definition in tail of --out FILE :

define("X",["require","exports","lib1","lib2"],function(require,exports,lib1,lib2){
      exports.X = X;
});

3. Compiler should hide var X from global scope, result file must be rewrited :
(function(){
var X;
(function (X) {
   //BODY OF FILE A
})(X|| (X= {}));
var X;
(function (X) {
   //BODY OF FILE B
})(X|| (X= {}));
define("X",["require","exports","lib1","lib2"],function(require,exports,lib1,lib2){
      exports.X = X;
});
})();

What is the result:
1) Sources of module are TypeScript-style - no order, no dependency - simple merging
2) Export AMD module as result of compilation
3) No artifacts in global scope

It can be cause some issues to provide "lib1","lib2" instances to internal module, but i think it can be solved with "export initializer" something like that:
///<amd-dependency path="lib1"/>
///<amd-dependency path="lib2"/>
export module X(lib1, lib2) {
      A.setServices(lib1,lib2);
}
So in result it must generate something like that:
define("X",["require","exports","lib1","lib2"],function(require,exports,lib1,lib2){
       X.A.setServices(lib1,lib2);
       exports.X = X;
});

So we can provide any module-level initialization module on real phisical module binding.

For now i do it manually and then minimize.

[AsherBarak]

Why not consider tooling? Group .ts files into modules using directory structure or comments or whatever code-indifferent mechanism, and have the tool rewrite the imports and the exports into a new .ts file to keep everything going. In case of conflicts, emit errors.
Tooling should also emit source maps.
This eliminates the need for braking changes in the code and allows room for errors in the merging process.
As all current and proposed modules loading relies on files location, and loading files is something you might want to manage to balance files sized with number of calls, this will keep with same principal while separating the development directory structure from the delivered modules structure.

[kripod]

Proposal suggestion

(Based on #5085, inspired by the @import function of SASS)

I would like to propose a mechanism for creating partial files, which are not compiled by default, but embedded into files which reference them. It's just like bundling, but with less hassle and in a more unified approach.

As a brief example, suppose that we have 4 files:

• index.ts
• user.ts
• _auth.ts
• _position.ts

The index.ts and user.ts files can reference contents of a partial file by using simple imports. For example, if we want index.ts to reference _auth.ts and _position.ts, use the following code in the non-partial index.ts file:

import auth = require('auth');
import position = require('position');

The content of partial files should be copied and instantly evaluated in their container file. For instance, whether we would also like to use the _position.ts partial file in user.ts, use an import just like above:

import position = require('position');

Although this may seem like a regular import, it's basically an include function which copies the partial file's full content at the place desired. That means, partial files should not be reused in multiple containers, but should be used for code organisation purposes. When both index.ts and user.ts are referenced in a HTML file, the content of _position.ts will be duplicated and inserted into the 2 containers separately.

Last but not least, in order to keep backwards compatibility, there should be a tsconfig setting (compilerOptions.ignorePartialFiles) which could be set to true in order to keep compiling files which start with a _.

[ToddThomson]

@kripod https://github.com/toddthomson/tsproject already does this. It adds bundling to the tsconfig.json file and will build a bundle from modules using ES6 syntax. TsProject is a gulp adapter that can be installed with npm.

[kripod]

@ToddThomson I have seen that project before proposing my specification, but bundling from a single file seems to be overcomplicated for such a little effect.

[ToddThomson]

@kripod No that is not how it works. Your Typescript ES6 exported modules are defined in separate files. When you specify a bundle it works out all the dependencies and build a single bundled ts, js and d.ts. Take a look at the sample on the github tsproject repository.

[mironx]

I am in the same situation as @park9140 and I am little tried.
I trayed the same solution (and also others).
I have more experience in C# and Java and probably my notation about how it could works has influence of it.

But I dream about something similar to .net assembly which could have multiple files and I could load it.

Thant means would be good have that java script external module be like assembly.
Even I our external module is big we could use in it namespaces [Namespace in module].
All files "are compiled" to one file. We could tell compilator:
ok: it is my new super external module and it consists these 350 files. [Tell that this files build module]
And these elements please make available see by other modules [Export Items]
Between files should be possibility to have references [Reference inside module].
And supporting compilation exception when e.g. in one module would define
two classes with the same name [Supporting Compilation Exception].

That meas we could indemnify 4 issues:
*Namespace in module
*Tell that this files build module
*Export Items
*Reference inside module
*Supporting Compilation Exception

Generally I would like to focus on developing business logic, provide solution than
focus on compilation process.

[mhegazy]

@mironx have you looked at browserify or webpack?

[mironx]

@mhegazy
Thank your encouragement. I had know this tool and I had afraid bit of configuration - that it could be to complicated.

Thanks your encouragement I the ice has broken up :-) and
I have spend several weeks with webpack and also systemjs.

What I would like to achieve?

But first what I would describe what I need to achieve.

I would like to create my suite of component which would exists in java script file.
E.g.:

• lib1.js, lib.d.ts
• lib2.js, lib.d.ts
• lib3.js, lib.d.ts
  this library could be depend on angular, reactjs, kendo and others.

apps
and apps which could consume this libraries and could have own libraries.

• app1.js
  libapp1.js
  libapp2.js
  and dependencies: lib1.js with lib1.d.ts, angular etc....
• app2.js ... silimar like app1

It would be like: software suite, framework or components.

1. One library could contain many external modules.
2. app it is just main js file

My adventure with webpack

I have tested webpack with grunt. I create GruntFile.js which could be use by each project.
This GruntFile.js use project.json which indicate how to process structure of project (solution line in Visual Studio)

Problems

• a) I could join js files (after compilation of typescript) but I have problem with generated one
  type script date type file d.ts for library
  As I wrote I would like to have references between libraries - I need also d.ts
• b) I have problem to dynamic loading js to achieve the same as in systemjs.
  That mean to have one file in html.
• c) And with references. because I had to use external - I don't wont bundle vendors and library to one file.

Summarization

It this moment the biggest problem for me is point "a".

Main requriment
I could compile whole libraries for each app but I need that my library work as plugin.
Thant means for existing application someone else could provide library as plugin
which extend functionality. This library would implement app interfaces to extend functionality.

New version of typescript compilator

I looked on roadmap of typescript compilator and I noticed interesting issues:

• 2.1 Support for project references
• 2.0 Improve lib.d.ts modularity
• 1.8 Concatenate module output with --outFile (only for amd and commonjs)

Conclusion

• Maybe in this moment it is difficult to achieve this structure which I was described.
• Maybe I am on wrong path because I have habit e.g form .NET or java and I would
  like transfer "assembly" to java script world
  But maybe no - typescript roadmap looks like small steps in this direction.

Maybe crazy idea for typescript compilator

What do you think to put to to output java script file declaration?
As metadata in comments. It would transparent for java script.
But js could be add as reference to typescript project - similar to assembly or jar file
which has own metadata.

Architecture for enterprise, long term solution

In new year I and my team I will have make some architect decision -
and I ma responsible of consequences.

I really appreciate you for each comments and criticism in this topic.
Typescript architecture for applications and libraries. Thant means how to organize compilation process to achieve these goals.

I think thant others have similar problems with typescript projects - how to organize it
when structure is more complicate, e.g.: #909 , #557

Thank you and sorry that I extended this issue of details
which could be out of scope

[mhegazy]

@mironx, would not #5090 be sufficient for your needs?

[mhegazy]

ES6 modules provide the ability to build modules from smaller ones using export * from "mod" and export { a as b } from "mod". This allows for having the multiple modules for implementation purposes, but then exposing a single "entry point" module that collates all of the smaller ones into a complete unit. This would be the recommended route for this scenario.

A follow up feature for the TS compiler is #4433, to enable publishing a single .d.ts file that has the "shape" of the "entry point" module, and thus completely hide the internal modules from consumers.

[Clark159]

This is my solution uses TypeScript 1.8, The solution build multi-Class in a module.

• sample : https://github.com/Clark159/clark159.github.io/raw/master/_posts/%5BTypeScript%5D/%5BTypeScript%5D%20%E5%BB%BA%E7%AB%8B%E8%88%87%E4%BD%BF%E7%94%A8AMD%20Library/Lab.zip
• document : https://dotblogs.com.tw/clark/2016/04/25/124833

[BSick7]

@mhegazy can you elaborate on this approach more of using modules for implementation, but smashing into 1 file with 1 entry point?

[mhegazy]

@mhegazy can you elaborate on this approach more of using modules for implementation, but smashing into 1 file with 1 entry point?

there is no "smashing" happening. Each file is emitted as a module, but you have one entry point.

e.g.:

// a.ts
export var a = 0;

// b.ts
export var b = 0;

// index.ts
export * from "./a";
export * from "./b";

compiling with --module amd and --outFile.:

define("a", ["require", "exports"], function (require, exports) {
    "use strict";
    exports.a = 0;
});
define("b", ["require", "exports"], function (require, exports) {
    "use strict";
    exports.b = 0;
});
define("index", ["require", "exports", "a", "b"], function (require, exports, a_1, b_1) {
    "use strict";
    function __export(m) {
        for (var p in m) if (!exports.hasOwnProperty(p)) exports[p] = m[p];
    }
    __export(a_1);
    __export(b_1);
});

your users would import index and not a or b.

[BSick7]

Doesn't this force me to use amd module loading?
Wouldn't UMD be an alternative that allows choice of module loading?

[mhegazy]

if you are using Node, then there is no need to bundle i would say. the issue with UMD is that it needs to work the same way on node and amd. since the way you can require an internal module is different from amd to node, we can not grantee that the transformation would work.
if you want to have a bundle that works in both amd and node, i would recommend looking at browserify or webpack.

[MortenHoustonLudvigsen]

I think rollup.js looks very promising as well. I haven't tried it yet though.

[BSick7]

Those work well for applications, but I am looking to build a library that works with amd and vanilla js.

Isn't the whole purpose of UMD to be agnostic between commonjs and amd?

I imagine that UMD would work like module: none with the whole thing wrapped with boilerplate umd.