New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Documentation - Creating a 'secure' package. #24580

Open
mattezell opened this Issue Jun 19, 2018 · 6 comments

Comments

Projects
None yet
4 participants
@mattezell

mattezell commented Jun 19, 2018

I'm submitting a...


[ ] Regression (a behavior that used to work and stopped working in a new release)
[ ] Bug report  
[ ] Performance issue
[ ] Feature request
[X] Documentation issue or request
[ ] Support request => Please do not submit support request here, instead see https://github.com/angular/angular/blob/master/CONTRIBUTING.md#question
[ ] Other... Please describe:

Current behavior

Not an issue - sharing a process.

Expected behavior

Minimal reproduction of the problem with instructions

What is the motivation / use case for changing the behavior?

@alxhub asked me to pop by and share.

Environment


Angular version: 6.0.8


Browser:
- [ ] Chrome (desktop) version XX
- [ ] Chrome (Android) version XX
- [ ] Chrome (iOS) version XX
- [ ] Firefox version XX
- [ ] Safari (desktop) version XX
- [ ] Safari (iOS) version XX
- [ ] IE version XX
- [ ] Edge version XX
 
For Tooling issues:
- Node version: 8.11.3  
- Platform:  Windows 10 Pro

Others:

Creating a 'Secure' Angular Library Package

Background/Motivations

I am an an enterprise architect who currently calls the arena of Fintech home. Years ago, we set out to modernize our main client facing systems and their supporting APIs. For the front end, we set our targets and AngularJS and didn't look back. Our challenge wasn't a small one - coming from a .Net and Java heavy world in terms of our existing code bases, part of our systems included a 3rd party ecosystem which allowed partner developers to extend our base functionality. Not only could partner developers extend the system's core functionality, but they could also package up their extensions in a manner compatible with our systems so that they could be resold to other organizations who also benefited from this new functionality.

The initial release of our revamped, albiet feature-limited system, was very well recieved - and almost immediately the ecosystem began to grown around it at a pace nearly exceeding our ability to continue to build out our initial system... A good problem to have - and so we continued to iterate and release.

As anyone who works in "Enterprise Development" knows, IP is an important point of focus - companies often require first and foremost that trade industries be protected, and that key implementation details be locked away under lock and key. Of course due to the core realities of JavaScript, this posed us an interesting challenege - in that you're highly limited in the ways that you can protect your propritary logic without breaking things. But ultimately we persisted and prevailed - obfuscation and minification got us past our requirements so that we could forge ahead with our new stack based on 'web technologies'.

For our system, which was based on AngularJS 1.5, we found the decorator pattern to suit us quite nicely when faced with the requirements of 'code protection'. Using the decorator pattern, we were able to provide developer guidance documents which served to expose extenibility points via documentation. While less than perfect, it worked pretty well for us and our clients - particularly when you consider the reality of JavaScript development, where API documentation was king, vs TypeScript development, where Typings are king (and more or less a requirement).

Fast foward a year+ and several iterations later, we found ourselves with a functional codebase now ported to Angular 4. Angular 4 provided us some new and interesting challenges - namely that the ballgame, and its associated rules which we had build our system around had changed. While we had completely redesigned our extensibility approach, which is outside of the scope of this document, we were faced with the very real problem of the 3rd party developer flow, which had seemingly degraded from Angular 1.x to 4.x under our design.. While I can't go too far into the details, a lot of the 3rd party work had to be done out of band - integration testing with the core platform required round trips to the build server under our design just to get a working environment to test your enhancements (which may have been broken due to an errant keystroke that had gone unnoticed before pushing for the build - so you had to do it all over again).

As a stop-gap, we devised a new approach; though also far from optimal. As we should have, we shifted the burden from the 3rd party developers back to ourselves. How we accomplished this initially was via 2 different Angular npm packages of our core functionality. While we tried time and time again, we could never get "one library to rule them all". What we could come up with was a system backed by 2 different builds of our core library. 1) The developer pacakge, which was capable of JIT with typing support, but only contained minified/obfuscated transpiled JavaScript housing the implementation details. 2) The build package, which supported AOT production builds, but contained unobfuscated logic and so could only be distributed to partnered builders with whom we had a legal agreement with concerning our IP.

This approach 'worked', but was never quite favored - it felt hacky and was difficult to maintain multiple builds...

Needless to say, we had to find a better way.

A Better Way

Enter Angular 6 - with it's fancy first class support of library generation. Not quite knowing what we were getting into, a port of our 4.x to 6.x was undertaken - and went surprisingly well. From here we worked to break out all of our implementation which was intended to be 'protected' in to Angular Libraries (with the help of the Angular CLI and ng-packagr). Once our cleanup efforts were complete, we had a handful of Angular 6 feature libraries and what is essentially a shell application to tie them together.

While I am still struggling to grok the hows and whys that this works, it seemingly just does for now. In Angular 4 Land, we could never quite get our obfuscated/minified package to work for AOT Prod builds, though it worked perfectly fine for JIT builds; hence our need for 2 pacakges. But I am happy to report that the rules are apparently different in Angular 6 Land - as we are now able to create this 'one package to rule them all' in a relatively straight forward manner.

During some of the exploratory phase, I'd spoken on gitter with Alex Rickabaugh (@alxhub) about what we were trying to accomplish. While our interactions were small in regards to our Angular 4 efforts, I popped back in to update Alex on the success we'd found with Angular 6 - at which time he requested that I do a quick writeup in the case that it might assist others....

...so here we are...

Here be dragons...

As previously mentioned, the steps are currently pretty straight forward. For the following overview, let's assume we arrived here from "ng g library awesomesauce".

  • "ng build awesomesauce --prod"
  • Navigate to /dist/awesomesauce/
  • Delete the esm5, esm2015, esm5, and fesm2015 folders.
  • Navigate to /dist/bundles/ and delete the non .min. resources and the map associated with your minified bundle.
  • Back in /dist/awesomesauce/, open up package.json and make the following edits:
    • update 'main' to reference the *.min.js bundle in /bundles/
    • Delete 'module', 'es2015', 'esm5', 'esm2015', 'fesm5' and 'fesm2015' key/value pairs
  • "ng pack"

You are now the proud parent of a 'secured' Angular 6 library packaged for installation via NPM into an Angular 6 application - your bouncing new Angular library provides developer typing support for your public APIs, and which more importantly only contains minified/obfuscated implementation logic. Of course this isn't 100% secure - you still have a considerable amount of information which is exposed in your *.d.ts typing files, and there's still quite a bit that can be inferred in combination with your included metadata file (which is required for this all to work as described)... This said, I do believe that there's little/nothing more exposed by this package than has always been the case with JavaScript libraries that provide typing support.

Note: Again, here be dragons. I am not a web security expert. The process described here may or may not stand up to rigorous scrutiny. We're in the process of vetting this and working to clear the security requirements imposed upon us - but so far this appears to be fitting the bill for us, so perhaps it will for you too.

@mattezell

This comment has been minimized.

mattezell commented Jun 19, 2018

Here's a tldr version:

  • "ng build awesomesauce --prod"
  • Navigate to /dist/awesomesauce/
  • Delete the esm5, esm2015, esm5, and fesm2015 folders.
  • Navigate to /dist/bundles/ and delete the non .min. resources and the map associated with your minified bundle.
  • Back in /dist/awesomesauce/, open up package.json and make the following edits:
    • update 'main' to reference the *.min.js bundle in /bundles/
    • Delete 'module', 'es2015', 'esm5', 'esm2015', 'fesm5' and 'fesm2015' key/value pairs
  • "ng pack"

@ngbot ngbot bot added this to the needsTriage milestone Jun 21, 2018

@jasonaden

This comment has been minimized.

Contributor

jasonaden commented Jun 21, 2018

Note: This might be something we could/should add to the Angular Package Format. It's also possible we would want to have a CLI build that would produce assets as described above.

@mattezell

This comment has been minimized.

mattezell commented Jun 25, 2018

Yes, agreed @jasonaden .

Currently we're automating this pre-packaging process via gulp tasks and associated npm scripts in our projects, but it would definitely be nice to have this get first class support in the CLI using flags and/or via package configs.

If we could get first class support in the CLI and the Angular Package Format, it would enable folks seeking to accomplish this to possibly speed up their production build time in that non-desired assets wouldn't have to actually be generated ( esm5, esm2015, esm5, and fesm2015 in our case, for example).

An additional sticking area for us in this exercise was that validation applied at build time varies a bit for application builds vs library builds. For example, the same codebase that builds as an application without error results in TS4023 (likely unavoidable), TS4033, TS2307 (likely unavoidable), additional validation checks seem to be in place for lib builds to ensure functional implementation matches template calls in regards to arguments, and TS0 warnings from tsickle resulting in build failure in ng-packagr due to invalid JSDoc markup...

That last one (TS0 warnings breaking the build) was our biggest one - lots of 'invalid JSdoc markup' in our codebase that wasn't a build error issue as an application, but that completely breaks the ability to build the library with ng-packagr (I believe I read this is due to additional strictness being required for closure compiler, where things will apparently break with some of these invalid types in JSDocs - though this may not be 100% accurate as it was just my take away from reading the various open issues on this matter in Angular CLI, ng-packagr, and tsickle).

Likely related to this request; a feature request for package support for assets, scripts and styles in the same way as is supported with application projects: angular/angular-cli#11317

Thanks!

@jenniferfell

This comment has been minimized.

Contributor

jenniferfell commented Jun 26, 2018

@hansl @Brocco : Hi. Interesting thread here. This may be a two-stage thing: stage one add more info to docs, stage 2 add features to CLI? How would you like me to triage this? Keep as a doc feature request? Or if there's related WIP on the eng side, hand off to y'all? Thanks.

@mattezell

This comment has been minimized.

mattezell commented Jun 27, 2018

Howdy again, folks.

While I'm sure this won't be the approach if this winds up evolving into CLI / APF as an enhancement, I created a minimal project to demonstrate the approach in the case that it assists: https://github.com/mattezell/SecureAngularLibrary

Let me know if there is any way that I can actually contribute to this :)

Thanks!

@wbhob

This comment has been minimized.

wbhob commented Jul 6, 2018

CodeTriage sent me here. Just want to touch base and see what the status of this issue is, and whether there is a PR in progress.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment