cli: support catch all routes (i.e. to a custom 404 page)

[Splaktar]

Steps to Reproduce

1. ng add @scullyio/init
2. npm i --save-dev fs-extra
3. ng build
4. npm run scully

Error

Cleaned up /Users/splaktar/Git/.../dist/static/ folder.
started servers in background
Invalid content-type.
Expected application/json but received text/html; charset=UTF-8
<repeated about 30 times>
Invalid content-type.
Expected application/json but received text/html; charset=UTF-8
Could not connect to server

The debugger indicates that this is caused by the following line

scully/scully/utils/waitForServerToBeAvailable.ts

Line 13 in eba6fba

httpGetJson('http://localhost:1864/_pong', {suppressErrors: true})

That URL (http://localhost:1864/_pong) seems to be returning my Angular app (in a broken state) rather than some JSON content.

Root Cause

A route like {path: '**', component: PageNotFoundComponent} breaks the ability to run Scully. Removing it works around the problem, but breaks the app's custom 404 page.

[SanderElias]

@Splaktar I'm unable to reproduce this, and would love to debug this together with you.
I have a faint idea what might be happening, but going over the code doesn't bring me any clue here.

[Splaktar]

I restarted from scratch with 0.0.8 and I can still reproduce this.

[Splaktar]

Angular CLI: 9.0.0-rc.6
Node: 12.13.0
OS: darwin x64

Angular: 9.0.0-rc.6
... animations, cli, common, compiler, compiler-cli, core, forms
... language-service, platform-browser, platform-browser-dynamic
... router, service-worker
Ivy Workspace: Yes

Package                           Version
-----------------------------------------------------------
@angular-devkit/architect         0.900.0-rc.6
@angular-devkit/build-angular     0.900.0-rc.6
@angular-devkit/build-optimizer   0.900.0-rc.6
@angular-devkit/build-webpack     0.900.0-rc.6
@angular-devkit/core              9.0.0-rc.6
@angular-devkit/schematics        9.0.0-rc.6
@angular/cdk                      9.0.0-rc.5
@angular/fire                     5.2.3
@angular/material                 9.0.0-rc.5
@ngtools/webpack                  9.0.0-rc.6
@schematics/angular               9.0.0-rc.6
@schematics/update                0.900.0-rc.6
rxjs                              6.5.3
typescript                        3.6.4
webpack                           4.41.2

As you can see above, I didn't mention that my app is a PWA. I'm not sure if that breaks Scully, but I'll try to disable service-worker and see how that goes.

[Splaktar]

However, I have SW disabled for dev builds like ng build, so it shouldn't be a problem here.

    ServiceWorkerModule.register('ngsw-worker.js', {
      enabled: environment.production,
      registrationStrategy: 'registerWithDelay:5000'
    }),

[Splaktar]

Opening http://localhost:1864/ gives me a broken page with the following errors

Failed to load module script: The server responded with a non-JavaScript MIME type of "text/html". Strict MIME type checking is enforced for module scripts per HTML spec.
polyfills-es2015.js:1 Failed to load module script: The server responded with a non-JavaScript MIME type of "text/html". Strict MIME type checking is enforced for module scripts per HTML spec.
styles-es2015.js:1 Failed to load module script: The server responded with a non-JavaScript MIME type of "text/html". Strict MIME type checking is enforced for module scripts per HTML spec.
vendor-es2015.js:1 Failed to load module script: The server responded with a non-JavaScript MIME type of "text/html". Strict MIME type checking is enforced for module scripts per HTML spec.
main-es2015.js:1 Failed to load module script: The server responded with a non-JavaScript MIME type of "text/html". Strict MIME type checking is enforced for module scripts per HTML spec.
manifest.webmanifest:1 Manifest: Line: 1, column: 1, Syntax error.

For each of these failed resources, the server is sending back the contents of index.html instead of the proper file. This is likely happening with the _pong endpoint where it is getting the contents of index.html instead of a JSON payload.

[Hypercubed]

I see the same issue... nothing special in my test app. Generated a new ng8 app using the cli, ng updated to ng9-rc, then ng added scully.

Running npm run scully throws the Invalid content-type errors.

I see lots of js files in dist/static, however, all js requests on http://localhost:1864/ return contents of index.html.

[Splaktar]

I wondering if there might be some dependance or issue with having Angular CLI version 8 installed globally, even if v9 is installed in the workspace. However, installing Angular CLI v9 globally didn't change anything.

I also tried ng build --prod instead of ng build, but that also had no effect.

[Hypercubed]

@Splaktar what routes do you have in your app?

I've think I have discovered if you try adding a route with only a slug (path: ":page"), I get the errors above. I assume because http://localhost:1864/_pong is being treated like a angular defined route.

Similarly, looks like http://localhost:1864/killMe doesn't work so the Scully server never shuts down and all subsequent runs of npm run scully will fail.

Kill the appropriate node process make all routes static (for the moment) and try again?

[Splaktar]

export const AppRoutes: Routes = [
  {
    path: '',
    pathMatch: 'full',
    component: AboutComponent,
    children: []
  },
  {path: '**', component: PageNotFoundComponent}
];

[Splaktar]

I've noticed that the node processes stay around and have to be manually killed, otherwise they sit on those two ports indefinitely.

[Splaktar]

@Hypercubed this video covers how to set things up with the route parameters. However, I'm running into this issue w/o using any route params.

[Hypercubed]

Try removing the ** catch all....

[Splaktar]

That worked (after killing the node process that was still around from the previous run)!

Cleaned up /Users/splaktar/Git/.../dist/static/ folder.
started servers in background
servers available
Finding all routes in application.
Pull in data to create additional routes.
Route list created in files:
      src/assets/scully-routes.json
      /Users/splaktar/Git/.../dist/static/assets/scully-routes.json
Route "" rendered into file: "/Users/splaktar/Git/.../dist/static/index.html"
Route list created in files:
      src/assets/scully-routes.json
      /Users/splaktar/Git/.../dist/static/assets/scully-routes.json

Generating took 9.32 seconds for 1 pages:
  That is 0.11 pages per second,
  or 9327 milliseconds for each page.

[Hypercubed]

I'm pretty sure just need to move the _pong and killMe routes above the routes loop here:

scully/scully/utils/staticServer.ts

Line 37 in 596d092

routes.forEach(route => {

I haven't yet been able to build scully for dev so I can't test that.

Edit: PR submitted. Tested locally.

[SanderElias]

@Hypercubed Yes, that is part of the issue, but it seems to go a bit deeper. I have a PR in flight that will address this. But as I'm officially on holiday, I'm not entirely sure when I will be able to complete this one. Might be next year ☺

[Hypercubed]

@SanderElias Are you able to share your PR? I'd like to test it if I can.

[jorgeucano]

the fix is in master :)

[Splaktar]

Was this fixed in PR #146 or another PR?

[SanderElias]

@Splaktar Yes, a partial fix might be in a bit earlier, but that one closed it completely.