CDN-in-a-Box for Developers #6532

ocket8888 · 2022-01-31T23:21:42Z

This PR contains a new "CDN-in-a-Box" that's better suited to rapid changes. It will rebuild components when they change, supports debug connections by default, and uses lighter images than regular CiaB - based on Alpine. No rebuilding when making changes, no creating RPM packages only to unpack them a moment later.

This new setup is not complete. There are a few problems. The t3c/cache server only runs t3c on-demand, which is not how that's typically done in production environments. Also only one t3c sub-command can be debugged at a time. Plus, Traffic Router just doesn't work at all. Currently, the stack builds TR and starts tomcat, but I can't get it to do anything meaningful.

There is no documentation and there are no tests. I'm opening this because I was told it could be helpful, but it does not represent a completed body of work. I personally tend to use it for Traffic Ops development.

Which Traffic Control components are affected by this PR?

Traffic Portal (package-lock.json changes)

What is the best way to verify this PR?

To use it, from the repo's root: source ./dev/atc.dev.sh and then run atc -h to see usage.

PR submission checklist

This PR has tests
This PR has documentation
This PR has a CHANGELOG.md entry
This PR DOES NOT FIX A SERIOUS SECURITY VULNERABILITY

zrhoffman

As a general comment, all build or run commands that can be run as an unprivileged user should be to avoid creating files owned by root like #6454. This wasn't really an issue for the CentOS CDN-in-a-Box because the only bind mount it modifies is the certificates directory, which is out-of-the-way, but root-owned files make a much larger impact in CDN-in-a-Box for Developers.

One way to create a user with the right user ID would be making a SERVICE_UID: ${UID:-1000} build arg for each docker-compose service, which should cover both Linux and macOS (hopefully, Windows is less picky), then running adduser -Du${SERVICE_UID} my_user as a build step in each Dockerfile.

dev/traffic_router/conf/trafficrouter.dev.ciab.test.crt

zrhoffman · 2022-02-01T08:37:25Z

dev/traffic_router/conf/traffic_ops.properties

+# See the License for the specific language governing permissions and
+# limitations under the License.
+traffic_ops.username=admin
+traffic_ops.password=twelve12


IMO the password should just be "twelve". I know the TO API itself does not allow a user to set passwords to "twelve", but because of that, there is no risk of re-using "twelve" as an insecure password. However, the same cannot be said for "twelve12": Since TO validates it, it is in danger of making its way into prod environments.

Also, "twelve" is the standard dev project everywhere except CDN in a Box.

Actually, IMO neither this nor CiaB should use twelve12, since that's not accepted by the API either - twelve12! is, which makes it a viable candidate. The password used here should ideally match the current CiaB for ease of use, but CiaB should not contain data we know to be invalid. If you are using the development or testing/demo setups in production, you have ignored all instructions to the contrary and in CiaB's case multiple warnings. We cannot reasonably prevent someone so determined to do improper things from doing those things.

The password used here should ideally match the current CiaB for ease of use, but CiaB should not contain data we know to be invalid.

Okay, PRed making the password twelve in both places in #7149

dev/traffic_router/Dockerfile

dev/atc.dev.sh

docker-compose.yml

traffic_portal/package-lock.json

dev/traffic_router/run.sh

dev/t3c/Dockerfile

zrhoffman · 2022-02-01T15:43:38Z

CDN-in-a-Box CI / ciab-build (pull_request) Failing after 1m — ciab-build

The makefiles and the entire dev directory should be in the CDN in a Box GitHub Actions workflow's paths-ignore lists

ocket8888 · 2022-02-01T17:34:32Z

I don't have time in the coming weeks to figure out how to make the Traffic Router service work, so I'd rather not spend any time making adjustments to it since I know it's not going to work anyway.

zrhoffman · 2022-02-01T18:02:14Z

I had done that, and it would start but it was not servicing any HTTP or DNS requests I threw at it.

That is normal if it cannot fetch the CRConfig yet, it only serves HTTP and DNS responses after that point. Once the comments I have made on #6532 are addressed, TR prints only this message repeatedly:

trafficcontrol-trafficrouter-1  | 17:43:00.361 [pool-3-thread-1] ERROR org.apache.traffic_control.traffic_router.core.secure.CertificatesClient - No traffic ops hostname yet!

which makes sense, since TM does not serve a CRConfig.

trafficrouter:/# curl -v trafficmonitor/publish/CrConfig;
*   Trying 172.25.0.4:80...
* Connected to trafficmonitor (172.25.0.4) port 80 (#0)
> GET /publish/CrConfig HTTP/1.1
> Host: trafficmonitor
> User-Agent: curl/7.80.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 503 Service Unavailable
< Date: Tue, 01 Feb 2022 17:59:21 GMT
< Content-Length: 19
< Content-Type: text/plain; charset=utf-8
<
* Connection #0 to host trafficmonitor left intact
Service Unavailable

Corresponding message from Traffic Monitor logs:

trafficcontrol-trafficmonitor-1  | ERROR: datareq.go:161: 2022-02-01T18:00:21.366963347Z: Request Error: /publish/CrConfig: service still starting, some caches unpolled: map[]

So, the key to getting TR to serve HTTP and DNS responses is to get Traffic Monitor to serve the CRConfig. The TR piece works fine with the suggested changes.

ocket8888 · 2022-02-01T18:05:32Z

So Traffic Router will not respond with even a 503 to requests made to its API until it has a CDN snapshot on which to operate? I've been spinning my wheels trying various weird and unlikely things because I expected it to at least be capable of telling me that it wasn't working right - I think we ought to change that behavior if possible because it seems very confusing and annoying for people trying to set up components for the first time. I can open an issue for that (once I confirm one doesn't already exist)

zrhoffman

Now the TR web server starts, but since Tomcat can't find its configs, it's just a web server on port 8080

trafficcontrol-trafficrouter-1  | 02-Feb-2022 22:25:23.817 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["http-nio-8080"]

dev/t3c/run.sh

dev/traffic_router/run.sh

mitchell852 · 2022-02-02T22:54:16Z

@ocket8888 - when this is merged will we have 2 "cdn in a box"s? aka 2 code bases to maintain for ciab? should the other one be deprecated in favor of this one? what are your thoughts on that?

dev/traffic_router/Dockerfile

ocket8888 · 2022-02-02T23:41:43Z

When I first started working on the Docker version of CDN-in-a-Box, this is much more what I had in mind than what we currently know as "CiaB" (although I can't speak for Dan who was working on it in some form or another before I knew what ATC was) - probably because as a developer I only had a developer's needs in mind. And that's likely why the decision was made to do things the way they currently are, with RPMs and running on CentOS and installing everything under /opt/ etc. It's "prod-like" and that's highly valued for testing, which is a primary use-case of our current CiaB.

mitchell852 · 2022-02-03T02:21:47Z

When I first started working on the Docker version of CDN-in-a-Box, this is much more what I had in mind than what we currently know as "CiaB" (although I can't speak for Dan who was working on it in some form or another before I knew what ATC was) - probably because as a developer I only had a developer's needs in mind. And that's likely why the decision was made to do things the way they currently are, with RPMs and running on CentOS and installing everything under /opt/ etc. It's "prod-like" and that's highly valued for testing, which is a primary use-case of our current CiaB.

yeah, seems there are 2 different use cases:

a ciab for testing where a ciab most like prod is desired (rpms, centos, etc)
a ciab for development (lightweight, fast build and rebuild, etc)

i just worry about having to maintain the codebase of 2 ciab's. anyhow, that's for you guys to figure out. maybe there's a way to abstract out similar pieces? maybe not? i'll leave it to the experts but i think this is great and much needed for a long time. hopefully, it can help ease development of TC and encourage more code contributions from our open source community.

ocket8888 · 2022-02-03T15:57:11Z

maybe there's a way to abstract out similar pieces?

The only truly similar parts are in the data-sets, and that similarity is honestly merely skin-deep. Other than that they don't run the same way, aren't packaged the same, and the service containers differ all the way down to the level of which OS they run on (distro, really). The way the CiaB "enroller" works is meant to be generic, it allows loading (somewhat) arbitrary data sets using the TO API - because of that, it has to wait for TO to be available and then each service that wants to exist is responsible for using it to "enroll" itself. The dev environment has some static data being inserted directly into the database, which allows each container to be started and stopped independently without creating any conflicting data, whereas CiaB needs to be entirely stopped and started together, but doesn't allow for arbitrary extension e.g. via the "optional" docker-compose files found in the CiaB directory.

zrhoffman · 2022-02-03T18:15:41Z

maybe there's a way to abstract out similar pieces?

The only truly similar parts are in the data-sets, and that similarity is honestly merely skin-deep. Other than that they don't run the same way, aren't packaged the same, and the service containers differ all the way down to the level of which OS they run on (distro, really). The way the CiaB "enroller" works is meant to be generic, it allows loading (somewhat) arbitrary data sets using the TO API - because of that, it has to wait for TO to be available and then each service that wants to exist is responsible for using it to "enroll" itself. The dev environment has some static data being inserted directly into the database, which allows each container to be started and stopped independently without creating any conflicting data, whereas CiaB needs to be entirely stopped and started together, but doesn't allow for arbitrary extension e.g. via the "optional" docker-compose files found in the CiaB directory.

Agreed that we shouldn't try to combine the CiaB and dev-CiaB code. Abstractions would get messy fast, since they are built differently as @ocket8888 mentioned, as well as distro differences. dev-CiaB doesn't even have Bash, and most of the CiaB scripts require Bash, not just POSIX sh.

jhg03a · 2022-02-04T00:19:26Z

The way the CiaB "enroller" works is meant to be generic, it allows loading (somewhat) arbitrary data sets using the TO API - because of that, it has to wait for TO to be available and then each service that wants to exist is responsible for using it to "enroll" itself

This was a big difference in all the ansible work from CiaB from the beginning. There's no concept of dynamically add/remove pieces. There's a definition of what you want generated at the very beginning of the lifecycle so it knows what all it has to wait through and shape the dataset accordingly.

zrhoffman

Once the suggested changes are in place, Traffic Router will almost accept the CRConfig. It gets the CRConfig from Traffic Monitor and tries to handle it, but that errors out because some required parameters do not exist. ConfigHandler exits with this error:

failed querying JSON for key: geolocation.polling.url

as a result, it does not save the CRConfig on the disk and tries again in 60 seconds.

I didn't bother checking what all parameters are required, but I noticed that no CCR profile exists. So maybe adding these would do it?

dev/traffic_router/Dockerfile

dev/traffic_ops/Dockerfile

dev/traffic_monitor/Dockerfile

dev/t3c/Dockerfile

dev/traffic_router/Dockerfile

dev/traffic_router/run.sh

zrhoffman · 2022-02-04T01:13:03Z

The way the CiaB "enroller" works is meant to be generic, it allows loading (somewhat) arbitrary data sets using the TO API - because of that, it has to wait for TO to be available and then each service that wants to exist is responsible for using it to "enroll" itself

This was a big difference in all the ansible work from CiaB from the beginning. There's no concept of dynamically add/remove pieces. There's a definition of what you want generated at the very beginning of the lifecycle so it knows what all it has to wait through and shape the dataset accordingly.

The enroller takes about 5 seconds on my workstation to complete. The ansible dataset loader takes 20 minutes

dev/traffic_router/Dockerfile

jhg03a · 2022-02-04T16:42:31Z

The enroller takes about 5 seconds on my workstation to complete. The ansible dataset loader takes 20 minutes

That's really an apples/oranges type of comparison. While they might have similar goals, the implementation and scope are different. I'm not suggesting trying to use it in this capacity, but rather addressing the simplicity/consistency that came from specifically not trying to support dynamic system addition/removal.

zrhoffman

In order for the CRConfig to be accepted, the geolocation.polling.url and coveragezone.polling.url parameters need to be added to CRConfig.json in a profile for the TR server.

Also, the CRConfig needs to include the .response.stats.tm_host property, the hostname of the TO server. In order for it to be added, the global profile needs to include the tm.url property.

With those changes made, TR will write the CRConfig to the disk. That said, it still doesn't get to the point where it writes to /opt/traffic_router/var/log/traffic_router.log

dev/traffic_router/conf/startup.properties

zrhoffman

The dev.ciab.test domain does not seem to be working:

[zrhoffman@computer trafficcontrol]$ docker-compose exec db ping edge.dev.ciab.test
ping: bad address 'edge.dev.ciab.test'

With these changes, it works for me

[zrhoffman@computer trafficcontrol]$ docker-compose exec db ping edge.dev.ciab.test
PING edge.dev.ciab.test (192.168.64.6): 56 data bytes
64 bytes from 192.168.64.6: seq=0 ttl=64 time=0.267 ms

docker-compose.yml

zrhoffman · 2022-02-07T22:59:16Z

Once these changes are made, TR responds on port 80 and port 3333:

[zrhoffman@computer trafficcontrol]$ curl localhost:3333/crs/stats | jq
{
  "app": {
    "buildTimestamp": "2022-02-07",
    "name": "traffic_router",
    "deploy-dir": "/opt/traffic_router",
    "git-revision": "${buildNumber}",
    "version": "6.2.0"
  },
  "stats": {
    "dnsMap": {},
    "httpMap": {},
    "totalDnsCount": 0,
    "totalHttpCount": 0,
    "totalDsMissCount": 1,
    "appStartTime": 1644274479863,
    "averageDnsTime": 0,
    "averageHttpTime": 0,
    "updateTracker": {
      "lastHttpsCertificatesCheck": 1644274540167,
      "lastCacheStateCheck": 1644274653886,
      "lastCacheStateChange": 1644274480649,
      "lastLetsEncryptDnsChallengeWatcherCheck": 1644274600401,
      "lastHttpsCertificatesFetchSuccess": 1644274540346,
      "lastConfigCheck": 1644274540186,
      "lastConfigChange": 1644274540200,
      "lastSteeringWatcherCheck": 1644274651137,
      "lastNetworkUpdaterCheck": 1644274480509,
      "lastGeolocationDatabaseUpdaterCheck": 1644274480509,
      "lastHttpsCertificatesFetchAttempt": 1644274540167
    }
  }
}

zrhoffman

With these changes, Traffic Router gets closer to serving over port 443 and port 3443. The exception they still run into:

(Click to expand)

08-Feb-2022 00:52:15.124 SEVERE [main] org.apache.catalina.util.LifecycleBase.handleSubClassException Failed to start component [Connector[org.apache.traffic_control.traffic_router.protocol.LanguidNioProtocol-3443]]

08-Feb-2022 00:52:07.869 WARNING [main] org.apache.traffic_control.traffic_router.protocol.LanguidNioProtocol.<clinit> Adding BouncyCastle provider
08-Feb-2022 00:52:08.070 WARNING [main] org.apache.traffic_control.traffic_router.protocol.LanguidNioProtocol.<init> Serving wildcard certs for multiple domains
08-Feb-2022 00:52:08.086 WARNING [main] org.apache.tomcat.util.digester.SetPropertiesRule.begin Match [Server/Service/Connector] failed to set property [sendReasonPhrase] to [true]
08-Feb-2022 00:52:08.086 WARNING [main] org.apache.traffic_control.traffic_router.protocol.LanguidNioProtocol.<init> Serving wildcard certs for multiple domains
08-Feb-2022 00:52:08.088 WARNING [main] org.apache.traffic_control.traffic_router.protocol.LanguidNioProtocol.<init> Serving wildcard certs for multiple domains
08-Feb-2022 00:52:08.089 INFO [main] org.apache.traffic_control.traffic_router.protocol.LanguidNioProtocol.setSslImplementationName setSslImplementation: org.apache.traffic_control.traffic_router.protocol.RouterSslImplementation
08-Feb-2022 00:52:08.093 WARNING [main] org.apache.tomcat.util.digester.SetPropertiesRule.begin Match [Server/Service/Connector] failed to set property [sendReasonPhrase] to [true]
08-Feb-2022 00:52:08.094 WARNING [main] org.apache.traffic_control.traffic_router.protocol.LanguidNioProtocol.<init> Serving wildcard certs for multiple domains
08-Feb-2022 00:52:08.094 INFO [main] org.apache.traffic_control.traffic_router.protocol.LanguidNioProtocol.setSslImplementationName setSslImplementation: org.apache.traffic_control.traffic_router.protocol.RouterSslImplementation
08-Feb-2022 00:52:08.095 WARNING [main] org.apache.tomcat.util.digester.SetPropertiesRule.begin Match [Server/Service/Connector] failed to set property [sendReasonPhrase] to [true]
08-Feb-2022 00:52:08.151 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server version name:   Apache Tomcat/9.0.43
08-Feb-2022 00:52:08.151 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server built:          Jan 28 2021 20:25:45 UTC
08-Feb-2022 00:52:08.151 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server version number: 9.0.43.0
08-Feb-2022 00:52:08.151 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log OS Name:               Linux
08-Feb-2022 00:52:08.151 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log OS Version:            5.12.19-2-mbp
08-Feb-2022 00:52:08.152 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Architecture:          amd64
08-Feb-2022 00:52:08.152 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Java Home:             /usr/lib/jvm/java-11-openjdk
08-Feb-2022 00:52:08.152 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Version:           11.0.14+9-alpine-r0
08-Feb-2022 00:52:08.152 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Vendor:            Alpine
08-Feb-2022 00:52:08.152 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log CATALINA_BASE:         /root/go/src/github.com/apache/trafficcontrol/dev/traffic_router
08-Feb-2022 00:52:08.152 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log CATALINA_HOME:         /opt/tomcat
08-Feb-2022 00:52:08.152 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.base/java.lang=ALL-UNNAMED
08-Feb-2022 00:52:08.152 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.base/java.io=ALL-UNNAMED
08-Feb-2022 00:52:08.152 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.base/java.util=ALL-UNNAMED
08-Feb-2022 00:52:08.153 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.base/java.util.concurrent=ALL-UNNAMED
08-Feb-2022 00:52:08.153 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED
08-Feb-2022 00:52:08.153 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.base/java.lang=ALL-UNNAMED
08-Feb-2022 00:52:08.153 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.base/java.io=ALL-UNNAMED
08-Feb-2022 00:52:08.153 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.base/java.util=ALL-UNNAMED
08-Feb-2022 00:52:08.153 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.base/java.util.concurrent=ALL-UNNAMED
08-Feb-2022 00:52:08.153 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED
08-Feb-2022 00:52:08.153 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.util.logging.config.file=/opt/traffic_router/conf/logging.properties
08-Feb-2022 00:52:08.153 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
08-Feb-2022 00:52:08.153 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.awt.headless=true
08-Feb-2022 00:52:08.153 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.security.egd=file:/dev/./urandom
08-Feb-2022 00:52:08.153 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djdk.tls.ephemeralDHKeySize=2048
08-Feb-2022 00:52:08.153 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.protocol.handler.pkgs=org.apache.catalina.webresources
08-Feb-2022 00:52:08.154 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dorg.apache.catalina.security.SecurityListener.UMASK=0027
08-Feb-2022 00:52:08.154 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -agentlib:jdwp=transport=dt_socket,address=*:5005,server=y,suspend=n
08-Feb-2022 00:52:08.154 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Xms2g
08-Feb-2022 00:52:08.154 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Xmx8g
08-Feb-2022 00:52:08.154 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dlog4j.configurationFile=file:/opt/traffic_router/conf/log4j2.xml
08-Feb-2022 00:52:08.154 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.library.path=/usr/lib
08-Feb-2022 00:52:08.154 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dorg.apache.catalina.connector.Response.ENFORCE_ENCODING_IN_GET_WRITER=false
08-Feb-2022 00:52:08.154 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:+UseG1GC
08-Feb-2022 00:52:08.154 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:+UnlockExperimentalVMOptions
08-Feb-2022 00:52:08.154 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -XX:InitiatingHeapOccupancyPercent=30
08-Feb-2022 00:52:08.154 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dignore.endorsed.dirs=
08-Feb-2022 00:52:08.154 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcatalina.base=/opt/traffic_router
08-Feb-2022 00:52:08.154 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcatalina.home=/opt/tomcat
08-Feb-2022 00:52:08.155 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.io.tmpdir=/opt/traffic_router/temp
08-Feb-2022 00:52:08.162 INFO [main] org.apache.catalina.core.AprLifecycleListener.lifecycleEvent Loaded Apache Tomcat Native library [1.2.31] using APR version [1.7.0].
08-Feb-2022 00:52:08.162 INFO [main] org.apache.catalina.core.AprLifecycleListener.lifecycleEvent APR capabilities: IPv6 [true], sendfile [true], accept filters [false], random [true].
08-Feb-2022 00:52:08.162 INFO [main] org.apache.catalina.core.AprLifecycleListener.lifecycleEvent APR/OpenSSL configuration: useAprConnector [false], useOpenSSL [true]
08-Feb-2022 00:52:08.166 INFO [main] org.apache.catalina.core.AprLifecycleListener.initializeSSL OpenSSL successfully initialized [OpenSSL 1.1.1l  24 Aug 2021]
08-Feb-2022 00:52:08.763 INFO [main] org.apache.traffic_control.traffic_router.protocol.LanguidNioProtocol.init Init called; creating thread to monitor the state of Traffic Router
08-Feb-2022 00:52:08.770 INFO [main] org.apache.traffic_control.traffic_router.protocol.LanguidNioProtocol.init Init called; creating thread to monitor the state of Traffic Router
08-Feb-2022 00:52:08.781 INFO [main] org.apache.traffic_control.traffic_router.protocol.LanguidNioProtocol.init Init called; creating thread to monitor the state of Traffic Router
08-Feb-2022 00:52:08.785 INFO [main] org.apache.traffic_control.traffic_router.protocol.LanguidNioProtocol.init Init called; creating thread to monitor the state of Traffic Router
08-Feb-2022 00:52:08.855 INFO [main] org.apache.catalina.startup.Catalina.load Server initialization in [2171] milliseconds
08-Feb-2022 00:52:08.901 INFO [main] org.apache.catalina.core.StandardService.startInternal Starting service [traffic_router_core]
08-Feb-2022 00:52:08.902 INFO [main] org.apache.catalina.core.StandardEngine.startInternal Starting Servlet engine: [Apache Tomcat/9.0.43]
08-Feb-2022 00:52:11.257 INFO [main] org.apache.catalina.core.ApplicationContext.log No Spring WebApplicationInitializer types detected on classpath
08-Feb-2022 00:52:11.351 INFO [main] org.apache.jasper.servlet.TldScanner.scanJars At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time.
08-Feb-2022 00:52:11.373 INFO [main] org.apache.catalina.core.ApplicationContext.log Initializing Spring root WebApplicationContext
08-Feb-2022 00:52:14.210 INFO [main] org.apache.catalina.core.ApplicationContext.log default: DefaultServlet.init:  input buffer size=2048, output buffer size=2048
08-Feb-2022 00:52:14.219 INFO [main] org.apache.catalina.core.ApplicationContext.log Initializing Spring DispatcherServlet 'status'
08-Feb-2022 00:52:14.286 INFO [Thread-2] org.apache.traffic_control.traffic_router.protocol.LanguidNioProtocol.init Traffic Router SSL Protocol is ready; calling super.init()
08-Feb-2022 00:52:14.286 INFO [Thread-3] org.apache.traffic_control.traffic_router.protocol.LanguidNioProtocol.init Traffic Router SSL Protocol is ready; calling super.init()
08-Feb-2022 00:52:14.288 INFO [Thread-3] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["http-nio-3333"]
08-Feb-2022 00:52:14.286 INFO [Thread-5] org.apache.traffic_control.traffic_router.protocol.LanguidNioProtocol.init Traffic Router SSL Protocol is ready; calling super.init()
08-Feb-2022 00:52:14.289 INFO [Thread-2] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["http-nio-80"]
08-Feb-2022 00:52:14.289 INFO [Thread-4] org.apache.traffic_control.traffic_router.protocol.LanguidNioProtocol.init Traffic Router SSL Protocol is ready; calling super.init()
08-Feb-2022 00:52:14.290 INFO [Thread-5] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["https-openssl-nio-443"]
08-Feb-2022 00:52:14.290 INFO [Thread-4] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["https-openssl-nio-3443"]
08-Feb-2022 00:52:15.084 INFO [main] org.apache.traffic_control.traffic_router.protocol.LanguidNioProtocol.start LanguidNioProtocol Handler Start called; waiting for initialization to occur
08-Feb-2022 00:52:15.084 INFO [main] org.apache.traffic_control.traffic_router.protocol.LanguidNioProtocol.start LanguidNioProtocol Handler Initialization complete; calling super.start()
08-Feb-2022 00:52:15.085 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["http-nio-80"]
08-Feb-2022 00:52:15.096 INFO [main] org.apache.traffic_control.traffic_router.protocol.LanguidNioProtocol.start LanguidNioProtocol Handler Start called; waiting for initialization to occur
08-Feb-2022 00:52:15.096 INFO [main] org.apache.traffic_control.traffic_router.protocol.LanguidNioProtocol.start LanguidNioProtocol Handler Initialization complete; calling super.start()
08-Feb-2022 00:52:15.096 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["http-nio-3333"]
08-Feb-2022 00:52:15.119 INFO [main] org.apache.traffic_control.traffic_router.protocol.LanguidNioProtocol.start LanguidNioProtocol Handler Start called; waiting for initialization to occur
08-Feb-2022 00:52:15.119 INFO [main] org.apache.traffic_control.traffic_router.protocol.LanguidNioProtocol.start LanguidNioProtocol Handler Initialization complete; calling super.start()
08-Feb-2022 00:52:15.119 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["https-openssl-nio-3443"]
08-Feb-2022 00:52:15.124 SEVERE [main] org.apache.catalina.util.LifecycleBase.handleSubClassException Failed to start component [Connector[org.apache.traffic_control.traffic_router.protocol.LanguidNioProtocol-3443]]
	org.apache.catalina.LifecycleException: Protocol handler start failed
		at org.apache.catalina.connector.Connector.startInternal(Connector.java:1074)
		at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
		at org.apache.catalina.core.StandardService.startInternal(StandardService.java:451)
		at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
		at org.apache.catalina.core.StandardServer.startInternal(StandardServer.java:930)
		at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
		at org.apache.catalina.startup.Catalina.start(Catalina.java:772)
		at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
		at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
		at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
		at java.base/java.lang.reflect.Method.invoke(Method.java:566)
		at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:342)
		at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:473)
	Caused by: java.lang.IllegalArgumentException: No SSLHostConfig element was found with the hostName [_default_] to match the defaultSSLHostConfigName for the connector [https-openssl-nio-3443]
		at org.apache.tomcat.util.net.AbstractJsseEndpoint.initialiseSsl(AbstractJsseEndpoint.java:76)
		at org.apache.traffic_control.traffic_router.protocol.RouterNioEndpoint.initialiseSsl(RouterNioEndpoint.java:56)
		at org.apache.tomcat.util.net.NioEndpoint.bind(NioEndpoint.java:246)
		at org.apache.tomcat.util.net.AbstractEndpoint.bindWithCleanup(AbstractEndpoint.java:1193)
		at org.apache.tomcat.util.net.AbstractEndpoint.start(AbstractEndpoint.java:1279)
		at org.apache.coyote.AbstractProtocol.start(AbstractProtocol.java:608)
		at org.apache.traffic_control.traffic_router.protocol.LanguidNioProtocol.start(LanguidNioProtocol.java:88)
		at org.apache.catalina.connector.Connector.startInternal(Connector.java:1071)
		... 12 more
08-Feb-2022 00:52:15.124 INFO [main] org.apache.traffic_control.traffic_router.protocol.LanguidNioProtocol.start LanguidNioProtocol Handler Start called; waiting for initialization to occur
08-Feb-2022 00:52:15.124 INFO [main] org.apache.traffic_control.traffic_router.protocol.LanguidNioProtocol.start LanguidNioProtocol Handler Initialization complete; calling super.start()
08-Feb-2022 00:52:15.124 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["https-openssl-nio-443"]
08-Feb-2022 00:52:15.125 SEVERE [main] org.apache.catalina.util.LifecycleBase.handleSubClassException Failed to start component [Connector[org.apache.traffic_control.traffic_router.protocol.LanguidNioProtocol-443]]
	org.apache.catalina.LifecycleException: Protocol handler start failed
		at org.apache.catalina.connector.Connector.startInternal(Connector.java:1074)
		at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
		at org.apache.catalina.core.StandardService.startInternal(StandardService.java:451)
		at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
		at org.apache.catalina.core.StandardServer.startInternal(StandardServer.java:930)
		at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
		at org.apache.catalina.startup.Catalina.start(Catalina.java:772)
		at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
		at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
		at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
		at java.base/java.lang.reflect.Method.invoke(Method.java:566)
		at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:342)
		at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:473)
	Caused by: java.lang.IllegalArgumentException: No SSLHostConfig element was found with the hostName [_default_] to match the defaultSSLHostConfigName for the connector [https-openssl-nio-443]
		at org.apache.tomcat.util.net.AbstractJsseEndpoint.initialiseSsl(AbstractJsseEndpoint.java:76)
		at org.apache.traffic_control.traffic_router.protocol.RouterNioEndpoint.initialiseSsl(RouterNioEndpoint.java:56)
		at org.apache.tomcat.util.net.NioEndpoint.bind(NioEndpoint.java:246)
		at org.apache.tomcat.util.net.AbstractEndpoint.bindWithCleanup(AbstractEndpoint.java:1193)
		at org.apache.tomcat.util.net.AbstractEndpoint.start(AbstractEndpoint.java:1279)
		at org.apache.coyote.AbstractProtocol.start(AbstractProtocol.java:608)
		at org.apache.traffic_control.traffic_router.protocol.LanguidNioProtocol.start(LanguidNioProtocol.java:88)
		at org.apache.catalina.connector.Connector.startInternal(Connector.java:1071)
		... 12 more
08-Feb-2022 00:52:15.125 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in [6269] milliseconds

Maybe it doesn't like the SSL cert?

dev/traffic_router/Dockerfile

dev/traffic_router/conf/startup.properties

zrhoffman · 2022-02-08T02:46:57Z

With these changes, Traffic Router gets closer to serving over port 443 and port 3443. The exception they still run into:
[...]
Maybe it doesn't like the SSL cert?

certificateRegistry.getHandshakeData() is empty at

trafficcontrol/traffic_router/connector/src/main/java/org/apache/traffic_control/traffic_router/protocol/RouterNioEndpoint.java

Line 53 in 3682943

replaceSSLHosts(certificateRegistry.getHandshakeData());

, so it looks like the cert is not getting imported successfully.

zrhoffman · 2022-02-10T14:07:03Z

There are conflicts now that #6513 is merged

…files being created

…export them This is likely to fail on systems where UID isn't actually set or is somehow set improperly as a shell variable, but it fixes compatibility issues for a few major OSs.

zrhoffman

Awesome! TR routes over DNS and HTTP, confirmed TR, TO, and TM debugging work, and documentation looks good.

phusinh · 2022-07-02T15:26:04Z

TR routes DNS requests but does not route HTTP requests:

[zrhoffman@computer traffic_router]$ curl -vH 'Host: cdn.dev.ciab.test' localhost:3080
*   Trying 127.0.0.1:3080...
* Connected to localhost (127.0.0.1) port 3080 (#0)
> GET / HTTP/1.1
> Host: cdn.dev.ciab.test
> User-Agent: curl/7.81.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 503
< Content-Type: text/html;charset=utf-8
< Content-Language: en
< Content-Length: 732
< Date: Wed, 23 Feb 2022 23:13:06 GMT
< Connection: close
<
* Closing connection 0
<!doctype html><html lang="en"><head><title>HTTP Status 503 – Service Unavailable</title><style type="text/css">body {font-family:Tahoma,Arial,sans-serif;} h1, h2, h3, b {color:white;background-color:#525D76;} h1 {font-size:22px;} h2 {font-size:16px;} h3 {font-size:14px;} p {font-size:12px;} a {color:black;} .line {height:1px;background-color:#525D76;border:none;}</style></head><body><h1>HTTP Status 503 – Service Unavailable</h1><hr class="line" /><p><b>Type</b> Status Report</p><p><b>Description</b> The server is currently unable to handle the request due to a temporary overload or scheduled maintenance, which will likely be alleviated after some delay.</p><hr class="line" /><h3>Apache Tomcat/9.0.43</h3></body></html>

I made such a request to the dns port but returned this error, did I make a mistake, please reply.I did not get a response from the dns

ocket8888 · 2022-07-05T17:27:07Z

@phusinh

For SUPPORT QUESTIONS, use the #traffic-control channel on the ASF slack (https://s.apache.org/tc-slack-request)
or the Traffic Control Users mailing list (send an email to users-subscribe@trafficcontrol.apache.org to subscribe).

ocket8888 added improvement The functionality exists but it could be improved in some way. experimental a feature/component not directly supported by ATC process labels Jan 31, 2022

zrhoffman requested changes Feb 1, 2022

View reviewed changes

ocket8888 force-pushed the dev-ciab branch from 6438fde to 1223cf7 Compare February 2, 2022 20:46

zrhoffman requested changes Feb 2, 2022

View reviewed changes

dev/t3c/run.sh Outdated Show resolved Hide resolved

dev/traffic_router/run.sh Outdated Show resolved Hide resolved

zrhoffman reviewed Feb 2, 2022

View reviewed changes

dev/traffic_router/Dockerfile Outdated Show resolved Hide resolved

ocket8888 force-pushed the dev-ciab branch from 1223cf7 to 7b30bab Compare February 3, 2022 16:52

zrhoffman requested changes Feb 4, 2022

View reviewed changes

zrhoffman reviewed Feb 4, 2022

View reviewed changes

dev/traffic_router/Dockerfile Outdated Show resolved Hide resolved

zrhoffman reviewed Feb 7, 2022

View reviewed changes

dev/traffic_router/conf/startup.properties Outdated Show resolved Hide resolved

zrhoffman requested changes Feb 7, 2022

View reviewed changes

zrhoffman requested changes Feb 8, 2022

View reviewed changes

dev/traffic_router/Dockerfile Outdated Show resolved Hide resolved

dev/traffic_router/conf/startup.properties Outdated Show resolved Hide resolved

ocket8888 force-pushed the dev-ciab branch from 7b30bab to 863a402 Compare February 14, 2022 17:45

ocket8888 added 20 commits February 24, 2022 13:24

remove unused code

19bf1d9

Add TR to the README

51de2ba

Fix spelling error

46b38a3

Fix atc-restart only respecting args for the start portion

477a367

Fix non-idempotent database seeding

10b16b3

Add proper documentation

5e2f359

Fix TR working directory

fff9a77

Fix incorrect build directory

c92a9f8

Fix user ID not properly set in Traffic Router to avoid inaccessible …

49781eb

…files being created

Add setting/exporting UID and USER for systems/shells that don't set/…

bd28407

…export them This is likely to fail on systems where UID isn't actually set or is somehow set improperly as a shell variable, but it fixes compatibility issues for a few major OSs.

Add TR logs to gitignore

e24c2ee

Set default user identification values

b215638

Add more info to TR logs

3a70eed

Fix TR not able to find its logging configuration

c5de27f

Remove extraneous escapism

a27b0fc

Add 'clean' target to Makefile to destroy built binaries

93ac83c

Change DS XMLID to dev-ds and add coveragezone file

5619116

Fix using apparently invalid latitude/longitude

7b90add

Fix DS not associated with edge server

504dd68

fix cached snapshot not updated for recent changes

a1e9dff

ocket8888 force-pushed the dev-ciab branch from dfdf7d4 to a1e9dff Compare February 24, 2022 20:45

zrhoffman approved these changes Feb 24, 2022

View reviewed changes

zrhoffman merged commit b1976d7 into apache:master Feb 24, 2022

ocket8888 deleted the dev-ciab branch February 25, 2022 16:02

zrhoffman added the go version label Feb 28, 2022

This was referenced Mar 1, 2022

Fix incorrect template strings #6595

Merged

Fix the "PR-to-Update-Go" GitHub Action #6605

Merged

zrhoffman added the dev-ciab Related to CDN in a Box for Developers label Apr 7, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CDN-in-a-Box for Developers #6532

CDN-in-a-Box for Developers #6532

ocket8888 commented Jan 31, 2022

zrhoffman left a comment

zrhoffman Feb 1, 2022

ocket8888 Feb 1, 2022 •

edited

zrhoffman Oct 19, 2022

zrhoffman commented Feb 1, 2022

ocket8888 commented Feb 1, 2022

zrhoffman commented Feb 1, 2022 •

edited

ocket8888 commented Feb 1, 2022

zrhoffman left a comment

mitchell852 commented Feb 2, 2022

ocket8888 commented Feb 2, 2022

mitchell852 commented Feb 3, 2022

ocket8888 commented Feb 3, 2022 •

edited

zrhoffman commented Feb 3, 2022

jhg03a commented Feb 4, 2022

zrhoffman left a comment •

edited

zrhoffman commented Feb 4, 2022

jhg03a commented Feb 4, 2022

zrhoffman left a comment

zrhoffman left a comment

zrhoffman commented Feb 7, 2022 •

edited

zrhoffman left a comment

zrhoffman commented Feb 8, 2022

zrhoffman commented Feb 10, 2022

zrhoffman left a comment

phusinh commented Jul 2, 2022 •

edited

ocket8888 commented Jul 5, 2022

CDN-in-a-Box for Developers #6532

CDN-in-a-Box for Developers #6532

Conversation

ocket8888 commented Jan 31, 2022

Which Traffic Control components are affected by this PR?

What is the best way to verify this PR?

PR submission checklist

zrhoffman left a comment

Choose a reason for hiding this comment

zrhoffman Feb 1, 2022

Choose a reason for hiding this comment

ocket8888 Feb 1, 2022 • edited

Choose a reason for hiding this comment

zrhoffman Oct 19, 2022

Choose a reason for hiding this comment

zrhoffman commented Feb 1, 2022

ocket8888 commented Feb 1, 2022

zrhoffman commented Feb 1, 2022 • edited

ocket8888 commented Feb 1, 2022

zrhoffman left a comment

Choose a reason for hiding this comment

mitchell852 commented Feb 2, 2022

ocket8888 commented Feb 2, 2022

mitchell852 commented Feb 3, 2022

ocket8888 commented Feb 3, 2022 • edited

zrhoffman commented Feb 3, 2022

jhg03a commented Feb 4, 2022

zrhoffman left a comment • edited

Choose a reason for hiding this comment

zrhoffman commented Feb 4, 2022

jhg03a commented Feb 4, 2022

zrhoffman left a comment

Choose a reason for hiding this comment

zrhoffman left a comment

Choose a reason for hiding this comment

zrhoffman commented Feb 7, 2022 • edited

zrhoffman left a comment

Choose a reason for hiding this comment

zrhoffman commented Feb 8, 2022

zrhoffman commented Feb 10, 2022

zrhoffman left a comment

Choose a reason for hiding this comment

phusinh commented Jul 2, 2022 • edited

ocket8888 commented Jul 5, 2022

ocket8888 Feb 1, 2022 •

edited

zrhoffman commented Feb 1, 2022 •

edited

ocket8888 commented Feb 3, 2022 •

edited

zrhoffman left a comment •

edited

zrhoffman commented Feb 7, 2022 •

edited

phusinh commented Jul 2, 2022 •

edited