tools: improved newgtlds.go, removed replace-between. (#1204)

== tools: improved newgtlds.go, removed replace-between. - 

d996dad

Previously the `tools/newgtlds.go` utility was difficult to use with the new Github action based pull-request workflow because it _always_ updated the timestamp in the header comment in the gTLD section of `public_suffix_list.dat` when it was run, even if no gTLD data changed. This meant there was a diff in the workdir after the tool ran that would cause a PR to be opened by the action. See #1166 for more discussion/background.

In the old travis version of the automation I used a crude shell pipeline to exclude the header comment when deciding if there was a diff or not. With the new action it's not possible to change how the diff status is determined (without re-inventing the pull request action in-repo). So instead we need to abandon the timestamp in the gTLD section header comment (that would be sad) or, we need the script to be idempotent when there's no true data change.

To make the tooling smarter about when it makes changes I reworked `newgtlds.go` to be able to consume the existing `public_suffix_list.dat` and to split apart the data templating and the header comment templating. Now the tool can exclude the header comment and compare against the existing data to determine if there is a real data change or not. If there's no change in the data then the file is left untouched. If there is a change then the file is updated and only then is the header comment with the date stamp updated along with the new data. I've included unit tests that get 80% statement coverage of this new tooling and also tested it locally with success. Conveniently this rework also lets us remove the `tools/replace-between` Perl script reducing the number of languages in play for this process.

== CI: re-enable github tld-update workflow - 

7856839

Now that the tooling won't produce PRs without meaningful data updates we can re-enable the `tld-update.yml` workflow.

== CI: run go unit tests in tld-update workflow - 

f6749a5

Before using the `tools/newgtlds.go` tooling to open a PR we should run the unit tests to make sure the code works the way we expect. This will also help catch any bitrot.

.github/workflows/tld-update.yml.hold → .github/workflows/tld-update.yml

@@ -19,6 +19,9 @@ jobs:
         with:
           go-version: ^1.15
 
+      - name: Run unit tests
+        run: go test tools/*.go
+
       - name: Set current date
         id: get-date
         run: echo "::set-output name=now::$(date +'%Y-%m-%dT%H:%M:%S %Z')"

.gitignore

@@ -1,3 +1,3 @@
 linter/log
 libpsl
-
+coverage.out

.travis.yml

@@ -1,7 +1,12 @@
 language: c
 compiler: gcc
 
-script: make
+script: 
+  - make
+  - go test -v -coverprofile=coverage.out tools/*.go
+
+go:
+  - "1.15.x"
 
 addons:
     apt:
@@ -18,4 +23,3 @@ addons:
             - libicu-dev
             - libunistring0
             - libunistring-dev
-