admin tool no seed on upgrade (#7614)

* Move seed to the database creation step

* seeding must only be done after schema is loaded

* update documentation

* Update changelog

* Add manual seeding to t3c and tc-health-client tests

CHANGELOG.md

@@ -7,6 +7,9 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
 ### Added
 - [#7812](https://github.com/apache/trafficcontrol/pull/7812) *Traffic Portal*: Expose the `configUpdateFailed` and `revalUpdateFailed` fields on the server table.
 
+### Changed
+- [#7614](https://github.com/apache/trafficcontrol/pull/7614) *Traffic Ops* The database upgrade process no longer overwrites changes users may have made to the initially seeded data.
+
 ## [8.0.0] - 2023-09-20
 ### Added
 - [#7672](https://github.com/apache/trafficcontrol/pull/7672) *Traffic Control Health Client*: Added peer monitor flag while using `strategies.yaml`.

cache-config/testing/docker/traffic_ops/run.sh

@@ -149,6 +149,7 @@ cd /opt/traffic_ops/app
 (
 db/admin --env=production reset
 db/admin --env=production upgrade
+db/admin --env=production seed
 db/admin --trafficvault --env=production reset
 db/admin --trafficvault --env=production upgrade
 ) >> /var/log/traffic_ops/to_admin.log 2>&1) || {

docs/source/admin/traffic_ops.rst

@@ -226,7 +226,7 @@ Upgrading
 =========
 To upgrade from older Traffic Ops versions, stop the service, use :manpage:`yum(8)` to upgrade to the latest available Traffic Ops package, and use the :ref:`admin <database-management>` tool to perform the database upgrade.
 
-.. tip:: In order to upgrade to the latest version of Traffic Ops, please be sure that you have first upgraded to the latest available minor or patch version of your current release. For example, if your current Traffic Ops version is 3.0.0 and version 3.1.0 is available, you must first upgrade to 3.1.0 before proceeding to upgrade to 4.0.0. (Specifically, this means running all migrations, :file:`traffic_ops/app/db/seeds.sql`, and :file:`traffic_ops/app/db/patches.sql` for the latest of your current major version - which should be handled by the :program:`admin` tool). The latest migration available before the release of 4.0.0 (pending at the time of this writing) was :file:`traffic_ops/app/db/migrations/20180814000625_remove_capabilities_for_reseed.sql`, so be sure that migrations up to this point have been run before attempting to upgrade Traffic Ops.
+.. tip:: In order to upgrade to the latest version of Traffic Ops, please be sure that you have first upgraded to the latest available minor or patch version of your current release. For example, if your current Traffic Ops version is 3.0.0 and version 3.1.0 is available, you must first upgrade to 3.1.0 before proceeding to upgrade to 4.0.0. (Specifically, this means running all migrations, :atc-file:`traffic_ops/app/db/seeds.sql`, and :file:`traffic_ops/app/db/patches.sql` for the latest of your current major version - which should be handled by the :program:`admin` tool). The latest migration available before the release of 4.0.0 (pending at the time of this writing) was :file:`traffic_ops/app/db/migrations/20180814000625_remove_capabilities_for_reseed.sql`, so be sure that migrations up to this point have been run before attempting to upgrade Traffic Ops.
 
 .. seealso:: :ref:`database-management` for more details about :program:`admin`.
 

docs/source/development/traffic_ops.rst

@@ -208,7 +208,7 @@ Options and Arguments
 	The :option:`command` specifies the operation to be performed on the database. It must be one of:
 
 	createdb
-		Creates the database for the current environment
+		Creates the database for the current environment.
 	create_migration
 		Creates a pair of timestamped up/down migrations titled NAME.
 	create_user
@@ -222,7 +222,7 @@ Options and Arguments
 	drop_user
 		Drops the user defined for the current environment
 	load_schema
-		Sets up the database for the current environment according to the SQL statements in ``app/db/create_tables.sql`` or ``app/db/trafficvault/create_tables.sql`` if the ``--trafficvault`` option is used
+		Sets up the database for the current environment according to the SQL statements in :atc-file:`traffic_ops/app/db/create_tables.sql` or :atc-file:`traffic_ops/app/db/trafficvault/create_tables.sql` if the ``--trafficvault`` option is used
 	migrate
 		Runs a migration on the database for the current environment
 	patch
@@ -232,13 +232,13 @@ Options and Arguments
 	reset
 		Creates the user defined for the current environment, drops the database for the current environment, creates a new one, loads the schema into it, and runs a single migration on it
 	seed
-		Executes the SQL statements from the ``app/db/seeds.sql`` file for loading static data. This command is not supported when using the ``--trafficvault`` option
+		Executes the SQL statements from the ``app/db/seeds.sql`` file for loading static data. This command is not supported when using the ``--trafficvault`` option. The seed data is constructed under the assumption all migrations for the release have been run, so ``migrate``/\ ``upgrade`` *must* be run first.
 	show_users
 		Displays a list of all users registered with the PostgreSQL server
 	status
 		Deprecated, ``status`` is now an alias for ``dbversion`` and will be removed in a future Traffic Control release.
 	upgrade
-		Performs a migration on the database for the current environment, then seeds it and patches it using the SQL statements from the ``app/db/patches.sql`` file
+		Performs a migration on the database for the current environment, then patches it using the SQL statements from the :atc-file:`traffic_ops/app/db/patches.sql` file.
 
 .. code-block:: bash
 	:caption: Example Usage

tc-health-client/testing/docker/traffic_ops/run.sh

@@ -7,9 +7,9 @@ set -x
 # to you under the Apache License, Version 2.0 (the
 # "License"); you may not use this file except in compliance
 # with the License.  You may obtain a copy of the License at
-# 
+#
 #   http://www.apache.org/licenses/LICENSE-2.0
-# 
+#
 # Unless required by applicable law or agreed to in writing,
 # software distributed under the License is distributed on an
 # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
@@ -102,7 +102,11 @@ source /etc/environment
 if [ -z "$INITIALIZED" ]; then init; fi
 
 # create the 'traffic_ops' database, tables and runs migrations
-(cd /opt/traffic_ops/app && db/admin --env=production reset > /admin.log 2>&1)
+pushd /opt/traffic_ops/app;
+db/admin --env=production reset > /admin.log 2>&1
+db/admin --env=production upgrade >> /admin.log 2>&1
+db/admin --env=production seed >> /admin.log 2>&1
+popd;
 
 # start traffic_ops
 start

traffic_ops/app/db/admin.go

@@ -353,7 +353,6 @@ func reset() {
 func upgrade() {
 	runMigrations()
 	if !trafficVault {
-		seed()
 		patch()
 	}
 }
@@ -459,7 +458,7 @@ func seed() {
 	if trafficVault {
 		die("seed not supported for trafficvault environment")
 	}
-	fmt.Println("Seeding database w/ required data.")
+	fmt.Println("Seeding database with required data.")
 	seedsBytes, err := ioutil.ReadFile(dbSeedsPath)
 	if err != nil {
 		die("unable to read '" + dbSeedsPath + "': " + err.Error())
@@ -507,9 +506,9 @@ func patch() {
 	cmd.Stdin = bytes.NewBuffer(patchesBytes)
 	cmd.Env = append(os.Environ(), "PGPASSWORD="+dbPassword)
 	out, err := cmd.CombinedOutput()
-	fmt.Printf(string(out))
+	fmt.Println(string(out))
 	if err != nil {
-		die("Can't patch database w/ required data")
+		die("Can't patch database with required data")
 	}
 }
 
@@ -559,19 +558,22 @@ OPERATION      The operation to perform; one of the following:
     dbversion   - Prints the current migration timestamp
     drop_user   - Execute 'drop_user' the user for the current environment
                   (traffic_ops).
+    load_schema - Loads the database schema.
     patch       - Execute sql from db/patches.sql for loading post-migration data
                   patches (NOTE: not supported with --trafficvault option).
     redo        - Roll back the most recently applied migration, then run it again.
     reset       - Execute db 'dropdb', 'createdb', load_schema, migrate on the
                   database for the current environment.
     seed        - Execute sql from db/seeds.sql for loading static data (NOTE: not
-                  supported with --trafficvault option).
+                  supported with --trafficvault option). This MUST ONLY be done
+				  after the schema is loaded and migrations have been run.
     show_users  - Execute sql to show all of the user for the current environment.
     status      - Prints the current migration timestamp (Deprecated, status is now an
                   alias for dbversion and will be removed in a future Traffic
                   Control release).
-    upgrade     - Execute migrate, seed, and patches on the database for the current
-                  environment.`)
+    upgrade     - Execute migrate and patch on the database for the current
+                  environment.`,
+	)
 	return buff.String()
 }