Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Verification of Installation, Back- Up and Clean Up Steps necessary for Successful install of v1.8 #196

Open
cflottorp opened this issue Apr 19, 2017 · 51 comments
Assignees

Comments

@cflottorp
Copy link

Need to confirm the order for the installation, back up of the existing pl_fit-db and clean up of database using the various scripts. Due to the various scripts that have been delivered, the order of the recommended deployment needs to be verified. Please confirm the following order or update as needed for a "clean", rebuild of the ISS FIT for version 1.8.

  1. Run this database migration script https://github.com/NASA-Tournament-Lab/NTL-ISS-Food-Intake-Tracker/blob/master/Backend Services/Migration.md
  2. Run the CleanUpISSFITDatabase script
  3. Run the 0100_fit_v1 script
  4. Run the 0100_fit_v2 script
  5. Using the Admin Tool, upload a Food CSV file and keep the checkbox "mark all food as removed" checked
@pizza-and-dog
Copy link
Contributor

I would make number 1 "back up database".

For the others, I'll ask @coderReview to confirm.

@coderReview
Copy link
Contributor

@cflottorp
if the server is already running the FIT application step 3 is not necessary. The v1 script is to setup the first version of the FIT application which is running already.

Since step 4 will delete the database, the migration step (1) won't do any good. If you want I can remove the database removal from the v2 script.

I don't know what is the script in step 2.

In my opinion, if you want to keep the data for the next version, you will need to:

1- Run the database migration
2- Run the 0100_fit_v2 script
3- Using the Admin Tool, upload a Food CSV file and keep the checkbox "mark all food as removed" checked

Otherwise you will need to:

1- Run the 0100_fit_v2 script
2- Using the Admin Tool, upload a Food CSV file

@cflottorp
Copy link
Author

The script in step 2 was to clean up the ISS FIT database to remove duplication in the database. With the first recommendation to run the migration, 0100_fit_v2 script , will that include cleaning up the database to remove the duplicates?

@pizza-and-dog
Copy link
Contributor

@coderReview we need to push to resolve this as soon as possible - please confirm.

@coderReview
Copy link
Contributor

@cflottorp can you confirm if you want to keep the current data? From the other issue I understood you would like to start fresh.

@cflottorp
Copy link
Author

With the V1 and v2 scripts there is no need to run the clean up script in step 2 that was received previously? This was a script to remove duplicates in the database. https://drive.google.com/file/d/0B3nmbMgH4_oxWVV6bFdVZkRGWmM/view

@cflottorp
Copy link
Author

We will be doing a fresh re-load, so we should only need to be running the 0100_fit_v2 script. Then we will use Admin Tool to upload a Food CSV file and keep the checkbox "mark all food as removed" checked and also a User Data CSV file for user log in. Correct?

@coderReview
Copy link
Contributor

Correct. You may just want to make a backup of the existing data for historical reason. I've sent the command on how to do that some time ago. Do you need me to send again? @rsial2 since the old data won't be kept the script is good to go for testing.

@pizza-and-dog
Copy link
Contributor

We need @szwart to confirm if the old data is needed or not.

@cflottorp
Copy link
Author

Yes, we use the "pg_dump -h $' IP address' -p xxxxx -U pl_fit_db -W -Fc -f pl_fit.bak pl_fit"

@szwart
Copy link

szwart commented Apr 21, 2017

We are good with a total wipe and reload. We get reports down every week so we have all of the historical data we need.

@briandmartin
Copy link

briandmartin commented Jul 20, 2017

I attempted the latest installation instructions provided by Cheryl and they were no joy. I started with LS1 loaded with the FIT version installed in the LS1 Fajita SP. I then ran the 0100_fit_v2 script.

The first error was bash refused to run the script because it's a Windows style file format. I converted the file to Unix file format and proceeded.

The next error was a series of errors related to one topic and that is that multiple commands fail due to a missing directory structure. These three lines throughout the script have non zero exit code:
unzip -q -o "${OTHER_FILES_DIR}/${ADMIN_TOOL_ZIP_NAME}" -d "${FIT_APP_DIR}"
chown -R ${ACCT} "${FIT_APP_DIR}/${FIT_FOLDER_NAME}"
sed -i.bak ${DATASOURCESJSON} \
-e "s/CHANGEME/$DB_ACCT_PASSWD/g" \
-e '/"host": ".*"/s/"host": ".*"/"host": "'"${DB_IP}"'"/g'

The problem comes from a combination of these two variables definitions and the FIT app zip file ( NTL-ISS-Food-Intake-Tracker-master.zip).

FIT_APP_DIR=/home/app/${ACCT}
FIT_FOLDER_NAME="NTL-ISS-Food-Intake-Tracker-master"

The zip file no longer encodes the directory name "NTL-ISS-Food-Intake-Tracker-master" in the zip file path.

I do not have a suggested fix for these issues.

That's what I have to report at this time. I'm fairly certain the FIT installer made it about 40 % through the installation process before exiting out with these errors. No other errors to report for this first part of the script (0100_fit_v2) that ran as expected.

@pizza-and-dog
Copy link
Contributor

@coderReview please treat this as high priority :)

@coderReview
Copy link
Contributor

@rsial2 will look into this. But just to remember that the v2 of the script must be applied to a system that had v1 of the script already applied. Thanks

@pizza-and-dog
Copy link
Contributor

@briandmartin can you confirm that the above script was run first?

@briandmartin
Copy link

FIT was installed on LS1 by the Fajita Service Pack and this SP was deploy On-orbit ~May 2016. I don't have the SP with me now but I think the installer filename was 1100_fit (without a v1 in the filename). A previous integration attempt of FIT v. 1.8 was attempted with LS1 Fajita SP + 0100_fit_v1 + 0100_fit_v2). This failed and Cheryl was sent the installation errors. I was told to instead try LS1 Fajita SP + 0100_fit_v2 (without deploying 0100_fit_v1). That was what was tested today, failed and are the errors sent today.

@cflottorp
Copy link
Author

Brian,
So if you are starting with the Fajita Service Pack for LS1 server, then does it NOT contain the dependency files?

  1.  Configparser
    
  2.  Orderreddict
    
  3.  Python crontab
    
  4.  Setuptools
    
  5.  Uuid
    
  6.  Uuid-ossp
    

@briandmartin
Copy link

I'm not sure I understand your question, Cheryl. Let me post directory listings of what I installed (both in Fajita SP and yesterday's v. 1.8 failed installation attempt).

@pizza-and-dog
Copy link
Contributor

Just to confirm folks, the V1 script must be run first, followed by V2.

If the V1 script fails, please send the errors.

If it passes, please continue to the V2 script. If V2 fails, please send the errors.

Thanks!

@briandmartin
Copy link

I had sent Cheryl the V1 deploy errors previously. I will look for that Email and post the errors to this thread.

@pizza-and-dog
Copy link
Contributor

pizza-and-dog commented Jul 21, 2017

Awesome, thanks Brian!

@briandmartin
Copy link

Directory listing of installation files for FIT version installed by Fajita Service Pack (Released to Operations May 2016, deployed On-orbit June 2016):
ssc_system/trunk/linux/chroot_os/os_build/build_scripts_ls1/service_pack/fajita$ tree
.
├── build_record
│ └── 1100_fit
├── other_files
│ ├── Admin Tool.zip
│ ├── Data.zip
│ ├── node-0.10.29-release.zip
│ └── nodejs
├── other_pkgs
│ ├── jade_1.2.1-47_amd64.deb
│ ├── libsp1c2_1.3.4-1.2.1-47_amd64.deb
│ ├── python-egenix-mxdatetime_3.1.3-4_amd64.deb
│ ├── python-egenix-mxtools_3.1.3-4_amd64.deb
│ └── python-psycopg2_2.2.1-1_amd64.deb

ssc_system/trunk/linux/chroot_os/os_build/build_scripts_ls1/service_pack/fajita$ find . -type f -exec md5sum {} ;
d078be12264042c59fa4a87ca6301b44 ./other_files/Admin Tool.zip
936ba66d3c0c92d12b5616ea6c2be4f9 ./other_files/node-0.10.29-release.zip
1ced2dba49526349828f418c5c8c7e15 ./other_files/Data.zip
4d16def245ab49b985940c4ca6853f13 ./other_files/nodejs
1d7ea036d7ae33154fe1bf9fce750d66 ./Makefile
2a221b166eb11f4a71f2dd643a455d76 ./sp_header
445fa9a5d0cce767697804c750fb326b ./sp_trailer
4c042c82b0266852079c4a97cb54e9da ./undo.sh
29a7afe6e21345ce3d79374a64bf465c ./other_pkgs/python-egenix-mxdatetime_3.1.3-4_amd64.deb
1f09d4832ecacf1e59e72c9bfcea6f1f ./other_pkgs/libsp1c2_1.3.4-1.2.1-47_amd64.deb
59dba4142639bc5cad8b06f642648757 ./other_pkgs/python-egenix-mxtools_3.1.3-4_amd64.deb
8f8b757375bf80535cc180129d6ff174 ./other_pkgs/jade_1.2.1-47_amd64.deb
36a2ea65e4f745cd19c6282e2c2102b9 ./other_pkgs/python-psycopg2_2.2.1-1_amd64.deb
5ef4fd02d9b74463cfd9e421c9df09a2 ./build_record/1100_fit

FIT v. 1.8 installation files directory listing:

├── kimchi
│ ├── build_scripts
│ │ ├── 0100_fit_v2
│ ├── other_files
│ │ ├── configparser-3.3.0r2.zip
│ │ ├── node-0.10.29-release.zip
│ │ ├── NTL-ISS-Food-Intake-Tracker-master.zip
│ │ ├── ordereddict-1.1.zip
│ │ ├── postgres_uuid-ossp_extn_squeeze-amd64.tar.gz
│ │ ├── python-crontab-1.6.0.zip
│ │ ├── setuptools-0.6c11.zip
│ │ ├── uuid-1.6.2.zip
│ │ ├── uuid-ossp.so
│ │ └── uuid-ossp.zip
│ └── other_pkgs
│ ├── jade_1.2.1-47_amd64.deb
│ ├── libossp-uuid16_1.6.2-1_amd64.deb
│ ├── libossp-uuid-dev_1.6.2-1_amd64.deb
│ ├── libsp1c2_1.3.4-1.2.1-47_amd64.deb
│ ├── python-egenix-mxdatetime_3.1.3-4_amd64.deb
│ ├── python-egenix-mxtools_3.1.3-4_amd64.deb
│ └── python-psycopg2_2.2.1-1_amd64.deb

418d58d280fdffd189b8bbe1b4eee76e ./kimchi/build_scripts/0100_fit_v2
04fe9470fc04ab931a1cca56a9da3fc7 ./kimchi/other_files/configparser-3.3.0r2.zip
936ba66d3c0c92d12b5616ea6c2be4f9 ./kimchi/other_files/node-0.10.29-release.zip
5c01a2c1965ad5fd1c01ad87fdbb1d82 ./kimchi/other_files/NTL-ISS-Food-Intake-Tracker-master.zip
cf3bfc54176b9468f7872720db95b859 ./kimchi/other_files/ordereddict-1.1.zip
37d82ab3c5fe32b18a760733ce6125f2 ./kimchi/other_files/postgres_uuid-ossp_extn_squeeze-amd64.tar.gz
3b9cfa733a763af8e6e826ad458673f2 ./kimchi/other_files/python-crontab-1.6.0.zip
b2e133838129f1609af5ba091a36be7f ./kimchi/other_files/setuptools-0.6c11.zip
aa3989366b2d933d1ec70672fbbf8d5e ./kimchi/other_files/uuid-1.6.2.zip
fa8a312b4ec954bff1ac148b6944704e ./kimchi/other_files/uuid-ossp.so
70a327b221aeb3bcbd666664fada48f9 ./kimchi/other_files/uuid-ossp.zip
8f8b757375bf80535cc180129d6ff174 ./kimchi/other_pkgs/jade_1.2.1-47_amd64.deb
5ab28dbd3f0c76379c76187d7c463764 ./kimchi/other_pkgs/libossp-uuid-dev_1.6.2-1_amd64.deb
3dc3bb98eeb211bee185d6c1fe92895a ./kimchi/other_pkgs/libossp-uuid16_1.6.2-1_amd64.deb
1f09d4832ecacf1e59e72c9bfcea6f1f ./kimchi/other_pkgs/libsp1c2_1.3.4-1.2.1-47_amd64.deb
29a7afe6e21345ce3d79374a64bf465c ./kimchi/other_pkgs/python-egenix-mxdatetime_3.1.3-4_amd64.deb
59dba4142639bc5cad8b06f642648757 ./kimchi/other_pkgs/python-egenix-mxtools_3.1.3-4_amd64.deb
36a2ea65e4f745cd19c6282e2c2102b9 ./kimchi/other_pkgs/python-psycopg2_2.2.1-1_amd64.deb

Also there is the 0100_fit_v1 script. Per the direction I was told, this was not installed yesterday. It's MD5 checksum is 5c83a236277be7396bed0a444d3ee213. There was a previous installation attempt of Fajita + V1 + V2. I look for the error report from that attempt and post the info.

@briandmartin
Copy link

Here's the Fajita SP + V1 + V2 errors from the failed attempt on 05/26/2017:

Copy the required installation order of Fajita FIT, 0100_fit_v1 and then 0100_fitv2. I tested that install order and the 0100_fit_v1 script is failing on the copy command in this series of three UUID OSSP install commands:
# Install UUID postgresql extension
unzip -o ${OTHER_DIR}/uuid-ossp.zip -d /usr/lib/postgresql/9.2/share/extension

cp -fv ${OTHER_DIR}/uuid-ossp.so /usr/lib/postgresql/9.2/lib
chmod a+r /usr/lib/postgresql/9.2/lib/uuid-ossp.so

The error is:
Archive: /admin/pluto/sscadmin/fit/jalapeno/other_files/uuid-ossp.zip
inflating: /usr/lib/postgresql/9.2/share/extension/uuid-ossp--1.0.sql
inflating: /usr/lib/postgresql/9.2/share/extension/uuid-ossp--unpackaged--1.0.sql
inflating: /usr/lib/postgresql/9.2/share/extension/uuid-ossp.control
/admin/pluto/sscadmin/fit/jalapeno/other_files/uuid-ossp.so' -> /usr/lib/postgresql/9.2/lib/uuid-ossp.so'
Error: cluster configuration already exists
[ /admin/pluto/sscadmin/fit/jalapeno/build_scripts ]:
root@ls1>#

@pizza-and-dog
Copy link
Contributor

@coderReview please review and provide feedback.

@coderReview
Copy link
Contributor

@rsial2 the error is at line 202 of the 0100_fit_v1 script. It looks like the database has already been created before the script is run. I created V1 for the enchilada image. And used the same image for the V2. Is it possible to share this new image with me?

@pizza-and-dog
Copy link
Contributor

@cflottorp & @briandmartin please confirm.

@cflottorp
Copy link
Author

Brain - is it possible to send a copy of image of via LFT?

@briandmartin
Copy link

Sending a newer LS1 load presents hurdles with export compliance issues. Also, the issue will arise again with future FIT updates. Because of these reasons we are going to integrate FIT on to its own VM that will be run on one of our On-orbit hypervisors.

The VM will run the 64-bit version of Debian Jessie. Though the VM will be only for FIT, it will still be built in a secure manner as far as permissions go and only opening up firewall rules that are necessary. We'll begin building the base VM now, send it to you when it's ready so you can install FIT and update your installation script. We'll be using the stable version of Debian packages available in the Jessie repo. I don't have all the version numbers yet since we're just starting the VM build now but the Jessie version is 8.9 and the Postgresql version is 9.4+165+deb8u2. For Node.JS we'll want to use a stable version compatible with Jessie (I don't know if there's a Debian package for that).

For LS1, the SSC team will build a deactivation script for LS1 that stops FIT and points DNS to the new FIT VM.

The time estimate is a couple of weeks for providing the FIT team with the baseline FIT VM.

@pizza-and-dog
Copy link
Contributor

Sounds good, thanks.

@coderReview
Copy link
Contributor

coderReview commented Jul 25, 2017

@rsial2 can we create a new VM in Amazon? I can start finding out what packages should we use and migrate the current web system to this new VM.

@pizza-and-dog
Copy link
Contributor

I think we should wait for @briandmartin to get the VM setup first.

@briandmartin
Copy link

@RSIAL @codereview OK thank you for your flexibility. I'll give you an update when we have the VM ready for your use and the Export Compliance is approved.

@pizza-and-dog
Copy link
Contributor

sounds good, thanks @briandmartin

@briandmartin
Copy link

@coderReview I sent you the initial FIT VM and documentation today so you can use it for updating the FIT application/installer. There should be an email inviting you to join the NASA NOMAD large file transfer service (sent to the email address pvmagacho@gmail.com). Then a second link to download the VM and documentation.

@pizza-and-dog
Copy link
Contributor

@briandmartin, confirming receipt of the VM. We will start working on the updates shortly.

@coderReview
Copy link
Contributor

@rsial2 downloaded the data. Started to work creating the new scripts

@pizza-and-dog
Copy link
Contributor

Awesome, thanks Paulo.

@coderReview
Copy link
Contributor

@briandmartin can you give me a hand on how to connect using ssh to the VM? It's seems the ssh configuration is not default one.

@briandmartin
Copy link

The primary IP address of the FIT VM is 192.168.65.102. Assign a NIC on your dev machine with an IP address on the 192.168.65.x subnet. Any number for x will work fine except for 102 since that address is in use by the FIT VM. Then you should have SSH access to the FIT VM and can SSH and login to the sscadmin account. Let me know if that does not work and then we can troubleshoot the issue.

@coderReview
Copy link
Contributor

@briandmartin had to run dpkg-reconfigure openssh-server to get the servers keys back.

@briandmartin
Copy link

OK great. Let me know if you have more questions.

@coderReview
Copy link
Contributor

@briandmartin have you changed in the new VM the authentication method from SASL to LDAP?

@coderReview
Copy link
Contributor

@rsial2 I have the script ready but I had to remove the site authentication because it's not using SASL anymore.

@pizza-and-dog
Copy link
Contributor

@briandmartin can you confirm if that's ok? Once you do, we'll send the script over.

@pizza-and-dog
Copy link
Contributor

Hi @briandmartin and @cflottorp, the new script is ready. The following MUST be done manually before running the script:

  1. Setup pl_fit account
  2. Setup certificates
  3. Setup firewall
  4. Setup DNS

The site authentication has been removed because SASL is not being used as authentication method anymore. To download the new script, please click here: https://drive.google.com/file/d/0B3nmbMgH4_oxOXdFNTdEUlJCMjQ/view?usp=sharing

@briandmartin
Copy link

briandmartin commented Sep 18, 2017

have you changed in the new VM the authentication method from SASL to LDAP?

No, I think LDAP is only scarred in for future use. For now, we still want to use Apache basic auth over SASL. I'll double check LDAP is still planned for future use. @coderReview

@briandmartin
Copy link

@rsial2

To download the new script, please click ...

Copy thanks. I've downloaded the ZIP file from Google Drive and will evaluate the install on the FIT VM.

@pizza-and-dog
Copy link
Contributor

Hi @briandmartin & @cflottorp, how did this go?

@briandmartin
Copy link

@RSIAL we have not started the evaluation due to development on other projects. We estimate early the week of Oct 16th we'll be able to evaluate the latest FIT delivery.

@pizza-and-dog
Copy link
Contributor

Hey @briandmartin and @cflottorp, just checking in on this. Please let us know. CC @nasarader and @szwart

@briandmartin
Copy link

@rsial2 and @cflottorp, we expect to do the load developer evaluation tomorrow, Wed 10/25. Getting the Kimchi Delta Service Pack ready for CM has been holding this activity up but we expect to complete that CM activity today to be freed up to do this evaluation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants