add setup scripts. Fixes #1467 #1627

psinghal20 · 2018-01-21T23:34:23Z

Fixes #1467. This adds the setup scripts for the repo. The main setup.py looks for the operating system of the user and executes the corresponding script. Right now, the script supports Debian based systems, OSX and Windows systems.

Add setup directory having the setup files for the corresponding operating system

ragesoss · 2018-01-22T21:02:37Z

Why is the main script in python?

psinghal20 · 2018-01-22T22:00:16Z

@ragesoss I tried to create a single script which could detect the operating system of the user and run the appropriate script/batch file. I thought python was decent option for that as it is widely used. I am not sure if it is the right way but I thought it could work.

ragesoss · 2018-01-23T20:46:46Z

setup.py

+                   shell=True, check=True)
+
+if platform.platform().lower().find('ubuntu') != -1 \
+        or platform.platform.lower().find('debian') != -1:


Should be platform()

ragesoss · 2018-01-23T20:54:54Z

setup/deb-setup.sh

+echo 'Setting up your developmental environment. This may take a while.'
+
+echo '[*] Adding keys...'
+curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -


curl may be absent. If so, it should exit at this point with instructions to install curl.

Should it exit the script or try to install curl also?

Either way would be okay.

ragesoss · 2018-01-23T21:00:13Z

setup/deb-setup.sh

+gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3 7D2BAF1CF37B13E2069D6956105BD0E739499BDB && echo '[*] Installing rvm...' && \curl -sSL https://get.rvm.io | bash -s stable
+
+echo '[*] Installing Ruby-2.5.0...'
+sudo rvm install ruby-2.5.0


Ruby 2.5.0 will not yet be active on a fresh install. When RVM installer exits, it provides a necessary step in its output, along the lines of source /home/sage/.rvm/scripts/rvm.

Only once you do that (or log out and log in again) will rvm be active, and then you can install the required ruby version.

ragesoss · 2018-01-23T21:01:33Z

setup/deb-setup.sh

+echo '[*] Installing GNUpg...'
+sudo apt-get install gnupg
+
+echo '[*] Adding Keys for rvm...'


This command both gets keys and installs RVM.

It also overwrites any previous installation and reinstalls it if already present. It should skip installation if rvm is already installed.

ragesoss · 2018-01-23T21:03:50Z

setup/deb-setup.sh

+gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3 7D2BAF1CF37B13E2069D6956105BD0E739499BDB && echo '[*] Installing rvm...' && \curl -sSL https://get.rvm.io | bash -s stable
+
+echo '[*] Installing Ruby-2.5.0...'
+sudo rvm install ruby-2.5.0


RVM is not installed with sudo, so this command should not be sudo either.

ragesoss · 2018-01-23T21:11:56Z

setup/deb-setup.sh

+cp config/database.example.yml config/database.yml
+
+echo '[*] Installing Mysql-server...'
+sudo apt-get install mysql-server


On recent debian, at least, this needs to be mariadb-server or default-mysql-server.

ragesoss · 2018-01-23T21:14:14Z

setup/deb-setup.sh

+gem install bundler
+
+echo '[*] Installing Gems and dependencies...'
+sudo apt-get install libmysqlclient-dev libpq-dev libqtwebkit-dev


I'm not sure any of these are actually necessary any more.

Strike that... libmysqlclient-dev is needed to build the mysql2 gem, but it actually is now provided by libmariadbclient-dev on debian.

I think they are needed when used with MySQL. At least I needed them when I was setting up my developmental environment.

ragesoss · 2018-01-23T21:19:39Z

setup/deb-setup.sh

+sudo apt-get install pandoc
+
+echo '[*] Creating configuration files...'
+cp config/application.example.yml config/application.yml


These two cp steps should only be done if the relevant files already present already. Otherwise, they'll overwrite what is there.

ragesoss · 2018-01-23T21:22:50Z

setup/deb-setup.sh

+
+echo '[*] Creating configuration files...'
+cp config/application.example.yml config/application.yml
+cp config/database.example.yml config/database.yml


I think what we want to do here is to change the example database.yml file to use a different user instead of root, and then create that user in mysql with the password specified in the file and grant ownership to that user for the new databases.

Otherwise, this will break at the rake step unless the root mysql password has been set to the same one in database.yml.

psinghal20 · 2018-01-23T21:33:31Z

Thanks for the review. I will look through all the changes carefully.

ragesoss · 2018-01-23T21:55:49Z

setup/deb-setup.sh

+bundle install
+
+echo '[*] Installing yarn...'
+sudo apt-get install yarn


This doesn't work without first adding a repo that has (the right) yarn. On debian, there may be a default package, cmdtest, which provides yarn and conflicts with the yarn deb repo.

I shifted the command for adding the repo for yarn on the top so that for all the keys required we need to call apt-get update only once.

ragesoss · 2018-01-23T22:15:24Z

This is a promising start! I'll merge it once I can run it on a clean debian system, and once we get reports of it working successfully on windows and OSX. (I might be able to test those in a VM once the debian version works.)

A few ideas that would help make this easier to use, maintain, and troubleshoot:

Organize the installations by stack: start with the basic system programs that need to be installed (all the things added via apt on debian), then try to install ruby and bundle install, then create the database and run the migrations, then install the frontend dependencies and build the assets.
Avoid redo-ing any steps that are already done, whenever that can be detected.
Reduce the amount of permanent output, so that the current task is always visible and only errors and warnings end up printed on the terminal at the end. This has some nice tricks to that end: https://robots.thoughtbot.com/shell-script-suggestions-for-speedy-setups
Extract common parts that don't depend on a particular system — the mysql and bundle and rake commands, at least — to their own scripts that can be called from the OS-specific scripts, so that they can be updated in just one place where possible.

This is a big job, so thanks for taking this on!

Makes the script silent ie. script doesn't print unnecessary installation output statements and prints only the working task. Errors are still logged on the shell for refrence for the user.

Add the default credentials for the user in database.example.yml and create user in debian setup script.

ragesoss · 2018-01-30T21:08:06Z

setup/deb-setup.sh

+else
+  printf "${CLEAR_LINE}Application configurations not found!\n"
+  printf '[*] Creating Application configurations...'
+  cp config/application.example.yml config/application.yml && printf "${CLEAR_LINE}Application configurations created!\n"


This must be done before db:migrate, or the migrations will fail.

…gnore.

psinghal20 · 2018-02-13T09:31:37Z

Hi @ragesoss , I had made the changes for the ruby error and exit. Please look into this and suggest any changes.

ragesoss · 2018-02-13T21:21:49Z

setup/deb-setup.sh

+# Add Keys for MariaDB
+output_line "sudo apt-get install -y software-properties-common"
+output_line "sudo apt-key adv --recv-keys --keyserver hkp://keyserver.ubuntu.com:80 0xF1656F24C74CD1D8" && \
+output_line "sudo add-apt-repository 'deb [arch=amd64,i386,ppc64el] http://sgp1.mirrors.digitalocean.com/mariadb/repo/10.2/ubuntu xenial main'"


This is the problem. We should rely on whatever package provides mysql from the system default repositories. This repo is not compatible with my debian buster VM.

ragesoss · 2018-02-14T22:21:51Z

Got through the migrations this time, but now it's got problems probably because of too-old node (4.8.4 from the debian repo) which is only available via nodejs and not node like expected.

ragesoss · 2018-02-15T16:14:34Z

The debian script now works. I think before we merge this, though, we should add some directions for use to the script... something saying 'this is a work in progress; please report your setup results and any error messages at [link to issue on github]'.

It almost certainly doesn't work on Windows or OSX yet, but we can keep the current scripts as starting points.

psinghal20 added 5 commits January 21, 2018 23:48

feat: add setup.py and debian setup script

71d2eb7

Add setup directory having the setup files for the corresponding operating system

feat: add OSX setup script

6f6db69

feat: add windows setup batch file.

21a5ed3

feat: add calls for OSX script and windows batch file

1fd52ba

Minor formatting changes in scripts

e527fdd

ragesoss reviewed Jan 23, 2018

View reviewed changes

psinghal20 added 5 commits January 25, 2018 01:23

fix typos, add reload source for rvm, add installation for curl

36058ea

makes the script silent

590ae49

Makes the script silent ie. script doesn't print unnecessary installation output statements and prints only the working task. Errors are still logged on the shell for refrence for the user.

add default mysql user in debian script.

d88a1da

Add the default credentials for the user in database.example.yml and create user in debian setup script.

organize installations by stack

c148141

apply rvm source, -y argument, clear line approach

940427f

psinghal20 force-pushed the setup-script branch from 45f476b to 940427f Compare January 30, 2018 17:59

ragesoss reviewed Jan 30, 2018

View reviewed changes

format output and reorder script

66a26e0

psinghal20 force-pushed the setup-script branch 3 times, most recently from 1edc9a9 to dfff87f Compare February 2, 2018 22:09

modify output_line function for piped command, enter log file in giti…

09e998f

…gnore.

psinghal20 force-pushed the setup-script branch from dfff87f to 09e998f Compare February 3, 2018 10:29

refactor code, update mariadb installation keys

85c7185

psinghal20 force-pushed the setup-script branch from 284fcad to 85c7185 Compare February 3, 2018 18:42

remove ruby installation, add checks for more steps

4123b2a

psinghal20 added 2 commits February 14, 2018 00:52

fix: change exit status of script when ruby is not found

011cd81

fix: grep pattern for ruby check

29201a9

ragesoss reviewed Feb 13, 2018

View reviewed changes

fix: remove dependent keys and set flag for exit on error

e70e8d6

fix: fix nodejs installation

3e87cf2

psinghal20 added 2 commits February 15, 2018 23:45

feat: add instructions and warning for setup script

fd20cd4

feat: add issue url and shift check for ruby to the top

087383e

ragesoss merged commit da6c4d4 into WikiEducationFoundation:master Feb 15, 2018

psinghal20 deleted the setup-script branch March 5, 2018 20:10

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add setup scripts. Fixes #1467 #1627

add setup scripts. Fixes #1467 #1627

psinghal20 commented Jan 21, 2018

ragesoss commented Jan 22, 2018

psinghal20 commented Jan 22, 2018 •

edited

Loading

ragesoss Jan 23, 2018

ragesoss Jan 23, 2018

psinghal20 Jan 23, 2018

ragesoss Jan 23, 2018

ragesoss Jan 23, 2018

ragesoss Jan 23, 2018

ragesoss Jan 23, 2018

ragesoss Jan 23, 2018

ragesoss Jan 23, 2018

ragesoss Jan 23, 2018

psinghal20 Jan 23, 2018

ragesoss Jan 23, 2018 •

edited

Loading

ragesoss Jan 23, 2018

psinghal20 commented Jan 23, 2018

ragesoss Jan 23, 2018

psinghal20 Jan 24, 2018

ragesoss commented Jan 23, 2018

ragesoss Jan 30, 2018

psinghal20 commented Feb 13, 2018

ragesoss Feb 13, 2018

ragesoss commented Feb 14, 2018

ragesoss commented Feb 15, 2018

add setup scripts. Fixes #1467 #1627

add setup scripts. Fixes #1467 #1627

Conversation

psinghal20 commented Jan 21, 2018

ragesoss commented Jan 22, 2018

psinghal20 commented Jan 22, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ragesoss Jan 23, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

psinghal20 commented Jan 23, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ragesoss commented Jan 23, 2018

Choose a reason for hiding this comment

psinghal20 commented Feb 13, 2018

Choose a reason for hiding this comment

ragesoss commented Feb 14, 2018

ragesoss commented Feb 15, 2018

psinghal20 commented Jan 22, 2018 •

edited

Loading

ragesoss Jan 23, 2018 •

edited

Loading