Developing with PyCharm Professional and the pre built virtual machine

Stephanie Mills edited this page May 29, 2014 · 30 revisions

This document is part of the Hydroshare Developers' Guide


Table of Contents

1. Set up Virtual Machine

1.1 Download the virtual machine

Get the virtual machine.

1.2 Add the machine to VirtualBox

The virtual machine file available at the link above may change, and the snapshot history and datestamp may change if there are updates to the machine. This is not important and will not affect you loading the machine into VirtualBox. Select the VirtualBox image similar to hydroshare_docker_vm_###### and add it to VirtualBox, as pictured below:

Image 1:

add machine1

Start the hydroshare_docker VM in VirtualBox, as pictured below:

Image 2:

add machine2

1.3 Connect to the vagrant host

The VM should now be running in a terminal on your desktop. You can safely minimize the VM, as you will no longer need to connect to it directly. Instead, open up a new terminal and SSH to it:

localhost ~$ ssh vagrant@localhost –p 2222
localhost ~$ password vagrant

You should now be faced with a prompt like this, pictured in img. 3:

vagrant@precise64:~$ docker ps

Image 3:

connect to host

From now on, you will need to pay careful attention to the characters before the $. This will show you which virtual machine you should be entering commands on. For example, in the above image, the information you are looking for is the text vagrant@precise64. Working with multiple virtual machines on a single desktop can be confusing, so please check and double check this information in every step.

1.4 Start hydroshare-dev docker container

In this step, you will check containers and start webserver in hydroshare-dev docker container.

The redis, postgres and hydroshare containers should be already be running in the VM. However, the webserver in the hydroshare-dev container needs to be started.

To verify that all is running as expected:

vagrant@precise64:~$ docker ps

e275e0d724c0 	/bin/bash		12 days ago	Up 6 days,	hydroshare-dev
543ed21ddd84		/		12 days ago	Up 6 days	5432/tcp				hydroshare-dev/postgis,hydroshare-postgis
be966c197c48		/bin/sh -c /usr/bin	12 days ago	Up 6 days 	6379/tcp				hydroshare-dev/redis,hydroshare-redis

This should be (roughly) what you see when you enter docker ps. The important bits are under the IMAGE header and the NAMES header. Those should be exactly as you see above.

1.5 Finish starting hydroshare-dev container

Now that you've verified that everything is running, you will want to attach to the hydroshare container and get the webserver going.

vagrant@precise64:~$ docker attach hydroshare-dev     # this is like Docker's own form of screen or SSH, you will be connected to a terminal
(hit return again, as docker will appear to hang now)

root@275e0d724c0:~$ cd /home/docker/hydroshare        # NOTE the new prompt!  This means you're in the hydroshare-dev container.
root@275e0d724c0:~$ env | grep _ >> /etc/environment  # This takes the current environment and makes it exist for later SSH connections to the machine.  Do this.  You will need it later.
root@275e0d724c0:~$ ./init                            # This starts the webserver and synchs the database

You should see some console messages, ending with:

*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 225)
spawned uWSGI worker 1 (...)
spawned uWSGI worker 2 (...)
spawned uWSGI worker 3 (...)
spawned uWSGI worker 4 (...)

If you see this, then the webserver is now working. It is bound to your local host machine at :8001.

In a browser on your own desktop (not the VM or anywhere else), you should now surf to http://localhost:8001.

1.6 Change the site record

  • Login to the admin interface on http://localhost:8001/admin (username: hs password: water)
  • Click "sites" in the header
  • Click the first record
  • Edit it to be localhost:8001
  • Click "save"

1.7 Test your local machine's connection to hydroshare-dev

This is on your own local host. Open a new, clean terminal and ssh to the container with this command:

localhost ~$ ssh docker@localhost -p 1338

The password is docker. If this doesn't work, email, as PyCharm depends on an SSH connection.

1.8 Get a local copy of hydroshare2 from GitHub

Do not use the PyCharm Tools to pull from GitHub as these do not currently support submodules, which Hydroshare makes heavy use of. Use the command line or SourceTree instead.

Using command line on your local host (not the VM, not the Hydroshare container). Open a fresh terminal, cd to the directory where you want your source tree to live, and:

localhost ~$ git clone hydroshare
localhost ~$ git submodule init
localhost ~$ git submodule update

1.9 Using SourceTree (this section is untested):

Image 4:

sourcetree screencap

First, add submodules:

Image 5:

sourcetree submodules

Then, open each submodule:

Image 6:

open each submodule

Final state (will change as new modules are added):

Image 7:

sourcetree final state

2. Start Developing with PyCharm

2.1 Configure Source Control

In the File menu, click Open directory and select the hydroshare repository you just cloned in the section 1.8 Get a local copy of hydroshare2 from GitHub. In the example pictured in img. 8 it is hydroshare_pycharm, but for you it is just where-ever you cloned hydroshare2.

Image 8:

open directory

2.2 Configure Git project roots

At this point, PyCharm knows where your code is, but it doesn't know about GitHub. To correct that, you will want to register your Git roots with PyCharm. Open the "Settings" menu. It may be called "Preferences" on a Mac. If the screen that comes up doesn't look like the img. 9 image below when you open it, click the "Version Control" item from the menu on the left hand and click "add root" on all the red boxes. This process is pictured in img. 10.

Image 9:

configure roots

Image 10:

configure roots

2.3 Configure Deployment

This section helps you configure PyCharm so that the code you develop in the IDE gets deployed to the container running on your virtual machine when you want to test it. All the steps in this subsection (steps a through e) require that you are in the "Settings" (or "Preferences" on some systems) dialog.

(a.) Open the Settings dialog. Pick the item Project > Deployment > Options, and follow the instructions below:

  • Check “Create empty directories”
  • Check “Delete target items when source ones do not exist”
  • Switch “Upload changed files automatically to the default server” to “Always” (pictured in img. 11)
  • Click "Apply"

Image 11:

general settings

(b.) Add a server for Deployment:

  • Pick the item Project > Deployment.
  • Click the + sign to add a new Deployment target. The dialog pictures in img. 12 below will show up.
  • Give it a name (the one we used is appropriate.
  • Pick the SFTP protocol if it is not already picked.
  • Click OK.

Image 12:

server for deployment

(c.) Edit the connection settings:

  • Click the Connection tab in the dialog.
  • Make sure that the settings look exactly like they do in img. 13 below (the password should be "docker"):
  • Click "Test SFTP Connection" to make sure that settings work. If they do not, send a support request to hydroshare-dev
  • Click "Apply"

Image 13:

completed connection settings

(d.) Setup the deployment mapping:

  • Pick the item Project > Deployment.
  • Click the Mappings tab in the dialog.
  • Set the local path to the path were you cloned Hydroshare2
  • Set the deployment path on the server to /hydroshare
  • Set the web path on the server to /
  • Click "Apply"

The results should look like this:

Image 14:

completed mapping setting

(e.) Enable Django support.

  • Pick the item Django Support
  • Enable Django support by checking on the box.

2.4 Configure remote interpreter

This section helps you configure PyCharm so that you can run and test code in an interpreter and so that PyCharm can syntax check and perform autocompletion using the code and libraries in the container. All the steps in this subsection require that you are in the Settings (or Preferences on some systems) dialog.

  • Pick the item Project Interpreter
  • Click the + item to add a new interpreter (see img. 15)
  • Configure it as a "remote interpreter" (see img. 16)
  • Click "fill from deployment server settings" (see img. 17)
  • Click the name of the deployment server you setup
  • Click OK (see img. 18)

Image 15:

project interpreter

Image 16:

project interpreter

Image 17:

project interpreter

Image 18:

project interpreter

2.5 Push hydroshare to your container

This section assumes you have closed the "Settings" dialog and are looking at the Hydroshare project in the main IDE window.

Sync: Right click on hydroshare project in the sidebar and select "Synchronize Hydroshare" (this performs a two way synchronization). To perform a one-way synchronization and replace everything on the Hydroshare container, select "Upload to localhost_1338"

3. Common Tasks

Explore the Tools and VCS menus. The Tools menu contains a number of entries related to Django and Python commands. The VCS menu, specifically the Git submenu contains a lot of the commands needed to interact with GitHub and commit your code.

3.1 Using sync-migrations

Sync-migrations will simplify using migrations in this workflow. It will will create migrations on the server side, apply them to your database, and then copy them back into your Hydroshare project tree at the appropriate place. You should run the following script anytime you edit to add or remove a database field:

~/hydroshare $ ./sync-migrations (app-name)

Where app-name is the name of the Hydroshare sub-project (hs_*) whose has changed. When you run this script, the migrations will get copied back to your app's migrations directory and you will need to add them in git yourself.

Also, when you pull the latest versions of hs_core and hydroshare2, please run sync-migrations, as the resource type has been updated to support URL shortening, and all AbstractResource subclasses will have a new field added to them via migration.

4. Customizing Hydroshare with an Extension

Please see the "Working on your own Hydroshare extension" section of the Customizing Hydroshare document for instructions on how to add a module.

5. Troubleshooting and References

5.1 Errors

If you find yourself accidentally in a detached HEAD state, open a terminal window and type this:

git checkout -b fixit1 
git checkout master
git pull origin master
git merge fixit1
git branch -d fixit1

Essentially, by entering these commands, you are saving your changes make while in a detached HEAD state to a temporary branch ("fixit1"), merging them with the master branch, and then deleting your temporary branch. You will need to complete this process in all branches that are in this state.

You can read more about detached heads in this git-notes article.

If you get a 404 or 502 error from loading http://localhost:8001

  • Switch to the terminal where you are attached to the docker container
  • Hit ctrl-c to kill uwsgi
  • Type uwsgi --ini uwsgi.ini
  • If you still have a problem, type ./init

If this still results in a problem, type:

localhost ~$ git checkout .
localhost ~$ git submodule update
localhost ~$ ./init

5.2 Passwords

localhost ~$ ssh vagrant@localhost: