Skip to content

Commit

Permalink
Make writable directory location configurable, closes #959
Browse files Browse the repository at this point in the history
* Created new FileDataManager class, moved data directory-specific Utils functions into it
* When config.inc.php datadir_path value doesn't exist, default to source_root_path/data/
* Avoid "Can't write to file" data folder permissions problems during export and backup
* Gracefully handle missing config file during installation, and missing datadir_path value in existing config
* Automatically create compiled_view folder if it doesn't already exist
* Expand and clarify documentation
  • Loading branch information
ginatrapani committed Jan 30, 2012
1 parent 7c2c77d commit b56dcab
Show file tree
Hide file tree
Showing 40 changed files with 321 additions and 171 deletions.
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
webapp/config.inc.php
webapp/config.inc.bak.php
webapp/data/*
webapp/data/logs/*
data/
tests/config.tests.inc.php
.*
extras/cron/config
Expand Down
1 change: 0 additions & 1 deletion data/logs/archive/README

This file was deleted.

1 change: 0 additions & 1 deletion data/plugins/README.md

This file was deleted.

3 changes: 2 additions & 1 deletion docs/source/install/backup.rst
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,8 @@ the "Backup ThinkUp's database" link. On the Backup page, click on the "Backup N
The web based backup tool has two permissions requirements.

1. Your ThinkUp installation's database user must have "GRANT FILE ON" permissions
2. The MySQL user must have write permissions to the ``compiled_view`` directory under the data directory defined in $THINKUP_CFG['datadir_path'].
2. The MySQL user must have write permissions to the data directory (``data`` by default, or defined in
``config.inc.php``'s ``$THINKUP_CFG['datadir_path']`` value).

If you don't have those permissions, you can use `mysqldump` or a tool like phpMyAdmin to back up your database
manually.
Expand Down
21 changes: 16 additions & 5 deletions docs/source/install/perms.rst
Original file line number Diff line number Diff line change
@@ -1,20 +1,31 @@
Understanding ThinkUp's Folder Permissions
==========================================

In order for ThinkUp to run, it must be able to write caches, template compilations, and sometimes plugin data
to a specific data directory defined in $THINKUP_CFG['datadir_path']. By default this is ``/your_path_to_thinkup/webapp/data``.
In order to run, ThinkUp must be able to write files to a specific data directory defined in the
``config.inc.php`` file as ``$THINKUP_CFG['datadir_path']``. By default the data directory is located in the root of
the thinkup web-accessible directory and is named ``data``.

The recommended and most secure way to grant ThinkUp write access to these folders is to change the owner of this
The recommended and most secure way to grant ThinkUp write access to this folder is to change the owner of this
folder to the web server user. The command for doing this is:

``chown -R apache your_datadir_path``

Where your_datadir_path is your installation path and apache is the name of the web server user.
Where your_datadir_path is your installation's data directory path and apache is the name of the web server user.
(Note that this username could vary depending on your server.)

If you are unable to change owner (chown) the folder, a less secure but just as effective method is to make the folder
writable by the world. To do that, you can run this command:

``chmod -R 777 your_datadir_path``

If possible, change the folder's owner to the web server user or group instead of setting its permissions to world-writable.
If possible, change the folder's owner to the web server user or group instead of setting its permissions to
world-writable.

Change the Location of ThinkUp's Data Directory
-----------------------------------------------

For security-related or other reasons, you may not want ThinkUp's data directory to live in a web-accessible folder.
To change the location ThinkUp's data directory, open the ``config.inc.php`` in any text editor and set your
desired location in the ``$THINKUP_CFG['datadir_path']`` value. Note that ThinkUp's data directory must be writable
by the web server and MySQL user for ThinkUp to function. Grant ThinkUp those access permissions using the
instructions above.
4 changes: 4 additions & 0 deletions docs/source/install/security.rst
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,10 @@ ThinkUp database, not any others.

**Make sure no ThinkUp files are writable** except the ones :doc:`required by the application </install/perms>`.

**Move ThinkUp's data directory.** By default ThinkUp's writeable data directory is located in a web-accessible
folder. Move that folder to a more secure location :doc:`by setting its path in ThinkUp's config file.</install/perms>`


**Use strong, unique passwords** for your ThinkUp user account as well as all your social network accounts.


Expand Down
6 changes: 3 additions & 3 deletions docs/source/troubleshoot/common/backupcannotwrite.rst
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,11 @@ or
or

``PDOException: SQLSTATE[HY000]: General error: 1 Can't create/write to file
'<thinkup>/webapp/data/compiled_view/tu_encoded_locations.txt' (Errcode: 13)``
'<thinkup>/data/backup/tu_encoded_locations.txt' (Errcode: 13)``

ThinkUp's backup or export tool doesn't have the permissions it needs to back up your files. Make sure that the MySQL
user has GRANT FILE and LOCK TABLE privileges in the database as well as write privileges to the compiled_view
directory.
user has GRANT FILE and LOCK TABLE privileges in the database as well as write privileges to :doc:`ThinkUp's data
directory </install/perms>`.

`Find out more about MySQL GRANT permission <http://dev.mysql.com/doc/refman/5.1/en/grant.html>`_.

Expand Down
6 changes: 3 additions & 3 deletions docs/source/troubleshoot/messages/mysqlfile.rst
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
The MySQL user does not have the proper file permissions to backup/export data
==============================================================================

The Thinkup MySQL user does not have the proper file permissions to backup or export data. Make Sure the MySQL
server has write privileges to the ThinkUp's ``compiled_view`` directory under the data directory
defined in $THINKUP_CFG['datadir_path'].
ThinkUp's MySQL user does not have the proper file permissions to back up or export data. Make sure the MySQL
user has write privileges to ThinkUp's ``backup`` folder under the data directory
defined in ``config.inc.php``'s ``$THINKUP_CFG['datadir_path']`` value.
5 changes: 3 additions & 2 deletions docs/source/troubleshoot/messages/upgrading.rst
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,9 @@ A message with a link to complete the upgrade process should be in your inbox.

If your ThinkUp server's email function doesn't work, do the following:

Open the ``.htupgrade_token`` in your compiled_view directory under the data directory specified in $THINKUP_CFG['datadir_path']
and copy the upgrade token to your clipboard.
Open the ``.htupgrade_token`` file in ThinkUp's data directory and copy its contents to your clipboard. By default,
the data directory is called ``data`` and is located in ThinkUp's root folder; if not it is specified in ThinkUp's
config.inc.php file as the value of $THINKUP_CFG['datadir_path'].

Then, enter the upgrade token into ThinkUp's form and click on the "Submit Token" button to continue the upgrade
process.
6 changes: 3 additions & 3 deletions tests/TestOfBackupController.php
Original file line number Diff line number Diff line change
Expand Up @@ -39,9 +39,9 @@ public function setUp() {
new BackupMySQLDAO();
$this->config = Config::getInstance();
$this->pdo = BackupMySQLDAO::$PDO;
$this->backup_file = Utils::getDataPath('.htthinkup_db_backup.zip');
$this->backup_test = Utils::getDataPath('thinkup_db_backup_test.zip');
$this->backup_dir = Utils::getBackupPath() . '/';
$this->backup_file = FileDataManager::getDataPath('.htthinkup_db_backup.zip');
$this->backup_test = FileDataManager::getDataPath('thinkup_db_backup_test.zip');
$this->backup_dir = FileDataManager::getBackupPath() . '/';
}

public function tearDown() {
Expand Down
4 changes: 2 additions & 2 deletions tests/TestOfBackupMySQLDAO.php
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,8 @@ public function setUp() {

public function tearDown() {
parent::tearDown();
$zipfile = Utils::getBackupPath('.htthinkup_db_backup.zip');
$backup_dir = Utils::getBackupPath();
$zipfile = FileDataManager::getBackupPath('.htthinkup_db_backup.zip');
$backup_dir = FileDataManager::getBackupPath();
if (file_exists($zipfile)) {
unlink($zipfile);
}
Expand Down
42 changes: 22 additions & 20 deletions tests/TestOfExportMySQLDAO.php
Original file line number Diff line number Diff line change
Expand Up @@ -88,8 +88,8 @@ protected function buildData() {
$source = 'web';
}
$builders[] = FixtureBuilder::build('posts', array('post_id'=>$counter, 'author_user_id'=>13,
'author_username'=>'ev', 'author_fullname'=>'Ev Williams', 'author_avatar'=>'avatar.jpg',
'post_text'=>'This is post '.$counter, 'source'=>$source, 'pub_date'=>'-1h',
'author_username'=>'ev', 'author_fullname'=>'Ev Williams', 'author_avatar'=>'avatar.jpg',
'post_text'=>'This is post '.$counter, 'source'=>$source, 'pub_date'=>'-1h',
'reply_count_cache'=>rand(0, 4), 'is_protected'=>0,
'retweet_count_cache'=>floor($counter/2), 'network'=>'twitter',
'old_retweet_count_cache' => floor($counter/3), 'in_rt_of_user_id' => null,
Expand All @@ -99,43 +99,43 @@ protected function buildData() {

//add 2 replies to post 2
$builders[] = FixtureBuilder::build('posts', array('post_id'=>41, 'author_user_id'=>10,
'author_username'=>'ev_replies', 'author_fullname'=>'Ev Williams\' replier', 'author_avatar'=>'avatar.jpg',
'post_text'=>'@ev This is a reply to post 2', 'source'=>'web', 'pub_date'=>'-10m',
'author_username'=>'ev_replies', 'author_fullname'=>'Ev Williams\' replier', 'author_avatar'=>'avatar.jpg',
'post_text'=>'@ev This is a reply to post 2', 'source'=>'web', 'pub_date'=>'-10m',
'reply_count_cache'=>rand(0, 4), 'is_protected'=>0,
'retweet_count_cache'=>0, 'network'=>'twitter',
'old_retweet_count_cache' => 0, 'in_rt_of_user_id' => null,
'in_reply_to_post_id'=>2, 'in_retweet_of_post_id'=>null, 'is_geo_encoded'=>0));

$builders[] = FixtureBuilder::build('posts', array('post_id'=>42, 'author_user_id'=>9,
'author_username'=>'ev_replies2', 'author_fullname'=>'Ev Williams\' replier 2', 'author_avatar'=>'avatar.jpg',
'post_text'=>'@ev This is another reply to post 2', 'source'=>'web', 'pub_date'=>'-11m',
'author_username'=>'ev_replies2', 'author_fullname'=>'Ev Williams\' replier 2', 'author_avatar'=>'avatar.jpg',
'post_text'=>'@ev This is another reply to post 2', 'source'=>'web', 'pub_date'=>'-11m',
'reply_count_cache'=>rand(0, 4), 'is_protected'=>0,
'retweet_count_cache'=>0, 'network'=>'twitter',
'old_retweet_count_cache' => 0, 'in_rt_of_user_id' => null,
'in_reply_to_post_id'=>2, 'in_retweet_of_post_id'=>null, 'is_geo_encoded'=>0));

//add a retweet of post 3
$builders[] = FixtureBuilder::build('posts', array('post_id'=>43, 'author_user_id'=>8,
'author_username'=>'ev_replies2', 'author_fullname'=>'Ev Williams\' retweeter', 'author_avatar'=>'avatar.jpg',
'post_text'=>'RT @ev: This is post 3', 'source'=>'web', 'pub_date'=>'-11m',
'author_username'=>'ev_replies2', 'author_fullname'=>'Ev Williams\' retweeter', 'author_avatar'=>'avatar.jpg',
'post_text'=>'RT @ev: This is post 3', 'source'=>'web', 'pub_date'=>'-11m',
'reply_count_cache'=>rand(0, 4), 'is_protected'=>0,
'retweet_count_cache'=>0, 'network'=>'twitter',
'old_retweet_count_cache' => 0, 'in_rt_of_user_id' => 13,
'in_reply_to_post_id'=>null, 'in_retweet_of_post_id'=>3, 'is_geo_encoded'=>0));

//add a mention
$builders[] = FixtureBuilder::build('posts', array('post_id'=>44, 'author_user_id'=>8,
'author_username'=>'ev_replies2', 'author_fullname'=>'Ev Williams\' retweeter', 'author_avatar'=>'avatar.jpg',
'post_text'=>'Yo, @ev is da man!', 'source'=>'web', 'pub_date'=>'-11m',
'author_username'=>'ev_replies2', 'author_fullname'=>'Ev Williams\' retweeter', 'author_avatar'=>'avatar.jpg',
'post_text'=>'Yo, @ev is da man!', 'source'=>'web', 'pub_date'=>'-11m',
'reply_count_cache'=>rand(0, 4), 'is_protected'=>0,
'retweet_count_cache'=>0, 'network'=>'twitter',
'old_retweet_count_cache' => 0, 'in_rt_of_user_id' => null,
'in_reply_to_post_id'=>null, 'in_retweet_of_post_id'=>null, 'is_geo_encoded'=>0));

//add a non-Ev post
$builders[] = FixtureBuilder::build('posts', array('post_id'=>45, 'author_user_id'=>8,
'author_username'=>'ev_replies2', 'author_fullname'=>'Ev Williams\' retweeter', 'author_avatar'=>'avatar.jpg',
'post_text'=>'Who\'s da man?', 'source'=>'web', 'pub_date'=>'-11m',
'author_username'=>'ev_replies2', 'author_fullname'=>'Ev Williams\' retweeter', 'author_avatar'=>'avatar.jpg',
'post_text'=>'Who\'s da man?', 'source'=>'web', 'pub_date'=>'-11m',
'reply_count_cache'=>rand(0, 4), 'is_protected'=>0,
'retweet_count_cache'=>0, 'network'=>'twitter',
'old_retweet_count_cache' => 0, 'in_rt_of_user_id' => null,
Expand Down Expand Up @@ -225,8 +225,8 @@ public function testExportMentionsOfServiceUser() {
public function testExportPostsServiceUserRepliedTo() {
$more_data = array();
$more_data[] = FixtureBuilder::build('posts', array('post_id'=>46, 'author_user_id'=>13,
'author_username'=>'ev', 'author_fullname'=>'Ev Williams', 'author_avatar'=>'avatar.jpg',
'post_text'=>'I am?', 'source'=>'web', 'pub_date'=>'-1h',
'author_username'=>'ev', 'author_fullname'=>'Ev Williams', 'author_avatar'=>'avatar.jpg',
'post_text'=>'I am?', 'source'=>'web', 'pub_date'=>'-1h',
'reply_count_cache'=>rand(0, 4), 'is_protected'=>0,
'retweet_count_cache'=>0, 'network'=>'twitter',
'old_retweet_count_cache' =>0, 'in_rt_of_user_id' => null,
Expand All @@ -242,7 +242,9 @@ public function testExportFavoritesOfServiceUser() {
$more_data[] = FixtureBuilder::build('favorites', array('post_id'=>44, 'fav_of_user_id'=>13,
'author_user_id'=>8, 'network'=>'twitter'));

$favorites_file = Utils::getBackupPath('favorites.tmp');
$bkdir = FileDataManager::getBackupPath();

$favorites_file = FileDataManager::getBackupPath('favorites.tmp');
if (file_exists($favorites_file)) {
unlink($favorites_file);
}
Expand All @@ -254,7 +256,7 @@ public function testExportFavoritesOfServiceUser() {
}

public function testExportGeoToFile() {
$file = Utils::getBackupPath('encoded_locations.tmp');
$file = FileDataManager::getBackupPath('encoded_locations.tmp');
if (file_exists($file)) {
unlink($file);
}
Expand All @@ -264,9 +266,9 @@ public function testExportGeoToFile() {
}

public function testExportToFile() {
$posts_table_file = Utils::getBackupPath('posts.tmp');
$links_table_file = Utils::getBackupPath('links.tmp');
$users_table_file = Utils::getBackupPath('users.tmp');
$posts_table_file = FileDataManager::getBackupPath('posts.tmp');
$links_table_file = FileDataManager::getBackupPath('links.tmp');
$users_table_file = FileDataManager::getBackupPath('users.tmp');
if (file_exists($posts_table_file)) {
unlink($posts_table_file);
}
Expand All @@ -289,7 +291,7 @@ public function testExportToFile() {

public function testGetExportFields() {
$test_table_sql = 'CREATE TABLE tu_test_table(' .
'id int(11) NOT NULL AUTO_INCREMENT PRIMARY KEY,' .
'id int(11) NOT NULL AUTO_INCREMENT PRIMARY KEY,' .
'test_name varchar(20),' .
'test_id int(11),' .
'unique key test_id_idx (test_id)' .
Expand Down
4 changes: 2 additions & 2 deletions tests/TestOfExportServiceUserDataController.php
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ public function setUp() {
new ExportMySQLDAO();
$this->config = Config::getInstance();
$this->pdo = ExportMySQLDAO::$PDO;
$this->export_test = Utils::getDataPath('thinkup_user_export_test.zip');
$this->export_test = FileDataManager::getDataPath('thinkup_user_export_test.zip');

$hashed_pass = ThinkUpTestLoginHelper::hashPasswordUsingDeprecatedMethod("secretpassword");

Expand Down Expand Up @@ -79,7 +79,7 @@ public function tearDown() {
}

private function deleteFile($file) {
$file = Utils::getBackupPath($file);
$file = FileDataManager::getBackupPath($file);
if (file_exists($file)) {
unlink($file);
}
Expand Down
93 changes: 93 additions & 0 deletions tests/TestOfFileDataManager.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
<?php
/**
*
* ThinkUp/tests/TestOfFileDataManager.php
*
* Copyright (c) 2012 Gina Trapani
*
* LICENSE:
*
* This file is part of ThinkUp (http://thinkupapp.com).
*
* ThinkUp is free software: you can redistribute it and/or modify it under the terms of the GNU General Public
* License as published by the Free Software Foundation, either version 2 of the License, or (at your option) any
* later version.
*
* ThinkUp is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied
* warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more
* details.
*
* You should have received a copy of the GNU General Public License along with ThinkUp. If not, see
* <http://www.gnu.org/licenses/>.
*
*
* @author Gina Trapani <ginatrapani[at]gmail[dot]com>
* @license http://www.gnu.org/licenses/gpl.html
* @copyright 2012 Gina Trapani, Guillaume Boudreau
*/
require_once dirname(__FILE__).'/init.tests.php';
require_once THINKUP_ROOT_PATH.'webapp/_lib/extlib/simpletest/autorun.php';
require_once THINKUP_ROOT_PATH.'webapp/config.inc.php';

class TestOfFileDataManager extends ThinkUpBasicUnitTestCase {

public function testGetDataPathNoConfigFile() {
Config::destroyInstance();
$this->removeConfigFile();

//test just path
$path = FileDataManager::getDataPath();
$this->assertEqual($path, THINKUP_WEBAPP_PATH.'data/');

//test path with file
$path = FileDataManager::getDataPath('myfile.txt');
$this->assertEqual($path, THINKUP_WEBAPP_PATH.'data/myfile.txt');
$this->restoreConfigFile();
}

public function testGetDataPathConfigExistsWithoutDataDirValue() {
Config::destroyInstance();
$this->removeConfigFile();
$cfg_values = array("table_prefix"=>"thinkupyo", "db_host"=>"myserver.com");
$config = Config::getInstance($cfg_values);

//test just path
$path = FileDataManager::getDataPath();
$this->assertEqual($path, THINKUP_WEBAPP_PATH.'data/');

//test path with file
$path = FileDataManager::getDataPath('myfile.txt');
$this->assertEqual($path, THINKUP_WEBAPP_PATH.'data/myfile.txt');
$this->restoreConfigFile();
}

public function testGetDataPathConfigExistsWithDataDirValue() {
require THINKUP_ROOT_PATH.'webapp/config.inc.php';

//if test fails here, the config file doesn't have datadir_path set
$this->assertNotNull($THINKUP_CFG['datadir_path']);

//test just path
$path = FileDataManager::getDataPath();
$this->assertEqual($path, $THINKUP_CFG['datadir_path']);

//test path with file
$path = FileDataManager::getDataPath('myfile.txt');
$this->assertEqual($path, $THINKUP_CFG['datadir_path'].'myfile.txt');
}

public function testGetBackupPath() {
require THINKUP_ROOT_PATH.'webapp/config.inc.php';

//if test fails here, the config file doesn't have datadir_path set
$this->assertNotNull($THINKUP_CFG['datadir_path']);

//test just path
$path = FileDataManager::getBackupPath();
$this->assertEqual($path, $THINKUP_CFG['datadir_path'].'backup/');

//test just path
$path = FileDataManager::getBackupPath('README.txt');
$this->assertEqual($path, $THINKUP_CFG['datadir_path'].'backup/README.txt');
}
}
2 changes: 1 addition & 1 deletion tests/TestOfMailer.php
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ public function setUp() {
public function tearDown(){
parent::tearDown();
// delete test email file if it exists
$test_email = THINKUP_WEBAPP_PATH . '_lib/view/compiled_view' . Mailer::EMAIL;
$test_email = FileDataManager::getDataPath(Mailer::EMAIL);
if (file_exists($test_email)) {
unlink($test_email);
}
Expand Down
2 changes: 1 addition & 1 deletion tests/TestOfSmartyThinkUp.php
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ public function testSmartyThinkUpDefaultValues() {
$this->assertEqual($v_mgr->template_dir[1], '/path/to/thinkup/tests/view');
$this->assertTrue(sizeof($v_mgr->plugins_dir), 2);
$this->assertEqual($v_mgr->plugins_dir[0], 'plugins');
$this->assertEqual($v_mgr->cache_dir, Utils::getDataPath('compiled_view/cache'));
$this->assertEqual($v_mgr->cache_dir, FileDataManager::getDataPath('compiled_view/cache'));
$this->assertEqual($v_mgr->cache_lifetime, $cfg->getValue('cache_lifetime'));
$this->assertTrue($v_mgr->caching);
}
Expand Down
Loading

0 comments on commit b56dcab

Please sign in to comment.