GSoc 2016: Admin console and Elasticsearch Integration (WIP)

.travis.yml

@@ -11,6 +11,7 @@ addons:
 services:
   - mysql
   - postgresql
+  - elasticsearch  
 
 env:
   global:
@@ -42,6 +43,7 @@ install:
   - luarocks install luacov
   - luarocks install luacov-coveralls
   - luarocks make rockspecs/sailor-current-1.rockspec
+  - luarocks install --server=http://luarocks.org/dev elasticsearch
 
 before_script:
   - source .travis/setdatabases.sh

docs/admin_center.md

@@ -0,0 +1,36 @@
+##Admin Center
+
+The controller and views for admin center are pre loaded with the blank-app. So, when you create a new app using `sailor create "app_name"` these files are within the new folders.
+
+###Admin center structure
+
+There is one controller named `admin.lua` and a views folder named "admin" inside which are three files. One is `dashboard.lp` which is the views file for the autogen and config editor functions. `index.lp` is for logging into the admin center. Third file is `error.lp`. 
+
+###Getting started
+First open `conf/conf.lua`. Change these two parameters:
+
+	-- the default value for this is "false"
+	enable_admin = "true"
+
+	-- the defaullt password is empty
+	admin_password = "someDifficultPassword"
+
+After running the `start-server.lua`, visit <a href="http://localhost:8080/?r=admin">here</a>. Enter the password set it in the config file in the above setup. You would be
+re-directed to `localhost:8000/?r=admin/dashboard`. 
+The autogen functions are located on the left and the fields from the config file are on the right. You can edit any of the config editor values and then press `submit` button 
+at the bottom of the screen. 
+
+###Autogen Functions
+
+The first autogen function is to Generate a model based on a table existing in the database. This is a very handy function to automatically generate the model file. 
+Enter the table name and press enter. 
+
+There would be a success message ` Model generated with success.`. 
+To create the methods for CRUD operations in the controller, use the next autogen function. 
+To Generate CRUD, enter the name of the `model` created in the previous step. This would be same as the name of the table. 
+
+###Config editor
+
+Config editor is basically a long form which when saved, writes data into the config file. The existing values can be edited and then saved at the end. The main file edited here is `conf/conf.lua`. 
+
+After making necessary changes, don't forget to logout and also disable admin in production servers.

docs/manual_ES_model.md

@@ -0,0 +1,147 @@
+##Reference Manual
+### Elastic Search Model
+
+This module makes it possible to store, retrieve and search data from <a href = "https://www.elastic.co/downloads/elasticsearch">Elasticsearch </a>. It uses the <a href = "github.com/dhavalkapil/elasticsearch-lua">elasticsearch-lua</a> client for Lua. Refer the client <a href="http://elasticsearch-lua.readthedocs.io/en/latest/">here</a> to know the parameters which can be sent through the model functions if any. Return responses are also same as the one offered by this client. 
+
+###Installation and getting started.
+
+To get started, install the elasticsearch-lua rock using
+
+	luarocks install --server=http://luarocks.org/dev elasticsearch
+
+Make sure you have elasticsearch installed and running. Refer to the instructions given on the official <a href = "https://www.elastic.co/downloads/elasticsearch">website</a>.
+
+The `es_model.lua` file is the main module. After importing "es_model" we proceed to create a new instance of the model. There should be a file inside models/ which contains the keys and types defined for the particular model. 
+
+	local es_model = require "sailor.es_model"
+	-- test.lua should be inside models/ folder.
+	local contact = es.model.new("test")
+
+Now, you can assign attributes to `contact` like 
+
+	contact.name = "test name"
+	contact.email = "test@test.com"
+
+And save it using,
+
+	msg , code = contact.save{id = 1}
+
+The `arg` represented here is named arguments which have to be passed into the function. For list of args that can be passed check the elasticsearch-lua reference.
+
+The return value for these functions is same as elasticsearch-lua. 
+Example:
+
+	local msg , err = contact.save{id = 2, routing = "test.com", timout = "2m"}
+
+If the call succeeds then `msg` contains the result table and `err` contains the status code.
+If the call fails then `msg` contains `nil` and `err` contains the error message.
+
+###Elasticsearch administration
+
+To use administrative functionalities, 
+
+	local es_model = require("sailor.db.es_model")
+	-- To use the namespaces
+	local res, err = es_model.client.cluster:stats()
+
+Check this <a href="http://elasticsearch-lua.readthedocs.io/en/latest/namespaces/">page</a> for more functions.
+
+###model.save(arg) 
+
+This function takes a list of paramters which would be same as for the elasticsearch-lua client. It saves the attributes assigned to the model if the `body` parameter is not present. Else it just saves the array passsed in the body parameter. The below example saves the `name` attribute into the ES
+
+	contact.name = "test"
+	contact.save = {id = 1}
+
+The index can be preset in the config file which can also be over written when passing the parameters.
+
+###model.getCount()
+
+This functions returns the number of documents present in the given index and type. You don't need to pass any parameter. 
+Example:
+
+	local count = contact.getCount()
+
+###model.getAll()
+
+This function returns a table of all the documents present in the given index and type. No parameter is required. It uses the `client:search()` function from elasticsearch-lua on the inside. 
+
+	local data, err = contact.getAll()
+
+###model.update(arg)
+
+This is used for updating documents already in the index. The id has to be specified.
+
+	contact.update{id = 1, body = {doc = { name = "new name" }}}
+
+###model.delete(arg)
+
+Deletes a particular document with the given id. 
+
+	contact.delete{id = 1}
+
+###model.search(arg)
+
+Search the given index and type. The arg list supports a lot of parameters. Read more about it <a href="https://dhavalkapil.com/elasticsearch-lua/docs/classes/Client.html#Client:search">here.</a> 
+
+	contact.search{q = "queryString", size = 15}
+
+###model.get(arg)
+
+Get's documents from the database based on the parameters provided. Full list of args can be found <a href="https://dhavalkapil.com/elasticsearch-lua/docs/classes/Client.html#Client:get">here.</a>
+
+	contact.get{id = 1, fields = {name, email}}
+
+###model.bulkIndex(arg)
+
+For indexing multiple documents at once. Uses `client:bulk()`. The index and type have to set in the parameters. For a list of parameters check <a href="https://dhavalkapil.com/elasticsearch-lua/docs/classes/Client.html#Client:bulk">here.</a>
+
+	contact.bulkIndex{
+  		body = {
+		    -- First action
+		    {
+		      index = {
+		        ["_index"] = "my_index1",
+		        ["_type"] = "my_type1"
+		      }
+		    },
+		    -- First body
+		    {
+		      my_key1 = "my_value1",
+		    },
+		    -- Second action
+		    {
+		      index = {
+		        ["_index"] = "my_index2",
+		        ["_type"] = "my_type2"
+		      }
+		    },
+		    -- Second body
+		    {
+		      my_key2 = "my_value2",
+	    	}
+  		}		
+	}
+
+###model.mget(arg)
+
+You can get multiple documents in the form of nested table using the `client:mget()`. The allowed parameters are <a href="https://dhavalkapil.com/elasticsearch-lua/docs/classes/Client.html#Client:mget">here.</a>
+
+	contact.mget{
+		  body = {
+		    docs = {
+		      -- First document
+		      {
+		        ["_index"] = "my_index1",
+		        ["_type"] = "my_type1",
+		        ["_id"] = "my_id1"
+		      },
+		      -- Second document
+		      {
+		        ["_index"] = "my_index2",
+		        ["_type"] = "my_type2",
+		        ["_id"] = "my_id2"
+		      }
+		    }
+		  }
+	}

docs/tutorial_testing.md

@@ -51,7 +51,7 @@ Let's suppose we have a User model and we want to unit test some of the User met
 
 The fixture is a regular Lua script that will be required. This for multiple possibilities, such as making loops for inserting many entries, as long as in the end you return a table containing samples of your model.
 
-After creating your fixture, you must add this on the bottom of your bootstrap file so they are loaded before running your tests.
+After creating your fixture, you must add this on the bottom of your bootstrap file so they are loaded before running your tests. Also don't forget to set the package.path to the sailor directory you want to test.
 
     --/tests/bootstrap.lua
     ...

rockspecs/sailor-current-1.rockspec

@@ -33,9 +33,10 @@ build = {
    modules = {
       sailor = "src/sailor.lua",
       ['sailor.access'] = "src/sailor/access.lua",
-      ['sailor.autogen'] = "src/sailor/autogen.lua",
       ['sailor.cookie'] = "src/sailor/cookie.lua",
       ['sailor.db'] = "src/sailor/db.lua",
+      ['sailor.autogen'] = "src/sailor/autogen.lua",
+      ['sailor.db.es_model'] = "src/sailor/db/es_model.lua",     
       ['sailor.db.resty_mysql'] = "src/sailor/db/resty_mysql.lua",
       ['sailor.db.luasql_common'] = "src/sailor/db/luasql_common.lua",
       ['sailor.blank-app.conf.conf'] = "src/sailor/blank-app/conf/conf.lua",

src/sailor.lua

@@ -103,17 +103,6 @@ function sailor.init(r)
     return p
 end
 
-
--- Auxiliary function to open the autogen page for models and CRUDs
--- page: our page object
-local function autogen(page)
-    local autogen = require "sailor.autogen"
-
-    local src = autogen.gen()
-    src = lp.translate(src)
-    page:render('sailor/autogen',{page=page},src)
-end
-
 -- Gets parameter from url query and made by mod rewrite and reassembles into page.GET
 -- TODO - improve
 local function apache_friendly_url(page)
@@ -174,14 +163,6 @@ function sailor.route(page)
             controller, action = match(route_name, "([^/]+)/?([^/]*)")
         end
 
-        if controller == "autogen" then 
-            if conf.sailor.enable_autogen then
-                local _,res = xpcall(function () autogen(page) end, error_handler)
-                return res or httpd.OK or page.r.status or 200
-            end
-            return error_404()
-        end
-
         local ctr
         local _, res = xpcall(function() ctr = require("controllers."..controller) end, error_handler)
         if ctr then

src/sailor/access.lua

@@ -104,4 +104,8 @@ function access.logout()
 	return session.destroy(sailor.r)
 end
 
+function access.is_loggedin(login)
+	if (session.data.login == login) then return true else return false end
+end
+
 return access

src/sailor/autogen.lua

@@ -9,50 +9,6 @@ local lfs = require "lfs"
 
 local M = {}
 
---The code for the autogen page
-function M.gen()
-	local code = [[
-<?lua
-local model =  require "sailor.model"
-local mogelgen = false
-local crudgen = false
-if next(page.POST) then
-
-	if page.POST.table_name then
-		modelgen = model.generate_model(page.POST.table_name)
-	end
-
-	if page.POST.model_name then
-		crudgen = model.generate_crud(page.POST.model_name)
-	end
-
-end
-?>
-
-<h2>Generate model</h2>
-
-<form method="post">
-	<input type=text placeholder="Table Name..." name="table_name"/>
-	<input type="submit" />
-</form>
-<?lua if modelgen then ?>
-	Model generated with success!
-<?lua end ?>
-<br/><br/>
-
-<h2>Generate CRUD</h2>
-
-<form method="post">
-	<input type=text placeholder="Model Name..." name="model_name"/>
-	<input type="submit" />
-</form>
-<?lua if crudgen then ?>
-	CRUD generated with success!
-<?lua end ?>
-]]
-	return code
-end
-
 -- The code for the generated controller
 -- model: the model from which the generated CRUD will refer to
 function M.generate_controller(model)

src/sailor/blank-app/conf/conf.lua

@@ -9,11 +9,12 @@ local conf = {
 		layout = 'main',
 		route_parameter = 'r',
 		default_error404 = 'error/404',
-		enable_autogen = false, -- default is false, should be true only in development environment
 		friendly_urls = false,
-		max_upload = 1024 * 1024, 
+		max_upload = 1024 * 1024,
 		environment = "development",  -- this will use db configuration named development
-		hide_stack_trace = false -- false recommended for development, true recommended for production
+		hide_stack_trace = false, -- false recommended for development, true recommended for production
+		enable_admin=false, -- set true only in development. Set password below too.
+		admin_password=""
 	},
 
 	db = {
@@ -28,8 +29,8 @@ local conf = {
 
 	smtp = {
 		server = '',
-		user = '',
-		pass = '',
+		smtpuser = '',
+		smtppass = '',
 		from = ''
 	},