Example GSoC proposal: Revamp astropython.org website

Project Proposal

" Revamping astropython.org for Astropy "

Astropython.org acts as a community knowledge source for performing astronomy research with Python. Its main components are a forum for advice and discussion, tutorials on specific topics, a wiki of useful resources and storing of code snippets from users for future reference. It is currently slow, outdated and difficult to maintain. Through my proposal , I wish to underline a comprehensive plan to revamp the website from its roots up using the latest and stable web technologies available to us making the application fast, secure, easy to maintain and scalable across all devices. In addition to redesigning and revamping each of its original components, I propose certain new features that would be of tremendous productive value to the community.

Features & Implementation

The backend of the web application would be primarily based on Django 1.7 (Python’s high level web framework). I have particularly stuck with Django because of its strong community and exceptional support and update policies. With Django, be rest assured that astropython will not need to be revamped from its roots. The core front end would be completely responsive and implemented in Bootstrap and JQuery.The design will only be finalized after consulting the astropy developers and community participants. This new backend structure will be guided by its old design.The entire code will be modular - and a detailed documentation and instruction set will be provided for easy maintenance and future updates. I have broken it down into main steps that need to be taken care of:-

1. Creating Tutorial Section

Front End i. A markdown editor would be used as the primary front end method for creating tutorials. After going through a list of such editors, I would like to suggest a Github flavoured markdown/Epic Editor for the purpose. It will be finalized after thorough discussion with my mentor. ii. Tutorials can be created and modified by logged in users. However, to moderate content of the page, no tutorial would be added or modified without site moderators approving it. Back End i. The high level models would include a basic model with title of tutorial , the original input(markdown), writer, slugs, hits and timestamps. Updations model would include successive updations to the tutorial including the modified code, Categories model would help in categorization, and Moderation model would include the status of submission/modification and person who approved the addition/modification of tutorials. More entities may be added/reduced later. ii. The markdown would be dynamically converted to html on runtime. This helps us in allowing the modification the original script whenever the user wants.

2. Creating Code Snippet section

i. For writing new code snippets and their descriptions, the primary method of input would be markdown editor. The users need to be logged in to do so. Common aspects such as comments,tags,etc remain the same.

ii. On the backend side,the high level models would are similar to those above. They include a basic model with title of the snippet, the original input(markdown), writer, slugs, timestamps.Categories model would help in categorization, and Moderation model would include the status of submission and the person who approved the addition/modification of tutorials. More entities may be added/reduced later.

iii. If it is planned to use Pygments or other syntax highlighting libraries we could also store the highlighted html text directly in the basic model itself.

3. Creating Question and Answer Forum

i. It would be best to integrate a stable Django Package such as AskBot (https://github.com/ASKBOT/askbot-devel)

ii. Admins and moderators have the power to remove/modify/close a question on the forum.The backend models would provide complete information for each question and answer including the writers ,open/closed status, timestamps,etc. Categories and Moderations can be pushed to separate models.

4. Creating Blog Section

i. The blogs section is perhaps new to the website. This would be the place to post and discuss scientific work or anything that the scientific community finds intriguing and worth sharing with the community. Each blog would be powered by tinyMCE plugin for richer text functionality

ii. The backend for the blog would have a title, writer,approval status,input data, timestamps.Categories and Moderations may have models of their own .

5. Creating Resources and Tools Section

i. Same as “existing resources and tools” but with a compulsory title , url for the resource and a description for the same. This would only have default textboxes for input

ii. If required, an iframe can open the url on the page itself.

iii. The backend would be similar to those above- having the same three models-basic functionality , Moderation and Categories

6. Porting old website’s content to new one

• To port the old website’s content into a new one , I would like to first store the data from the old database into a CSV file and then use the CSV file to load data into a new database using Google Cloud Storage Bucket or I could do the same by storing the entire DB into Google Cloud Storage and sharing it with the new application.

• Proper design of database and SQL commands would ensure that null values do not enter the database and there is consistency between the old and new databases. Hence the new models and database structure would be guided by their old structure.

• There might be some issues with the tags and categorization. Some form of markup language may be necessary. This would be decided by me and my mentor once I get to see current storage structure and page rendering mechanisms.

7. Admin Functionality : Maintaining the site

• Using grappelli or django-admin-tools to revamp and add functionality to Django’s admin interface.This would allow us to add new tools to do common tasks with ease.

• The standard admin interface rules/protocols/conventions are to be followed.A common stream of posts from various sections are waiting for moderation would be maintained.

• The users would be categorized into the following categories :-

i. Admins - The Almighty Powerful ones -"With great power comes great responsibility"

ii. Moderators

Essentially all moderators belong to the “trusted” category also. Hence bypass moderation- Their posts can be deleted by admins only. Moderators are created by the admins They have the power to approve a modification /addition of content to the website They have the power to block a particular user from a particular section or all sections as they deem fit. A proper interface would be provided for the same They are able to delete posts from any of the above sections (blogs, tutorials, code snippets, resources, etc) including all moderation rights on the forum. However a log would be maintained of their actions. They have the ability to make a user “trusted”

iii. Trusted Users

These users are “trusted” by admins or moderators, hence they do not need to make a request for posting something on any of the sections. However, people above them in the hierarchy can delete their posts or reduce their status to normal users.

iv. Normal Users

For each user we maintain his access grant on various sections and overall permission to sign in. This is done by extending functionality into Django’s existing User models They require permissions from their superiors to add content.

v. Blocked Users: Banned from posting on particular sections / logging in.

8. General Section

• OAuth authentications for Login and SignUp

• A “search bar” would be available at the navigation bar of the website at all times other than the home page. However the “Search” functionality will be limited only to the section of the website the user is currently in. If the user searches for a particular #tag , results from all the sections are collected for the tag.

• Tags will be used to implement the basic classification and searching mechanism using Django’s built-in references. I suggest using django-taggit package for the same.

• Comments across all sections (blogs, tutorials, code snippets, tools) would be powered by Disqus for authorized users only

And yes ,there is going to be an About / Contribute. The contribute section will also contain the links to github address for convenience. The Atom Feed will be ported from the old website. Optionally , if time remains , more features would be added to the Landing Page and it would be made better!

9. Deployment – As a testing/production host, I would like to work on Heroku or PythonAnywhere as it tremendously reduces deployment overhead and is easy to use and also GAE doesn't allow running "normal" Django with a relational backend using join, hence it possess additional difficulty.

10. Optional – Advanced

• Share posts on blogs on Facebook or other popular blog sharing sites.

• Add rating mechanisms or upvoting schemes to all sections.

• Better search functionality by displaying sorted results using popularity,ratings,etc

• Create a newsletter delivery system using one of Django’s prebuilt packages.

Timeline for Execution

• 27th April - 25th May: Community bonding Period. Through detailed interaction with the mentor and community, refine the general idea, acquaint myself with the expectations and essential requirements. Get acquainted to using AskBot and other Django packages I plan to use in the project by making basic applications of each. Finalize the design and layout of the entire website with my mentor and create a base template. Finalize the models,views, database schemas and tables.

Note : My end semester college exams are from 5th May,2015 to 18th May,2015. I will not be completely available during this time. • 26th May –1st June: Environment setup. Creation of separate apps for each section. Creation of models of all apps. Testing Database connections, Moderator/Admin permissions and retrieval of values through admin panel. Integrating django-admin-tools.

• 2nd June –11th June: Get started and complete the Tutorials Section. On the back end it is checked that the values are being forwarded through the front end are being correctly entered in the database tables. Views are properly framed. Check if changes made by users on a higher level are being reflected across.Devise a method to display correct HTML output from markdown text.

• 12th June – 22nd June : Create basic login/register functionalities on the front end. Create and complete “Code Snippets” section. Update the views.Check for correct formatting and highlighting on the front end and check for correct storage results on the backend.

• 23rd June –3rd July: Add login and register (authentication) functionality to the web application. Start and complete the “Resource and Tools” and “Blogs” section of the website. On the backend check for entries entering the correct columns through the front end.

• 3rd July -Mid Term Evaluation

• 4th July – 10th July: Build the Question and Answer Forum implementing AskBot. Check if all questions and answers front end components are working smoothly and the values are being correctly stored in the database.

Open website to the astropy community and some other organizations for testing and reviews

• 11th July – 17th July: Add a common stream of posts from various sections waiting for moderation on the admin panel. Complete the admin panel.

• 18th July – 24th July: Add Disqus comments , OAuth and Search functionality.Check and improve on all CSS components. Complete and improve on the landing page but strictly within this time frame.

• 25th July – 31st July: Store old database contents and migrate to new one using one of the methods described in the implementation.

Note : My university begins for the next semester from 1st August. There might be a slight hit in productivity after this,but I will try to my best to avoid it !

• 1st August – 7th August: Complete deployment. Testing of all components.Get the Atom feed ported.

• 8th August – 21st August: Add documentation. Final testing. Buffer Period if any of the above spill over. If spare time is left work on optional components.

• After GSOC: Keep working on and complete all of the optional components