Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Clone this wiki locally
A note from the dev and design team: we're still madly working on code for launch, so there's not necessarily a lot here for launch but we'll be adding more docs while we're in code freeze...
Guiding ethos for development
Assume that action is better than inaction; ask for feedback or review of the results of your actions rather permission to act on something that needs doing. Assume goodwill in all interactions! If you pick up a task from the Issues list (or from the Wishlist) then assign it to yourself. If you see something unassigned that needs doing, assign it to the most appropriate person. If you get something assigned to you but you can't manage it in the short-term, assign it to the next most appropriate person, or if all else fails, to a PM lead (Meghan or Brian).
Technical documentation for developers/tweakers/forkers of the tool
Relevant Python standards are...
Backend site structure
Copied from https://github.com/chnm/serendipomatic/issues/103#issuecomment-22299813 as the latest documentation I can find... NOTE for anyone who needs to edit templates/html/css:
- site_base.html should have elements for all common elements across the site and is the basis for all other pages
- page_base.html extends site_base.html and is the basis for all secondary pages (results page, about, connect, etc)
- index_base.html extends site_base.html and is the basis for the site index page (doesn't really need to be separate, but I left it there since I think we're kind of used to that)
The exception to this is the 500 error page, which is an entirely static page (so it can still be served if all else goes wrong) - significant changes to the page template need to be manually copied over to templates/500.html.
JQuery and JQuery UI are loaded in site_base.html. These are needed for the "Feed the Machine" Hippo dialog on submission and for the tooltip on the save link. These two libraries are loaded from the Google CDN with a fallback to a local version. The JQuery UI theme css is loaded from a local copy, but a link to the JQuery UI CDN is commented out should we need it.
How we use GitHub
During OWOT most people developed on their laptops then pushed their changes to the master repo. However, some people also made changes via the website, particularly small text, HTML or CSS tweaks, so our workflow should allow for that.
It's probably a good idea to merge master into your local branch first (if it's been around long enough / enough changes on master) and do any updates & conflict resolution there before merging back into master.
Changes that are pushed to master will automatically be pushed to our dev server and will be visible at http://dev-serendipomatic.herokuapp.com/, though there may be a small time delay. You can check on the status of recent builds at https://travis-ci.org/chnm/serendipomatic/builds, which should give you some idea which version of the code is on the dev server.
Changes to code can be automatically linked back to GitHub issues by including the issue number (e.g. Issue #113 from the top RHS of the issue page) in your commit message in the format '#nn' (e.g. '#113'). You can also do the reverse - if you copy the git commit hash and add it to a comment, github will make a nice link (e.g. the number that appears on code commits as 'commit 260d625' or 'latest commit 260d625' - you can also use the 'copy to clipboard' icon to get it).
Getting set up with the development environment and code base
Check and pull in notes from https://github.com/chnm/serendipomatic/blob/master/README.md
Software libraries used
- Natural Language Toolkit, Apache2 license
- Beautiful Soup, MIT License
- Django, custom license?
- Isotope, MIT license
- DBpedia Spotlight, Apache v.2 with other small thing
- Skeleton.css, MIT license
- Requests, Apache2 license
- bibs, GPLv3 license
- libZotero, no license listed
- Flickr API, Flickr license
- dicttoxml, GPL v.2
- xmltodict, MIT license
- simplejson, MIT License
Note during the build: some were listed at https://github.com/chnm/petulant-adventure/issues/9 which could now be closed. The developers have to add libraries to the requirements.txt file so that's probably the best place to pull the list from.
Top tip: be consistent in your use of localhost vs 127.0.0.1.
UX documentation for developers/forkers of the tool
Including graphic design, functionality, usability, content and micro-copy tone
- The app uses American English spellings
The results page layout works best when images are sized between 200 and 400 pixels wide (starter values only - we need to check this). Where it's possible to request specific image sizes from a provider API, please do so. Images can also be resized server-side.
Deployment notes - GitHub to AWS
To dev (heroku)
Changes to master should automatically be pushed to dev. See Heroku setup instructions if you want to set up your own heroku development instance. Older notes: see https://github.com/chnm/serendipomatic/issues/96 and https://github.com/chnm/serendipomatic/issues/106
Continuous integration and build status
To live (AWS)
(Mia's version of Rebecca's notes - to be fixed) SSH into the AWS instance. Git pull. Check env settings: /$path/%to/$envgoeshere Check for new requests Sync db in case of database changes Run collect static (for CSS, images, etc) Touch apache wsgi.py
Documentation for repositories
If you'd like your content to be included in the Serendip-o-matic, the first step is to have a good API that also has a clear statement of how your content is licensed for re-use. Then you can either
- fork the code, update it to include your API and issue a pull request for review by the core team OR
- fund project developers to do that for you.
If you update the code yourself, be careful around the 'magic box' dynamic query generation code.
If you're an individual institution, we'd love you to fork our code to integrate your API. That being said, we're still working out how we'll manage the mix of big aggregators and individual institutions, so your institution's representation in the mix of results will vary over time as we manage the mix of technical and broader issues. Get in contact with us at email@example.com if you'd like to chat about your options. We will be reviewing and adjusting the mix of results so they match the vision of the project and the needs of our wide range of users.
Please check that the code will recognise when results have been pulled from your repository and link to the appropriate catalogue search page from the 'Sources used' line on the results page.
Individual results always link to the original online source, not the repository's URI for the resource.
Note for everyone: if you change the number of results per page (or move to dynamic page load by device) then adjust the query logic re: the mix of results per aggregator.
Documentation for end users
Hopefully everything you need to know is at About Serendip-o-matic
We don't store your input texts and we don't store your results.