(Unofficial) Project Gutenberg search widget (under development)
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
_data Go tooling should ignore data Mar 24, 2018
cli Fix finder with errgroup Nov 11, 2018
db/migrate Author alias search result item table Jul 21, 2018
gutenfinder-bindata Update bindata May 2, 2018
gutenfinder Debug many author/category repository funcs Jun 13, 2018
integration Use the correct catalogue update type for Authors Dec 30, 2018
protocol Uniform fail test function Jun 17, 2018
script Fix clean db migrate script Dec 30, 2018
static Implement landing page Apr 20, 2018
template Study EBook template Apr 22, 2018
test_data/gutenberg/mirror Use mocks for TemplateView unit test Apr 26, 2018
vendor
web_frontend Uniform fail test function Jun 17, 2018
.dockerignore Rename to Gutenfinder Apr 16, 2018
.gitignore
Dockerfile Implement landing page Apr 20, 2018
Gopkg.lock CatalogueUpdates include EBookFiles Aug 27, 2018
Gopkg.toml CatalogueUpdates include EBookFiles Aug 27, 2018
LICENSE Initial commit Mar 16, 2018
README.md Fix README typo Apr 26, 2018
api_route_controller.go Debug many author/category repository funcs Jun 13, 2018
api_route_controller_test.go Fix protocol package lint errors Apr 26, 2018
app.go Finder has extensible sources Oct 26, 2018
app_test.go Fix tests for DB injection Oct 26, 2018
asset_bundle.go Move Bindata to separate package Apr 26, 2018
author.go CatalogueUpdates include EBookFiles Aug 27, 2018
author_alias_search_result_item.go
author_alias_search_result_item_repository.go
author_indexer.go Fix tests for DB injection Oct 26, 2018
author_matcher.go
author_repository.go Rename combine interface/impl files Oct 13, 2018
author_test.go Uniform fail test function Jun 17, 2018
bindata_dir.go Use error package that gives stacktrace May 13, 2018
bindata_dir_test.go Uniform fail test function Jun 17, 2018
catalogue_update.go CatalogueUpdates include EBookFiles Aug 27, 2018
catalogue_update_repository.go Rename combine interface/impl files Oct 13, 2018
catalogue_update_test.go
catalogue_update_type_id.go CatalogueUpdates are each for one type of data only May 1, 2018
controller_util.go Debug many author/category repository funcs Jun 13, 2018
controller_util_test.go
database_postgres.go Use sqlx for tricky search result join Aug 27, 2018
doc.go Rename to Gutenfinder Apr 16, 2018
docker-compose.yml
ebook.go OK for book to have no author Dec 30, 2018
ebook_category.go Debug outstanding author repository funcs Jun 13, 2018
ebook_category_member.go Implement EBookCategory repository Jun 3, 2018
ebook_category_member_repository.go Rename combine interface/impl files Oct 13, 2018
ebook_category_repository.go Rename combine interface/impl files Oct 13, 2018
ebook_category_test.go Debug outstanding author repository funcs Jun 13, 2018
ebook_category_type_id.go Implement author related funcs May 6, 2018
ebook_category_type_id_test.go Implement author related funcs May 6, 2018
ebook_file.go Debug many author/category repository funcs Jun 13, 2018
ebook_file_repository.go Rename combine interface/impl files Oct 13, 2018
ebook_file_test.go Uniform fail test function Jun 17, 2018
ebook_indexer.go Fix tests for DB injection Oct 26, 2018
ebook_repository.go Rename combine interface/impl files Oct 13, 2018
ebook_test.go OK for book to have no author Dec 30, 2018
ebook_text_loader.go
ebook_text_loader_test.go
elastic.dev.env
elastic_author.go Implement AuthorIndexer tests Jul 25, 2018
elastic_author_repository.go Use debug log level for indexing log Dec 30, 2018
elastic_ebook.go
elastic_ebook_repository.go Use debug log level for indexing log Dec 30, 2018
elastic_ebook_test.go Uniform fail test function Jun 17, 2018
elastic_entity.go Tidy up ElasticAuthor entity and friends Jun 17, 2018
elastic_repository.go Implement AuthorIndexer tests Jul 25, 2018
encoded_file_path.go Use error package that gives stacktrace May 13, 2018
entity.go Tidy up new Identity methods Apr 20, 2018
entity_test.go Implement Entity unit tests Apr 24, 2018
finder.go Rename finder drain channel method Dec 30, 2018
finder_test.go
generate_bindata.go No metadata for bindata Apr 28, 2018
generate_mock.go Update Finder related integration test for author Nov 11, 2018
gutenberg_entity.go Implement author related funcs May 6, 2018
gutenberg_entity_test.go Implement author related funcs May 6, 2018
helper_test.go Use injected service interfaces Oct 14, 2018
landing_controller.go
log.go Debug many author/category repository funcs Jun 13, 2018
log_test.go Uniform fail test function Jun 17, 2018
mock_asset_bundle_test.go Use mocks for TemplateView unit test Apr 26, 2018
mock_closer_test.go Unit tests for Finder Nov 11, 2018
mock_finder_result_cache_test.go Unit tests for Finder Nov 11, 2018
mock_finder_source_test.go
mock_request_handler_test.go Implement WebRouteController unit tests Apr 26, 2018
mock_response_writer_test.go
mock_sqldb_test.go Unit tests for Finder Nov 11, 2018
mock_view_test.go Generate mocks Apr 26, 2018
mock_web_request_handler_test.go Implement WebRouteController unit tests Apr 26, 2018
postgres.dev.env Rename to Gutenfinder Apr 16, 2018
property_map.go
repository.go Fix tests for DB injection Oct 26, 2018
root_controller.go Move Bindata to separate package Apr 26, 2018
scanner.go Rename combine interface/impl files Oct 13, 2018
scanner_test.go
search.go Tidy up new Identity methods Apr 20, 2018
search_controller.go Remove author alias results from search controller Dec 30, 2018
search_repository.go Rename combine interface/impl files Oct 13, 2018
search_result.go Tweak SearchResult size Apr 21, 2018
search_result_item.go Implement Author Alias search repository Jul 21, 2018
search_result_item_controller.go Fix tests for DB injection Oct 26, 2018
search_result_item_repository.go Rename combine interface/impl files Oct 13, 2018
search_result_item_test.go Uniform fail test function Jun 17, 2018
search_result_item_type_id.go Change search result item precedence Dec 30, 2018
search_result_item_type_id_test.go Change search result item precedence Dec 30, 2018
search_result_repository.go Rename combine interface/impl files Oct 13, 2018
search_result_test.go Implement SearchResult unit tests Apr 26, 2018
search_result_view_model.go Build SearchResultVIewModel with Authors Dec 30, 2018
search_result_view_model_test.go
search_test.go Tidy up new Identity methods Apr 20, 2018
static_file_controller.go Use error package that gives stacktrace May 13, 2018
static_file_controller_test.go Implement unit test for IsPermittedStaticFile Apr 25, 2018
study_ebook_view_model.go Move Bindata to separate package Apr 26, 2018
study_ebook_view_model_test.go Implement StudyEBookViewModel unit test Apr 25, 2018
template_view.go Debug many author/category repository funcs Jun 13, 2018
template_view_test.go Uniform fail test function Jun 17, 2018
text_encoding_id.go Define new (failing) unit tests Apr 22, 2018
text_encoding_id_test.go Uniform fail test function Jun 17, 2018
text_search_result_item.go Rename to Gutenfinder Apr 16, 2018
text_search_result_item_repository.go Rename combine interface/impl files Oct 13, 2018
text_search_result_item_test.go Implement Title & Text SearchResultItem unit tests Apr 26, 2018
title_search_result_item.go Implement Title & Text SearchResultItem unit tests Apr 26, 2018
title_search_result_item_repository.go Rename combine interface/impl files Oct 13, 2018
title_search_result_item_test.go Implement Title & Text SearchResultItem unit tests Apr 26, 2018
util.go Debug many author/category repository funcs Jun 13, 2018
util_test.go Uniform fail test function Jun 17, 2018
view.go Fix lint errors and tidy Apr 20, 2018
view_controller.go Define new (failing) unit tests Apr 22, 2018
view_test.go Uniform fail test function Jun 17, 2018
web_api_controller.go Fix protocol package lint errors Apr 26, 2018
web_controller.go Split WebRouteController from WebController Apr 26, 2018
web_route_controller.go Use error package that gives stacktrace May 13, 2018
web_route_controller_test.go Implement WebRouteController unit tests Apr 26, 2018
websocket_controller.go

README.md

Gutenfinder: search widget for Project Gutenberg

Gutenfinder is a little project to make a search widget for Project Gutenberg.

It is under development.

CLI examples:

Serve a webapp on localhost, port 8080:

$ gutenfinder web serve \
	--postgres="user=username dbname=gutenfinder host=dbhost sslmode=disable" \
	--elastic="http://gutenfinder_dev_elastic:9200"

Search for eBooks on the command line:

$ gutenfinder search --term beancurd
{
	"id": 9,
	"term": "beancurd",
	"result_count": 1,
	"items": [
		{
			"id": 467,
			"ebook_id": 724,
			"ebook_title": "Have We No Rights? A frank discussion of the \"rights\" of missionaries",
			"html_snippet": "be made of a couple of\r\nplanks laid on sawhorses, and you may have to eat boiled rice, greens,\r\nand \u003cem\u003ebeancurd\u003c/em\u003e"
		}
	]
}

Results are returned based on titles, the full text of the book, and any other properties currently indexed.

Getting started

Dependencies

Install the following dependencies:

Due to an issue with the dockerized Elasticsearch, the following tweak must be made on Linux:

sudo sysctl -w vm.max_map_count=262144

This setting will be lost on reboot, but can be permanently set by adding it to /etc/etc/sysctl.conf

Initialize the project

$ ./script/init
$ ./script/download

This will download a lot of data from a Project Gutenberg archive.

Run with docker

The following sequence of commands will build and launch the app, run migrations, read the eBook catalogue, and index the eBooks on Elasticsearch.

$ ./script/docker-compose-up
$ ./script/db-migrate
$ ./script/catalogue-load-xml
$ ./script/catalogue-load-text

Credits

Johnny Morrice

I am not affiliated with Project Gutenberg in any way. This is just for fun.

License

MIT-style, see the LICENSE file.