Skip to content
Permalink
Browse files

Join robinho, api, site and extension projects in a single monorepo

  • Loading branch information...
rogeriochaves committed May 1, 2018
1 parent 0fc5a32 commit b5f7a34bd45adf285ee664bee41ae4926dc9ae09
Showing with 35,170 additions and 0 deletions.
  1. +29 βˆ’0 api/.circleci/config.yml
  2. +3 βˆ’0 api/.dockerignore
  3. +6 βˆ’0 api/.gitignore
  4. +132 βˆ’0 api/CONTRIBUTING.md
  5. +1,553 βˆ’0 api/Cargo.lock
  6. +22 βˆ’0 api/Cargo.toml
  7. +40 βˆ’0 api/Dockerfile
  8. +102 βˆ’0 api/README.md
  9. +4 βˆ’0 api/Rocket.toml
  10. +101 βˆ’0 api/admin/Data/Category.elm
  11. +403 βˆ’0 api/admin/Links.elm
  12. +176 βˆ’0 api/admin/Login.elm
  13. +105 βˆ’0 api/admin/Main.elm
  14. +37 βˆ’0 api/admin/Stylesheet.elm
  15. +303 βˆ’0 api/dev/postman_collection.json
  16. +19 βˆ’0 api/docker-compose.yml
  17. +22 βˆ’0 api/elm-package.json
  18. +2 βˆ’0 api/migrations/00000000000000_diesel_initial_setup/down.sql
  19. +36 βˆ’0 api/migrations/00000000000000_diesel_initial_setup/up.sql
  20. +1 βˆ’0 api/migrations/2017-10-01-203443_create_categories/down.sql
  21. +10 βˆ’0 api/migrations/2017-10-01-203443_create_categories/up.sql
  22. +1 βˆ’0 api/migrations/2017-10-02-125941_create_links/down.sql
  23. +10 βˆ’0 api/migrations/2017-10-02-125941_create_links/up.sql
  24. +1 βˆ’0 api/migrations/2017-10-02-130231_create_votes/down.sql
  25. +9 βˆ’0 api/migrations/2017-10-02-130231_create_votes/up.sql
  26. +1 βˆ’0 api/migrations/2017-10-13-165149_add_not_news_category/down.sql
  27. +1 βˆ’0 api/migrations/2017-10-13-165149_add_not_news_category/up.sql
  28. +1 βˆ’0 api/migrations/2017-10-23-111129_add_content_to_links/down.sql
  29. +1 βˆ’0 api/migrations/2017-10-23-111129_add_content_to_links/up.sql
  30. +1 βˆ’0 api/migrations/2017-12-30-220053_add_verified_category_id_to_links/down.sql
  31. +1 βˆ’0 api/migrations/2017-12-30-220053_add_verified_category_id_to_links/up.sql
  32. +1 βˆ’0 api/migrations/2018-01-17-233852_create_hoaxes/down.sql
  33. +39 βˆ’0 api/migrations/2018-01-17-233852_create_hoaxes/up.sql
  34. +41 βˆ’0 api/migrations/2018-01-20-211841_drop_hoaxes/down.sql
  35. +1 βˆ’0 api/migrations/2018-01-20-211841_drop_hoaxes/up.sql
  36. +1 βˆ’0 api/migrations/2018-03-25-125500_separate_clickbait_title/down.sql
  37. +2 βˆ’0 api/migrations/2018-03-25-125500_separate_clickbait_title/up.sql
  38. +3 βˆ’0 api/migrations/2018-03-26-111749_add_verified_clickbait_title_and_removed/down.sql
  39. +3 βˆ’0 api/migrations/2018-03-26-111749_add_verified_clickbait_title_and_removed/up.sql
  40. +3 βˆ’0 api/src/commons/mod.rs
  41. +59 βˆ’0 api/src/commons/pg_pool.rs
  42. +23 βˆ’0 api/src/commons/remote_ip.rs
  43. +43 βˆ’0 api/src/commons/responders.rs
  44. +8 βˆ’0 api/src/data/category.rs
  45. +91 βˆ’0 api/src/data/link.rs
  46. +5 βˆ’0 api/src/data/mod.rs
  47. +30 βˆ’0 api/src/data/schema.rs
  48. +72 βˆ’0 api/src/data/verified_domains.rs
  49. +224 βˆ’0 api/src/data/vote.rs
  50. +20 βˆ’0 api/src/dummy.rs
  51. +113 βˆ’0 api/src/endpoints/admin.rs
  52. +19 βˆ’0 api/src/endpoints/categories.rs
  53. +22 βˆ’0 api/src/endpoints/healthcheck.rs
  54. +9 βˆ’0 api/src/endpoints/index.rs
  55. +63 βˆ’0 api/src/endpoints/links.rs
  56. +6 βˆ’0 api/src/endpoints/mod.rs
  57. +161 βˆ’0 api/src/endpoints/votes.rs
  58. +26 βˆ’0 api/src/jobs.rs
  59. +63 βˆ’0 api/src/lib.rs
  60. +13 βˆ’0 api/src/main.rs
  61. +1 βˆ’0 api/src/scrapper/mod.rs
  62. +210 βˆ’0 api/src/scrapper/package-lock.json
  63. +23 βˆ’0 api/src/scrapper/package.json
  64. +70 βˆ’0 api/src/scrapper/scrapper.rs
  65. +7 βˆ’0 api/src/scrapper/unfluff.js
  66. +15 βˆ’0 api/src/views/admin.hbs
  67. +8 βˆ’0 api/start.sh
  68. +39 βˆ’0 api/tests/admin_test.rs
  69. +16 βˆ’0 api/tests/healthcheck_test.rs
  70. +31 βˆ’0 api/tests/index_test.rs
  71. +3 βˆ’0 extension/.babelrc
  72. +36 βˆ’0 extension/.circleci/config.yml
  73. +4 βˆ’0 extension/.gitignore
  74. +101 βˆ’0 extension/CONTRIBUTING.md
  75. +21 βˆ’0 extension/README.md
  76. +8 βˆ’0 extension/_locales/en/messages.json
  77. +8 βˆ’0 extension/_locales/es/messages.json
  78. +8 βˆ’0 extension/_locales/pt_BR/messages.json
  79. +8 βˆ’0 extension/_locales/pt_PT/messages.json
  80. +21 βˆ’0 extension/elm-package.json
  81. BIN extension/icons/trump-128.png
  82. BIN extension/icons/trump-48.png
  83. BIN extension/icons/trump-96.png
  84. +25 βˆ’0 extension/manifest.json
  85. +15,252 βˆ’0 extension/package-lock.json
  86. +39 βˆ’0 extension/package.json
  87. +103 βˆ’0 extension/src/Data/Category.elm
  88. +163 βˆ’0 extension/src/Data/Votes.elm
  89. +285 βˆ’0 extension/src/FlagPopup.elm
  90. +35 βˆ’0 extension/src/Helpers.elm
  91. +100 βˆ’0 extension/src/Locale/English.elm
  92. +7 βˆ’0 extension/src/Locale/Languages.elm
  93. +63 βˆ’0 extension/src/Locale/Locale.elm
  94. +100 βˆ’0 extension/src/Locale/Portuguese.elm
  95. +100 βˆ’0 extension/src/Locale/Spanish.elm
  96. +35 βˆ’0 extension/src/Locale/Words.elm
  97. +210 βˆ’0 extension/src/StoryVotes.elm
  98. +93 βˆ’0 extension/src/Stylesheet.elm
  99. +11 βˆ’0 extension/src/index.html
  100. +43 βˆ’0 extension/src/index.js
  101. +29 βˆ’0 extension/src/injectors/common.js
  102. +22 βˆ’0 extension/src/injectors/extensionPopup.js
  103. +33 βˆ’0 extension/src/injectors/facebook.js
  104. +31 βˆ’0 extension/src/injectors/facebook/externalLinks.js
  105. +27 βˆ’0 extension/src/injectors/facebook/pagePost.js
  106. +26 βˆ’0 extension/src/injectors/facebook/sharePagePost.js
  107. +11 βˆ’0 extension/src/injectors/twitter.js
  108. +18 βˆ’0 extension/src/injectors/twitter/externalLinks.js
  109. +60 βˆ’0 extension/test/Data/VotesTest.elm
  110. +24 βˆ’0 extension/test/elm-package.json
  111. +3 βˆ’0 extension/test/fixtures/facebook/advertisement.html
  112. +3 βˆ’0 extension/test/fixtures/facebook/nestedStory.html
  113. +3 βˆ’0 extension/test/fixtures/facebook/postFromPage.html
  114. +3 βˆ’0 extension/test/fixtures/facebook/postFromProfile.html
  115. +258 βˆ’0 extension/test/fixtures/facebook/postFromTwitter.html
  116. +1,106 βˆ’0 extension/test/fixtures/facebook/postWithExternalLink.html
  117. +3 βˆ’0 extension/test/fixtures/facebook/sharedPostFromPage.html
  118. +3 βˆ’0 extension/test/fixtures/facebook/sharedPostFromProfile.html
  119. +78 βˆ’0 extension/test/fixtures/twitter/embededLinkIframe.html
  120. +244 βˆ’0 extension/test/fixtures/twitter/tweetWithEmbededLink.html
  121. +14 βˆ’0 extension/test/helpers.js
  122. +20 βˆ’0 extension/test/injectors/facebook.spec.js
  123. +95 βˆ’0 extension/test/injectors/facebook/externalLinks.spec.js
  124. +37 βˆ’0 extension/test/injectors/facebook/pagePost.spec.js
  125. +37 βˆ’0 extension/test/injectors/facebook/sharePagePost.spec.js
  126. +35 βˆ’0 extension/test/injectors/twitter/externalLinks.spec.js
  127. +26 βˆ’0 extension/webpack.config.js
  128. +32 βˆ’0 robinho/.circleci/config.yml
  129. 0 { β†’ robinho}/.dockerignore
  130. 0 { β†’ robinho}/.flake8
  131. 0 { β†’ robinho}/.gitignore
  132. 0 { β†’ robinho}/Dockerfile
  133. 0 { β†’ robinho}/Makefile
  134. 0 { β†’ robinho}/README.md
  135. 0 { β†’ robinho}/__main__.py
  136. 0 { β†’ robinho}/dev/postman_collection.json
  137. 0 { β†’ robinho}/links.csv
  138. 0 { β†’ robinho}/notebooks/.ipynb_checkpoints/ExtremelyBiased-checkpoint.ipynb
  139. 0 { β†’ robinho}/notebooks/.ipynb_checkpoints/Keywords-checkpoint.ipynb
  140. 0 { β†’ robinho}/notebooks/.ipynb_checkpoints/Lime-checkpoint.ipynb
  141. 0 { β†’ robinho}/notebooks/ExtremelyBiased.ipynb
  142. 0 { β†’ robinho}/notebooks/Keywords.ipynb
  143. 0 { β†’ robinho}/notebooks/Lime.ipynb
  144. 0 { β†’ robinho}/output/.gitkeep
  145. 0 { β†’ robinho}/requirements.txt
  146. 0 robinho/{ β†’ robinho}/categories.py
  147. 0 robinho/{ β†’ robinho}/classifiers/base.py
  148. 0 robinho/{ β†’ robinho}/classifiers/click_bait.py
  149. 0 robinho/{ β†’ robinho}/classifiers/extremely_biased.py
  150. 0 robinho/{ β†’ robinho}/classifiers/fake_news.py
  151. 0 robinho/{ β†’ robinho}/classifiers/keywords.py
  152. 0 robinho/{ β†’ robinho}/marisa_vectorizers.py
  153. 0 robinho/{ β†’ robinho}/model.py
  154. 0 robinho/{ β†’ robinho}/server.py
  155. 0 { β†’ robinho}/tests/__init__.py
  156. 0 { β†’ robinho}/tests/classifiers/__init__.py
  157. 0 { β†’ robinho}/tests/classifiers/test_base.py
  158. 0 { β†’ robinho}/tests/classifiers/test_click_bait.py
  159. 0 { β†’ robinho}/tests/classifiers/test_extremely_biased.py
  160. 0 { β†’ robinho}/tests/classifiers/test_fake_news.py
  161. 0 { β†’ robinho}/tests/classifiers/test_keywords.py
  162. 0 { β†’ robinho}/tests/helpers.py
  163. 0 { β†’ robinho}/tests/test_model.py
  164. +31 βˆ’0 site/.circleci/config.yml
  165. +3 βˆ’0 site/.gitignore
  166. +2 βˆ’0 site/CNAME
  167. +8 βˆ’0 site/README.md
  168. +37 βˆ’0 site/ROADMAP.md
  169. +19 βˆ’0 site/deploy.sh
  170. +22 βˆ’0 site/elm-package.json
  171. +20 βˆ’0 site/elm-static-html.json
  172. +119 βˆ’0 site/more.html
  173. +8,786 βˆ’0 site/package-lock.json
  174. +43 βˆ’0 site/package.json
  175. +103 βˆ’0 site/src/Data/Category.elm
  176. +191 βˆ’0 site/src/Data/Votes.elm
  177. +237 βˆ’0 site/src/FlagLink.elm
  178. +35 βˆ’0 site/src/Helpers.elm
  179. +231 βˆ’0 site/src/Locale/English.elm
  180. +7 βˆ’0 site/src/Locale/Languages.elm
  181. +63 βˆ’0 site/src/Locale/Locale.elm
  182. +230 βˆ’0 site/src/Locale/Portuguese.elm
  183. +233 βˆ’0 site/src/Locale/Spanish.elm
  184. +50 βˆ’0 site/src/Locale/Words.elm
  185. +354 βˆ’0 site/src/Main.elm
  186. +94 βˆ’0 site/src/Stylesheet.elm
  187. +49 βˆ’0 site/src/index.ejs
  188. +36 βˆ’0 site/src/index.js
  189. BIN site/src/static/add-to-chrome.png
  190. BIN site/src/static/add-to-firefox.png
  191. BIN site/src/static/avatar.png
  192. +1 βˆ’0 site/src/static/avatar.svg
  193. BIN site/src/static/clickbait.png
  194. BIN site/src/static/favicons/android-chrome-192x192.png
  195. BIN site/src/static/favicons/android-chrome-512x512.png
  196. BIN site/src/static/favicons/apple-touch-icon.png
  197. +9 βˆ’0 site/src/static/favicons/browserconfig.xml
  198. BIN site/src/static/favicons/favicon-16x16.png
  199. BIN site/src/static/favicons/favicon-32x32.png
  200. BIN site/src/static/favicons/favicon.ico
  201. +20 βˆ’0 site/src/static/favicons/manifest.json
  202. BIN site/src/static/favicons/mstile-144x144.png
  203. BIN site/src/static/favicons/mstile-150x150.png
  204. BIN site/src/static/favicons/mstile-310x150.png
  205. BIN site/src/static/favicons/mstile-310x310.png
  206. BIN site/src/static/favicons/mstile-70x70.png
  207. +57 βˆ’0 site/src/static/favicons/safari-pinned-tab.svg
  208. BIN site/src/static/robinho.jpg
  209. +42 βˆ’0 site/src/static/searchResults.html
  210. +7 βˆ’0 site/sw-precache-config.js
  211. +60 βˆ’0 site/test/Data/VotesTest.elm
  212. +25 βˆ’0 site/test/elm-package.json
  213. +66 βˆ’0 site/webpack.config.js
@@ -0,0 +1,29 @@
version: 2
jobs:
build:
docker:
- image: docker
working_directory: ~/app
steps:
- checkout
- setup_remote_docker
- run:
name: fetch existing image
command: docker pull fakenewsdetector/api
- run:
name: build
command: docker build --cache-from fakenewsdetector/api -t fakenewsdetector/api .
- run:
name: test
command: docker run --rm fakenewsdetector/api cargo test
- deploy:
name: deploy
command: |
if [ "${CIRCLE_BRANCH}" == "master" ]; then
docker login --username=$DOCKERHUB_USER --password=$DOCKERHUB_PASS
docker push fakenewsdetector/api
docker login --username=_ --password=$HEROKU_KEY registry.heroku.com
docker tag fakenewsdetector/api registry.heroku.com/fake-news-detector-api/web
docker push registry.heroku.com/fake-news-detector-api/web
fi
@@ -0,0 +1,3 @@
target
.git
.env
@@ -0,0 +1,6 @@
target/
**/*.rs.bk
target/
**/*.rs.bk
node_modules*
elm-stuff
@@ -0,0 +1,132 @@
---

[Pre-Requirements](#pre-requirements) | [Running](#running) | [Testing](#testing) | [Postman](#postman) | [Running outside docker](#running-outside-docker) | [Tech Stack](#tech-stack) | [Formatting](#formatting) | [Deploy](#deploy) | [Jobs](#jobs)
---

# Contributing

Contributions are always welcome, no matter how large or small.

## Pre-Requirements

You need to have installed:

* [docker](https://www.docker.com/docker)
* optional: rust nightly (install using [rustup](https://www.rustup.rs/))
* optional: [elm](https://guide.elm-lang.org/install.html)

## Running

```
docker-compose up
```

It will take a while, about 20 minutes to download an compile all rust dependencies.

But that's it! Now go to `http://localhost:8000/admin` and it should be running.

The default credentials for logging in the admin area are email: `admin@fakenewsdetector.org`, password: `123`

## Testing

To run the tests, with `docker-compose up` running, fire up another terminal and execute:

`docker-compose exec web cargo test`

If you want to run it in watch mode, use:

`docker-compose exec web cargo watch -x test`

## Postman

Postman is an app that helps you make requests when developing APIs.

We have a postman collection ready for you to use for debugging the API, just [download postman](https://www.getpostman.com/) and import the `dev/postman_collection.json` file.

<img width="310" src="https://user-images.githubusercontent.com/792201/31867369-ff4e8060-b76c-11e7-9ea8-2ddbfebec215.png" alt="Postman Collection">

## Running Outside Docker

If you want to run the app outside docker for faster development flow, you will need to install [nightly rust](https://doc.rust-lang.org/1.2.0/book/nightly-rust.html), [cargo watch](https://github.com/passcod/cargo-watch) and [nodejs](https://nodejs.org/en/).

Also, you will need to have a postgres database running, if you don't, you can run it with docker-compose:

```
docker-compose up database
```

Install the rust and js dependencies:

```
cargo build
cd src/scrapper/
npm install
cd -
```

Then start the app:

```
cargo watch -w src -x run
```

For compiling the admin area:

```
cargo watch -w admin -s "elm-make --yes admin/Main.elm --output target/admin.js"
```

To run the tests:

```
cargo watch -x test
```

## Tech Stack

This project is done in Rust, if you don't know Rust yet, the [official guide](https://doc.rust-lang.org/stable/book/) is a good place to start. It shouldn't take very long before you can start contributing, just a few first chapters and you should be good to go, the compiler helps a lot.

For the web framework we use [rocket](https://rocket.rs/), for starting a web server, routing, json-responses, etc.

We use [PostgreSQL](https://www.postgresql.org/) for the database, with [Diesel](http://diesel.rs/) as the ORM for connecting with Rust and run the migrations.

Finally, we use [docker-compose](https://docs.docker.com/compose/) to launch the database and build an image with everything we need to run the project. By using docker we ensure that what's running locally is as close as possible from production.

## Formatting

We use [rustfmt](https://github.com/rust-lang-nursery/rustfmt) to automatically format the source code, please use it as well when developing new features, it integrates with most text editors and IDEs and keep a standard across the codebase.

Also, we add no configuration to it, just use the standard behaviours, this way we eliminate the discussions about formatting on PRs.

## Configuring your IDE

To develop with Rust, having a well configured IDE helps A LOT, once you have a well configured one, you won't be able to code Rust without it anymore, so we recomment to spend a few minutes doing it.

[Visual Studio Code](https://code.visualstudio.com/) has a good support for Rust. After installing it add the [Rust Lang Extension](https://marketplace.visualstudio.com/items?itemName=kalitaalexey.vscode-rust), it should take you about half an hour to configure it well. Things to check:

* Autocomplete is working
* Code autoformats on save (rustfmt)
* Compilation errors and warnings show up on the IDE
* Code compiles on save

## Deploy

The deploy is automatically done by [the CircleCI pipeline](https://circleci.com/gh/fake-news-detector/api) after merging to master.

If you want to learn more how it works, read about deploying to heroku with docker [on the oficial docs](https://devcenter.heroku.com/articles/container-registry-and-runtime).

## Jobs

There are some jobs for helping administrating the app, for creating and running migrations, check out [diesel's docummentation](http://diesel.rs/). If you are running any diesel's job outside docker, you have to `export DATABASE_URL=postgres://postgres:password@localhost:5432/fakenews`.

Custom jobs are added on the `src/jobs.rs` file, and ran like this:

```
cargo run -- --scrape-missing-data
```

or, if the app is installed:

```
fake-news-api --scrape-missing-data
```

0 comments on commit b5f7a34

Please sign in to comment.
You can’t perform that action at this time.