Milou-backend

​ This is the repository for the backend which powers Milou, running on an Elastic Beanstalk environment on AWS. It includes a MongoDB database connection and can be interfaced via HTTPS requests. It is also fetching from different financial APIs such as Alpha Vantage, Finnhub, Benzinga and News API to retrieve information/news about stocks and companies. For a proper manipulation of different currencies real time rate data is collected from Rates API to convert prices to Euro. Furthermore, finAPI is used to enable users to connect their bank-accounts, which allows the application to import their real portfolios. Besides, the Analytics part of Milou is integrated in a sub-folder called data-analytics and one file called analyticsRoutes.js, which offers a connection to the Analyzer part of Milou.

Progress

​ What's not working yet?

• automatic tests for the routes need to be implemented
• the formula for calculating the portfolio score needs to be revised

Dependencies

​ Among others, we use express, swagger, passportJS, fetch and axios, aws-sdk, dotenv and mongoose.

Express is our main backend-application framework. We use Swagger 3.0 OpenAPI for the documentation of the API. PassportJS is used for user-authorization. Fetch is used by the backend application to retrieve data from other APIs, the analytics team uses Axios to do so. Aws-sdk and dotenv are used to access keys and secrets from AWS Secret Manager. Mongoose is used to access our MongoDB.

You can install our dependencies by running npm install.

Installation

​ Except for importing the secrets (next step), no major effort is required to run the project locally.

npm i
npm start

For local development you should keep in mind, that the application is connected to the database (if your aws-secrets are correct). Thus, you are able to run CRUD operations on the real database using Postman, for example. ​

Credentials

​ In order to connect to AWS and retrieve the secrets (in local development), the application needs to get the AWS credentials. It looks for them in the file ~/.aws/credentials on Linux/macOS and in the file C:\Users\USER_NAME\.aws\credentials. That file must contain aws_access_key_id and aws_secret_access_key. ​

Pipeline

​ As already mentioned, our code is deployed on an Elastic Beanstalk instance. To do so a Connection is set up, deploying changes on master within seconds using AWS CodePipeline. ​

Database

The backend's database is running on MongoDB Atlas. It is used to cache information about stocks, companies and currencies, enabling us to update analysis of portfolios, even tough we are limited in terms of calls per minute to our financial APIs.

It also stores the user profiles and all the portfolios of the users, as well as the analysis values of the portfolios, calculated by the analysis team.

All data is stored encrypted (MongoDB automatically encrypts all data), and passwords are stored hashed. ​

Cronjobs

These two endpoints¹ and these five fields² of stocks are updated every night at 4:00 am via CronJobWorker from Alpha Vantage API, because they are important for frontend and require daily updates. Other fields/endpoints of the stocks do not require daily update and hence information is static.

The information about stock values in the portfolio and the analysis of portfolios are updated every night at 2:00 am, in order to keep the values of the stock prices and of the performance chart up to date.

¹ Endpoints: /stocks/charts/historic , /stocks/charts/analysts
² Fields: price, per1d, per7d, per30d, date

Authentication

The Authentication of users can be viewed at auth/auth.js. It uses a generator as well as a secret to generate a JSON web token (JWT), which has to be passed as a bearer token in the header of requests, that should only work for signed-in users. This JWT token is valid for 24 hours, afterwards it has to be renewed.

Endpoints

Stocks

Endpoint	Parameters	Description	Example response
/stocks/list	country, currency, industry, mc	Returns a stock list according to filter.	Click
/stocks/search	id (can be name, ISIN1, WKN1, symbol)	Returns stock list with search parameter.	Click
/stocks/overview	id (symbol of a stock)	Returns a stock overview with given id.	Click
/stocks/details	id (symbol of a stock)	Returns details of a stock with given id.	Click
/stocks/charts/historic	id (symbol of a stock), max (all data points if true, else only last 5 years)	Get the performance of the stock from beginning or last 5 years.	Click
/stocks/charts/key_figures	id (symbol of a stock), max (all data points if true, else only last 5 years)	Get the key figures of the stock from beginning or last 5 years.	Click
/stocks/charts/dividend	id (symbol of a stock), max (all data points if true, else only last 5 years)	Get the key figures of the stock from beginning or last 5 years.	Click
/stocks/charts/detailed-analysts	id (symbol of a stock)	Get the analysts recommendation from Benzinga 2.	Click
/stocks/charts/analysts	id (symbol of a stock)	Get the analysts recommendation from finAPI.	Click
/stocks/news	id (symbol of a stock)	Returns news list with given stock symbol from News API.	Click
/stocks/balanceSheet	id (symbol of a stock)	Get balance sheet.	Click
/stocks/incomeStatement	id (symbol of a stock)	Get income statement.	Click
/stocks/cashFlow	id (symbol of a stock)	Get cash flow.	Click
/stocks/finanzen	id (symbol e.g. dax, sp500)	Get finanzen data.	Click

¹ Since ISIN and WKN are licensed and are not provided by any of free APIs, we mocked ISIN and WKN fields of stocks with random data in a proper format.
² Benzinga is on free student trial currently and valid until 19.04.2021. After this date, detailed-analysts endpoint will not be updated.

Portfolios

Endpoints for reading, modifying, creating and deleting portfolios. For accessing these routes, a valid access_token of a specific user must be passed in the header of the request. A user can view/modify only their own portfolios. Only virtual portfolios can be modified. These routes are used by the frontend portfolio application.

Endpoint	Parameters	Description
/portfolio/list	-	Returns all portfolios of the authorized user
/portfolio/details/:portfolioId	ID of a portfolio	Returns a global overview of a portfolio, positions, risk, analytics etc.
/portfolio/performance/:portfolioId	ID of a portfolio	Returns data points to display a performance chart.
/portfolio/create	Name of a portfolio to create	Creates a new, empty, virtual portfolio and saves the timestamp of the portfolio creation in the portfolio's history.
/portfolio/:portfolioId	ID of a portfolio to delete	A request to delete a portfolio with a given ID, if exists.
/portfolio/rename/:portfolioId	ID of a portfolio to rename, a new name	Renames a specific portfolio with a given name.
/portfolio/modify/:portfolioId	ID of a portfolio to modify, symbols of positions and a new quantity for them.	Modifies a virtual portfolio, if exists, by adjusting a quantity of specified positions and updates timestamp of the last change.
/portfolio/duplicate/:portfolioId	ID of a portfolio to duplicate, name for a new one.	Creates a new virtual portfolio as a duplicate of a real or virtual portfolio. Changes to a real portfolio will not be tracked in the duplicated version.
/portfolio/stock/:symbol	Symbol/name/isin/wkn (symbol preferred) of the specified stock	Get request. After adding a stock to his portfolios returns portfolio name and quantity of a specified stock for all portfolios of the authorized user.
/portfolio/stock/:symbol	Symbol/name/isin/wkn (symbol preferred) of the specified stock	Put request. Modifies a stock's quantity within multiple portfolios simultaneously.

Users

The User Routes enable the user to interact with his identity model, which consists of an email and a password. First- and lastname are optional and can be set later on via edit. In this part of the documentation it can be reviewed. Users can register, login and confirm their emails via a token that gets send, once registered. Also the token can be resent. If a password was forgotten, a reset process can be started, wich sends an email with a link in it, including a token used for verification of the user.

Besides that main user functionality also a user is able to search for banks and add, view, remove and refresh bank-connections.

Of course a user can delete the account, including all user-related data on the database, such as portfolios, user-information, bank-information as well as the account on finAPI.

Domains

​ The application can be found at:

• https://milou.io/

Our documentation can be viewed at:

• https://api.milou.io/api-docs