Data preprocessing and feature engineering 🚜

[KarelZe]

• Write proposal for feature sets based on EDA / literature research
• Write script for feature engineering
• Create more features from quote and price data. See e. g., Rosenthal or Prado
• Combine relative bid and ask into one measure. Look at distribution first. Look at information density feature found Rosenthal paper.
• Perform adversarial validation on newly created feature sets e. g., with min-max-scaling, w /o log-transform etc. and get feature importances
• Create features that are hard to learn for neural nets and gradient boosting machines
• When loading data, verify wandb hashes
• Add economic intuition to each feature. Which paper suggest the feature and why does it make sense? (Feedback from @CaroGrau). Research why log-transform on prices makes sense from a theoretical perspective?