Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Run chunked pipeline #1811
Running a pipeline in chunks can be more efficient when running over a long time period. This pr ultimately introduces a function to do just that, but other required functions are included.
Pipelines can sometimes have a column that is of type 'categorical', e.g. a country code. In order to combine the separate pipeline chunks, the categories of each column must be consolidated because pd.concat cannot concatenate two DataFrame columns with different categories. For this the function
To roll back input to the last trading day (inclusive), the function
Tests for each new function are included.