Luminaire to support higher dependent package versions

[shahsmit14]

Right now the dependent packages defined in requirements.txt file are to a specific version. So, we should test and support future versions too.

[snazzyfox]

Maybe consider using dependabot

[Aristarhys]

@shahsmit14 @snazzyfox

• Remove unused pyramid-arima dependency #74
• Remove unused pyramid web framework dependency #75
• Bump minor versions of dependencies #76

dependabot is nice but i think cleanup of existing deps should be performed

[Aristarhys]

@shahsmit14 to unblock python 3.9 scikit-learn https://github.com/zillow/luminaire/runs/1973722601?check_suite_focus=true wheel should building OK
Bumping scikit-learn will bump some other dependencies like numpy and scipy (cython should be installed also for never versions)
But probably it's possible to find minimal bump of scikit-learn and related dependencies for 3.9 which will not involve bumping everything 10+ versions (still supporting 3.6, 3.7 and 3.8) and will not break much in existing logic so we don't need to do full regression testing of lib manually

[rafaelleinio]

Hi! First of all great open-source tool you guys are developing!

Going straight to the point: I'm having trouble using luminaire because of this issue. The requirements are too specific, so I'm having a hard time installing luminaire with other libs in my project. A common block is that a lot of tools nowadays already require pandas >=1.0.0. But here we have a strict dependency on an old version of pandas. In my case, I'm trying to benchmark Luminaire and Prophet and I'm falling into these requirements conflicts.

Making requirements more flexible, so people could easily integrate luminaire in their projects, is in the roadmap for the next releases? While we don't have this issue fixed, how you guys recommend for me to test luminaire? Install it with no-deps and try to adjust dependencies on my own fork? 🤔

Thanks!

[sayanchk]

@rafaelleinio Thanks for bringing up the strict package dependency issue and we are aware of it. This is one of the highest priorities for the next major release. Another major issue we are planing to fix is to support higher python versions beyond 3.6.

At this moment there are couple ways you can test luminaire with other packages:

1. Install Luminaire in a different environment and perform testing.
2. Install luminaire without the dependencies and install the packages manually. This is something I have tried too and you should be able to upgrade many package versions from my experience (except sklearn probably which was throwing some build error).

[sayanchk]

@shahsmit14 @Aristarhys @rafaelleinio We have started supporting latest version of most of the critical dependency packages except scipy and statsmodels. I have created a separate issue to address those two libraries and closing this one: #98