URGENT HELP - conda distribution

[pycaret]

upload pycaret on conda cloud so that people can do conda install just like pip install.

[fcakyon]

tried to add auto-publish to anaconda cloud via github actions in my fork but having conflicts between package versions https://github.com/fcakyon/pycaret/runs/611534708?check_suite_focus=true

[pycaret]

@fcakyon Can you please try uploading pycaret-nightly on conda and let me know if that works?

[pycaret]

URGENT HELP NEEDED.

Can somebody please create a conda distribution for pycaret==2.0 so that users can also install pycaret using "conda install pycaret". Please reach out on LinkedIn or email moez.ali@queensu.ca if discussion is needed.

[HenriqueAJNB]

Can you assign me this issue? I'll make my best to solve it.

[pycaret]

Assigned. Thanks for contributing.

[pycaret]

@HenriqueAJNB Any progress made on ths?

[HenriqueAJNB]

I tried to add the pycaret to anaconda but I couldn't.

The main reason is because all dependencies must also be uploaded to conda, otherwise it will return dependency error...

[HenriqueAJNB]

I'll list those dependencies that are not in conda environment yet

[HenriqueAJNB]

I followed those steps at building conda packages with conda skeleton and create the meta.yaml file.

The next step was to run conda-build where meta.yaml was created. I've got this error related to package dependencies:

... raise ResolvePackageNotFound(bad_deps)
conda.exceptions.ResolvePackageNotFound:
  - mlflow
  - textblob
  - lightgbm[version='>=2.3.1']
  - imbalanced-learn
  - catboost
  - wordcloud
  - pyldavis
  - pyod
  - cufflinks[version='>=0.17.0']
  - mlxtend
  - umap-learn
  - pandas-profiling[version='>=2.3.0']
  - xgboost[version='>=0.90']
  - datetime[version='>=4.3']
  - kmodes[version='>=0.10.1']
  - yellowbrick[version='>=1.0.1']
  - datefinder[version='>=0.7.0']
...
raise DependencyNeedsBuildingError(exc, subdir=subdir)
conda_build.exceptions.DependencyNeedsBuildingError: Unsatisfiable dependencies for platform win-64: {'wordcloud', "lightgbm[version='>=2.3.1']", 'imbalanced-learn', "cufflinks[version='>=0.17.0']", 'pyod', 'catboost', 'mlflow', "kmodes[version='>=0.10.1']", "pandas-profiling[version='>=2.3.0']", 'textblob', "xgboost[version='>=0.90']", 'umap-learn', 'mlxtend', "yellowbrick[version='>=1.0.1']", 'pyldavis', "datetime[version='>=4.3']", "datefinder[version='>=0.7.0']"}

After some research, I found, with help of my opened a question in Stack Overflow community, the best practices about external dependencies in conda-forge channel documentation.

Avoid external dependencies
As a general rule: all dependencies have to be packaged by conda-forge as well. This is necessary to assure ABI compatibility for all our packages.
There are only a few exceptions to this rule:

1. Some dependencies have to be satisfied with CDT packages (see Core Dependency Tree Packages (CDTs)).
2. Some packages require root access (e.g. device drivers) that cannot be distributed by conda-forge. These dependencies should be avoided whenever possible.

I discussed this issue with @moezali1 and my final suggestion is:

Check if the listed dependencies can be dropped/removed or changed for some that have been already uploaded into conda (without affecting the package, of course).

Any other suggestions?

[murari-goswami]

@HenriqueAJNB Can - you try conda-forge instead ?

[HenriqueAJNB]

I'm generated the meta.yaml file with grayskull pypi pycaret command instead.

And the great news are that conda-forge could identify almost all of the listed packages, with just one exception:

• datetime >=4.3

Now it's much easier to deal with just one exception not installed in conda-forge.

I really don't know the reason conda-build was throwing all those exceptions.

Thanks @murari-goswami

[pycaret]

@HenriqueAJNB This is great progress. Do you think it would be a big deal to upload datetime in conda-forge. Unfortunately datetime parsing and feature engineering in preprocess.py has dependency on datetime package.

[synapticarbors]

pycaret is already available on conda-forge:

https://anaconda.org/conda-forge/pycaret
https://github.com/conda-forge/pycaret-feedstock

You can see the recipe here:
https://github.com/conda-forge/pycaret-feedstock/blob/master/recipe/meta.yaml

If you would like to be a co-maintainer, please open a pull request there.

[HenriqueAJNB]

Someone uploaded it 3 days ago. I've tested it in Windows platform and still not working. Can someone test it in Linux platform?

I tried using python 3.7.

[synapticarbors]

I've tested on OSX and Linux and both work fine. The issue with windows is due to xgboost not being available on that platform on either the conda-forge or anaconda defaults channels.

For some reason the recipes on conda-forge does not include DateTime as a dependency. I've grep'd through the code in the tarball on pypi for pycaret for both import DateTime and from DateTime and couldn't find either in the source code. I'm not familiar with the package so I'm not sure if it's being used or called in some non-typical way. That said, I've opened a pull request to add it to conda-forge:

conda-forge/staged-recipes#12390

Once it's available on the conda-forge channel, we can open a PR on the pycaret-feedstock to fix the dependency if needed.

[HenriqueAJNB]

I send this issue to Google Groups "conda-forge" group to ask for any help. Here is the answer of Mr. Chris Barker:

If I followed the issue thread correctly, all you are missing is "datetime". Is this it?
https://pypi.org/project/DateTime/
If so, that hasn't seem a release in almost 2 years, and it sure looks like it duplicates what is provided by other more common packages, including the stdlib datetime.
So I'd see if you can remove the dependency altogether, in favor of the built in datetime, or one of the other widely used time handling libs.
Otherwise, it should package up fine for conda-forge. This is how you do that:
https://conda-forge.org/docs/maintainer/adding_pkgs.html
-CHB

Is it possible to change the actual DateTime package used for the built-in one? This would solve the problem in Windows platform.

[synapticarbors]

The DateTime package is not the problem on windows. See my comment above (#21 (comment)).

[pycaret]

@synapticarbors @HenriqueAJNB Thank you for your patience.

I have now removed DateTime from requirements.txt.

Hope it's possible to build conda now?

[HenriqueAJNB]

I've tested on OSX and Linux and both work fine. The issue with windows is due to xgboost not being available on that platform on either the conda-forge or anaconda defaults channels.

The dependency on DateTime package is solved.

Now I need to deal with xgboost mentioned by @synapticarbors in comments above. I still can't install xgboost on Windows platform because is not available in any conda channels

There is another package with same name that seems to work on Windows and I could install it normally. It's called py-xgboost. See this question in Stackoverflow for more details.

How could I deal with xgboost vs py-xgboost on Windows? Any ideias?

[pycaret]

Does this help: https://anaconda.org/conda-forge/xgboost ?

[HenriqueAJNB]

Does this help: https://anaconda.org/conda-forge/xgboost ?

This link just confirm that xgboost is not supported in Windows platform. See image below

If you check other package (pandas for example) there is a symbol indicating that it is supported in Windows:

[pycaret]

@Yard1 Should we hold on this until the xgboost issue on conda is sorted. I don't think we should shift to py-xgboost. I don't think update cycles for py-xgboost and xgboost is in sync. Your thoughts?

[Yard1]

Agreed, I'd rather not try to support both libraries. We should be fine with just a Linux conda for pycaret right now, tbh

[jamesmyatt]

py-xgboost is the right conda dependency for a python library. xgboost is just the C library, as far as I know.

[Yard1]

I believe that to be incorrect. pip install xgboost does work and is separate from py-xgboost. If I recall correctly, the latter was a third party project to create a Python wrapper around Xgboost, before it itself started providing one.

[pycaret]

@jamesmyatt I also think that is incorrect. xgboost is the right dependency and is unfortunately not available on conda hence became a show stopper for pycaret to be conda install compatible. In the future, we are planning to slim down our mandatory dependencies in which case this problem will be resolved since we will remove xgboost and a few other libraries from our requirements.txt.

[jamesmyatt]

So I actually read the conda recipe: https://github.com/conda-forge/xgboost-feedstock/blob/master/recipe/meta.yaml, and it shows that the following commands will give you the same xgboost python module (assuming they're all at the same version for your platform):

• pip install xgboost
• conda install -c conda-forge py-xgboost: https://github.com/conda-forge/xgboost-feedstock/blob/c3e34bfea8d6b9d4250398ad37f475f593f7d0ed/recipe/meta.yaml#L54
• conda install -c conda-forge xgboost: https://github.com/conda-forge/xgboost-feedstock/blob/c3e34bfea8d6b9d4250398ad37f475f593f7d0ed/recipe/meta.yaml#L82
• conda install -c defaults py-xgboost

i.e. the conda-forge xgboost package just install the py-xgboost package, which, in turn, builds and installs the python setuptools package directly from https://github.com/dmlc/xgboost. So I'd say that the correct conda dependency is py-xgboost (since it's also in defaults for people who prefer defaults), but using xgboost works too, but only with conda-forge, since it's provided as a convenience. You can't assume that pypi and conda package names are always the same, unfortunately.

The C version is called libxgboost, so I was wrong about that.

The conda-forge xgboost packages are linux and osx only though. But I agree about silmming down the requireents.

[Yard1]

I see. Well, that does clear it up. We should use that, then.

[pycaret]

@Yard1 Does this mean pycaret can be made available on conda now?

[Yard1]

@pycaret I believe so, though this is not my area of expertise.

[pycaret]

Hi All,

@HenriqueAJNB @Yard1 @jamesmyatt

Long time - Since now xgboost is no more a dependency in requirements.txt, can we look at this issue again and try to make pycaret available on conda?

[jamesmyatt]

xgboost is not a problem on conda-forge win64 anymore.

[pycaret]

Thanks, @jamesmyatt for the update.

@HenriqueAJNB Are you still available to work on this issue?

[HenriqueAJNB]

Thanks, @jamesmyatt for the update.

@HenriqueAJNB Are you still available to work on this issue?

I'm not available at moment

[pycaret]

@HenriqueAJNB Thanks for letting us know.

[pycaret]

HELP WANTED

[iki77]

Pycaret can be installed using conda from conda-forge channel since 9 months ago
https://anaconda.org/conda-forge/pycaret

[melonhead901]

Here's the recipe used for conda-forge
https://github.com/conda-forge/pycaret-feedstock/blob/master/recipe/meta.yaml

[jmakov]

Is it possible to get a pre release (full) version also via conda?

[jmakov]

The last conda version is 3.0.4. Thought that was solved already.

[melonhead901]

The latest version is now also available on conda-forge.

[jmakov]

@melonhead901 thanks. Is the plan to also officially support conda-forge?

[moezali1]

This is no more an issue. We officially do not have conda builds.

https://anaconda.org/conda-forge/pycaret