How to contribute to berryconda or new packages?

[step21]

I would like to contribute missing packages and make berryconda behave more like normal conda. Would it be ok to just open a PR?

[jjhelmus]

Thanks for your interest. Unfortunately, this project is no longer activate. I will not be updating recipes or adding and new packages to the rpi channel. I'll update the README to highlight the un-active status.

[step21]

Hey, thanks for your reply. I was actually thinking about making my own channel/installer at least for my own use... so if you do not mind or prefer I could also take over maintaining this repo and channel.

[deeplook]

Thanks for your interest. Unfortunately, this project is no longer activate. I will not be updating recipes or adding and new packages to the rpi channel. I'll update the README to highlight the un-active status.

That's really a pity! Would you be interested to share your experience with others who want to continue on the project?

[step21]

That would be great. I actually thought about starting my own channel... but prob better if @jjhelmus would be willing to pass on this project and the channel. Building installers with constructor and making packages already works for me. I actuallly also started a conda-build docker container that could be used to automate things. If we can get it to use balena based images, or include the compatibility layer manually, they could even run on standard ci/cd.

[mathematicalmichael]

@deeplook it'd be awesome if you could add @step21 as maintainer to the repo if you're not active anymore. there's a growing interest in using ARM in supercomputing and a robust conda is really a missing feature for many computational scientists interested.

[mathematicalmichael]

That would be great. I actually thought about starting my own channel... but prob better if @jjhelmus would be willing to pass on this project and the channel. Building installers with constructor and making packages already works for me. I actuallly also started a conda-build docker container that could be used to automate things. If we can get it to use balena based images, or include the compatibility layer manually, they could even run on standard ci/cd.

this is the dream.

[deeplook]

@deeplook it'd be awesome if you could add @step21 as maintainer to the repo if you're not active anymore. there's a growing interest in using ARM in supercomputing and a robust conda is really a missing feature for many computational scientists interested.

I'm not a maintainer of this repo. Or which one do you mean?

I did start this tiny one recently for more conveniently installing MiniConda, also on ARM using Berryconda (a new release coming out these days): https://github.com/deeplook/mcinstall

[step21]

I think he meant @jjhelmus - but do you know him or anyone else at anaconda by any chance?
Thanks for the pointer to your repo.

[deeplook]

@step21 I do know some people at Anaconda and will point them to this discussion.

[deeplook]

What I understood is that Anaconda is very interested in the ARM platform, but at the moment only with external funding and not alone. It's kind of hard not to see the potential of a sophisticated, standardized AI/ML stack on a number of 100M or so existing smartphones with 4-6 cores. Maybe we should run a Kickstarter campaign to raise $100K say?

[step21]

@deeplook Thanks for that insight. At the moment, I wouldn't need funding, but obviously it is much easier if they would just continue to host the rpi channel on anaconda cloud.

The issue with your use case probably is that rpi stuff doesn't run on phones, and especially sth like conda or python AI/ML is really hard to transfer to phones, even if berryconda was optimized for the arm cores used in phones.

[mathematicalmichael]

@deeplook apologies for the misunderstanding and big thanks for the clarifications + insight!

@step21 I'd love to know more about why the leap from rpi to android is difficult, is it just that very little has been done to allow support of the projects for that architecture? or some more fundamental challenge?

One thing that comes to mind is webassembly... If it can run in-browser, maybe there's more promise in having the AI/ML stack support that instead?

I have some experience in high-performance scientific computing and I suspect there are enough unused state-of-the-art-but-kinda-old servers in basements of universities that could provide more value in the immediate term than the investment required to transfer architectures (i.e. maintain old intel for longer, make it more accessible, rather than build out capabilities in hopes of distributing to cellphones).
I know from personal experience that at my university, we have better resources than we have people to support them, or finances. Total lack of knowledge of the software challenges involved, so lots of servers just sit under-utilized, not fulfilling the promises of the grant that paid for it.

[step21]

@step21 I'd love to know more about why the leap from rpi to android is difficult, is it just that very little has been done to allow support of the projects for that architecture? or some more fundamental challenge?

Well it depends on what we're talking about - for berryconda we're talking about a full python environment, with some native code in certain dependencies, that in itself is not suited to run on phones and would blow things up considerably for any app. What can work, you create and train your model with berryconda/python, then load that model into the requisite app. That can then run on node, in browser or inside an app.
Regarding architecture, I know that at least some parts, phone chips have specific optimizations that are not necessarily present on rpi - so to use them, you have to code or compile for them, probably.
Example of AI/ML running in browser: https://github.com/chaosmail/caffejs

[jjhelmus]

I've created issue #83 for those who are interested to discuss the future of Berry conda. I am happy to provide some guidance on how the existing packages were built and how this might be done differently.