Api changes

[eddiebergman]

A major API overhaul to make cleaner separation between different OS's and to make it easier to document and understand all the things going on.

For reviewing, I'd highly recommend just reading the code outside of the github differ as there no diffable changes.

The README.md is a good spot to start to see the changes/features.

[eddiebergman]

It appears so and I don't know why, would you be able to try it out? I removed all occurences of time.process_time yet mac still seems to timeout

[mfeurer]

It appears so and I don't know why, would you be able to try it out? I removed all occurences of time.process_time yet mac still seems to timeout

Hey, how about we get going with a version that doesn't yet include OSX and make OSX a second PR? Keeping an overview in this one is becoming hard.

[eddiebergman]

I agree, I removed mac from the tests. Windows and Linux should pass and then we can merge and leave for another time.

[eddiebergman]

@renesass @mfeurer Finished, mac testing is disabled for now and a note was added to the README. I think it's ready for merging if you are both okay with it. I know auto-sklearn has the version locked <0.7 but there might be other libraries effected. We should warn them before doing any merge?

I would also propose to make it a 1.0.0 release?

[eddiebergman]

Here is a search within our org:
https://github.com/search?q=pynisher+user%3Aautoml&type=Code&ref=advsearch&l=&l=

There are a few libraries new and old that use it, most don't have any upper bound on Pynisher so I would go and raise an issue with each of them that there is major API breaking changes.

[renesass]

Sounds good. But let me fix the Mac tests. I think it shouldn't be much overhead. Is tomorrow fine?

[eddiebergman]

Sounds good. But let me fix the Mac tests. I think it shouldn't be much overhead. Is tomorrow fine?

Sure, You'll have to re-enable them in the pytest.yml once you're done.

[renesass]

And we definitely should push for 1.0.0. :)

[eddiebergman]

The autopytorch team currently have multiple branches and version for benchmarking, I left an issue but I would hold off with merging until they are done as it would break all the experiments.

[mfeurer]

Sounds good. But let me fix the Mac tests. I think it shouldn't be much overhead. Is tomorrow fine?

Sure, You'll have to re-enable them in the pytest.yml once you're done.

Sound good to me, too. I'll wait then with the final review until I get an okay from you.

The autopytorch team currently have multiple branches and version for benchmarking, I left an issue but I would hold off with merging until they are done as it would break all the experiments.

We don't have to immediately release the new version and can give them a bit of time. Also, as they use the pynisher via SMAC, the next SMAC version could introduce an upper bound on the pynisher version to simplify migration.

[eddiebergman]

Using this code search filter, there's about 10 libraries in AutoML that need to be informed as they explicitly state a dependancy on it

https://github.com/search?q=pynisher+user%3Aautoml+filename%3Arequirements.txt&type=Code&ref=advsearch&l=&l=

[mfeurer]

I'd say only HPOLib1.5 is a real library, but that's neither used nor maintained any more.

[renesass]

I'm still not really satisfied that in the MacLimit we raise a RuntimeException now if memory is not supported.

[eddiebergman]

I'm still not really satisfied that in the MacLimit we raise a RuntimeException now if memory is not supported.

I'm not sure what we should do to be honest, I good application will check support before trying to pynish anything using support. Otherwise, I think it's better we fail then to silently pretend anything is limited. Open to suggestions though

Btw, So happy to see those mac tests pass, just pre-commit and we're good :)

[eddiebergman]

I'm going to make this a 1.0.0 release just to signify the API changes and the support for windows and Mac. I will ask the autopyotrch team today about when is a good time to release, as they would have to update all of their branches they're benchmarking.

[mfeurer]

@renesass would this be ready for a final review?

[renesass]

I had a quick look and I'm fine with how it is rn :)
I will prepare an update for SMAC accordingly.

[eddiebergman]

Got the go ahead to go break AutoPytorch, we can merge whenever we're ready

[renesass]

SMAC is ready to merge.