publish a new release on pypi?

[bdewilde]

Given the huge number of changes to dragnet since the last release (v1.1, Jan '17), are there near-term plans to publish a new release on PyPi? I see that the failing test got removed, but don't know if there are other critical to-do's before everything's release-ready.

[pakelley]

I'd like to get up those models I said I'd train in #52 (thinking I'll have that done today, but maybe tomorrow), but aside from that there's nothing I can think of that needs to be done before a release

[pakelley]

@bdewilde , I'm using the GridSearchCV you added in to try and get some decent params for these models, but when I try to run the grid search with more than one worker I get an error about not being able to pickle the lxml Element objects that are passed into fit. (I can copy in a stack trace, etc. if it's helpful)

From what I can tell, this is expected behavior for multiprocessing with joblib, but do you know of a way around this? Do you remember this being an issue for you when you were working on your changes?

btw, thanks a ton for all your contribution here!

[bdewilde]

Glad that the updated model training code is being put to good use! Unfortunately, I don't have a solution for this problem — I never tried training in parallel (n_jobs > 1, right?), and instead just bit the bullet on training time. Just checking: Is the latter option prohibitively slow?

[pakelley]

Yeah exactly, n_jobs>1, but no it's not terribly slow, just wondering if there's an obvious solution I was missing. I'll get some models trained and uploaded asap!

One more thing on the models: would it make sense to use LFS to commit them?

[bdewilde]

re: git-lfs: I'm not sure. How large are the pickled models? I remember the updated ones being larger than the old ones, but not the absolute values. If the files are > 100MB or so, I think git-lfs may be a good idea, but since we won't be modifying these models often, it probably won't make a huge difference in developers' lives.

[pakelley]

The models are only 40-50MB, so that seems fine to commit them normally. I wasn't sure what a good threshold for that was, but if it seems good to everyone working on this it's good with me.

[nehalecky]

@bdewilde / @b4hand. With #63 and #65 merged, we're now good with a new release of dragnet being published to pypi. I recall a discussion of someone around here having access to the dragnet pypi account, but don't recall who specifically. Any help here would be appreciated, thanks! And thanks @pakelley for the effort getting the python 3 port to be release-ready!

[b4hand]

I currently don't have access to the pypi release for that project, but it appears that @dlecocq and @matt-peters do. Could one of you grant me access?

[dlecocq]

That'll have to be @matt-peters - it doesn't appear that I have access, either.

[nehalecky]

thanks for the replies, all. @matt-peters, let us know if we can help. We look forward to this so we can streamline our dependencies! Also, FWIW, we're also interested in maintaining a conda package per release, and will be submitting a pull request for that in the near future. :)

[matt-peters]

I'll dig up the pypi access information this evening and should have a chance to do a new release tomorrow. @b4hand We can also coordinate to grant you access.

[matt-peters]

OK, new version (2.0.2) is available on via pip. Check it out and let me know if it you run into problems.

[b4hand]

It looks like the version in GitHub wasn't actually bumped in setup.py. I'll go ahead and update that as well.

[nehalecky]

hey @b4hand and @matt-peters, great to hear! FWIW, the changes made in #63 #52 by @bdewilde and @pakelley we're substantial, including breaking external API changes. I don't think we mentioned it specifically, however, with semver, you're likely going to want to do a major version bump! Dragnet 3.0.0, now supporting Python 3! :)