New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Versions and Releases - New Standards #223

Closed
joewheaton opened this Issue Oct 28, 2018 · 10 comments

Comments

Projects
None yet
5 participants
@joewheaton
Copy link
Contributor

joewheaton commented Oct 28, 2018

Hey @banderson1618. Just a heads up that I DO NOT think we should be tagging every release as 'Latest Release'. Lets reserve that for releases we are confident in (i.e. we've completed testing on) and appear to be stable.

Instead, please flag every Pre Release that we do internally. When we are finally happy with testing for the BRAT 3.0... we will release a 3.1.0 that we will flag as latest release. If we do testing on releases to the point we're happy with them, that's fine we can release a 3.1.x as Latest. But on all these releases that we are making for our own purposes, just flag them as Pre Release as to not give any of the users out there the wrong idea that they are supported.

For now, I've put the Latest Release back to BRAT 3.0.1. I know there are a bunch of improvements that we're confident in, but I don't want to mislead users.

This was prompted from @ctobalsk who reasonably inquired:

Hey Joe,

I’m currently re-running BRAT for Montana using the BRAT I downloaded on October 19th. It runs really fast (compared to 2.0!). I did get slightly worried by your GitHub answer ([…] “until we get through testing the beta releases of 3.2 and we are happy with the outputs to the point that code development has slowed down”).

Is there any chance that BRAT will change enough, within the next few weeks, that outputs would be significantly different from what I’m getting now? If that’s the case I’ll slow down my reruns; we don’t want to release a statewide model to the BLM and FWP and have it be obsolete within a few weeks!

Thanks,

Claudine

Claudine, FYI we're pretty confident in where things are getting to with these 'pre-releases' or BETA stuff, but we've not yet tested them thoroughly nor have we gotten to the documentation. Not that we ever actually give any warranty for open-source software (you get what you pay for), but I don't want to pretend that we have the funding or staff to support other people's timelines. We will move at the pace we move at relative to our paying sponsors and grants and contracts. We take it upon ourselves to make this all freely available in the spirit of transparency and sharing. However, our budgets are skimpy, we're an entirely soft-money funded shop, and we can only do what we have time and budget to do. I'm sorry we were flagging these as 'latest releases' . I've rectified that now.

As for

Is there any chance that BRAT will change enough, within the next few weeks, that outputs would be significantly different from what I’m getting now?

There will be some additional pre-releases but I can't guarantee they'll be finalized. I think the safest thing to do is just be clear what version of BRAT you are delivering to them is from. There is never a 'final', just a 'latest and greatest' version or realization.

And as for your concern:

If that’s the case I’ll slow down my reruns; we don’t want to release a statewide model to the BLM and FWP and have it be obsolete within a few weeks!

BRAT capacity runs won't be obsolete. The proper thing for you do is just report what version you ran it off of. Please note that that the capacity model is not fundamentally changing. Almost all the changes you will have noticed there are strictly to make pre-processing and running and housekeeping more efficient. Most of the development improvements and real changes are on the 'management' outputs and layers. I would NOT suggest you deliver those (or at least don't highlight them) to your client. I saw the RFP from BLM you responded to and I think they just asked for the BRAT capacity runs.

@banderson1618 confirm this makes sense to you, post any questions/comments, but otherwise close issue.

@joewheaton

This comment has been minimized.

Copy link
Contributor Author

joewheaton commented Oct 29, 2018

@banderson1618 and @wally-mac please check out this new Internal Release Standard and edit if necessary. Add comments/questions here. I want to start giving folks examples of moving things we've made decisions on from issues into documentation...

@banderson1618

This comment has been minimized.

Copy link
Collaborator

banderson1618 commented Oct 30, 2018

While I like the direction that this takes us in, I think that we need to formalize a system of testing pyBRAT if we're going to require testing to release it. Even something as simple as "BRAT needs to be run on these three watersheds by at least two people on different machines before being released" would be a significant improvement.

If we're doing things this way, I wonder if there would be a way to set one branch as our "Release" version, and one branch as our "Internal" version. That way, we could implement quick hotfixes (if necessary) without having to include features that are still in development. I'll have to look more into whether or not this is possible, but if it is, I think it'd be good.

@banderson1618

This comment has been minimized.

Copy link
Collaborator

banderson1618 commented Nov 5, 2018

@joewheaton, what testing steps do you want to be taken before a new release is officially accepted? @wally-mac, @bangen, @Albonicomt, and @MatthewMeier probably have some input on this as well, so I'll tag them as well in this.

I'd like to put our latest release as 3.0.17, since that's the most recent build before the management model changes were made (I think). What are the testing steps should we take to test this release?

@ctobalsk

This comment has been minimized.

Copy link

ctobalsk commented Nov 5, 2018

From a user's perspective, it would be extremely useful to have something on the download page that gives an idea of the timeframe of updates. What I mean is, my version (3.0.20, downloaded October 15) is gone and replaced by 3.0.01 (which I assume is a more stable version). However my understanding is that there are changes in the making, especially with regards to the management model, and an upcoming 3.1 version. Since I'm going to be running BRAT for 97 HUCs in Montana and 13 HUCs in South Dakota, I would hate getting started only to have some of the modules change part way through. Thanks!

@banderson1618

This comment has been minimized.

Copy link
Collaborator

banderson1618 commented Nov 5, 2018

@ctobalsk, that's certainly fair. You caught us at a bad time, unfortunately, because we're switching between management models at the moment. Hopefully we'll be able to get this resolved soon, but getting everyone on one page is always difficult, in my experience.

@wally-mac

This comment has been minimized.

Copy link
Collaborator

wally-mac commented Nov 6, 2018

@banderson1618, I think that version 3.0.17 has been tested by @MatthewMeier and @Albonicomt in the Brunt River and John Day Basin in Oregon and the Middle Columbia-Hood in Washington. Correct me if I'm mistaken Matt and Mic. If I'm correct then 3.0.17 has been run on three watersheds by two people on different machines.

@MatthewMeier

This comment has been minimized.

Copy link
Contributor

MatthewMeier commented Nov 6, 2018

@wally-mac
@Albonicomt and I have ran BRAT on these watersheds with version 3.0.19 and I feel like this is the latest and best working version. Changes to the new management model have been currently underway and are not included in this version but I assume will be in the next version. All of the edits that @joewheaton , @banderson1618, and myself conducted for the management model are post version 3.0.19 but I would say this is the version that we have been able to truth the most lately.

@banderson1618

This comment has been minimized.

Copy link
Collaborator

banderson1618 commented Nov 6, 2018

@MatthewMeier, thanks for the info. I'll go through 3.0.19 and see if it does what we need. If everything checks out, @joewheaton, would you be OK with us setting 3.0.19 as our latest version?

@banderson1618

This comment has been minimized.

Copy link
Collaborator

banderson1618 commented Nov 7, 2018

@joewheaton, v3.0.19 seems to fill our needs. It has the management layers, and it seems stable, according to Mic and Matt. If you don't respond in the next few days, I'm planning to set that as our most recent stable release, as long as @wally-mac agrees.

@wally-mac

This comment has been minimized.

Copy link
Collaborator

wally-mac commented Nov 7, 2018

@wally-mac wally-mac closed this Jan 17, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment