New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
TLS for VectorPostprocessors #7819
Conversation
@@ -47,5 +48,8 @@ VectorPostprocessor::getVector(const std::string & vector_name) | |||
VectorPostprocessorValue & | |||
VectorPostprocessor::declareVector(const std::string & vector_name) | |||
{ | |||
return _vpp_fe_problem->declareVectorPostprocessorVector(_vpp_name, vector_name); | |||
if (_vpp_tid) | |||
return _thread_local_vector; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How are you envisioning these being used and joined by users?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Exactly as they are in UserObjects and Postprocessors. We already have a threadJoin() routine that's being called on these objects. The content of these extra vectors is not important. It's just there as scratch space as the VectorPostprocessor executes.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There needs to be one local vector for each vector_name
, right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed
@@ -47,5 +48,8 @@ VectorPostprocessor::getVector(const std::string & vector_name) | |||
VectorPostprocessorValue & | |||
VectorPostprocessor::declareVector(const std::string & vector_name) | |||
{ | |||
return _vpp_fe_problem->declareVectorPostprocessorVector(_vpp_name, vector_name); | |||
if (_vpp_tid) | |||
return _thread_local_vector; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There needs to be one local vector for each vector_name
, right?
Also: there should be a test or two showing how to use the new functionality. |
Yes - we definitely do need multiple vectors. I'll also add a test object. |
So I decided not to add a dummy test object for this. I reverted Daniel's "fix" instead so we have a real object and a real test in the framework already with the Histogram VPP. Also, I rewrote SamplerBase to get rid of all the temporaries so we can verify that this all works with threads properly. |
The revert of the fix also stripped the |
Well you can't have your cake and eat it too, but apparently you "can has" all the donuts you want (and do want). I'll add those back into the separate fixup commit that already sits on top of the revert commit and repush. |
/// Values of the post-processors at the time t-1 | ||
std::map<std::string, std::map<std::string, VectorPostprocessorValue*> > _values_old; | ||
std::set<std::string> _requested_items; | ||
std::set<std::string> _supplied_items; | ||
}; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this for a planned dependency resolution?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes and no. Right now there's already dependency resolution on the objects themselves because they are UserObjects and sorted in the warehouse. The intention for this was to add in a new sanity check for the vectors themselves if you couple to vectors, there is no check on whether a vector that you couple to exists right now. I started on this but need to flesh it out quite a bit more and want to unify the way it works between VectorPostprocessors and Postprocessors. I can either continue to work on this or continue that work in another PR. I kind of want to handle it separately since this is already getting to be a substantial PR.
@permcody BTW: I'm somewhat abusing the fact that there is no sanity checking on VPP vector declarations for my own purposes. I don't declare the vectors until Please don't undo my ability to do that 😉 |
Ha! Well your "we allow that" may have to take a back seat to having MOOSE do something intelligent for our users. We can still support late creation of vectors with additional Actions or something. At some point though we really should be able to check the sanity of the system before beginning execution. I already got bit on this on one of my first uses of VectorPostprocessors since I renamed my vector during development and my coupled object worked just fine and drove me mad while I tried to figure out while I only had zeros in my vector (or nothing at all). BTW - @YaqiWang abuses Postprocessors in a similar fashion so I don't have satisfactory checks like I want to. I'm supportive of late declarations, we might just need to design that in to the system so that we can still check them. |
Sounds good. I was just throwing the idea out there. Since there aren't On Tue, Oct 11, 2016 at 2:41 PM Cody Permann notifications@github.com
|
421472f
to
56cb41b
Compare
/// sample count per bin | ||
VectorPostprocessorValue _count_tmp; | ||
std::vector<unsigned int> _counts; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note to reviewer: I think we should avoid creating VectoPostprocessor instances in our objects. We may eventually want to make the constructor private like we have for Material and Variable containers to help prevent users from missing the reference in normal declarations. This particular vector is not a declared vector so I switched it to C++ vector.
After this is merged we can go through the existing samplers and remove temporary vectors or other workarounds to not having TLS. refs idaholab#7809
This reverts commit 513c5df.
Also add override keywords
@friedmud - care to review this? |
Looking |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm marking this as Request changes
... but really it's ok the way it is.
I made a few suggestions... but it's mergeable as is.
Overall this is a really great leap forward in code quality in this area. Thanks for taking the time to do it really well!
@@ -31,6 +31,12 @@ class VectorPostprocessorData : public Restartable | |||
*/ | |||
VectorPostprocessorData(FEProblem & fe_problem); | |||
|
|||
struct VPPVectors |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe name it VectorPostprocessorState
or something like that?
|
||
FormattedTable & table = _vector_postprocessor_tables[vpp_name]; | ||
auto table_it = _vector_postprocessor_tables.lower_bound(vpp_name); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This set of 3 lines could use a bit of documentation. Took me a minute to figure out that you're just looking for or adding the table.
I mean: is this really worth removing the "accidental" insertion side effect? The old code was so simple...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's a fair criticism. I'm very, very tainted on RHS use of std::map::operator[]. I would say that I track down a bug about every 4-6 weeks that's related to careless use of that operator. I'd almost rather eradicate it's use anywhere in MOOSE but the syntax is very convenient which makes that hard to do.
This particular case is actually one of the few cases where RHS brackets are OK because we want to always create tables so I'm torn. In general, this is the syntax I'd like to move to because for the normal access case, you either add a straight up error when the item isn't found or often just an assertion when you are "sure" the item is there (yeah right...)
Here's another crazy idea... We use maps a lot in MOOSE. Maybe it's time to think about replacing our use of maps with a better container. We could start with just aliasing our container to std::map but quickly move to hiding implementation details such as the code you see there behind familiar operators like brackets. I've been using this lower_bound
code lately because it's the best way to check for an item and insert if if necessary because you can use hint
. Using find
doesn't work because you always end up with a iterator to the end of the map meaning you have to do two tree searches to handle the insert case. Is it premature optimization? Maybe, but maybe not. I'm starting to put these in as I fix accidental insertion.
|
||
for (unsigned int i=0; i<vector.size(); i++) | ||
for (auto i = beginIndex(vector); i < vector.size(); ++i) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like this!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I do but I think we are in the minority. No other MOOSE devs are using this syntax, not even @dschwen. I wonder if we can do better still.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I used it in 513c5df after you made a comment to that effect :-D. It might just take a bit to catch on. I suggest we start converting loops (if we decide to stick with it). Seeing it in action in existing code may improve adoption.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks to me should be a method in vector, vector.begin_index(). This beginIndex looks ugly although correct.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, sadly there is no method for creating an enumeration. There are other options and they are all uglier.
auto i = decltype(foo)(0)
or if you don't have a const type
decltype(foo) i = 0
|
||
table.clear(); | ||
table.outputTimeColumn(false); | ||
|
||
for (const auto & vec_it : vectors) | ||
{ | ||
VectorPostprocessorValue vector = *(vec_it.second); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yikes! Was this doing a copy before?!?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yep
_found_points.assign(_points.size(), false); | ||
|
||
_point_values.resize(_points.size()); | ||
std::for_each(_point_values.begin(), _point_values.end(), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Well done
#ifndef NDEBUG | ||
for (const auto vec_ptr : vec_ptrs) | ||
if (vec_ptr->size() != vector_length) | ||
mooseError("Vector length mismatch"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah - it would nice to be able to actually handle this... like only sort the values that exist or something. But it's an edge case that can't happen with any of our existing objects that use this interface right now.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That'll take a different data structure because we don't have an association of which items go with which ids if the lengths are different. That's a problem for a different day.
_z[i] = _z_tmp[index]; | ||
_id[i] = _id_tmp[index]; | ||
// Swap vector storage with sorted vector | ||
vec_ptrs[i]->swap(tmp_vector); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Very nice.
VectorPostprocessorData::getVectorPostprocessorHelper(const VectorPostprocessorName & vpp_name, const std::string & vector_name, bool get_current) | ||
{ | ||
// Intentional use of RHS brackets on a std::map to do a multilevel retrieve or insert | ||
auto & vec_struct = _values[vpp_name][vector_name]; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You do it here and it's ok ;-)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Doing a double layer insert is really, really ugly with lower_bound
. It's actually more than 2*3. I tried it and thought it was atrocious.
|
||
if (pp_val == NULL) | ||
pp_val = &declareRestartableDataWithObjectName<VectorPostprocessorValue>(vpp_name + "_" + vector_name, "values_old"); | ||
_requested_items.emplace(vpp_name + "::" + vector_name); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This means that you will create a dependency on this VPP... even though it's only a dependency on the old value of this VPP.
That can lead to erroneous cyclic dependencies (like the kind we've had to clear up elsewhere).
I think that for now it's ok to just not note the dependency on the old value. It should always be available...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice catch, I'll fix this
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
oh wait... This might not be bad depending on how it's used. We actually need three different array, or at least one additional flag: these two for creating the sanity checking (making sure that supplied and requested are on parity), but we also need a flag or something to indicate whether or not a dependency should be created for a given pair. Since this implementation isn't finished, I'll leave it for now.
if (!vec_struct.current) | ||
{ | ||
mooseAssert(!vec_struct.old, "Uninitialized pointers in VectorPostprocessor Data"); | ||
vec_struct.current = &declareRestartableDataWithObjectName<VectorPostprocessorValue>(vpp_name + "_" + vector_name, "values"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yuck - does this have to be this way for "lazy" binding? It seems like the restartable data should be declared when the vector is declared.
(Note: I realize that the old code was doing the same)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm sure we can do better. This ...Data
class is isolated from users so I'm not worried about fixing this right now. These things should be on the radar later for refactoring opportunities.
Comments have been addressed. |
This PR should be ready to merge, any takers? |
@andrsd - Would you mind reviewing this PR and merging if appropriate? It's been sitting here for awhile. |
@permcody This PR breaks my application... Like I said: I need to be able to declare vectors after the solve. I thought this wasn't going to mess with that? Reverting this merge commit fixes my problem... How do you want to proceed? This is required for my application and my immediate deliverables. Can you see an easy path forward? |
I guess we need to get your App fixed and on our our testing list! I'm sure On Fri, Nov 11, 2016 at 3:44 PM Derek Gaston notifications@github.com
|
I get a segfault in opt. In dbg I get an error about a key missing in On Fri, Nov 11, 2016 at 7:41 PM Cody Permann notifications@github.com
|
Is that without threads? I'm surprised that anything changed for the On Fri, Nov 11, 2016 at 5:46 PM Derek Gaston notifications@github.com
|
Yep - without threads. On Fri, Nov 11, 2016 at 8:13 PM Cody Permann notifications@github.com
|
If you want to run it: https://github.com/friedmud/mocodile The test is: deterministic/parameter_study |
Here's what I get in
|
In dbg I get this:
|
That last one is spot on... there are no vectors for that VPP... because it hasn't declared any yet (this is |
The basic issue is that |
Trying to fix it - stay tuned |
After this is merged we can go through the existing samplers and remove temporary vectors
or other workarounds to not having TLS.
refs #7809
Ping @friedmud, @dschwen