Allow sampling of alternatives during prediction

[jiffyclub]

It's kind of limited sampling, not something you could use
during a location choice model. Alternatives may show up as
available for more than one chooser and alternatives may show up
more than once for a single chooser.

[jiffyclub]

@fscottfoti I noticed that mnl_interaction_dataset doesn't have replace=False when drawing samples so the same alternative can show up multiple times for the same chooser.

[fscottfoti]

That's definitely a bug. There shouldn't be any reason to sample the sample alternative twice.

[jiffyclub]

It's the same way in activitysim. I think we did it that way so that the same alternative can show up for different choosers, but it means the same alternative can also show up multiple times for the same chooser. To do this correctly I think we'd need to do a random draw with replace=False individually for each chooser.

[fscottfoti]

That makes sense. Unfortunately, I think we have to do a random draw with replace=False for each chooser. Is it a lot slower?

[waddell]

This needs to be an option. In some applications like shopping destination choice it is fine to re sample the same alternative destination. In others like residential location it may not be, depending on how the choice algorithm deals with competition, since a unit is generally constrained to only one household occupant.

Sent from my iPhone

On Apr 29, 2015, at 9:41 AM, Matt Davis notifications@github.com wrote:

It's the same way in activitysim. I think we did it that way so that the same alternative can show up for different choosers, but it means the same alternative can also show up multiple times for the same chooser. To do this correctly I think we'd need to do a random draw with replace=False individually for each chooser.

—
Reply to this email directly or view it on GitHub.

[jiffyclub]

@fscottfoti and I talked about this and we're explicitly not adding support for a situation like household location because it'd be a lot of work to have sampling while respecting supply constraints. So this sampling would only be used in situations where the alternatives are not supply constrained.

[fscottfoti]

@waddell I don't think there's any case where you sample the same alternative twice for a single chooser?

[waddell]

Not unless you have an aggregated alternative (residential units of a specific building type in a submarket) and are not accounting for the varying size of the alternatives in the specification of the model.

So I’d agree that sampling without replacement is the right general pattern, and that the other cases should be handled either in the utility function specification or on the choice algorithm.

Paul

On Apr 29, 2015, at 10:01 AM, Fletcher Foti notifications@github.com wrote:

@waddell https://github.com/waddell I don't think there's any case where you sample the same alternative twice for a single chooser?

—
Reply to this email directly or view it on GitHub #142 (comment).

[fscottfoti]

Makes sense.

[bridwell]

So does this mean that for situations like household location choice, interaction variables will not be supported for choices in the simulation? If this is the case, a boolean argument called 'is_constrained' or something would be helpful in the model constructor: if provided, this would automatically set the choice_mode to aggregate and the probability_mode to 'full_product', and would disable any interaction dataset functionality. This way we don't specify and estimate models in a manner that won't be supported by the simulation.

[jiffyclub]

@bridwell Right now probability_mode='single_chooser' must be used with choice_mode='aggregate'. This is checked by the system (as of 5087d76 in this PR).

Interaction variables are allowed in any mode because in either case the choosers and alternatives are being merged. However, it is not possible to use interaction_predict_filters if probability_mode='full_product' because UrbanSim can't handle the possibility of different choosers having different numbers of available alternatives.

[coveralls]

Changes Unknown when pulling 9c49885 on dcm-pred-sample into * on master*.

[jiffyclub]

Think this is ready to go.

[fscottfoti]

To be clear, UrbanSim can handle the possibility of different individuals having different numbers of alternatives. It can't handle interaction filters well because the filters are applied after sampling, and so might result in a poor sample. If that's OK with the modeler, it's something we could support pretty easily.

I actually don't know how the old UrbanSim did this. I suspect it would continue getting new samples until it had enough alternatives that passed the filters, which can be an expensive operation if there are few alternatives that do pass the filter.

Otherwise you have to merge every chooser with every alternative and then filter, and then sample, which gives you the right answer but uses too much memory during the merge.

[jiffyclub]

What I was seeing today was that if there was a mismatch between the number of alts passed to mnl_simulate and the number available to one of the choosers UrbanSim would throw an exception because it couldn't reshape the data into a rectangular array.

[fscottfoti]

That makes sense - in that case we should be able to fill in dummy alternatives though. In the other case it's not clear how to solve it.

[jiffyclub]

Do you want to do that now, or merge this and carry on with the release?

[fscottfoti]

Merge this and release. We can address this when we get the chance.