new RW_multinomial sampler

[danielturek]

The title says it all.

[danielturek]

A more complete description of this PR, for posterity, and for when @paciorek reads these to update NEWS (prior to release).

This PR reinstates a RW_multinomial sampler. The PR includes:

• A new (simpler) implementation of the RW_multinomial sampler
• MCMC configuration will, by default, detect and assign this sampler to latent dmulti distributions
• New tests of asymptotic convergence of the sampler

Details of the new sampler:

• Uses a simple MH update scheme
• On each attempt, a proposal is made to move a single observation from any (non-empty) category, to any other (randomly selected) category
• This proposal to move a single observation is accepted or rejected
• The only flair of this sampler is the presence of a tries control argument. Exactly following the RW_block sampler, if tries is greater than 1, then a total of tries independent propose/accept/reject steps (each one attempting to move a single observation) are made, on each invocation of the sampler.
• Differing from the RW_block sampler tries argument, here, the default value of tries is not 1. The default is (the ceiling of) the cube root of the multinomial size. That is, if the multinomial size is 10 (10 observations being allocated among the outcome groups), then tries has a default value of 3. If the multinomial size is 1000, then tries has a default value of 10.
• The value of tries can also be manually set, of course, using the control list.

A variety of tests, those which were added to test-mcmc.R, and also others not included, give strong evidence that the sampler implementation is correct.

[danielturek]

@paciorek @perrydv Testing passing. Welcome any review or comments.

@paciorek I'd like to try and include this in the upcoming release.

[perrydv]

Thanks @danielturek . I like this and it would provide a basic functionality. I am wondering about a couple of aspects.

• For a large N (size), would it be quite slow? Could we come up with approaches to propose moving around multiple outcomes perhaps in proportion to N? I have some brainstorms but will not burden this comment with them.
• We hear from multiple users that having latent N is an important use case. Then N and the multinomial vector need to be sampled jointly. We could potentially make that a sampler that will only ever be assigned by hand, rather than as part of any default sampler configuration. But I'm just wondering if we should cover that case while the problem is opened up, vs. trying to add it on later.

[paciorek]

Not getting into substance here, but we need submission of 1.2.1 to CRAN as of 2024-07-31. That means I'd like anything on this front in a passing PR by, say, this weekend. I agree that ideally we'd at least have a basic fix/functionality in the release.

[danielturek]

@perrydv I agree we can envision ways in which >1 observations are proposed to be moved on each "try", perhaps simply as some fraction of N, for example N/100, or randomly according to a discrete uniform distribution between 1 and ceiling(N/100). Anything along these lines would complicate the acceptance probabilities somewhat, although it could be handled. My goal with this PR was just to have something that works, to cover this base; then leave improvements for future work - perhaps that of a student, as well.

Noting that the tries argument, with default value scaling with N, mitigates this to some degree.

I agree also, the notion of "jointly sample N and the observation vector" is something else we could handle - I haven't given this as much thought, but agree it would be useful to have. (I can also envision how the need for such a sampler would be automatically detected and assigned).

@perrydv Do you want to discuss other ideas? My main ideas are in the first paragraph above.

[danielturek]

@perrydv Update to the RW_multinomial sampler:

Addition of maxMove control parameter, which allows for proposals that move >1 observation.

On each independent "try", a random (non-empty) category is selected. From this category, we propose moving move observations to another (randomly selected) category. The value of move is drawn from a discrete uniform distribution, with minimum 1, and maximum value given by maxMove, or the total number of observations in that category (if fewer than maxMove.

Again, each independent proposal is to move move observations, where move is drawn from the distribution:

DiscreteUniform(1, min(maxMove, current_number_of_observations_in_that_category))

The default value of maxMove, if not provided in the control list, is ceiling(N/20). Thus, for multinomial size N=100, we propose moving between 1 and 5 observations. But, if the "from" category only has (say) 2 observations in it, we propose moving either 1 or 2 observations.

An additional (more stringent) test is also added, for asymptotic correctness.

[perrydv]

@danielturek @paciorek OK if the goal is to get a basic sampler that at least works for latent multinomials, we can go with this. We know there is some risk that other priorities will get in the way of improving on the basic version in the future, so we'll have to keep it on the radar. A technical detail is that the maxMove approach, in the case that the total number of observations in the from category is less than maxMove, could introduce asymmetry in the proposal vs reverse proposal probabilities. It does appear to me from the code that that was recognized and handled, but as it is a slight subtely I just wanted to make sure that I am reading the code correctly and it is handled.

[danielturek]

@perrydv I believe that case is handled correctly by the calculations.

[danielturek]

Merging this in.