The maximum general network size that ERGM can support.

[mgg-new]

Given the increasing trend in the scale of network data that needs to be processed, I would like to inquire about the maximum general network size that ERGM can support. Is it capable of handling networks with hundreds or thousands of nodes? Can it support even larger network sizes?

[krivit]

In principle, vertex indices are passed via in 32-bit signed int types, so the maximum number of vertices that can conceivably be represented is 2,147,483,647. Some operations may try to calculate numbers of dyads, with dyad indices stored either in 64-bit ints or 64-bit double types, the latter of which can represent indices up to 2^53, and so the number of vertices would be the root of that, or about 95 million.

In practice, I don't know what the record is, but I believe it's in the high hundreds of thousands by the @statnet/epimodel group.

I hope this helps.

[mgg-new]

thanks

[AdrienLeGuillou]

@mgg-new currently for our HIV simulations with EpiModel, we use 100k nodes with 3 networks at once.

The full model runs in around 1.5 hours on a single CPU with 5GiB of RAM. We use the tergmLite formulation to speed up the simulation process

[mgg-new]

Thank you for your response. The reason I asked this question is that I intend to use ERGM to fit a bipartite network with a scale of about hundreds of thousands of nodes. This network involves some node covariates and edge covariates. However, in my previous experience, I found that ERGM is more suitable for handling networks of a few thousand nodes, and exceeding this scale significantly impacts running speed.

[AdrienLeGuillou]

It becomes indeed slower with size.
Using tergmlite helps a lot on this regard. Then working on HPCs is what allows us to produce true results

[mbojan]

Supplementing answers already given, do note that the running speed will very much depend on the model specification you will be fitting. Dyad-dependent models are much more demanding.

[martinamorris]

The other thing that can really speed things up is to make full use of the new MCMC tuning parameters in ergm 4.0. See https://arxiv.org/abs/2203.08198 for more details.