Flag to change size of egress gateway policy map

[pchaigno]

We should have an agent flag to change the default size of the egress gateway's policy map, similar to other such flags. This will be needed for users who run out of space in the map (currently sized to 16384).

Initially reported in #22805.

[cyclinder]

Hi @pchaigno , I'm new to cilium, can I have this ? I'd take this one 😄

[pchaigno]

@cyclinder Of course, feel free to submit a pull request. If you need any help, you can ask here or in the Slack #development channel.

[alex-hunt-materialize]

Are there any significant performance or memory limits to the size of this map? If we were to set it to 1000x the default value, would that be reasonable?

[pchaigno]

For the performance it's hard to say. I would expect performance to depend on the number of entries actually in the map, not its maximum size. It's an LPM map so you can check theoretical lookup and update times for that algorithm.

Regarding memory, the map has a key of size 12B and a value of size 8B. I would expect memory consumption to grow approximately linearly with the number of entries. So for a map of 16384000 entries, you'd have an in-kernel memory consumption of at least 312MiB. Note there's a hard limit on how much memory you can allocated for BPF maps in the kernel, so you wouldn't be able to grow the map to huge sizes (e.g., x100000 is probably not possible).