Merging topology fragments

[a-v-popov]

Is there a way to build topology from multiple fragments?
E.g. one file describes physical topology and another logical structure, so that the logical structure can be an overlay on physical topology which can be executed by different providers as needed.

[ipspace]

I had similar ideas when building BGP labs ;) Short answer: not at the moment (but you know things can happen ;)

The details:

• The _include directive can include another file (or a set of files) as a dictionary key, not at the top level, but that should be easy to fix. * However, _include merges dictionaries, and things that are lists will be overwritten, links being the prime example. However, if you want to write links as a dictionary (which is also an option), it should be possible.

What else have I missed?

[a-v-popov]

The _include directive can include another file

I was thinking more in terms of overlay topology rather namespace loader.

to write links as a dictionary (which is also an option)

What would be a dictionary value? A list of interfaces, which would be replaced on collision? Does it allow a single interface?

What else have I missed?

Maybe a stable merge key for links and binding for ports. E.g. something like

#topology.yml
links:
  - id: r1-r2-core
    name: primary-core
    interfaces: 
      - node: r1
        port: core
      - node: r2
        port: core

#physical.yml
nodes:
    r1:
        ports.core.ifname: GigabitEthernet0/0/3
        mgmt.ipv4: 10.0.0.101
    r2:
        ports.core.ifname: Ethernet7
        mgmt.ipv4: 10.0.0.102
        
links:
  - id: r1-r2-core
    mtu: 9000

and then either netlab up ... topology.yml for virtual environment or netlab (create|initial) ... topology.yml --overlay physical.yml for a physical one.

Or maybe an explicit port binding which can be selected by CLI:

bindings:
  physical:
    r1.core.ifname: GigabitEthernet0/0/3
    r2.core.ifname: Ethernet7

and then netlab ... --bindings physical topology.yml --overlay fancy.yml

Binding could also be a merge on an explicit interface id, but since interfaces seems to be internal data structure, it might be better be presented as a separate feature.

I am not sure about all of this though, more like investigating possibilities.

[jbemmel]

and then either netlab up ... topology.yml for virtual environment or netlab (create|initial) ... topology.yml --overlay physical.yml for a physical one.

We could enable netlab to accept multiple topology files, appending sections in the order they are listed. So in your case, the links from physical.yml would get appended to the links: section in topology.yml

No need for special --overlay flag - just multiple files merged in order of appearance

[jbemmel]

[jbemmel]

Something like this: #3592

[a-v-popov]

accept multiple topology files

Let's assume I am using some_netlab_cmd to start a topology. What would be the new command to run the same topology augmented with another fragment. With an explicit CLI switch that would be some_netlab_cmd --overlay augment.yml. Try to answer that question without one. But I don't think CLI is the problem here. Plus I see no reason why the two options would be mutually exclusive.

So in your case...

My case, or at least the one I had in mind, is to switch back and forth between virtual and physical setup, because those super fast magic containers can only get you so far without proper data plane. This seems to required a genuine merge, either explicitly via unique ids, or through bindings. At the moment I lean towards the latter, because I would prefer to express logical setup through abstract references and then bind them to actual identifiers like ifname or ifindex, if required. Although I think a general merging mechanism based on unique, potentially autogenerated, keys might be useful in other cases when one want to decompose information on about the same link, for example.