Multi card support - Prevent route override for primary ENI across multi-cards ENAs

[jayanthvn]

What type of PR is this?
Feature

Which issue does this PR fix:
No

What does this PR do / Why do we need it:
ENAs can be on multi-card (P4 family) and CNI currently doesn't support non-zero network card index. So on nodeInit, CNI thinks all the attached ENIs are on card 0, hence this PR will make CNI aware of that. But CNI doesnt allocate pods or secondary IPs on non-zero network cards. Support will be in future releases.

If an issue # is not available please add repro steps and logs from IPAMD/CNI showing the issue:
N/A

Testing done on this change:

NodeInit

{"level":"info","ts":"2021-03-02T19:04:57.922Z","caller":"ipamd/ipamd.go:363","msg":"Got network cardindex 0 for ENI eni-0aa1c65af9aa0f115"}
{"level":"info","ts":"2021-03-02T19:04:57.922Z","caller":"ipamd/ipamd.go:363","msg":"Got network cardindex 1 for ENI eni-0535397aaad78b390"}
{"level":"info","ts":"2021-03-02T19:04:57.922Z","caller":"ipamd/ipamd.go:363","msg":"Got network cardindex 2 for ENI eni-02142cb550d03b63b"}
{"level":"info","ts":"2021-03-02T19:04:57.922Z","caller":"ipamd/ipamd.go:363","msg":"Got network cardindex 3 for ENI eni-04e0ddb1cc00fca31"}
{"level":"debug","ts":"2021-03-02T19:04:57.922Z","caller":"ipamd/ipamd.go:312","msg":"DescribeAllENIs success: ENIs: 4, tagged: 0"}
{"level":"debug","ts":"2021-03-02T19:04:57.922Z","caller":"ipamd/ipamd.go:370","msg":"Skipping ENI eni-04e0ddb1cc00fca31: since on non-zero network card"}
{"level":"debug","ts":"2021-03-02T19:04:57.922Z","caller":"ipamd/ipamd.go:370","msg":"Skipping ENI eni-0535397aaad78b390: since on non-zero network card"}
{"level":"debug","ts":"2021-03-02T19:04:57.922Z","caller":"ipamd/ipamd.go:370","msg":"Skipping ENI eni-02142cb550d03b63b: since on non-zero network card"}
{"level":"debug","ts":"2021-03-02T19:04:57.922Z","caller":"ipamd/ipamd.go:312","msg":"Discovered ENI eni-0aa1c65af9aa0f115, trying to set it up"}
{"level":"debug","ts":"2021-03-02T19:04:57.922Z","caller":"ipamd/ipamd.go:768","msg":"DataStore Add an ENI eni-0aa1c65af9aa0f115"}

Reconciler -

{"level":"debug","ts":"2021-03-02T19:06:03.610Z","caller":"ipamd/ipamd.go:962","msg":"Skipping ENI eni-02142cb550d03b63b: since on non-zero network card"}
{"level":"debug","ts":"2021-03-02T19:06:03.610Z","caller":"ipamd/ipamd.go:962","msg":"Skipping ENI eni-04e0ddb1cc00fca31: since on non-zero network card"}
{"level":"debug","ts":"2021-03-02T19:06:03.610Z","caller":"ipamd/ipamd.go:962","msg":"Skipping ENI eni-0535397aaad78b390: since on non-zero network card"}
{"level":"debug","ts":"2021-03-02T19:06:03.610Z","caller":"ipamd/ipamd.go:504","msg":"Reconcile existing ENI eni-0aa1c65af9aa0f115 IP pool"}
{"level":"debug","ts":"2021-03-02T19:06:03.610Z","caller":"ipamd/ipamd.go:1007","msg":"Reconcile and skip primary IP 192.168.72.127 on ENI eni-0aa1c65af9aa0f115"}
{"level":"debug","ts":"2021-03-02T19:06:03.610Z","caller":"ipamd/ipamd.go:504","msg":"Successfully Reconciled ENI/IP pool"}
{"level":"debug","ts":"2021-03-02T19:06:03.610Z","caller":"ipamd/ipamd.go:504","msg":"IP Address Pool stats: total: 49, assigned: 0"}
{"level":"debug","ts":"2021-03-02T19:07:03.617Z","caller":"ipamd/ipamd.go:504","msg":"Reconciling ENI/IP pool info because time since last 1m0.079149379s <= 1m0s"}

RefreshSGIDs -

{"level":"debug","ts":"2021-03-02T19:04:58.014Z","caller":"awsutils/awsutils.go:592","msg":"Found CIDR 192.168.64.0/19 for ENI 0a:f7:72:1b:8c:8b"}
{"level":"debug","ts":"2021-03-02T19:04:58.018Z","caller":"awsutils/awsutils.go:592","msg":"Found IP addresses [192.168.70.10] on ENI 0a:f7:72:1b:8c:8b"}
{"level":"debug","ts":"2021-03-02T19:04:58.018Z","caller":"ipamd/ipamd.go:319","msg":"Update ENI eni-0aa1c65af9aa0f115"}
{"level":"info","ts":"2021-03-02T19:04:58.443Z","caller":"ipamd/ipamd.go:325","msg":"Found 192.168.0.0/16, added to ipamd cache"}
{"level":"info","ts":"2021-03-02T19:04:58.444Z","caller":"aws-k8s-agent/main.go:69","msg":"Serving RPC Handler version v1.8-dirty on 127.0.0.1:50051"}

Automation added to e2e:

Yes

Will this break upgrades or downgrades. Has updating a running cluster been tested?:
No

Does this change require updates to the CNI daemonset config files to work?:

No

Does this PR introduce any user-facing change?:

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.