Enable to use of an nginx ingress controller using an internal LB to access the cluster resources

[mosabami]

The build intelligent apps scenario's architecture diagram has Nginx used as an ingress controller to access the application. However our deployment steps currently expose the web application directly using a LB service. Make changes required so that an nginx ingress controller is used instead and have all the other services changed to ClusterIP. Here is a link to the scenario: https://github.com/Azure/AKS-Landing-Zone-Accelerator/tree/openai-embeddings/Scenarios/AKS-OpenAI-CogServe-Redis-Embeddings

You can make the Nginx LB use an internal LB by using the command below:

helm upgrade --install ingress-nginx ingress-nginx \
 --namespace ingress-basic \
 -f internal-ingress.yaml \
   --repo https://kubernetes.github.io/ingress-nginx \
   --set controller.replicaCount=2 

the internal-ingress.yaml file can be found in the kubernetes folder. We dont need to use https at this time

Acceptance criteria:

Ingress controller manifest file added using http
Ingress controller should have an internal IP address
Update the readme to deploy the ingress controller and also mention the fact that the app can only be access via app gateway as one of the features of this scenario
Include an az cli command to add the ingress controller's ip address (10.240.3.101) to the back end pool of the app gateway to the README or use bicep to do that
All other services are using ClusterIP service type
PR submitted to the opernai-embeddings branch