Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unclear whether / how it is possible to let App Service access resource (CosmosDB) that can only be accessed from vnet #28461

Closed
hansmbakker opened this issue Apr 1, 2019 — with docs.microsoft.com · 53 comments

Comments

Copy link
Contributor

(I'm not sure whether this is an App Service or Cosmos DB docs issue; this is a crosspost from the ticket for the App Service docs)

This issue is related to this post on Stackoverflow.

From the docs page that I'm creating this issue from, and from the VNet-related Cosmos DB docs page it is unclear to me whether the scenario below is supported.

I have an App Service and a Cosmos DB. I want to restrict access to the Cosmos DB - only the App Service should access it.

My solution was to:

  • create a vnet with one subnet called default
  • enable the firewall feature in Cosmos DB --> set Allow access from Selected networks
  • create service endpoint for Cosmos DB on the default subnet
  • apply the New VNet integration in the App Service and connected it to the same default subnet

Cosmos DB firewall setup:
image

default subnet setup:
image

Unfortunately, this does not work. How do I make the App Service correctly access Cosmos DB through the VNet?


Document Details

Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

@Mike-Ubezzi-MSFT
Copy link
Contributor

@hansmbakker Thank you for detailing the issue you are encountering, Are you saying that the 4 steps you detailed in "My solution was to:" still resulted in the following statement "Unfortunately, this does not work. How do I make the App Service correctly access Cosmos DB through the VNet?" ?

Or, you followed the 4 steps (as detailed) and got it working but when you follow the documentation, it DOES NOT work?

If you are completly stuck and your solution is not working at all, I suspect that you need to add the specific source address for your App Service to the Cosmos DB firewall Allow IP Address list. I do need to look into this more closely to understand the docs and identify any corrections that need to be made but did want to suggest pursuing this as a possibility. Please see the following docs pertaing to App Service IP addressing:

Inbound and outbound IP addresses in Azure App Service
How do I determine the outbound IP addresses of my Azure App Service
How can I determine the IP address of an Azure hosted WebApp

@hansmbakker
Copy link
Contributor Author

@Mike-Ubezzi-MSFT thank you for your response.

After the 4 steps of my solution (+whitelisting the ip of my development machine) it did not work.

So what I’m asking is:

  • is this solution supposed to work?
  • is there something I need to change to make this work?

I couldn’t find an answer to both questions in the docs.

The reason I prefer not to whitelist the IP from the app service is that that IP is not guaranteed to be fixed I believe.

@Mike-Ubezzi-MSFT
Copy link
Contributor

@hansmbakker You need to subscribe to App Service Environment to gain the static IP functionality. The App Service by itself does not allow for dedicated IP addresses. So, you can use the Allow access to Azure services to allow traffic to your Cosmos instance or build out a whitelist of specific IP addresses...or use VNet. The document you filed this issue against is a FAQ page and does not address your specific questions. This document walks you through the specific configuration steps for deploying/configuring a VNet for a Cosmos DB instance.

@hansmbakker
Copy link
Contributor Author

hansmbakker commented Apr 2, 2019

@Mike-Ubezzi-MSFT are you saying that my scenario / solution is not possible without an App Service Environment (ASE)?

If the idea is that

  • VNET Integration allows access from App Service to VNET
  • CosmosDB VNET service endpoint allows access from VNET to CosmosDB
  • access can be restricted by CosmosDB to traffic originating from the VNET

Then why can't I route the App Service --> CosmosDB traffic through the VNet?

I don't understand what the purpose of the (New) VNet Integration of the normal App Service is then?

@SnehaGunda
Copy link
Contributor

@hansmbakker To let appservice access Cosmos DB you should check the accept connections from Azure datacenters option as shown in the screen below:

image

Azure Cosmos DB account will be accessible from Azure datacenter IP range. This allows services like Search, Data factory, Stream analytics whose IP addresses can't be fixed - to talk to Cosmos DB account. 

@hansmbakker
Copy link
Contributor Author

hansmbakker commented Apr 3, 2019

@SnehaGunda I understand that would allow the App Service to access the CosmosDB instance. However, I use the option you recommend, then any random App Service or other component from within Azure can access the CosmosDB instance.

That does not match the reason behind this story - the IT department came up with the approach mentioned in the opening post and they required this setup in our architecture, to let only this App Service access the CosmosDB instance.

Now the point of this GitHub issue is to

  • validate the approach that the IT department of my company requires
  • check whether I understand the VNet integration of App Service and CosmosDB correctly

Therefore, I'm asking feedback on

@Mike-Ubezzi-MSFT
Copy link
Contributor

Mike-Ubezzi-MSFT commented Apr 3, 2019

@hansmbakker The Stack Overflow answer provided by Nancy Xiong is spot on. If you deploy the application tier in an App Service Environment, you gain true VNet functionality. Using Multi-tenent environment does not fully support VNet only access...as Nancy pointed out in the Stack Overflow answer.

There is a article written by Deloitte that provides and overview and a comparison between a Multi-tenant app service instance and an App Service Environment instance: When to Use an Azure App Service Environment v2 (App Service Isolated)

Private access only?

Multi-tenant environment: No.

App Services are always accessible via the public internet.
One way to restrict access to your App Service is using IP and Domain restrictions, but the App Service is still reachable from the internet.

App Service Isolated / App Service Environment v2: Yes.

An ASE can be deployed with an Internal Load Balancer, which will lock down your App Services to be accessible only from within your VNET or via ExpressRoute or Site-to-Site VPN.

@hansmbakker
Copy link
Contributor Author

@Mike-Ubezzi-MSFT I understand that ASE can be used to restrict access to the App Service. However, that is not what I'm trying to achieve.

I'm trying to let the App Service access another resource (in this case Cosmos DB, but also Blob Storage) routed through the VNet. I thought I could use the (new) VNet integration for that.

One of my questions is what the purpose of the App Service's VNet Integration is, if it can not do that?

@Mike-Ubezzi-MSFT
Copy link
Contributor

@hansmbakker The intent of the VNet Integration feature had the following requirements when the service was initially launched:

For Azure App Service, a new version of the VNet Integration capability enables access to resources across service endpoints or Azure ExpressRoute connections.
Like the earlier version, the new VNet Integration feature only enables your app to make calls into your virtual network. It does not affect inbound traffic to your app.
This feature is in preview in all public regions. It has the following characteristics:

(shortlist of items)

  • Your App Service plan must be a Standard, Premium, or Premium V2 plan.
  • Your app must be in an Azure App Service deployment that’s capable of scaling up to Premium v2.
  • The feature currently works just with Windows apps.
  • The new feature requires an unused subnet in your Azure Resource Manager virtual network.
  • The app and the virtual network must be in the same region.

Announcing the new VNet Integration feature in App Service

So, it is intended to integrate with a VNet and extend (front end) services across an Express Route connection or other peered network for app services calls into a virtual network.

What you are attempting to do is secure the communication the App Service needs to establish with the Cosmos DB instance, either through a direct connection or via REST.

It's not that your method will not work but the integration service is not the solution here.

@NateB2
Copy link

NateB2 commented Apr 5, 2019

I have verified that the service endpoints do not appear to be working for Key Vault and Blob storage as well, despite the last paragraph on this page that explicitly says:

The new VNet Integration feature enables you to use service endpoints. To use service endpoints with your app, use the new VNet Integration to connect to a selected VNet and then configure service endpoints on the subnet you used for the integration.

I've set up my app service exactly like the OP, and I get "unauthorized" errors. When I disable the VNET restrictions on the key vault/blob storage accounts, it works fine.

@Mike-Ubezzi-MSFT
Copy link
Contributor

Thanks for the additional detail! I have assigned the issue to the content author to evaluate and update as appropriate.

@hansmbakker
Copy link
Contributor Author

@Mike-Ubezzi-MSFT

#28461 (comment) it is intended (...) for app services calls into a virtual network

That is exactly what I'm trying: routing the calls from app services through the vnet to a service endpoint that is also connected to the vnet

app service --> vnet --> service endpoint --> cosmosdb

the integration service is not the solution here

This is where my confusion comes from. From the first quote it looks like it could/should work, but from the second quote it looks like it cannot work.

What is the way to go, then?

I have assigned the issue to the content author to evaluate and update as appropriate.

Thank you!

@Mike-Ubezzi-MSFT
Copy link
Contributor

@hansmbakker It has to do with inbound connectivity versus outbound connectivity, and the use of Site-to-Site/Point-to-Site VPN and Express Route scenarios for the creation of private addressing to extend on-premise to Azure, in cases where the VNet service endpoints are private. Not necessarily VNet to VNet integration, which is what you are attempting to implement. I agree the documentation needs to be enhanced to make things more clear but the MSDN and Stack Overflow forums have a few threads where this is being discussed:

App Service and VNet to VNet Connectivity
Cannot access app service website after adding vnet integration

I think if the document had a supported/unsupported scenario chart to help distill the difference between leveraging VNet service endpoints (public and private) and VNet integration.

@hansmbakker
Copy link
Contributor Author

Please also note that the App Service VNet Integration page suggests that an App Service can access other services through a VNet using VNet integration.

image

@hansmbakker
Copy link
Contributor Author

hansmbakker commented Apr 8, 2019

The "Learn more" link in the screenshot above points to https://docs.microsoft.com/en-us/azure/app-service/web-sites-integrate-with-vnet.

The "New VNet Integration" (which I'm using) section on that page says:

You can consume Service Endpoint secured resources using the new VNet Integration capability. To do so, enable service endpoints on the subnet used for VNet Integration.

@hansmbakker
Copy link
Contributor Author

@Mike-Ubezzi-MSFT thank you for the elaborate answer.

At minute 7:00 in the video, @ccompy discusses the New VNet Integration. That I believe is the functionality you desire.

Yes, that is exactly what I want, and I am already using the New VNet Integration (please see my repro steps in the opening post). I tried setting it up just as @ccompy presented, but it does not work as expected.

@NateB2 has the same issue in #28805.

I understand that it is still in preview, but how can we collaborate best to raise this issue? Is there any chance that @ccompy could comment here?

@NateB2
Copy link

NateB2 commented Apr 17, 2019

FWIW, on Monday, I had a phone/remote desktop sharing session with a support engineer on the App Service team, and he appeared to indicate that what we're trying to do should be possible with the new Vnet integration preview. At the end of the conversation, after I showed him what we had set up, he said he was going to reproduce it on his side and confer with the team to see what's going on.

@hansmbakker
Copy link
Contributor Author

@NateB2 were they able to reproduce it?

@NateB2
Copy link

NateB2 commented Apr 25, 2019

@hansmbakker - yes, and they said that it's not yet implemented, but going to be implemented before the new VNET integration reaches GA. They didn't give a timeline as to when that was going to take place, however.

@sarah026
Copy link

sarah026 commented Jul 5, 2019

@hansmbakker thanks for raising this, have you made any progress since then? Have you tried the two subnets scenario for the ServiceEndpoint and one for the Vnet?

Its been more than 2 months, and for the claims they've made in the video since 2018 and the updated website, this should be working by now!

@hansmbakker
Copy link
Contributor Author

hansmbakker commented Jul 5, 2019

@sarah026 it's a coincidence that you ask this today, as I succeeded with CosmosDB just two hours ago.

Things I had to do to make it work:

  • delete the existing VNET (incl. the subnet, CosmosDB and other service endpoints, VNET integration)
  • re-create the VNET with subnet. Integrate the CosmosDB using its Firewall/networking tab using a service endpoint.
  • setup the New VNET Integration again in App Service (if you do this using ARM templates, be aware of the different syntax from the old VNET integration! Use a child resource in the App Service for this)

Having two subnets in the vnet was not needed, one subnet is enough. The New VNET integration and the CosmosDB service endpoint are connected to the same subnet.

This made it work for CosmosDB for me, but Azure Storage does not work for me yet using the same approach.

@sarah026
Copy link

sarah026 commented Jul 5, 2019

Pleased to hear that @hansmbakker! Thanks for sharing. Our environment was set up yesterday, so it is a new build.
Which geo location is your service deployed to?

@hansmbakker
Copy link
Contributor Author

hansmbakker commented Jul 5, 2019

West Europe, hope it works for you as well.

@bowlerma
Copy link

Having the same problem with App Services and SQL Azure, this doesn't seem to work as the documentation and the various other demos indicate it should:

https://www.youtube.com/watch?v=wWX5iTQPp5M

https://www.youtube.com/watch?v=ic6Laiv9Jfc

@sarah026
Copy link

Having the same problem with App Services and SQL Azure, this doesn't seem to work as the documentation and the various other demos indicate it should:

https://www.youtube.com/watch?v=wWX5iTQPp5M

https://www.youtube.com/watch?v=ic6Laiv9Jfc

Mine is working fine now between AppService and SQL elasticpool. I suggest you raise a ticket with them. The issue with mine was datacentre level issue, they needed to whitelist something for it to work. Not something we've done wrong, and so I wouldn't suggest you waste as much time as I wasted trying to figure this out.
It is West Eu region as well! Why did it work for hansmbakker and not for us!

@bowlerma
Copy link

Will do, thanks for the advice @sarah026

Copy link

Got it working between Web app and cosmos table. Not sure what made it work but it took a couple of hours of deleting resources, and adding them back again. Might just have been the time that solved it, provisions etc maybe??

@syndicatedshannon
Copy link

syndicatedshannon commented Aug 25, 2019

Deleting and recreating also 'mystically' solved my problem.

I tried this route after I found that the servers I was initially provisioned did not support scaling to P1V2. It was referenced in some docs, but what I didn't realize is that it meant my SPECIFIC server may have that scaling option disabled (which it did), and that would be an indicator of a separate deficiency that would prevent this VNet solution from working.

What I think happened was I was assigned compute resources in an old rack, possibly because I carried that server forward from a free trial account (even though that was less than 2 months ago). The requirement/consequence is very hard to associate, an invisible thread that was keeping my VNet from working.

I did not realize that of two servers in the same compute location on the same plan, one may support scaling while the other did not. It finally dawned on me what they meant when I was watching the youtube video posted above.

@devedse
Copy link

devedse commented Jul 23, 2020

So I'm still running into this problem. Also, is the new VNET integration already released?

The steps I executed:

  1. Created a VNet
  2. I left the subnet as default
    image
  3. Setup cosmos
  4. Assign Cosmos to the VNET
    image
  5. Now I got some messages that the service endpoint did not exist and would be created. Which actually happened:
    image
  6. I now created an AppService and deployed a very simple asp.net core application to it that connects to the Cosmos DB on startup using the code below.
  7. I connected it to the VNET as seen here:
    image
  8. It then gave me a message about having to wait a few hours before the connection would work. (Is this honnestly the case???)
  9. When I now access the AppService it shows a 503:
    image
  10. If I then browse to Kudu and open D:\home\LogFiles\eventlog.xml I can see the following message:
Beginning operations...

Forbidden error occurred: Microsoft.Azure.Cosmos.CosmosException : Response status code does not indicate success: 403 Substatus: 0 Reason: (Microsoft.Azure.Documents.DocumentClientException: Request originated from client IP 52.136.230.110 through public internet. This is blocked by your Cosmos DB account firewall settings.ActivityId: b9bcb4b7-2ad4-4130-b592-fb70b0c18628, Microsoft.Azure.Documents.Common/2.11.0, Windows/10.0.14393 cosmos-netstandard-sdk/3.4.2   at Microsoft.Azure.Cosmos.GatewayStoreClient.ParseResponseAsync(HttpResponseMessage responseMessage, JsonSerializerSettings serializerSettings, DocumentServiceRequest request)   at Microsoft.Azure.Cosmos.GatewayAccountReader.GetDatabaseAccountAsync(Uri serviceEndpoint)   at Microsoft.Azure.Cosmos.Routing.GlobalEndpointManager.GetDatabaseAccountFromAnyLocationsAsync(Uri defaultEndpoint, IList`1 locations, Func`2 getDatabaseAccountFn)   at Microsoft.Azure.Cosmos.GatewayAccountReader.InitializeReaderAsync()   at Microsoft.Azure.Cosmos.CosmosAccountServiceConfiguration.InitializeAsync()   at Microsoft.Azure.Cosmos.DocumentClient.InitializeGatewayConfigurationReaderAsync()   at Microsoft.Azure.Cosmos.DocumentClient.GetInitializationTaskAsync(IStoreClientFactory storeClientFactory)   at Microsoft.Azure.Cosmos.DocumentClient.EnsureValidClientAsync()   at Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler.EnsureValidClientAsync(RequestMessage request)).
StatusCode = Forbidden;
SubStatusCode = 0;
ActivityId = b9bcb4b7-2ad4-4130-b592-fb70b0c18628;
RequestCharge = 0;
{"Summary":{"StartUtc":"2020-07-23T14:33:28.0440416Z","ElapsedTime":"00:00:00.5533417","UserAgent":"cosmos-netstandard-sdk/3.6.0|3.4.2|43726|X86|Microsoft Windows 10.0.14393|.NET Core 3.1.6|CosmosDBDotnetQuickstart"},"Context":[{"Id":"RequestInvokerHandler","ElapsedTime":"00:00:00.5533417"},{"Id":"PointOperationStatistics","ActivityId":"b9bcb4b7-2ad4-4130-b592-fb70b0c18628","StatusCode":403,"SubStatusCode":0,"RequestCharge":0.0,"RequestUri":"dbs/ToDoList","ErrorMessage":"Microsoft.Azure.Documents.DocumentClientException: Request originated from client IP 52.136.230.110 through public internet. This is blocked by your Cosmos DB account firewall settings.ActivityId: b9bcb4b7-2ad4-4130-b592-fb70b0c18628, Microsoft.Azure.Documents.Common/2.11.0, Windows/10.0.14393 cosmos-netstandard-sdk/3.4.2   at Microsoft.Azure.Cosmos.GatewayStoreClient.ParseResponseAsync(HttpResponseMessage responseMessage, JsonSerializerSettings serializerSettings, DocumentServiceRequest request)   at Microsoft.Azure.Cosmos.GatewayAccountReader.GetDatabaseAccountAsync(Uri serviceEndpoint)   at Microsoft.Azure.Cosmos.Routing.GlobalEndpointManager.GetDatabaseAccountFromAnyLocationsAsync(Uri defaultEndpoint, IList`1 locations, Func`2 getDatabaseAccountFn)   at Microsoft.Azure.Cosmos.GatewayAccountReader.InitializeReaderAsync()   at Microsoft.Azure.Cosmos.CosmosAccountServiceConfiguration.InitializeAsync()   at Microsoft.Azure.Cosmos.DocumentClient.InitializeGatewayConfigurationReaderAsync()   at Microsoft.Azure.Cosmos.DocumentClient.GetInitializationTaskAsync(IStoreClientFactory storeClientFactory)   at Microsoft.Azure.Cosmos.DocumentClient.EnsureValidClientAsync()   at Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler.EnsureValidClientAsync(RequestMessage request)","RequestSessionToken":null,"ResponseSessionToken":null}]}
   at Microsoft.Azure.Cosmos.ResponseMessage.EnsureSuccessStatusCode()
   at Microsoft.Azure.Cosmos.CosmosResponseFactory.ProcessMessageAsync[T](Task`1 cosmosResponseTask, Func`2 createResponse)
   at Microsoft.Azure.Cosmos.CosmosClient.<>c__DisplayClass41_0.<<CreateDatabaseIfNotExistsAsync>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at CosmosGettingStartedTutorial.Program.CreateDatabaseAsync() in C:\Users\davy\Downloads\DocumentDB-Quickstart-DotNet\CosmosGettingStartedTutorial\Program.cs:line 86
   at CosmosGettingStartedTutorial.Program.GetStartedDemoAsync() in C:\Users\davy\Downloads\DocumentDB-Quickstart-DotNet\CosmosGettingStartedTutorial\Program.cs:line 68
   at CosmosGettingStartedTutorial.Program.Main(String[] args) in C:\Users\davy\Downloads\DocumentDB-Quickstart-DotNet\CosmosGettingStartedTutorial\Program.cs:line 40
End of demo, press any key to exit.
...
...

@hansmbakker and the rest, do you see anything I missed?

Code to connect to cosmos:

    public class Program
    {
        // The Azure Cosmos DB endpoint for running this sample.
        private static readonly string EndpointUri = "https://davycosmos.documents.azure.com:443/";

        // The primary key for the Azure Cosmos account.
        private static readonly string PrimaryKey = "**********";

        // The Cosmos client instance
        private CosmosClient cosmosClient;

        // The database we will create
        private Database database;

        // The container we will create.
        private Container container;

        // The name of the database and container we will create
        private string databaseId = "ToDoList";
        private string containerId = "Items";

        // <Main>
        public static async Task Main(string[] args)
        {
            try
            {
                Console.WriteLine("Beginning operations...\n");
                Program p = new Program();
                await p.GetStartedDemoAsync();

            }
            catch (CosmosException de)
            {
                Exception baseException = de.GetBaseException();
                Console.WriteLine("{0} error occurred: {1}", de.StatusCode, de);
            }
            catch (Exception e)
            {
                Console.WriteLine("Error: {0}", e);
            }
            finally
            {
                Console.WriteLine("End of demo, press any key to exit.");
                Console.ReadKey();
            }
        }
        // </Main>

        // <GetStartedDemoAsync>
        /// <summary>
        /// Entry point to call methods that operate on Azure Cosmos DB resources in this sample
        /// </summary>
        public async Task GetStartedDemoAsync()
        {
            // Create a new instance of the Cosmos Client
            this.cosmosClient = new CosmosClient(EndpointUri, PrimaryKey, new CosmosClientOptions() { ApplicationName = "CosmosDBDotnetQuickstart" });
            await this.CreateDatabaseAsync();
            await this.CreateContainerAsync();
            await this.ScaleContainerAsync();
            await this.AddItemsToContainerAsync();
            await this.QueryItemsAsync();
            await this.ReplaceFamilyItemAsync();
            await this.DeleteFamilyItemAsync();
            await this.DeleteDatabaseAndCleanupAsync();
        }
        // </GetStartedDemoAsync>

        // <CreateDatabaseAsync>
        /// <summary>
        /// Create the database if it does not exist
        /// </summary>
        private async Task CreateDatabaseAsync()
        {
            // Create a new database
            this.database = await this.cosmosClient.CreateDatabaseIfNotExistsAsync(databaseId);
            Console.WriteLine("Created Database: {0}\n", this.database.Id);
        }
    }

@Mike-Ubezzi-MSFT
Copy link
Contributor

@devedse Thank you for providing the details specific to your deployment. What I do not see is a Privatelink associated with the VNET that you have created. A Private Endpoint establishes a private address space as defined by the VNET configuration. What you do not have is a Private IP Address issued for the Cosmos DB for SQL API service that has been setup in that VNET. Please see the following to complete the configuration: Configure Azure Private Link for an Azure Cosmos account.
If you do in fact have the Cosmos DB Privatelink configured, then you need to add the App Service NSG to your Cosmos DB VNET to allow traffic to pass, as you do not have Accept connections from within public Azure datacenters enabled. You will need to explicitly allow this service (NSG) to connect. The FAQ page does not capture all the required steps. Please let me know if you have additional questions. Regards, Mike.

Note: By working with NSGs instead of static IP and port values, when IP address changes occur, the NSG is automatically updated. Any manual configuration implemented outside an NSG will require intervention to update the correct values when each Azure maintenance update made.

@devedse
Copy link

devedse commented Jul 24, 2020

Hi @Mike-Ubezzi-MSFT , I've been trying to follow your steps but am still running into some new issues:

  1. I tried to setup the Private Endpoint but got this error:
    image

  2. I then wanted to delete the delegation, but this failed because the app service was still connected:
    image

  3. I then removed the VNET from the appservice
    image

  4. Then I tried to remove the delegate again which now worked
    image

  5. Then I tried to add the private endpoint again which now worked (Just make sure you take a good look at the settings because it keeps prefilling them with subnets from different resource groups for some reason):
    image

  6. When I now tried to re-add the AppService to the subnet I'm not allowed:
    image

  7. I then created a NSG as you mentioned and associated this with the VNET:
    image

I'm now however quite stumped on what to do next. You mention that the App Service should already have a NSG? So should I not create it? I'm not sure what to do next.

@devedse
Copy link

devedse commented Sep 2, 2020

@Mike-Ubezzi-MSFT , it's been a while since anyone reacted to this topic. Do you have an idea what steps I'm missing?

@devedse
Copy link

devedse commented Sep 2, 2020

Ok I finally managed to get it to work, what I did was the following:

  1. I tried removing the NSG as I don't think we need it, however to remove this I first wanted to disassociate the subnet. This gave me the following error:
    image
  2. Ignoring this error and continuing; I added a new subnet:
    image
  3. I connected the AppService to this new subnet:
    image
  4. I configured the DNS by setting the following app settings:
    image
    https://docs.microsoft.com/en-us/azure/app-service/web-sites-integrate-with-vnet#azure-dns-private-zones
  5. When I now opened KUDU and did an NSLOOKUP it finally resulted in the right IP address:
    image
  6. I then reran my code and suddenly items started popping up in the cosmos db :)

@dawwa
Copy link

dawwa commented Sep 25, 2020

@devedse you mentioned ended up creating another new subset in your most recent update, would you please clarify are the cosmos DB private endpoint and the web app service ended up with a same subset or not?

I went into the issue of CosmosDB requiring a subset without any dedication, but web app service requires a subset with a dedication to servicefarm, which means the two requirements are conflict to each other. @Mike-Ubezzi-MSFT

@devedse
Copy link

devedse commented Sep 25, 2020

@dawwa, they are now both on separate subnets.

@dawwa
Copy link

dawwa commented Sep 25, 2020

@devedse , thanks for the clarification, the section below mentioned the WEBSITE_VNET_ROUTE_ALL is currently not supported in Windows containers., but looks like you were on a Windows container?
https://docs.microsoft.com/en-us/azure/app-service/web-sites-integrate-with-vnet#how-regional-vnet-integration-works

@devedse
Copy link

devedse commented Sep 25, 2020

@dawwa, I was making use of Linux Containers. (I can really suggest to go for Linux containers unless you really have an application that only runs on Windows).

@Mike-Ubezzi-MSFT
Copy link
Contributor

Mike-Ubezzi-MSFT commented Sep 25, 2020

This issue needs to go to support for a deeper investigation as it appears there are some complications with deleting and adding the NSG, etc. Please make sure this step has been completed, if it applies.

Once service endpoint for Azure Cosmos DB is enabled on a subnet, the source of the traffic reaching the account switches from public IP to virtual network and subnet. If your Azure Cosmos account has IP-based firewall only, traffic from service enabled subnet would no longer match the IP firewall rules and therefore be rejected. Go over the steps to seamlessly migrate from IP-based firewall to virtual network-based access control.

Also, this one: Do the peered virtual networks also have access to Azure Cosmos account?

Only virtual network and their subnets added to Azure Cosmos account have access. Their peered VNets cannot access the account until the subnets within peered virtual networks are added to the account.

@devedse
Copy link

devedse commented Nov 30, 2020

@Mike-Ubezzi-MSFT , would it be possible for you to connect me with some Microsoft people. I'm basically trying to deploy the whole solution through Terraform but there's so much issues I'm running into, for example:

  1. Not able to deploy Azure Functions due to 503 errors
  2. Azure Function deployments taking more then 60 minutes and then running into a timeout
  3. The Azure Function runtime not working on the Azure Portal. (When I open an Azure Function it simply doesn't load the web page and shows an error in the portal)

All of it seems to be related to having some things in VNET's but I can't figure out how it should all integrate.

Mainly because sometimes the first deployment does work, so it seems my code is correct (sometimes it doesn't though). But when you do subsequent deployment everything breaks down.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests