Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Azure Search] Remap legacy DNS to point to Azure Search Service #7628

Closed
loic-sharma opened this issue Oct 15, 2019 · 8 comments
Closed

[Azure Search] Remap legacy DNS to point to Azure Search Service #7628

loic-sharma opened this issue Oct 15, 2019 · 8 comments
Assignees

Comments

@loic-sharma
Copy link
Member

@loic-sharma loic-sharma commented Oct 15, 2019

There is still large amounts of traffic going to legacy search (App Insights query). Things to do:

  • Understand CORS policy? Talk to Barry
  • Consider updating the DNS to point legacy search to azure search. See: NuGet/Engineering#2841

Raw query:

requests
| where timestamp > ago(1d)
| where name in ("GET /search/diag", "GET /search/query", "GET /query", "GET /autocomplete")
//| where url !contains "/search/query?q=packageid:E2E." // End-to-end tests
| extend dId = tostring(customDimensions.DeploymentId)
| extend Region =
  iff(dId == "a755594844164679a1ca264946f81f18" or dId == "233d868eef7b47ddacbc138a147a50c3", "USNC",
     iff(dId == "86b2d4f718ee498996291ab94a08febf" or dId == "c4e86a9343a44b7ba14593a720e5c13c", "USSC",
       iff(dId == "fe4df56377be435a97eded74a7529f98" or dId == "d293fb80632a41cbbfc535e9adeb5c2c", "EA",
         iff(dId == "15493afe371f43e99924731983887688" or dId == "e3f92c47a56342a0bd36f3f971700184", "SEA", "Unknown")
       )
     )
  )
| extend RegionAndRequestName = strcat(Region, " ", name) 
| summarize count() by bin(timestamp, 30m), RegionAndRequestName, Region, name
| order by count_ desc
| render barchart kind=stacked 

This investigation needs to be done before we can delete legacy search resources.

@joelverhagen

This comment has been minimized.

Copy link
Member

@joelverhagen joelverhagen commented Oct 31, 2019

For the past two days, the traffic is very low.

Pattern Number Remaining
Total 240656 240656
/search/diag 175056 65600
/ 41101 24499
404 1827 22672
Application Insights Monitoring 15051 7621
q=Microsoft.ApplicationInsights.SnapshotCollector 6965 656
Zabbix 299 357
/autocomplete 26 331
/query 159 172
Non-existent endpoints 172 0

The queries for Microsoft.ApplicationInsights.SnapshotCollector are a known issue that we are tracking with the Application Insights team.

The /autocomplete and /query hits are low and are all (except 1) to EA. My theory is that there is a proxy or copied service index that someone has set up -- perhaps the China replicas that we heard a lot about before we set up the China clone. The one hit to USSC was from Denmark and the remaining hits (184) were all from China.

My proposal is to wait for the Application Insights team to move off of legacy search (microsoft/ApplicationInsights-Home#397) then turn off all regions.

@joelverhagen

This comment has been minimized.

Copy link
Member

@joelverhagen joelverhagen commented Nov 12, 2019

This is blocked by NuGet/Engineering#2835.

@joelverhagen

This comment has been minimized.

Copy link
Member

@joelverhagen joelverhagen commented Nov 12, 2019

Both IIS logs and Application Insights corroborate the fact that SEA and EA traffic is extremely low. I will proceed with shutting these down.

@joelverhagen

This comment has been minimized.

Copy link
Member

@joelverhagen joelverhagen commented Nov 15, 2019

CORS is no longer supported. This may be a problem for Azure Portal:
Legacy CORS
Azure Search CORS

@loic-sharma

This comment has been minimized.

Copy link
Member Author

@loic-sharma loic-sharma commented Dec 3, 2019

Talked offline to Joel:

  1. Use IIS logs to find common referrers
  2. See Joel's notes here
  3. Reach out to Azure Site Extensions and App Insights to confirm domains for CORS policy
@joelverhagen

This comment has been minimized.

Copy link
Member

@joelverhagen joelverhagen commented Dec 3, 2019

If we can convince Barry that we can keep a cors policy for all domains we can probably save a lot of time with emails and analysis

joelverhagen added a commit to NuGet/NuGet.Services.Metadata that referenced this issue Dec 5, 2019
joelverhagen added a commit to NuGet/NuGet.Services.Metadata that referenced this issue Dec 5, 2019
@joelverhagen joelverhagen changed the title [Azure Search] Investigate traffic to legacy search [Azure Search] Deploy CORS and remap legacy DNS Dec 10, 2019
@joelverhagen joelverhagen changed the title [Azure Search] Deploy CORS and remap legacy DNS [Azure Search] Remap legacy DNS to point to Azure Search Service Dec 18, 2019
@joelverhagen

This comment has been minimized.

Copy link
Member

@joelverhagen joelverhagen commented Dec 18, 2019

CORS for all domains is deployed to all environments. The remapping can commence.

@joelverhagen

This comment has been minimized.

Copy link
Member

@joelverhagen joelverhagen commented Jan 13, 2020

This is done on both PROD USNC and PROD USSC.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.