Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Code-First Federation 2 support #3144

Closed
wants to merge 1 commit into from
Closed

Conversation

Mithras
Copy link
Contributor

@Mithras Mithras commented May 19, 2022

Subj.

Some details on what's in the PR:

  1. Added .devcontainer because I don't want to install 9000 SDKs and packages on my host. OmniSharp (Intellisense) is half broken though because it works only with NET 6. This is not related to Federation and I can remove it if you don't want it. Or I can try to fix OmniSharp if you are interested in developing in containers.
  2. Separate GraphQL.Federation.Tests project with a couple INTEGRATION tests (run in 75ms though). I'm not sure you want integration tests and you might prefer 9000 unit tests instead so I made it separate so that it's easy to get rid of. I don't mind adding unit tests, just didn't want to waste time as I'm not sure this PR will ever get reviewed/merged. The tests are pretty simple and self explanatory, see FederationFieldTests.
  3. A bunch of NEW stuff under GraphQL.Federation namespace.
    3.1. Attributes.* - All Federation 2 directives as attributes.
    3.2 Extensions.FederationFieldExtensions - All Federation 2 directives as FieldType/FieldBuilder extensions.
    3.3 Extensions.FederationGraphTypeExtensions - All Federation 2 directives as GraphType extensions. Also, .AddServices() and .AddEntities() to add Federation fields explicitly/manually (e.g. when you want to specify .AuthorizeWith() on them).
    3.4 Extensions.FederationGraphQLBuilderExtensions - AddFederation() extension. See src/GraphQL.Federation.Tests/ServiceProviderFixture.cs.
    3.5 Schema.* - A few output types to support Federation.
    3.6. A bunch of internal classes.

There are a few questionable hacks:

  1. EntityType adds pointless NeverType because that's the only way I can pass validation that triggers BEFORE I have a chance to add any Federated entities to the EntityType which I do in FederationEntitiesSchemaNodeVisitor.
  2. FederationQuerySchemaNodeVisitor adds _service and _entities in a bit of a weird way (injects GraphType instances in ctor) because it doesn't want to work otherwise. Probably something about order of when schema initializes and visitors are called.
  3. FederationQuerySchemaNodeVisitor.ResolveEntities() doesn't check authorization at an entity level. Only at a field level.
  4. [Ignore] I also copy-pasted GraphTypeToClrType() from our code which I feel like should be provided by graphql-dotnet by default but I'm not aware of any other ways of mapping Graph <-> CLR types. It's used by FederationQuerySchemaNodeVisitor.ResolveEntities().

Hope this won't get stuck for years like #1669 did.

@Shane32
Copy link
Member

Shane32 commented May 19, 2022

At first glance I like the look of it!

@joemcbride
Copy link
Member

Just popping in for a moment. This does look pretty interesting.

A reminder that there are already some federation types that look almost exactly the same as these used with schema-first. Probably want to figure out some reuse there.

https://github.com/graphql-dotnet/graphql-dotnet/tree/master/src/GraphQL/Utilities/Federation

@Mithras
Copy link
Contributor Author

Mithras commented May 19, 2022

A reminder that there are already some federation types that look almost exactly the same as these used with schema-first. Probably want to figure out some reuse there.

I think I re-used what I can (only AnyScalarGraphType really). The rest has breaking changes (I'm not sure how existing Federation is used if at all). I can merge them if we are ok with breaking changes:

  • Existing FederatedSchemaPrinter doesn't support new Federation 2 directives and doesn't add @link directive to schema. You can say that existing one is for Federation 1 and the new one is for Federation 2.
  • The main difference between existing IFederatedResolver and my IFederationResolver is that the later supports all 3 sync, async and DataLoader scenarios while the existing one ONLY supports async. See 3 different overloads here
  • Existing ServiceGraphType uses Federation 1 printer, while my ServiceGraphType uses Federation 2 printer.
  • The rest under Utilities/Federation are schema-first specific and I don't really need/have them.

Also, I'm not sure what's the story with schema-first approach in general. Is it something graphql-dotnet actively supports?

@joemcbride
Copy link
Member

joemcbride commented May 19, 2022

My point was - I think it would be fairly awkward to have two different implementations of federation in the same project. If we're upgrading federation support to add GraphType First support, it should be upgraded everywhere and not work differently when using Schema First vs. GraphType First.

Also, I'm not sure what's the story with schema-first approach in general. Is it something graphql-dotnet actively supports?

I certainly hope so since I actively use it. 😉

Schema First at it's core uses the same GraphQL Framework. It is just a light schema builder on top of the core engine. From that regard, it is basically the same as the AutoRegisteringGraphType. Schema First is where the GraphQLAttribute was first conceived and added, which is heavily used by this PR.

@Mithras
Copy link
Contributor Author

Mithras commented May 19, 2022

@joemcbride Let me first look into supporting schema-first in my implementation and then we can decide on either merging or deprecating the old one completely.

@Mithras
Copy link
Contributor Author

Mithras commented May 19, 2022

@joemcbride do you know why I see this changes when running tests locally but in CI they are not present?
/workspaces/graphql-dotnet/src/GraphQL.ApiTests/netstandard20+netstandard21/GraphQL.approved.txt
image

@Shane32
Copy link
Member

Shane32 commented May 19, 2022

@joemcbride do you know why I see this changes when running tests locally but in CI they are not present? /workspaces/graphql-dotnet/src/GraphQL.ApiTests/netstandard20+netstandard21/GraphQL.approved.txt image

It's because the latest version of Visual Studio was released a couple days ago. We need to merge this PR (and you need to be running the latest version of VS also, which presumably you are which is why you are having the issue):

I'll merge it now; then merge master into your PR.

@Mithras
Copy link
Contributor Author

Mithras commented May 19, 2022

Oh, ok. I rolled back these lines for now. I'll re-run tests after rebasing.

@Mithras
Copy link
Contributor Author

Mithras commented May 19, 2022

I also see these changes which also seems to be VS Code version related:
image

@Shane32
Copy link
Member

Shane32 commented May 19, 2022

I also see these changes which also seems to be VS Code version related

I can't help there; I never use VS Code. As far as I know, @sungam3r does not either; but perhaps @joemcbride does.

@Shane32
Copy link
Member

Shane32 commented May 19, 2022

Schema First is where the GraphQLAttribute was first conceived and added, which is heavily used by this PR.

Keep in mind that not all of GraphQLAttribute.Modify methods get applied to schema-first; and there are other Modify overloads that are called only for schema-first. I can't remember the exact details offhand, but a little testing can determine if the attributes you wrote are working as intended or not for schema-first. If not, you should be able to modify the attribute to support both.

@codecov-commenter
Copy link

Codecov Report

Merging #3144 (b3b193c) into master (11bbc31) will increase coverage by 0.06%.
The diff coverage is 82.59%.

@@            Coverage Diff             @@
##           master    #3144      +/-   ##
==========================================
+ Coverage   84.43%   84.49%   +0.06%     
==========================================
  Files         371      391      +20     
  Lines       16087    16541     +454     
  Branches     2599     2638      +39     
==========================================
+ Hits        13583    13977     +394     
- Misses       1882     1915      +33     
- Partials      622      649      +27     
Impacted Files Coverage Δ
src/GraphQL/Federation/AsyncFederationResolver.cs 0.00% <0.00%> (ø)
...GraphQL/Federation/Attributes/RequiresAttribute.cs 53.84% <53.84%> (ø)
src/GraphQL/Federation/Attributes/KeyAttribute.cs 57.14% <57.14%> (ø)
...ration/Extensions/FederationGraphTypeExtensions.cs 59.37% <59.37%> (ø)
...GraphQL/Federation/Attributes/ExternalAttribute.cs 60.00% <60.00%> (ø)
...hQL/Federation/Attributes/InaccessibleAttribute.cs 75.00% <75.00%> (ø)
...raphQL/Federation/Attributes/ShareableAttribute.cs 75.00% <75.00%> (ø)
...GraphQL/Federation/Attributes/OverrideAttribute.cs 80.00% <80.00%> (ø)
src/GraphQL/Federation/FuncFederationResolver.cs 80.00% <80.00%> (ø)
...Federation/Extensions/FederationFieldExtensions.cs 80.48% <80.48%> (ø)
... and 16 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 11bbc31...b3b193c. Read the comment docs.

@Mithras Mithras force-pushed the master branch 3 times, most recently from 8ed7f5b to e762183 Compare May 23, 2022 22:03
@Mithras
Copy link
Contributor Author

Mithras commented May 23, 2022

I've added schema-first Federation 2 support. Can you please review and let me know if it's ok to remove existing Federation classes?

@Mithras
Copy link
Contributor Author

Mithras commented Jun 1, 2022

bump

@Mithras
Copy link
Contributor Author

Mithras commented Jan 18, 2023

Hi,
I've changed teams and don't have any Federated services here for now so refactoring this PR wasn't a priority. I'll try to rebase/refactor it this or maybe next week. It shouldn't take more than an hour or two most.

@Shane32
Copy link
Member

Shane32 commented Jan 18, 2023

If @sungam3r does not have time soon (it sounds like he's got a lot on his plate), then maybe I can review it. We should try to get this in v8.

@sungam3r
Copy link
Member

Definetly in v8.

@dariuszkuc
Copy link

dariuszkuc commented Jan 19, 2023

BTW just an FYI, I recently externalized the Federation Compatibility Tests -> you can now run them locally using NPX (https://www.npmjs.com/package/@apollo/federation-subgraph-compatibility) or through Github Actions (https://github.com/marketplace/actions/apollo-federation-subgraph-compatibility).

Example GH action integration (workflow is invoked on the PRs) -> https://github.com/apollographql/federation-jvm/blob/master/.github/workflows/federation-compatibility.yaml#L35
Action will then comment on the PRs -> apollographql/federation-jvm#285 (comment)

@Mithras
Copy link
Contributor Author

Mithras commented Feb 2, 2023

@sungam3r

How does it feel on NET7 ?

Feels pretty good actually. Omnisharp works out of the box which has never happened with previous .NET versions.

I will prefer to accept backward incompatible changes than to have two different implementations on support.

As I have said, I've added new implementation for Feraration 2 with DataLoader, CodeFirst and SchemaFirst support. I intentionally haven't touched old implementation as there is pretty much zero overlap. Old implementation can be removed in like 10 minutes.

Regarding VS Code configuration files. Yes, I don't use VS Code and I'm fine to change these files in any way you see fit. Or simply remove them from this repo and add into gitignore.

I let VS Code update tasks.json and made sure it works.

@joemcbride said I certainly hope so since I actively use it. in #3144 (comment)

Well I'm not sure how it's possible to use existing implementation which doesn't work with DataLoaders for anything more than Hello World.


Anyway, I moved everything to GraphQL.Federation and GraphQL.Federation.Tests accordingly. There are minimal changes outside of the two.

@Mithras
Copy link
Contributor Author

Mithras commented Feb 2, 2023

@sungam3r

Regarding unit tests with spaghetti string literals. I recommend you to switch to raw string literals that are available since C#11.

done.

@sungam3r
Copy link
Member

sungam3r commented Feb 8, 2023

@Mithras thanks, I remember about your work and this one in my todo list just after OpenTelemetry PR.

@ViliusVZ
Copy link

@Mithras @sungam3r any plans to merge this anytime soon?

@Shane32
Copy link
Member

Shane32 commented May 30, 2023

If @sungam3r doesn't have time to merge it for v8, I’ll try to do so. However, v8 isn’t planned to be released anytime soon.

@michael-watson
Copy link

Is there anything I can do to help move this PR along? I'm happy to spend some cycles completing out any last items ❤️

@Shane32
Copy link
Member

Shane32 commented Aug 23, 2023

@michael-watson @Mithras Yes, I'm ready to help get this merged for v8. Recently in #3660 we have added CI tests for schema-first federation. We need similar projects added to this PR so that there is CI tests for code-first federation. We also need all new work to have XML comments. And then we need to review the level of support for Federation v2. (ok if not 100%, but we need it documented where we are at) Finally, we need approval from @joemcbride to add GraphQL.Federation nuget package. Then ready to review and merge.

@Mithras
Copy link
Contributor Author

Mithras commented Aug 23, 2023

There is an integration test that tests most of everything. Anyway, I'm on vacation and probably won't even have internet for 1.5 weeks

@Shane32
Copy link
Member

Shane32 commented Aug 23, 2023

No problem @Mithras

The purpose is to have end to end tests plus have the code double as a sample project. So the new schema-first tests demonstrate two fully functional GraphQL subgraphs created in GraphQL.NET, and the tests start hosting them at two endpoints and have Apollo Federation create a super graph by querying those endpoints, and then start Apollo Router, followed by actually running a query that pulls data from both subgraphs and verifying the result. I’d like to do the same for code-first federation. Sorry I’ve not looked this PR recently to see what tests exist.

@michael-watson michael-watson mentioned this pull request Oct 2, 2023
6 tasks
@michael-watson
Copy link

@Shane32 I've been working on bringing all of the Federation code together in one place and I put everything in PR #3723. I can't run the github actions locally because of NUGET_AUTH_TOKEN. How can I work around this to properly test my code with what you have today?

@Shane32
Copy link
Member

Shane32 commented Oct 2, 2023

@Shane32 I've been working on bringing all of the Federation code together in one place and I put everything in PR #3723. I can't run the github actions locally because of NUGET_AUTH_TOKEN. How can I work around this to properly test my code with what you have today?

The federation tests doesn't use the NUGET_AUTH_TOKEN -- it only uses the GITHUB_TOKEN and only to pull the repository.

See https://github.com/graphql-dotnet/graphql-dotnet/blob/master/.github/workflows/test-code.yml and scroll down to the testfederation part. Basically it just:

  1. Builds and spins up a couple of the sample projects
  2. Runs rover supergraph compose
  3. Starts router
  4. Runs a few test queries against router
  5. Validates the responses

All the rest of the tests you can run from Visual Studio.

@Shane32
Copy link
Member

Shane32 commented Oct 2, 2023

I'd suggest just writing a powershell script to perform the same actions, or something like that. Here's ChatGPT's rough draft:

# Change to src directory
Set-Location -Path .\src

# Install dependencies for Sample1
dotnet restore .\GraphQL.Federation.SchemaFirst.Sample1

# Build Sample1 and Sample2
dotnet build --no-restore .\GraphQL.Federation.SchemaFirst.Sample1
dotnet build .\GraphQL.Federation.SchemaFirst.Sample2

# Start Sample1 and Sample2
Start-Process dotnet -ArgumentList "run --no-build --project GraphQL.Federation.SchemaFirst.Sample1"
Start-Process dotnet -ArgumentList "run --no-build --project GraphQL.Federation.SchemaFirst.Sample2"

# Install Apollo Rover CLI
Invoke-RestMethod -Uri 'https://rover.apollo.dev/nix/latest' | Out-File -FilePath '.\rover_install.sh'
Start-Process "sh" -ArgumentList ".\rover_install.sh"

# Install jq
Start-Process "sudo" -ArgumentList "apt-get install -y jq"

# Wait for Sample1 and Sample2 to spin up
Function WaitForService ($url) {
    For ($i=1; $i -le 60; $i++) {
        Write-Host "Request $i to the URL..."
        $response = Invoke-RestMethod -Uri $url -Method Get -StatusCodeVariable statusCode
        if ($statusCode -eq 200) {
            Write-Host "Received 200 response, step completed."
            return
        }
        Write-Host "Did not receive a 200 response, sleeping for 0.5 second..."
        Start-Sleep -Milliseconds 500
    }
    Write-Host "Timed out after 30 seconds, step failed."
    Exit 1
}
WaitForService 'http://localhost:5601'
WaitForService 'http://localhost:5602'

# Change to src/Federation directory
Set-Location -Path .\Federation

# Build supergraph
& "$HOME/.rover/bin/rover" supergraph compose --config .\federation-$env:FEDERATION_VERSION-supergraph-config.yaml --elv2-license=accept | Out-File -FilePath '.\supergraph.graphql'

# Print supergraph
Get-Content -Path .\supergraph.graphql

# Download and Start router
Invoke-RestMethod -Uri 'https://router.apollo.dev/download/nix/latest' | Out-File -FilePath '.\router_install.sh'
Start-Process "sh" -ArgumentList ".\router_install.sh"
Start-Process .\router -ArgumentList "--dev --supergraph .\supergraph.graphql"

# Wait for router to spin up
WaitForService 'http://localhost:8088/health'

# Run GraphQL queries
Invoke-RestMethod -Uri 'http://127.0.0.1:4000' -Method Post -Headers @{"Content-Type" = "application/json"} -InFile '.\federation-request-1.json' | Out-File -FilePath '.\response-1.json'
Invoke-RestMethod -Uri 'http://127.0.0.1:4000' -Method Post -Headers @{"Content-Type" = "application/json"} -InFile '.\federation-request-2.json' | Out-File -FilePath '.\response-2.json'
Invoke-RestMethod -Uri 'http://127.0.0.1:4000' -Method Post -Headers @{"Content-Type" = "application/json"} -InFile '.\federation-request-3.json' | Out-File -FilePath '.\response-3.json'

# Function to compare JSON files
Function CompareJsonFiles ($actualFile, $expectedFile) {
    $actualContent = Get-Content -Path $actualFile | ConvertFrom-Json | ConvertTo-Json
    $expectedContent = Get-Content -Path $expectedFile | ConvertFrom-Json | ConvertTo-Json
    if ($actualContent -eq $expectedContent) {
        Write-Host "$actualFile and $expectedFile are identical."
    } else {
        Write-Host "$actualFile and $expectedFile differ."
        Exit 1
    }
}

# Compare query result 1 to expected response
$actualResponse1 = "actual-response-1.json"
$expectedResponse1 = "expected-response-1.json"
Get-Content -Path "response-1.json" | Out-File -FilePath $actualResponse1
Get-Content -Path "federation-response-1.json" | Out-File -FilePath $expectedResponse1
CompareJsonFiles $actualResponse1 $expectedResponse1

# Compare query result 2 to expected response
$actualResponse2 = "actual-response-2.json"
$expectedResponse2 = "expected-response-2.json"
Get-Content -Path "response-2.json" | Out-File -FilePath $actualResponse2
Get-Content -Path "federation-response-2.json" | Out-File -FilePath $expectedResponse2
CompareJsonFiles $actualResponse2 $expectedResponse2

# Compare query result 3 to expected response
$actualResponse3 = "actual-response-3.json"
$expectedResponse3 = "expected-response-3.json"
Get-Content -Path "response-3.json" | Out-File -FilePath $actualResponse3
Get-Content -Path "federation-response-3.json" | Out-File -FilePath $expectedResponse3
CompareJsonFiles $actualResponse3 $expectedResponse3

Obviously you need changes for running on Windows, etc, even if it didn't make any mistakes.

@Shane32
Copy link
Member

Shane32 commented Oct 2, 2023

If you devise a good powershell script, add some comments that explain how to use it, post a PR with the file, and I'll merge it in.

@Shane32
Copy link
Member

Shane32 commented Oct 2, 2023

@michael-watson By the way, I'm okay with having a separate NuGet package for federation, or having it in the main GraphQL NuGet package.

@Shane32
Copy link
Member

Shane32 commented Jun 28, 2024

Closing in favor of #3921, which extended this PR with additional functionality. Thank you for your contributions @Mithras !

@Shane32 Shane32 closed this Jun 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
federation Relates to GraphQL Federation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet