Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add XDP, AF_XDP, and XDP driver docs in markdown #344

Merged
merged 3 commits into from
Jul 21, 2023
Merged

Add XDP, AF_XDP, and XDP driver docs in markdown #344

merged 3 commits into from
Jul 21, 2023

Conversation

mtfriesen
Copy link
Contributor

Crude progress towards #12

  • Do not document experimental APIs yet
  • Many TODOs left

@mtfriesen mtfriesen requested a review from a team as a code owner July 20, 2023 20:20
@anrossi
Copy link

anrossi commented Jul 20, 2023

Just a couple things that should be 5 seconds to fix. Looks great! Thanks for doing this work

nibanks
nibanks reviewed Jul 21, 2023
View reviewed changes
docs/afxdp.md Outdated Show resolved Hide resolved
nibanks
nibanks reviewed Jul 21, 2023
View reviewed changes
docs/api/XDP_QUEUE_INFO.md Outdated Show resolved Hide resolved
nibanks
nibanks reviewed Jul 21, 2023
View reviewed changes
docs/api/XdpLoadApi.md Outdated Show resolved Hide resolved
nibanks
nibanks reviewed Jul 21, 2023
View reviewed changes
docs/api/XdpUnloadApi.md Outdated Show resolved Hide resolved
nibanks
nibanks reviewed Jul 21, 2023
View reviewed changes
docs/afxdp.md

> **TODO** Complete AF_XDP documentation.

## See Also
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see all other docs are dropped in the api directory. What do you say to splitting that out, instead having an afxdp directory for the UM API, and then putting the other related functions and types in a different relevant dir?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I've wondered about organization. Part of the issue is our long term goal is to share types across various XDP layers, so the same offload extensions get used by both native driver and AF_XDP, for example. I agree we have tons of docs here and some kind of grouping or sorting would be useful.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I definitely think it would be useful. Ideally adding some split now would be good, but I won't block the PR on it if you want to do it in a follow up.

Copy link
Contributor Author

@mtfriesen mtfriesen Jul 21, 2023 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The See Also section of afxdp.md was an attempt at providing that kind of index, since each header/struct/function/enum/define can belong to multiple API families by design.

@mtfriesen mtfriesen enabled auto-merge (squash) July 21, 2023 13:28
@mtfriesen mtfriesen merged commit 1b4d8a5 into main Jul 21, 2023
41 checks passed
@mtfriesen mtfriesen deleted the mtfriesen/docs branch July 21, 2023 14:19
brentyates-iex pushed a commit to swxtchio/xdp-for-windows that referenced this pull request Aug 22, 2023
@mafrankSWXTCH @dependabot
@mtfriesen @cdammanintopix @nibanks @csujedihy @gtrevi
Sync repo (#3)
3f35a34 
* Bump submodules/wil from `3e2ee37` to `0b2d6c2` (microsoft#175)

Bumps [submodules/wil](https://github.com/microsoft/wil) from `3e2ee37` to `0b2d6c2`.
- [Release notes](https://github.com/microsoft/wil/releases)
- [Commits](microsoft/wil@3e2ee37...0b2d6c2)

---
updated-dependencies:
- dependency-name: submodules/wil
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Upgrade to the latest prerelease build of eBPF (microsoft#180)

* upgrade to the latest prerelease build of eBPF, including minor version bump to 0.7

* eBPF: relax NPI client dispatch table requirement

* Bump actions/checkout from 3.3.0 to 3.5.0 (microsoft#181)

Bumps [actions/checkout](https://github.com/actions/checkout) from 3.3.0 to 3.5.0.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](actions/checkout@ac59398...8f4b7f8)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump submodules/wil from `0b2d6c2` to `1340b8a` (microsoft#176)

Bumps [submodules/wil](https://github.com/microsoft/wil) from `0b2d6c2` to `1340b8a`.
- [Release notes](https://github.com/microsoft/wil/releases)
- [Commits](microsoft/wil@0b2d6c2...1340b8a)

---
updated-dependencies:
- dependency-name: submodules/wil
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* consume new ebpf extension definitions (microsoft#183)

* improve tracing of XDP programs (microsoft#182)

* program: discard IPv4 packets with options (microsoft#184)

* Use volatile accessors for reads from user mode (microsoft#188)

* use volatile accessors for reads from user mode

* add gratuitous casts for VS2019 builds

* update to a newer eBPF 0.7.0 build (microsoft#191)

* Bump submodules/wil from `1340b8a` to `70155eb` (microsoft#190)

Bumps [submodules/wil](https://github.com/microsoft/wil) from `1340b8a` to `70155eb`.
- [Release notes](https://github.com/microsoft/wil/releases)
- [Commits](microsoft/wil@1340b8a...70155eb)

---
updated-dependencies:
- dependency-name: submodules/wil
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* assert MDL reuse correctness in generic XDP (microsoft#192)

* upgrade to eBPF 0.8.0 release (official) (microsoft#194)

* Bump submodules/wil from `70155eb` to `9db6276` (microsoft#199)

Bumps [submodules/wil](https://github.com/microsoft/wil) from `70155eb` to `9db6276`.
- [Release notes](https://github.com/microsoft/wil/releases)
- [Commits](microsoft/wil@70155eb...9db6276)

---
updated-dependencies:
- dependency-name: submodules/wil
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump actions/checkout from 3.5.0 to 3.5.2 (microsoft#200)

Bumps [actions/checkout](https://github.com/actions/checkout) from 3.5.0 to 3.5.2.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](actions/checkout@8f4b7f8...8e5e7e5)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump submodules/wil from `9db6276` to `fc5dbf5` (microsoft#202)

Bumps [submodules/wil](https://github.com/microsoft/wil) from `9db6276` to `fc5dbf5`.
- [Release notes](https://github.com/microsoft/wil/releases)
- [Commits](microsoft/wil@9db6276...fc5dbf5)

---
updated-dependencies:
- dependency-name: submodules/wil
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* collect full kernel dumps in Azure pipelines (microsoft#203)

* enable 1GB circular buffer for ebpf logging in spinxsk (microsoft#204)

* ensure EC passive worker gets boosted during cleanup (microsoft#205)

* Bump submodules/wil from `fc5dbf5` to `1c6126b` (microsoft#206)

Bumps [submodules/wil](https://github.com/microsoft/wil) from `fc5dbf5` to `1c6126b`.
- [Release notes](https://github.com/microsoft/wil/releases)
- [Commits](microsoft/wil@fc5dbf5...1c6126b)

---
updated-dependencies:
- dependency-name: submodules/wil
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* ingest eBPF 0.9.0 (microsoft#207)

* Bump submodules/wil from `1c6126b` to `57a57cd` (microsoft#209)

Bumps [submodules/wil](https://github.com/microsoft/wil) from `1c6126b` to `57a57cd`.
- [Release notes](https://github.com/microsoft/wil/releases)
- [Commits](microsoft/wil@1c6126b...57a57cd)

---
updated-dependencies:
- dependency-name: submodules/wil
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* submodule QEO and plumb stub interface (microsoft#211)

* Add XDPFNMP support for direct OIDs (microsoft#210)

* Add bare minimum QEO offload support (microsoft#214)

* collect complete crash dumps on local machines, too (microsoft#215)

* s/flavor/config/g (microsoft#216)

* Improve offload synchronization (microsoft#217)

* Bumb XDP Patch Version (microsoft#219)

* Bump submodules/quic-offloads from `d08bf41` to `fc6f00b` (microsoft#220)

Bumps [submodules/quic-offloads](https://github.com/microsoft/quic-offloads) from `d08bf41` to `fc6f00b`.
- [Commits](microsoft/net-offloads@d08bf41...fc6f00b)

---
updated-dependencies:
- dependency-name: submodules/quic-offloads
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Ensure offloads are cleaned up during detach (microsoft#222)

* many hours of work :) (microsoft#223)

* split offload file into per-offload-type modules (microsoft#224)

* enable functional and spinxsk tests to run with eBPF preinstalled (microsoft#226)

* Outline of an Initial Threat Model (microsoft#229)

* Initial Release and Support Documentation (microsoft#227)

* Fix deadlock in LWF unload (microsoft#230)

* Enforce QEO connection offload state (microsoft#231)

* disable untested, experimental features in official builds (microsoft#232)

* conditionally re-copy the pattern buffer into each RX packet (microsoft#235)

* Consume new NDIS OID definition and fall back to IPSec prototype OID (microsoft#236)

* Bump actions/setup-dotnet from 3.0.3 to 3.1.0 (microsoft#238)

Bumps [actions/setup-dotnet](https://github.com/actions/setup-dotnet) from 3.0.3 to 3.1.0.
- [Release notes](https://github.com/actions/setup-dotnet/releases)
- [Commits](actions/setup-dotnet@607fce5...aa983c5)

---
updated-dependencies:
- dependency-name: actions/setup-dotnet
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Update uninstall instructions (microsoft#239)

* upgrade to 0.9.0 postrelease fix (microsoft#237)

* configure linker to load xdpapi dependencies only from system32 (microsoft#240)

* Add some more detail to our threat model (microsoft#241)

* Bump submodules/net-offloads from `b5fc47d` to `02dbf4e` (microsoft#244)

Bumps [submodules/net-offloads](https://github.com/microsoft/net-offloads) from `b5fc47d` to `02dbf4e`.
- [Commits](microsoft/net-offloads@b5fc47d...02dbf4e)

---
updated-dependencies:
- dependency-name: submodules/net-offloads
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump submodules/wil from `57a57cd` to `08ce919` (microsoft#245)

Bumps [submodules/wil](https://github.com/microsoft/wil) from `57a57cd` to `08ce919`.
- [Release notes](https://github.com/microsoft/wil/releases)
- [Commits](microsoft/wil@57a57cd...08ce919)

---
updated-dependencies:
- dependency-name: submodules/wil
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump actions/setup-dotnet from 3.1.0 to 3.2.0 (microsoft#246)

Bumps [actions/setup-dotnet](https://github.com/actions/setup-dotnet) from 3.1.0 to 3.2.0.
- [Release notes](https://github.com/actions/setup-dotnet/releases)
- [Commits](actions/setup-dotnet@aa983c5...3447fd6)

---
updated-dependencies:
- dependency-name: actions/setup-dotnet
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Allow configuration of non-admin XDP users (microsoft#248)

* enable OS component tracing in functional tests (microsoft#253)

* Bump functional test timeouts (microsoft#250)

* wait for NDIS in restart test, outside of watchdog (microsoft#251)

* catch eBPF uninstall hangs (microsoft#255)

* Use 1ES hosted pools for GitHub actions (microsoft#257)

* avoid WMI query during eBPF uninstall (microsoft#260)

* bump miniport restart timeout (microsoft#261)

* always test eBPF in spinxsk CI (microsoft#263)

* force powershell job cleanup (microsoft#264)

* Fix the only AzWatson bugcheck in the last 7 days (microsoft#262)

* use default spinxsk queue count (microsoft#266)

* dust off the XSK perf tests and run them in CI (microsoft#268)

* Bump actions/checkout from 3.5.2 to 3.5.3 (microsoft#269)

Bumps [actions/checkout](https://github.com/actions/checkout) from 3.5.2 to 3.5.3.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](actions/checkout@8e5e7e5...c85c95e)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Add functional test watchdog (microsoft#267)

* use plain PowerShell in GitHub Actions (microsoft#271)

* cut perf matrix in half (microsoft#270)

* continue on error (microsoft#272)

* fix UDP checksum ipv6 logic (microsoft#273)

* Improve TCP test reliability (microsoft#274)

* continue to try to fix TCP test case (microsoft#276)

* add more perf coverage (microsoft#275)

* log system commands in functional tests (microsoft#277)

* remove TODO in RSS query initialization (microsoft#279)

* add more eBPF uninstall diagnostics (microsoft#280)

* consistently use xdp.cpp.default.props (microsoft#281)

* target WS2019 ABI

* cleanup

* Add downlevel VS2019 build (microsoft#282)

* dummy commit

* Upload perf data (microsoft#283)

* Depend on specific build of corenet-ci repo (microsoft#284)

* Link to Perf Dashboard (microsoft#287)

* Bump submodules/wil from `08ce919` to `9eb9851` (microsoft#289)

Bumps [submodules/wil](https://github.com/microsoft/wil) from `08ce919` to `9eb9851`.
- [Release notes](https://github.com/microsoft/wil/releases)
- [Commits](microsoft/wil@08ce919...9eb9851)

---
updated-dependencies:
- dependency-name: submodules/wil
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump actions/checkout from 3.5.2 to 3.5.3 (microsoft#290)

Bumps [actions/checkout](https://github.com/actions/checkout) from 3.5.2 to 3.5.3.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](actions/checkout@v3.5.2...c85c95e)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* don't out of bounds argv (microsoft#291)

* add new ForPerfTest profile to prepare-machine.ps1. (microsoft#292)

* Add simple AF_XDP sample (microsoft#295)

* Use enum for XdpCreateProgram flags (microsoft#296)

* remove nonstandard shared umem (microsoft#297)

* Delete changelog.md (microsoft#299)

* Move .props files to src directory (microsoft#298)

* Add new MSI installer project - `xdpinstaller` (microsoft#285)

* invoke prepare-machine from anywhere (microsoft#301)

* remove cleanup artifacts script (microsoft#302)

* Bump submodules/wil from `9eb9851` to `d784315` (microsoft#305)

Bumps [submodules/wil](https://github.com/microsoft/wil) from `9eb9851` to `d784315`.
- [Release notes](https://github.com/microsoft/wil/releases)
- [Commits](microsoft/wil@9eb9851...d784315)

---
updated-dependencies:
- dependency-name: submodules/wil
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* reduce spinxsk success threshold for msquic pool (microsoft#303)

* check drivers after prepare-machine (microsoft#304)

* Integrate installer with XDP project (microsoft#300)

* bugchecks aren't being captured, so try again with live dumps (microsoft#308)

* limit test runtime in GH Actions (microsoft#309)

* try again with GitHub actions timeouts (microsoft#313)

* Use XDP MSI in CI tests (microsoft#312)

* add IO work item logging (microsoft#317)

* Fix perf runs with MSI (microsoft#316)

* improve eBPF uninstall diagnostics (microsoft#318)

* Add MSI documentation (microsoft#314)

* more perf script fixes (microsoft#319)

* use regular, kernel-only crash dumps in CI (microsoft#322)

* Wait for work items before unloading XDP (microsoft#321)

* provide MSI in kits instead of raw binaries (microsoft#323)

* Bump submodules/wil from `d784315` to `f1b694f` (microsoft#324)

Bumps [submodules/wil](https://github.com/microsoft/wil) from `d784315` to `f1b694f`.
- [Release notes](https://github.com/microsoft/wil/releases)
- [Commits](microsoft/wil@d784315...f1b694f)

---
updated-dependencies:
- dependency-name: submodules/wil
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* consistently use PascalCase for fields and variables in published API (microsoft#327)

* Bump submodules/wil from `f1b694f` to `ae986e1` (microsoft#328)

Bumps [submodules/wil](https://github.com/microsoft/wil) from `f1b694f` to `ae986e1`.
- [Release notes](https://github.com/microsoft/wil/releases)
- [Commits](microsoft/wil@f1b694f...ae986e1)

---
updated-dependencies:
- dependency-name: submodules/wil
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Add minimal performance counters (microsoft#326)

* Remove dependencies on msquic CI pool and use prerelease images instead (microsoft#330)

* Actions: require all actions succeed (microsoft#332)

* Remove OID_GEN_DRIVER_VERSION from test miniports (microsoft#334)

* Enable developer builds of just one project (microsoft#335)

* Refactor packet parsing into separate file and cross compile in user mode (microsoft#333)

* try to start XDP 10x more times (microsoft#337)

* Create packet fuzzer (microsoft#336)

* specify config and arch during check-drivers.ps1 (microsoft#339)

* improve eBPF uninstall timeout diagnostics (microsoft#340)

* Move experimental (unsupported) APIs to separate headers (microsoft#342)

* use consistent argument types in NDIS request poll helper (microsoft#343)

* Add XDP, AF_XDP, and XDP driver docs in markdown (microsoft#344)

* add link to AF_XDP documentation from readme (microsoft#347)

* create test archive (microsoft#349)

* add xdpcfg.exe to MSI (microsoft#348)

* convert XSK buffer types from macro-based bit values to plain C bitfields (microsoft#346)

* yml (microsoft#350)

* fix devkit creation (microsoft#351)

* bump minor release (microsoft#352)

* bump patch number (microsoft#353)

* Fix creation of release artifacts and revert version number (microsoft#354)

* fix kit version numbering (microsoft#355)

* Stamp the XDP version number onto the MSI file name (microsoft#357)

* add a few more docs (microsoft#358)

* onebranch build changes (1/x), (2/x) (microsoft#360)

* Use warning level 4 by default (microsoft#359)

* set CET flag (microsoft#362)

* OneBranch Build Support (microsoft#361)

* Fix Official OneBranch Pipeline (microsoft#364)

* bump versions for 1.0 release (microsoft#365)

* bump main version to 1.1 (microsoft#366)

* bump main version to 1.0.1

* Update xdp.props

* Fix and verify test execution outside of CI (microsoft#368)

* Describe OneBranch authorizing the new branch to the release steps (microsoft#367)

* Mention v1.0 in Release Docs (microsoft#369)

* reduce artifacts in devkit (microsoft#372)

* Updated Installation Instructions (microsoft#371)

* Document updating aka.ms links (microsoft#374)

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Michael Friesen <3517159+mtfriesen@users.noreply.github.com>
Co-authored-by: cdammanintopix <63319198+cdammanintopix@users.noreply.github.com>
Co-authored-by: Nick Banks <nibanks@microsoft.com>
Co-authored-by: Yi Huang <huanyi@microsoft.com>
Co-authored-by: Gianni Trevisiol <gtrevi@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@anrossi anrossi anrossi left review comments

@nibanks nibanks nibanks approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@mtfriesen @anrossi @nibanks