Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mount udev devpath in Akri brokers #534

Merged

Conversation

kate-goldenring
Copy link
Contributor

What this PR does / why we need it:
fixes #521

Currently, Akri defines a udev device by its devnode; however, not all devices have nodes. This changes the device identifier to by devpath and creates a new environment variable UDEV_DEVPATH which is injected into brokers.

Special notes for your reviewer:
@agracey does this fix your issue of using the dev rule of SUBSYSTEM=="drm", ATTR{status}=="connected" with Akri?

Docs will need to be updated:

If applicable:

  • this PR has an associated PR with documentation in akri-docs
  • this PR contains unit tests
  • added code adheres to standard Rust formatting (cargo fmt)
  • code builds properly (cargo build)
  • code is free of common mistakes (cargo clippy)
  • all Akri tests succeed (cargo test)
  • inline documentation builds (cargo doc)
  • all commits pass the DCO bot check by being signed off -- see the failing DCO check for instructions on how to retroactively sign commits

Signed-off-by: Kate Goldenring kate.goldenring@gmail.com

@kate-goldenring
Copy link
Contributor Author

I was able to confirm that this change enables the dynamic discovery of an HDMI monitor.
Currently, the following does not discover the monitor because devpath /devices/pci0000:00/0000:00:02.0/drm/card0/card0-HDMI-A-1 does not have a devnode:

helm install akri akri-helm-charts/akri \
     $AKRI_HELM_CRICTL_CONFIGURATION \
     --set udev.configuration.enabled=true \
     --set udev.discovery.enabled=true \
     --set udev.configuration.name=akri-hdmi \
     --set udev.configuration.discoveryDetails.udevRules[0]='SUBSYSTEM=="drm"\, ATTR{status}=="connected"\, KERNEL=="*HDMI*"'

Running these changes locally and deploying the chart without the udev discovery handler:

helm install akri akri-helm-charts/akri \
     $AKRI_HELM_CRICTL_CONFIGURATION \
     --set udev.configuration.enabled=true \
     --set udev.configuration.name=akri-hdmi \
     --set udev.configuration.discoveryDetails.udevRules[0]='SUBSYSTEM=="drm"\, ATTR{status}=="connected"\, KERNEL=="*HDMI*"'

And in another terminal

sudo RUST_LOG=info DISCOVERY_HANDLERS_DIRECTORY=/var/lib/akri AGENT_NODE_NAME=kagold-thinkpad-x1-carbon-6th ./target/release/udev-discovery-handler

Resultant Akri instance:

apiVersion: v1
items:
- apiVersion: akri.sh/v0
  kind: Instance
  metadata:
    creationTimestamp: "2022-11-02T19:28:39Z"
    generation: 1
    name: akri-hdmi-cc5ea6
    namespace: default
    ownerReferences:
    - apiVersion: akri.sh/v0
      blockOwnerDeletion: true
      controller: true
      kind: Configuration
      name: akri-hdmi
      uid: 6d6714eb-d010-441c-9866-671366729f57
    resourceVersion: "5469"
    uid: be443857-889c-49dc-bb43-8f19abed3f1b
  spec:
    brokerProperties:
      UDEV_DEVPATH: /devices/pci0000:00/0000:00:02.0/drm/card0/card0-HDMI-A-1
    configurationName: akri-hdmi
    deviceUsage:
      akri-hdmi-cc5ea6-0: ""
    nodes:
    - kagold-thinkpad-x1-carbon-6th
    shared: false
kind: List
metadata:
  resourceVersion: ""

Instance disappears when HDMI is unplugged (as expected)

@kate-goldenring
Copy link
Contributor Author

/version patch

@github-actions github-actions bot added the version/patch Patch version change is needed label Nov 2, 2022
@kate-goldenring kate-goldenring force-pushed the udev-devpath-mount branch 2 times, most recently from 541b2f4 to fc2825c Compare November 2, 2022 19:39
.map(|device| {
(
get_devpath(&device).to_str().unwrap().to_string(),
get_devnode(&device).map(|devnode| devnode.to_str().unwrap().to_string()),
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think something like unwrap_or_default("") might be good for both of these in case udev doesn't have a devpath or devnode

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nvm... looking at your example, I think rust likely does the same thing under the covers

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that these are pretty, but they are transforming from OsStr to String. All devices have a devpath (get_devpath returns OsStr) while not all have devnodes as you pointed out (get_devnode returns Option<OsStr>)

@agracey
Copy link

agracey commented Nov 2, 2022

Yay!!! This looks like it'll fix my issue nicely

@adithyaj
Copy link
Collaborator

adithyaj commented Nov 2, 2022

I see K3s is failing here again and likely related to #531, I'll restart it and it should go away

@kate-goldenring
Copy link
Contributor Author

@adithyaj @bfjelds @romoh I wonder if this should be a minor (rather than patch) version change since it changes the way that we create Instance ids for udev devices (a hash of devpath rather than devnode). If someone has a pre-existing deployment based on assumed instance ids, this would break it.

@adithyaj
Copy link
Collaborator

adithyaj commented Nov 7, 2022

@adithyaj @bfjelds @romoh I wonder if this should be a minor (rather than patch) version change since it changes the way that we create Instance ids for udev devices (a hash of devpath rather than devnode). If someone has a pre-existing deployment based on assumed instance ids, this would break it.

I agree, since it would be a breaking change it may be better to do a minor bump to have some way to distinguish it. Also, last time there was a breaking change there was a minor version bump too (#437). Let's give this a minor version bump.

@kate-goldenring
Copy link
Contributor Author

/version minor

@github-actions github-actions bot added the version/minor Minor version change is needed label Nov 8, 2022
@kate-goldenring
Copy link
Contributor Author

@adithyaj @bfjelds @romoh I wonder if this should be a minor (rather than patch) version change since it changes the way that we create Instance ids for udev devices (a hash of devpath rather than devnode). If someone has a pre-existing deployment based on assumed instance ids, this would break it.

I agree, since it would be a breaking change it may be better to do a minor bump to have some way to distinguish it. Also, last time there was a breaking change there was a minor version bump too (#437). Let's give this a minor version bump.

I think this does point out the fact that we may need to decide scope of "Akri" do breaking changes in DHs mean breaking change to Akri? I'm imagining if the DH's lived in separate repos for example. For now, we can assume yes, but i expect this will evolve

@adithyaj adithyaj removed the version/patch Patch version change is needed label Nov 8, 2022
Signed-off-by: Kate Goldenring <kate.goldenring@gmail.com>
@kate-goldenring kate-goldenring added version/minor Minor version change is needed and removed version/minor Minor version change is needed labels Dec 7, 2022
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@kate-goldenring
Copy link
Contributor Author

kate-goldenring commented Dec 7, 2022

accidentally bumped version twice: do not merge good to merge

@kate-goldenring kate-goldenring merged commit 2efb89e into project-akri:main Dec 7, 2022
@kate-goldenring kate-goldenring deleted the udev-devpath-mount branch December 7, 2022 18:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
version/minor Minor version change is needed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Allow for udev rules without devnode
3 participants