Context-awareness in AX

[quarterback]

How would you define context awareness in AX? What does it mean for an agent to be "context-aware" when interacting with systems and humans? I've been thinking about this a lot, especially around agency and keeping humans in the driver's seat of agent intention vs letting agents operate in their own silos. Like, will agents actually understand the broader environment they're operating in? The downstream effects of their actions? The human systems they're meant to enhance?

The measurement piece is tricky too - how do we track agent effectiveness in real time and aggregate the impact of agent mistakes (not talking about hallucinations here, but actual decision errors)? Here's what keeps me up at night: when a human screws up, we have clear paths for accountability and recourse. But what happens when we've delegated responsibility to an inferred decisionmaker?

What if that system becomes effectively closed off from meaningful oversight?

Sure, letting a bot handle your shopping isn't the same as giving someone power of attorney - but as these tools proliferate and handle increasingly complex decisions, we need real mechanisms for dealing with these questions.

If we don't figure this out now, integrating these tools into the working world is going to be a mess.

[ndaeila]

You're talking about a couple interesting things here:

• For a different discussion post: When a human screws up, we have clear paths for accountability and recourse. But what happens when we've delegated responsibility to an inferred decisionmaker?

• What does it mean for an agent to be "context-aware" when interacting with systems and humans?

Context awareness is something that could cause the issues that initially make us think about how we should handle accountability and recourse, but handling accountability and recourse is its own topic.

[ndaeila]

I think both of these topics should be deeply explored, but they can fortunately be explored separately:

• How do you design a system that is context-aware (or can probe for missing context) and does what the human actually wants?
• How can we create systems where humans can still validate and be accountable, with the least amount of effort/friction/pain?

[quarterback]

What does it mean for an agent to be "context-aware" when interacting with systems and humans? Yes, I think that's the right split.