feat: Audit

[gewenyu99]

Problem

We wanna audit existing posthog integrations.

Changes

Here's a start. We just check against known best practices. This is an LLM powered static file checker basically. We don't yet check for data integrity, live events, etc. BUT this catches the most common foot-guns and is reasonable ~6 minutes on big projects like Cal.com

• Refactors some of the shared bits between revenue, posthog integration, and audit workflows
• Builds a custom progress screen for audit items, complete with expanding details and scrolling
• Spits out report at the end as a markdown report.

The skill PR

Test it on the workbench branch here

Test plan

Screen.Recording.2026-05-01.at.1.09.31.PM-converted.mov

[sarahxsanders]

looking good so far!!

a few things:

the audit info screen could be a little more descriptive. "audits" doesn't tell me much about what audit means. what is it looking for/improving on? this is good context for why I should run it/consent

the next screen is good as is if we give enough context in the intro. if not, I'm left looking at this like "what is it doing". there could also be room to make this more of a sequenced experience like some of the wizard install hints/tips/educational bits

not sure how long it takes on your end, but the "Loading audit skills" bit takes a longggg time on my machine. what's the benchmark or ideal time for this? are we aiming for a similar time as the wizard command, or something else?

I also see the event log says "Auditing PostHog integration" but seeing that and "Loading audit skills" at the same time makes me think it hasn't fully loaded the skills yet/hasn't started

[gewenyu99]

Yeah @sarahxsanders This taking 2-3 minutes to load audit skills is... well... what I find too 😢 . I'm not sure how much more I can compress it. I'm telling it to literally do nothing and check sdk install first and this is how long it takes :sigh:

[daniloc]

UI wise:

I think the level of detail is neat in that it conveys just how much we're doing here, and when there are problems, it surfaces them clearly.

one issue is that it feels NOISY. Getting everything corralled into some tidy columns or tables would go a long way to feeling like things are under control. Aligning "Up next" to "Identification" instead of the status bullet, for example, could go a long way.

I think the missing piece here is a bit of storytelling. a bit of setup like "We're checking your installation for common pitfalls…" that then set up the flavor text that's already scrolling through would be neat.

but generally: less is more, I bet you could trim quite a bit of the visual noise here and not lose much. the game is getting the output, and making the user confident that the process is running successfully to get that output.

nit: the outro screen boilerplate isn't right for this: we didn't make any changes. needs an override for tasks like these.

[sarahxsanders]

Yeah @sarahxsanders This taking 2-3 minutes to load audit skills is... well... what I find too 😢 . I'm not sure how much more I can compress it. I'm telling it to literally do nothing and check sdk install first and this is how long it takes :sigh:

oh robots >.< I get it! was more just curious how long it was expected to take!

[gewenyu99]

@daniloc I did a quick round on the UI. 2 Key differences

Brought back some fun visuals for learning. Made a bunch of these that I'm still just tweaking. If you have ideas, please just push them up

I cleaned up a lot of the noise on the audit plan screen

[daniloc]

I would actually suggest steering away from the visuals/attention holders we do in the main wizard flow. In that case we have a new relationship we're trying to keep warm.

in this case, the relationship is established. I think the user will care much, much more about what we're actually checking, and how it relates to their project. So instead of widgets, maybe a brief paragraph explaining the way each thing can go wrong? What we're checking, and why we're checking it out? just a gut thing but I think keeping the lens squarely on where the user is exactly right now is the move for this guise of the wizard. "we know the way things should work, and we're going to get you healthy."

[daniloc]

iow, this isn't the moment to be cute. We earn cute in the top of funnel flow because we're buying people time back and they're happy for it.

here, things might be a bit more sensitive. it's the moment to inspire confidence and project competent clarity. If you're a little annoyed with PostHog, because things aren't quite working or you think it's too complex, running this should be the moment where you feel "oh, these folks have this under control, my doubts were premature."

[gewenyu99]

@daniloc I love that idea. I'm gonna tear this up again. I actually think, this also fundamentally changes how our skills might look in the future for audits.

Like this ability to audit and clearly articulate what we're checking is just so amazing. I want to see (maybe not in this PR) if we can surface what the robot is checking for more dynamically through tools, driven the the agent. For now, I'm gonna hardcode some mappings

[edwinyjlim]

ship it!

we can iterate and decompose some of the bigger workflow pieces later. I added some small UX updates