Skill Being Reviewed
Skill name: hipaa-review
Skill path: skills/compliance/hipaa-review/
False Positive Analysis
Benign-looking HIPAA audit-control packet that can be over-scored as compliant:
technical_safeguards:
164_312_b_audit_controls: implemented
logging_enabled: true
siem_connected: true
retention_policy: six_years
sampled_ehr_login_event: present
events_not_mapped:
- ephi_view
- ephi_export
- ephi_modify
- ephi_delete
- break_glass_access
- admin_privilege_change
log_integrity: not_tested
review_cadence: undocumented
Why this is a false positive:
The current skill says to verify audit logging is enabled, reviewed, and retained, but does not define evidence that proves audit controls cover all systems that contain or use ePHI. A reviewer can mark 45 CFR 164.312(b) as compliant from one SIEM integration and one login event while ePHI read/export/delete, break-glass, privilege changes, API access, and failed access attempts are not logged or reviewed.
Coverage Gaps
Missed variant 1: login-only logging misses ePHI activity
system: ehr-prod
logged_events:
- user_login
- user_logout
missing_events:
- patient_record_view
- record_export
- record_update
- record_delete
- failed_access
- emergency_access
Why it should be caught:
164.312(b) requires mechanisms to record and examine activity in systems that contain or use ePHI. Login-only audit evidence does not prove ePHI activity can be examined.
Missed variant 2: log retention exists but integrity and time basis are unproven
retention: six_years
storage: siem_hot_warm_archive
immutability: missing
time_sync: unknown
chain_of_custody: missing
Why it should be caught:
Audit evidence is weak if timestamps are not reliable, logs can be altered, or exported evidence lacks integrity/chain-of-custody controls.
Missed variant 3: activity review is disconnected from audit controls
164_308_activity_review:
weekly_report: access_summary
164_312_b_audit_controls:
event_coverage_matrix: missing
reviewed_exceptions: none
Why it should be caught:
164.308(a)(1)(ii)(D) information system activity review and 164.312(b) audit controls should connect: reviewers need evidence that the collected audit events are actually examined and exceptions are tracked.
Edge Cases
- Some legacy clinical systems cannot emit all event types; the review should require compensating controls and Not Evaluable reasons rather than a blind pass.
- API gateways, patient portals, mobile apps, interface engines, data warehouses, billing systems, and Business Associate systems may all contain or use ePHI and need audit-control coverage decisions.
- Emergency access and break-glass events can be legitimate, but they need explicit logging, review owner, and follow-up evidence.
- Log retention alone does not prove review, integrity, timestamp reliability, or coverage of sensitive events.
Remediation Quality
Recommended output fields:
| Field |
Purpose |
| ePHI system |
System, API, app, warehouse, interface, device, or BA platform in scope. |
| Event coverage |
Login, ePHI view/export/modify/delete, failed access, break-glass, admin change, API access. |
| Log source |
Application audit log, database audit, cloud audit trail, EDR, SIEM, or BA report. |
| Integrity/time basis |
Time sync, immutability, hash/signature, archive control, or chain of custody. |
| Retention evidence |
Retention period, archive location, legal hold/exception, and restore/export test. |
| Review linkage |
Report/query reviewed under 164.308(a)(1)(ii)(D), owner, cadence, and exception disposition. |
| Decision |
Compliant, Partial, Non-Compliance, or Not Evaluable. |
Comparison to Other Tools
| Tool / Framework |
Catches this? |
Notes |
| eCFR 45 CFR 164.312(b) |
Partial |
Establishes the audit-control requirement but does not provide a portable evidence matrix. |
| eCFR 45 CFR 164.308(a)(1)(ii)(D) |
Partial |
Requires activity review; the skill should link reviewed activity to collected events. |
| NIST SP 800-66 Rev. 2 |
Partial |
Provides implementation guidance; the skill needs concrete review output fields and edge cases. |
| SIEM platforms |
Partial |
Can collect logs, but collection alone does not prove HIPAA audit-control coverage or review. |
Overall Assessment
Strengths:
- The skill covers all HIPAA Security Rule safeguard categories and correctly lists 164.312(b) as a required Technical Safeguard.
- It already emphasizes risk analysis, ePHI inventory, BAAs, documentation, and breach readiness.
Needs improvement:
- 164.312(b) is too thin for audit evidence. It lacks event coverage, system coverage, log integrity, retention proof, and activity-review linkage.
- The output format does not include an audit-control matrix, so a reviewer can pass a system from generic logging evidence.
- Not Evaluable handling is needed for legacy systems or Business Associate systems where event coverage is unknown.
Priority recommendations:
- Add a 164.312(b) audit-control evidence gate under Technical Safeguards.
- Add a HIPAA audit-control coverage matrix to the report output.
- Add edge-case fixtures for login-only logging, mutable logs, missing activity-review linkage, legacy compensating controls, and complete evidence.
- Cross-link 164.312(b) audit controls with 164.308(a)(1)(ii)(D) information system activity review and 164.316 documentation retention.
Sources Checked
This review is distinct from #1106/#1089/#1236 because it focuses on 164.312(b) audit controls and 164.308(a)(1)(ii)(D) activity-review linkage, not BAA clauses or resilience/addressable handling. It is distinct from #1326/#1367 because it does not address de-identification or re-identification evidence.
Bounty Info
Skill Being Reviewed
Skill name:
hipaa-reviewSkill path:
skills/compliance/hipaa-review/False Positive Analysis
Benign-looking HIPAA audit-control packet that can be over-scored as compliant:
Why this is a false positive:
The current skill says to verify audit logging is enabled, reviewed, and retained, but does not define evidence that proves audit controls cover all systems that contain or use ePHI. A reviewer can mark 45 CFR 164.312(b) as compliant from one SIEM integration and one login event while ePHI read/export/delete, break-glass, privilege changes, API access, and failed access attempts are not logged or reviewed.
Coverage Gaps
Missed variant 1: login-only logging misses ePHI activity
Why it should be caught:
164.312(b) requires mechanisms to record and examine activity in systems that contain or use ePHI. Login-only audit evidence does not prove ePHI activity can be examined.
Missed variant 2: log retention exists but integrity and time basis are unproven
Why it should be caught:
Audit evidence is weak if timestamps are not reliable, logs can be altered, or exported evidence lacks integrity/chain-of-custody controls.
Missed variant 3: activity review is disconnected from audit controls
Why it should be caught:
164.308(a)(1)(ii)(D) information system activity review and 164.312(b) audit controls should connect: reviewers need evidence that the collected audit events are actually examined and exceptions are tracked.
Edge Cases
Remediation Quality
Recommended output fields:
Comparison to Other Tools
Overall Assessment
Strengths:
Needs improvement:
Priority recommendations:
Sources Checked
This review is distinct from #1106/#1089/#1236 because it focuses on 164.312(b) audit controls and 164.308(a)(1)(ii)(D) activity-review linkage, not BAA clauses or resilience/addressable handling. It is distinct from #1326/#1367 because it does not address de-identification or re-identification evidence.
Bounty Info