Rewrite Home page to lead with value proposition vs built-in reports
Add a benefit-led intro, a head-to-head comparison against the built-in Microsoft 365 usage reports, and outcome-focused use cases above the existing navigation.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
a45cb70
Document that importer timestamps are stored in UTC
Adds a Time zones section to the Database Schema page clarifying that all event and activity timestamps (web tracking, audit log, usage reports, Teams/calls/email/Copilot) are stored in UTC.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
f33fc3e
Importer aggressiveness: add Teams interval, force flag, and 'Forcing a re-import' (clearing last-run flags) section
Reflects PR #162 review fixes: GraphTeamsImportIntervalHours, ForceGraphMetadataImport, High=every-cycle, in-memory fallback note, and redis-cli DEL instructions for the unprefixed gate keys.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2c9abeb
Document importer CPU aggressiveness settings (pnp/Microsoft365-Analytics-Insights#161/#162)
Add 'Reducing importer CPU usage (aggressiveness)' section to Configuration Reference covering ImportAggressiveness, MaxAuditReportLoadConcurrency, ImportCyclePauseMinutes, GraphMetadataImportIntervalHours and ImportStartStaggerMinutes; cross-link from Monitoring System Performance checklist.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
ac5722c
Remove Adoptify references; reframe as a single analytics solution
- Drop Adoptify from Home, Deployment Guidance, Prerequisites, Telemetry Service
and Architecture & Costs; delete the orphaned installer-targets-adoptify
screenshot.
- Reframe the Home intro away from "other solutions" now that there is just the
analytics solution. Historical Release Notes left untouched.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2d1d8dc
Add KQL queries for tracing Teams call webhook traffic
Document Log Analytics queries on the Monitoring page to trace all Teams call
records through the pipeline - both the webhook (CallRecordWebhookController)
and the calls importer (Office365CallsImporter), failures and successes:
all-traces, webhook-only, failures-only (severityLevel >= 3), an end-to-end
received-vs-saved join, and a volume timechart. Includes the workspace-based
table-name equivalents.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
5463a3f
Wiki: finish branding modernisation + fix two malformed Analytics tables
- Modernise generic 'Office 365' platform references to 'Microsoft 365' across
the wiki (44 occurrences), now that the product is 'Microsoft 365 Advanced
Analytics'. The real 'Office 365 Management [Activity] API' name and code
identifiers (Office365ActivityImporter, o365_client_applications) are preserved.
- Analytics - Data model: fix the malformed 'Usage metrics' table (the header had
two columns but the data row had four; the stray cells duplicated the
App-per-platform table below).
- Analytics: de-indent the Runbooks table so it renders reliably (it was indented
under a list item).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
6791bcd
Wiki: stage the manual install, nest monitoring, flesh out cert auth/product notes
Manual installation (447-line page) split into an overview/index plus three
ordered stage pages so each phase is digestible and navigable:
- Manual Install Azure Resources (storage, App Insights, App Service, SQL,
Redis, Service Bus, Cognitive)
- Manual Install Deploy Binaries (publish web-jobs/website, app settings)
- Manual Install Web Tracking and Config (AITracker, SPFx extension, reply
URLs, filtered URLs)
The parent keeps its filename so inbound links still resolve; each stage has
prev/next navigation; in-page section references became cross-page links; the
Uninstall deep link to #deploy-aitracker-via-powershell was repointed.
Sidebar: nest the three manual-install stages under Manual installation, and
nest Health alerts + System performance under Monitoring.
Certificate authentication: add intro, a How it works summary (Key Vault, the
required O365AdvancedAnalytics CN, the UseClientCertificate setting) and a
Prerequisites section; tidy the verify-logs heading.
Product notes: clearer intro and cross-links (anchors preserved for inbound
links from Configuration reference and Troubleshooting).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
8ebd1ac
Wiki: modernise product name to 'Microsoft 365 Advanced Analytics' + accuracy/structure polish
- Standardise the product name to Microsoft 365 Advanced Analytics (was a mix of
'Office 365 Advanced Analytics [Engine]', 'M365 Advanced Analytics', 'O365...').
Code identifiers (Office365ActivityImporter), the Office 365 Management Activity
API name and the literal 'O365 Adv Analytics Setup Rule' firewall rule are
deliberately preserved.
- Replace legacy docs.microsoft.com links with learn.microsoft.com.
- Tie the monitoring trio together: Health Alerts and Monitoring now cross-link
Monitoring system performance and vice-versa.
- Verify: fix informal wording and add a 'What good looks like' pass/fail checklist.
- Troubleshooting: point fatal-exception reports at GitHub Issues (with a
redact-secrets warning) instead of an unaddressed 'please email'.
- Manual setup: clarify the importer uses the App Insights connection string, with
the instrumentation key still needed for AITracker/legacy logging.
- Architecture & costs: document the basis for the ~EUR170/month estimate.
- CSP page: add deployment/context cross-links.
- Align the Analytics page title with the sidebar ('Activity & Usage Analytics').
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
baf3092
Wiki polish: remove empty Troubleshooting stub, frame Manual Setup, tidy CSP page
- Troubleshooting: delete the empty [TBD] 'Unexpected/Missing SharePoint Page
Properties' section (duplicate of the populated section; not in the TOC).
- Installation – Manual Setup: add a pre-flight intro pointing to Prerequisites,
App Registrations Setup and the Configuration reference, and flagging it as the
by-hand fallback to the installer-driven install.
- Enable CSP for AITracker: rewrite the stub with a proper heading, fix the
'enabled' typo, explain why it matters, and use a real markdown doc link.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
d449fd9
Wiki: split app-registration setup out of Deployment Guidance into its own page; update links + sidebar
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
bbd601f
Wiki: refresh installer screenshots in Deployment Guidance; add Networking tab + Test configuration steps
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
0731541
Wiki: add Configuration reference + Security & permissions model pages; link from sidebar and install flow
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
4e67135
Wiki: regroup sidebar, split Home into Data Collection + Architecture pages, fix stale/stub content
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
8820aec
Reorganize Prerequisites; document Azure RBAC roles
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
fd44971
Add plain-English 'How URLs are shortened' page
Non-technical explainer of the import-time URL cleaning that keeps SharePoint/Teams deep-link URLs within the urls.full_url (nvarchar(850)) column: strip the temporary xsdata token, else reduce to the page path, else truncate as a last resort. Adds a sidebar entry and cross-links with the technical 'URLs Full URL Shrink Upgrade' page.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
763d6ff
Update URLs Full URL Shrink Upgrade for nvarchar(850) (Greek/Unicode fix)
Corrects the operator guide for the nvarchar(850) change (issue #122 / PR #123): column is now nvarchar(850) not varchar(1700); the lossy/representability pre-flight check is removed (only the >850 length abort remains); documents the new UrlFullUrlNvarchar migration and the two lossless upgrade-from states; fixes the operator probes and post-upgrade sanity SQL; notes nvarchar storage is ~2 bytes/char so headroom budget is larger; benchmark annotated as measured on the superseded varchar form.
9961309
Add Telemetry Service page; link from Home and sidebar
Documents the new src/TelemetryService/ app as a related project:
private MSAL-protected dashboard with public source code that receives
anonymous, opt-in usage telemetry (AnonUsageStatsModel) from deployed
analytics engines. No tenant data is sent. Status: work in progress.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
eff1cc5
Add 'URLs Full URL Shrink Upgrade' page with measured 10M-row baseline
Companion guide for PR #108 (ShrinkUrlsFullUrlColumn migration), mirroring
the structure of the existing 'Audit Events FK Upgrade' page.
Headline measurement: ShrinkUrlsFullUrlColumn on 10M urls rows (avg 152 chars,
SQL 2025 Standard Developer Edition, Azure VM with Standard HDD) ran end-to-end
in 6 min 6 sec. Per-step: ~2.5s too-long scan, ~41s lossy-varchar scan,
~2m13s ALTER COLUMN, ~3m10s CREATE INDEX (offline). Storage: urls grows
from 3.27 GB to 8.19 GB (URLs move from LOB to inline + new 1.68 GB index).
Includes:
- Per-step measured breakdown and 'how many rows do you have' diagnostics
- Extrapolation tables for VM (1M to 500M rows on Standard HDD) and Azure SQL
DB (S3-S7 DTU tiers) using the measured ~6 GB log-write per 10M rows scaling
- Operator checklist (stop web jobs, data file headroom, log space, scale up)
- 'If the migration aborts' section for too-long or lossy URLs
- Post-upgrade sanity check, including dm_db_index_usage_stats query to
confirm IX_urls_full_url is actually being used by importers
Sidebar entry placed next to 'Audit Events FK upgrade' (both are
upgrade-time guides).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
9b53671
Audit Events FK Upgrade: clarify installer won't time out
CommandTimeout in the EF migrations Configuration is now 0 (infinite),
so the installer / DatabaseUpgrader.CheckDbUpgraded will sit on the
WITH CHECK CHECK CONSTRAINT validation for as long as it needs to,
even on multi-hour scans. Reflect that in sections 6 (During the
upgrade) and 8 (Quick reference) so operators know to just let it run.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
a9a6436
Document Power Platform auditing prerequisite for adoption events
ActivityFeed.Read lets the importer read Power Platform audit events, but those events only exist if Power Platform auditing is on. Adds a new Power Platform auditing must be turned on section to Prerequisites covering the two-step checklist (Microsoft Purview tenant audit + per-environment auditing for Power Apps / Power Automate / Copilot Studio / Dataverse) with per-workload Microsoft Learn links. Cross-references from Home.md (Power Platform Adoption Events) and Release Notes.md (Power Platform audit logging entry). Clarifies that Power BI / Fabric uses the unified Purview audit log and does NOT need a separate tenant toggle.
6125fd9
Wiki updates for next release: Power Platform, Sent Email, FK upgrade
New page
- Audit Events FK Upgrade: operator guide for the AddAuditEventsOperationIndex +
AddAuditEventsOperationFK migrations - what the upgrade does, measured
benchmark, time estimates for VM/HDD and Azure SQL DB tiers, row-count
probe SQL, pre/during/after steps, sanity-check script.
Updates
- Database Schema: new Power Platform section documenting all the new tables
(power_app_environments, power_apps, power_automate_flows, power_bi_*,
copilot_studio_bots, event_meta_*, connector junction tables). audit_events
row updated to reflect the restored FK and link to the FK upgrade page.
- Release Notes: new top entry covering the 6 migrations, Power Platform
import, Sent Email Import, Cognitive Services RBAC fallback, optional
Service Bus, license import + delta-token + Power BI merge + org-URL +
SentEmail cascade fixes, and the ImportTaskSettings opt-in behaviour change.
- Home: Power Platform Adoption Events and Sent Email entries added to the
Data Collection Map.
- Prerequisites: Microsoft.ServiceBus marked optional (only required when
Calls import is enabled); ActivityFeed.Read row extended to mention
Power Platform adoption events.
- Updating the solution: new Schema-change considerations section linking
to the FK upgrade page with the pre-upgrade checklist.
- _Sidebar: link to the new Audit Events FK Upgrade page.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2a66c5a
Use sys.tables size query instead of deprecated sys_telemetry_reports
The sys_telemetry_reports table is deprecated. Replace section 3 with a
direct sys.tables/sys.partitions query that returns per-table row count
and MB on disk, and add a mapping from each large table back to the
import area that fills it so admins can either clean up or disable an
import area they don't actually use.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
a174e70
Add Monitoring System Performance wiki page
Documents how to monitor import cycle duration via Application Insights,
how to diff sys_telemetry_reports snapshots to compare against previous
metrics as the DB grows, and how/when to run Clean Old Data Data.sql
followed by index rebuilds.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
09644f8
Split health alerts into its own page and refresh alert rules
Move the health-alerts content out of Monitoring.md into a dedicated
Health Alerts page, add it to the sidebar, and bring the alert rules
in line with the current codebase:
- Fix the `Waiting 2 mins...` Kusto example (the web-job actually logs
`Waiting 10 mins...`) and clarify the 24h import-cycle guidance.
- Drop the standalone Yammer-importer alerts; that web-job no longer
exists in the solution.
- Note that the `Service Bus` alerts only apply when the Teams calls
import is enabled, and that `System.Data.SqlClient.SqlException` is
still the correct outerType today.
- Add new data-flow alerts for audit events received from the Activity
API (`Audit events import: Got`), the user-metadata import
completion (`User import - complete`), and the Graph permissions
skip path (`ERROR: Can't access Teams user data`).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
3b16f5c
Docs: add Sent Email Import feature details and related permissions to database schema and prerequisites
d5d366a
Wiki audit: fix Copilot Graph permissions, runbook name, Home links
5864379
Wiki cleanup: fix broken links, missing headings, outdated paths
216f8ec
Align Private Endpoints docs with installer behavior
- Clarify SKU selection is only applied on creation; existing Basic Redis/Standard Service Bus/Basic SQL are not upgraded automatically
- Document that App Service outbound VNet integration and the dedicated app-integration subnet are now created automatically by the installer
- Add Automation Account private endpoint (DSCAndHybridWorker) and privatelink.azure-automation.net DNS zone
- Document DeployDnsZones and CustomEndpointNames config options
- Note storage PE is blob-only; clarify table sub-resource still needs manual setup
- Add Hybrid Runbook Worker to the installer-created list
0f68a7f
Docs: enhance Private Endpoints documentation with automated setup details and manual configuration steps
3fd74f5