Diagnostics & Usage Data

landonf edited this page Dec 30, 2014 · 3 revisions
Clone this wiki locally

This document will attempt to explain what is involved with the Diagnostics & Usage Privacy checkbox, and how it does its work. This focuses on the mechanics more than what gets submitted.

The process that drives submission is /System/Library/CoreServices/SubmitDiagInfo, which is run as an "always on" system daemon by /System/Library/LaunchDaemons/com.apple.SubmitDiagInfo.plist launchd daemon. If "Send diagnostic & usage data to Apple" was previously checked, unchecking it will not immediately disable this process. On the next reboot, SubmitDiagInfo will not launch.

Submission usually happens nightly, but SubmitDiagInfo can be run manually from the command line to trigger a submission of the information to Apple. If run with no arguments, a "normal" submission is made.

Submission information includes information from /Library/Logs/DiagnosticReports, ~/Library/Logs/DiagnosticReports, and summarized content from /var/log/DiagnosticMessages.

The DiagnosticReports directories contain crash logs, stack shots of processes spinning or hanging, and nothing directly normal use. Things typically only end up in those directories when something goes wrong. The files in that directory are plaintext and can be easily inspected by the user.

/var/log/DiagnosticMessages is an Apple System Logger store, best viewed with system utilities. To get a high level view of the messages, Console.app can show you generally what is there:

For a more detailed view, you can use the command line: syslog -d /var/log/DiagnosticMessages -F raw:

[ASLMessageID 558] [Time 1418074778] [TimeNanoSec 238185000] [Level 5] [PID 411] [UID 501] [GID 20] [ReadGID 80] [Host myhost.example.com] [Sender findNames] [Facility user] [com.apple.message.domain com.apple.AddressBook.accounts.summary] [com.apple.message.summarize YES] [com.apple.message.carddav_accounts 0] [com.apple.message.disabled_accounts 1] [com.apple.message.enabled_accounts 1] [com.apple.message.exchange_accounts 0] [com.apple.message.ldap_accounts 0] [com.apple.message.total_accounts 3] [SenderMachUUID E9B14040-9B6F-35F6-8D30-929C64D563DC]
[ASLMessageID 559] [Time 1418074784] [TimeNanoSec 936324000] [Level 3] [PID 406] [UID 501] [GID 20] [ReadGID 80] [Host myhost] [Sender Unknown] [Facility user] [com.apple.message.domain com.apple.assumes.failure] [com.apple.message.signature 9437C02E-A07B-38C8-91CB-299FAA63083D:97940] [com.apple.message.signature2 0x89] [com.apple.message.signature3 libxpc.dylib] [com.apple.message.summarize YES] [SenderMachUUID CFAEE3FE-C71A-3C9A-BEA1-B05500B70A54]

Other display formats are available with the syslog tool, including xml. Consult the syslog man page for more information on supported output formats.

Something to keep an eye on with these messages is, they will continue to be logged and stored locally even when "Send diagnostic & usage data to Apple" is unchecked and SubmitDiagInfo isn't running. If "Send diagnostic & usage data to Apple" is ever turned back on, the information that was collected while "Send diagnostic & usage data to Apple" was unchecked will be sent to Apple.

So, how do the logs get there?

These are just syslog messages, and some apps and system services have the logging scattered throughout their code. There's no real way to know beforehand what will or won't generate log messages, and it can change in updates. Some of the log messages are actually in lower level frameworks and libraries that will trigger a log message when a certain function is called, or under certain configurations. System Preferences for example logs the prefpane used and when it is opened. The MAS logs when and which updates are available.

However, some of the most interesting information is logged by /System/Library/CoreServices/diagnostics_agent, which is a per-user launchd agent, configured in /System/Library/LaunchAgents/com.apple.diagnostics_agent.plist. In releases prior to 10.10, the same functionality existed in a UserEventAgent. The diagnostics_agent is what logs user behavior. It only dumps its state to the log periodically, although it can be forced by logging out and then logging back in again. For example, here is some messages logged during a logout: 

This is my best guess of what the fields mean for the com.apple.usage.app_activetime, please update if you find more accurate descriptions:

  • com.apple.message.signature2: from the bundle's Info.plist CFBundleIdentifier ||| CFBundleShortVersionString (CFBundleVersion)
  • com.apple.message.value: total uptime of the app (accross launches, but since the last log time diagnostics_agent logged the app's usage)
  • com.apple.message.value2: total time diagnostics_agent believes the user is actively using the app
  • com.apple.message.value4: number of transitions from "active" -> "idle". Meaning, diagnostics_agent believes the app has gone from being actively used to unused. This seems to correspond to switching from foreground to background application, although if you leave the app in the foreground but not have any user input for a couple minutes, it will increment.
  • com.apple.message.value5: number of launches since the last com.apple.usage.app_activetime for this app
  • com.apple.message.value6: number of transitions from "idle" -> "active", including launch
  • SenderMachUUID: the LC_UUID macho load command of the executable generating the log message. Viewable with: otool -l /System/Library/CoreServices/diagnostics_agent | grep uuid

Since these messages are ASL messages, they are controlled by the ASL daemon. The MessageTracer log message location is configured by /etc/asl/com.apple.MessageTracer to log to /var/log/DiagnosticMessages. If you change this file, you change where the messages get logged. Additionally, SubmitDiagInfo does not look at the ASL configuration to see where the message tracer logs are being logged to, it assumes the hard coded path to /var/log/DiagnosticMessages.

ASL, the Apple syslog infrastructure, is open source and available here.

Hopefully the information can allow people to better understand what Diagnostics & Usage data is, where it comes from, how to inspect it, and how to control what gets sent.