Rewrite of the settings provider.

This change modifies how global, secure, and system settings are
managed. In particular, we are moving away from the database to
an in-memory model where the settings are persisted asynchronously
to XML.

This simplifies evolution and improves performance, for example,
changing a setting is down from around 400 ms to 10 ms as we do not
hit the disk. The trade off is that we may lose data if the system
dies before persisting the change.

In practice this is not a problem because 1) this is very rare;
2) apps changing a setting use the setting itself to know if it
changed, so next time the app runs (after a reboot that lost data)
the app will be oblivious that data was lost.

When persisting the settings we delay the write a bit to batch
multiple changes. If a change occurs we reschedule the write
but when a maximal delay occurs after the first non-persisted
change we write to disk no matter what. This prevents a malicious
app poking the settings all the time to prevent them being persisted.

The settings are persisted in separate XML files for each type of
setting per user. Specifically, they are in the user's system
directory and the files are named: settings_type_of_settings.xml.

Data migration is performed after the data base is upgraded to its
last version after which the global, system, and secure tables are
dropped.

The global, secure, and system settings now have the same version
and are upgraded as a whole per user to allow migration of settings
between these them. The upgrade steps should be added to the
SettingsProvider.UpgradeController and not in the DatabaseHelper.

Setting states are mapped to an integer key derived from the user
id and the setting type. Therefore, all setting states are in
a lookup table which makes all opertions very fast.

The code is a complete rewrite aiming for improved clarity and
increased maintainability as opposed to using minor optimizations.
Now setting and getting the changed setting takes around 10 ms. We
can optimize later if needed.

Now the code path through the call API and the one through the
content provider APIs end up being the same which fixes bugs where
some enterprise cases were not implemented in the content provider
code path.

Note that we are keeping the call code path as it is a bit faster
than the provider APIs with about 2 ms for setting and getting
a setting. The front-end settings APIs use the call method.

Further, we are restricting apps writing to the system settings.
If the app is targeting API higher than Lollipop MR1 we do not
let them have their settings in the system ones. Otherwise, we
warn that this will become an error. System apps like GMS core
can change anything like the system or shell or root.

Since old apps can add their settings, this can increase the
system memory footprint with no limit. Therefore, we limit the
amount of settings data an app can write to the system settings
before starting to reject new data.

Another problem with the system settings was that an app with a
permission to write there can put invalid values for the settings.
We now have validators for these settings that ensure only valid
values are accepted.

Since apps can put their settings in the system table, when the
app is uninstalled this data is stale in the sytem table without
ever being used. Now we keep the package that last changed the
setting and when the package is removed all settings it touched
that are not in the ones defined in the APIs are dropped.

Keeping in memory settings means that we cannot handle arbitrary
SQL operations, rather the supported operations are on a single
setting by name and all settings (querying). This should not be
a problem in practice but we have to verify it. For that reason,
we log unsupported SQL operations to the event log to do some
crunching and see what if any cases we should additionally support.

There are also tests for the settings provider in this change.

Change-Id: I941dc6e567588d9812905b147dbe1a3191c8dd68

core/java/android/util/AtomicFile.java

@@ -102,7 +102,7 @@ public FileOutputStream startWrite() throws IOException {
             str = new FileOutputStream(mBaseName);
         } catch (FileNotFoundException e) {
             File parent = mBaseName.getParentFile();
-            if (!parent.mkdir()) {
+            if (!parent.mkdirs()) {
                 throw new IOException("Couldn't create directory " + mBaseName);
             }
             FileUtils.setPermissions(

core/res/res/values/symbols.xml

@@ -2164,4 +2164,7 @@
 
   <java-symbol type="bool" name="allow_stacked_button_bar" />
   <java-symbol type="id" name="spacer" />
+
+  <java-symbol type="xml" name="bookmarks" />
+
 </resources>

core/res/res/xml/bookmarks.xml

@@ -0,0 +1,59 @@
+<?xml version="1.0" encoding="utf-8"?>
+<!-- Copyright (C) 2007 The Android Open Source Project
+
+     Licensed under the Apache License, Version 2.0 (the "License");
+     you may not use this file except in compliance with the License.
+     You may obtain a copy of the License at
+
+          http://www.apache.org/licenses/LICENSE-2.0
+
+     Unless required by applicable law or agreed to in writing, software
+     distributed under the License is distributed on an "AS IS" BASIS,
+     WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+     See the License for the specific language governing permissions and
+     limitations under the License.
+-->
+
+<!--
+     Default system bookmarks for AOSP.
+     Bookmarks for vendor apps should be added to a bookmarks resource overlay; not here.
+
+     Typical shortcuts (not necessarily defined here):
+       'a': Calculator
+       'b': Browser
+       'c': Contacts
+       'e': Email
+       'g': GMail
+       'l': Calendar
+       'm': Maps
+       'p': Music
+       's': SMS
+       't': Talk
+       'y': YouTube
+-->
+<bookmarks>
+    <bookmark
+        category="android.intent.category.APP_CALCULATOR"
+        shortcut="a" />
+    <bookmark
+        category="android.intent.category.APP_BROWSER"
+        shortcut="b" />
+    <bookmark
+        category="android.intent.category.APP_CONTACTS"
+        shortcut="c" />
+    <bookmark
+        category="android.intent.category.APP_EMAIL"
+        shortcut="e" />
+    <bookmark
+        category="android.intent.category.APP_CALENDAR"
+        shortcut="l" />
+    <bookmark
+        category="android.intent.category.APP_MAPS"
+        shortcut="m" />
+    <bookmark
+        category="android.intent.category.APP_MUSIC"
+        shortcut="p" />
+    <bookmark
+        category="android.intent.category.APP_MESSAGING"
+        shortcut="s" />
+</bookmarks>

packages/SettingsProvider/Android.mk

@@ -3,7 +3,8 @@ include $(CLEAR_VARS)
 
 LOCAL_MODULE_TAGS := optional
 
-LOCAL_SRC_FILES := $(call all-subdir-java-files)
+LOCAL_SRC_FILES := $(call all-subdir-java-files) \
+    src/com/android/providers/settings/EventLogTags.logtags
 
 LOCAL_JAVA_LIBRARIES := telephony-common ims-common
 

packages/SettingsProvider/src/com/android/providers/settings/DatabaseHelper.java

@@ -58,6 +58,7 @@
 import java.io.IOException;
 import java.util.HashSet;
 import java.util.List;
+import java.util.Set;
 
 /**
  * Database helper class for {@link SettingsProvider}.
@@ -78,6 +79,9 @@ public class DatabaseHelper extends SQLiteOpenHelper {
 
     private static final HashSet<String> mValidTables = new HashSet<String>();
 
+    private static final String DATABASE_JOURNAL_SUFFIX = "-journal";
+    private static final String DATABASE_BACKUP_SUFFIX = "-backup";
+
     private static final String TABLE_SYSTEM = "system";
     private static final String TABLE_SECURE = "secure";
     private static final String TABLE_GLOBAL = "global";
@@ -86,13 +90,13 @@ public class DatabaseHelper extends SQLiteOpenHelper {
         mValidTables.add(TABLE_SYSTEM);
         mValidTables.add(TABLE_SECURE);
         mValidTables.add(TABLE_GLOBAL);
-        mValidTables.add("bluetooth_devices");
-        mValidTables.add("bookmarks");
 
         // These are old.
+        mValidTables.add("bluetooth_devices");
+        mValidTables.add("bookmarks");
         mValidTables.add("favorites");
-        mValidTables.add("gservices");
         mValidTables.add("old_favorites");
+        mValidTables.add("android_metadata");
     }
 
     static String dbNameForUser(final int userHandle) {
@@ -118,6 +122,33 @@ public static boolean isValidTable(String name) {
         return mValidTables.contains(name);
     }
 
+    public void dropDatabase() {
+        close();
+        File databaseFile = mContext.getDatabasePath(getDatabaseName());
+        if (databaseFile.exists()) {
+            databaseFile.delete();
+        }
+        File databaseJournalFile = mContext.getDatabasePath(getDatabaseName()
+                + DATABASE_JOURNAL_SUFFIX);
+        if (databaseJournalFile.exists()) {
+            databaseJournalFile.delete();
+        }
+    }
+
+    public void backupDatabase() {
+        close();
+        File databaseFile = mContext.getDatabasePath(getDatabaseName());
+        if (!databaseFile.exists()) {
+            return;
+        }
+        File backupFile = mContext.getDatabasePath(getDatabaseName()
+                + DATABASE_BACKUP_SUFFIX);
+        if (backupFile.exists()) {
+            return;
+        }
+        databaseFile.renameTo(backupFile);
+    }
+
     private void createSecureTable(SQLiteDatabase db) {
         db.execSQL("CREATE TABLE secure (" +
                 "_id INTEGER PRIMARY KEY AUTOINCREMENT," +
@@ -1221,9 +1252,11 @@ public void onUpgrade(SQLiteDatabase db, int oldVersion, int currentVersion) {
                     // Migrate now-global settings. Note that this happens before
                     // new users can be created.
                     createGlobalTable(db);
-                    String[] settingsToMove = hashsetToStringArray(SettingsProvider.sSystemGlobalKeys);
+                    String[] settingsToMove = setToStringArray(
+                            SettingsProvider.sSystemMovedToGlobalSettings);
                     moveSettingsToNewTable(db, TABLE_SYSTEM, TABLE_GLOBAL, settingsToMove, false);
-                    settingsToMove = hashsetToStringArray(SettingsProvider.sSecureGlobalKeys);
+                    settingsToMove = setToStringArray(
+                            SettingsProvider.sSecureMovedToGlobalSettings);
                     moveSettingsToNewTable(db, TABLE_SECURE, TABLE_GLOBAL, settingsToMove, false);
 
                     db.setTransactionSuccessful();
@@ -1489,9 +1522,11 @@ public void onUpgrade(SQLiteDatabase db, int oldVersion, int currentVersion) {
                 db.beginTransaction();
                 try {
                     // Migrate now-global settings
-                    String[] settingsToMove = hashsetToStringArray(SettingsProvider.sSystemGlobalKeys);
+                    String[] settingsToMove = setToStringArray(
+                            SettingsProvider.sSystemMovedToGlobalSettings);
                     moveSettingsToNewTable(db, TABLE_SYSTEM, TABLE_GLOBAL, settingsToMove, true);
-                    settingsToMove = hashsetToStringArray(SettingsProvider.sSecureGlobalKeys);
+                    settingsToMove = setToStringArray(
+                            SettingsProvider.sSecureMovedToGlobalSettings);
                     moveSettingsToNewTable(db, TABLE_SECURE, TABLE_GLOBAL, settingsToMove, true);
 
                     db.setTransactionSuccessful();
@@ -1855,7 +1890,8 @@ public void onUpgrade(SQLiteDatabase db, int oldVersion, int currentVersion) {
                 try {
                     stmt = db.compileStatement("INSERT OR IGNORE INTO global(name,value)"
                             + " VALUES(?,?);");
-                    loadSetting(stmt, Settings.Global.ENHANCED_4G_MODE_ENABLED, ImsConfig.FeatureValueConstants.ON);
+                    loadSetting(stmt, Settings.Global.ENHANCED_4G_MODE_ENABLED,
+                            ImsConfig.FeatureValueConstants.ON);
                     db.setTransactionSuccessful();
                 } finally {
                     db.endTransaction();
@@ -1895,34 +1931,50 @@ public void onUpgrade(SQLiteDatabase db, int oldVersion, int currentVersion) {
             }
             upgradeVersion = 118;
         }
+
+        /**
+         * IMPORTANT: Do not add any more upgrade steps here as the global,
+         * secure, and system settings are no longer stored in a database
+         * but are kept in memory and persisted to XML. The correct places
+         * for adding upgrade steps are:
+         *
+         * Global: SettingsProvider.UpgradeController#onUpgradeGlobalSettings
+         * Secure: SettingsProvider.UpgradeController#onUpgradeSecureSettings
+         * System: SettingsProvider.UpgradeController#onUpgradeSystemSettings
+         */
+
         // *** Remember to update DATABASE_VERSION above!
 
         if (upgradeVersion != currentVersion) {
-            Log.w(TAG, "Got stuck trying to upgrade from version " + upgradeVersion
-                    + ", must wipe the settings provider");
-            db.execSQL("DROP TABLE IF EXISTS global");
-            db.execSQL("DROP TABLE IF EXISTS globalIndex1");
-            db.execSQL("DROP TABLE IF EXISTS system");
-            db.execSQL("DROP INDEX IF EXISTS systemIndex1");
-            db.execSQL("DROP TABLE IF EXISTS secure");
-            db.execSQL("DROP INDEX IF EXISTS secureIndex1");
-            db.execSQL("DROP TABLE IF EXISTS gservices");
-            db.execSQL("DROP INDEX IF EXISTS gservicesIndex1");
-            db.execSQL("DROP TABLE IF EXISTS bluetooth_devices");
-            db.execSQL("DROP TABLE IF EXISTS bookmarks");
-            db.execSQL("DROP INDEX IF EXISTS bookmarksIndex1");
-            db.execSQL("DROP INDEX IF EXISTS bookmarksIndex2");
-            db.execSQL("DROP TABLE IF EXISTS favorites");
-            onCreate(db);
-
-            // Added for diagnosing settings.db wipes after the fact
-            String wipeReason = oldVersion + "/" + upgradeVersion + "/" + currentVersion;
-            db.execSQL("INSERT INTO secure(name,value) values('" +
-                    "wiped_db_reason" + "','" + wipeReason + "');");
+            recreateDatabase(db, oldVersion, upgradeVersion, currentVersion);
         }
     }
 
-    private String[] hashsetToStringArray(HashSet<String> set) {
+    public void recreateDatabase(SQLiteDatabase db, int oldVersion,
+            int upgradeVersion, int currentVersion) {
+        db.execSQL("DROP TABLE IF EXISTS global");
+        db.execSQL("DROP TABLE IF EXISTS globalIndex1");
+        db.execSQL("DROP TABLE IF EXISTS system");
+        db.execSQL("DROP INDEX IF EXISTS systemIndex1");
+        db.execSQL("DROP TABLE IF EXISTS secure");
+        db.execSQL("DROP INDEX IF EXISTS secureIndex1");
+        db.execSQL("DROP TABLE IF EXISTS gservices");
+        db.execSQL("DROP INDEX IF EXISTS gservicesIndex1");
+        db.execSQL("DROP TABLE IF EXISTS bluetooth_devices");
+        db.execSQL("DROP TABLE IF EXISTS bookmarks");
+        db.execSQL("DROP INDEX IF EXISTS bookmarksIndex1");
+        db.execSQL("DROP INDEX IF EXISTS bookmarksIndex2");
+        db.execSQL("DROP TABLE IF EXISTS favorites");
+
+        onCreate(db);
+
+        // Added for diagnosing settings.db wipes after the fact
+        String wipeReason = oldVersion + "/" + upgradeVersion + "/" + currentVersion;
+        db.execSQL("INSERT INTO secure(name,value) values('" +
+                "wiped_db_reason" + "','" + wipeReason + "');");
+    }
+
+    private String[] setToStringArray(Set<String> set) {
         String[] array = new String[set.size()];
         return set.toArray(array);
     }
@@ -2639,7 +2691,8 @@ private void loadGlobalSettings(SQLiteDatabase db) {
 
             loadBooleanSetting(stmt, Settings.Global.GUEST_USER_ENABLED,
                     R.bool.def_guest_user_enabled);
-            loadSetting(stmt, Settings.Global.ENHANCED_4G_MODE_ENABLED, ImsConfig.FeatureValueConstants.ON);
+            loadSetting(stmt, Settings.Global.ENHANCED_4G_MODE_ENABLED,
+                    ImsConfig.FeatureValueConstants.ON);
             // --- New global settings start here
         } finally {
             if (stmt != null) stmt.close();

packages/SettingsProvider/src/com/android/providers/settings/EventLogTags.logtags

@@ -0,0 +1,5 @@
+# See system/core/logcat/e for a description of the format of this file.
+
+option java_package com.android.providers.settings;
+
+52100 unsupported_settings_query (uri|3),(selection|3),(whereArgs|3)

packages/SettingsProvider/src/com/android/providers/settings/SettingsBackupAgent.java

@@ -110,11 +110,7 @@ public class SettingsBackupAgent extends BackupAgentHelper {
 
     private static final String TAG = "SettingsBackupAgent";
 
-    private static final int COLUMN_NAME = 1;
-    private static final int COLUMN_VALUE = 2;
-
     private static final String[] PROJECTION = {
-        Settings.NameValueTable._ID,
         Settings.NameValueTable.NAME,
         Settings.NameValueTable.VALUE
     };
@@ -473,8 +469,8 @@ public void onRestore(BackupDataInput data, int appVersionCode,
             ParcelFileDescriptor newState) throws IOException {
 
         HashSet<String> movedToGlobal = new HashSet<String>();
-        Settings.System.getMovedKeys(movedToGlobal);
-        Settings.Secure.getMovedKeys(movedToGlobal);
+        Settings.System.getMovedToGlobalSettings(movedToGlobal);
+        Settings.Secure.getMovedToGlobalSettings(movedToGlobal);
 
         while (data.readNextHeader()) {
             final String key = data.getKey();
@@ -577,8 +573,8 @@ public void onRestoreFile(ParcelFileDescriptor data, long size,
         if (version <= FULL_BACKUP_VERSION) {
             // Generate the moved-to-global lookup table
             HashSet<String> movedToGlobal = new HashSet<String>();
-            Settings.System.getMovedKeys(movedToGlobal);
-            Settings.Secure.getMovedKeys(movedToGlobal);
+            Settings.System.getMovedToGlobalSettings(movedToGlobal);
+            Settings.Secure.getMovedToGlobalSettings(movedToGlobal);
 
             // system settings data first
             int nBytes = in.readInt();
@@ -824,11 +820,14 @@ private byte[] extractRelevantValues(Cursor cursor, String[] settings) {
             String key = settings[i];
             String value = cachedEntries.remove(key);
 
+            final int nameColumnIndex = cursor.getColumnIndex(Settings.NameValueTable.NAME);
+            final int valueColumnIndex = cursor.getColumnIndex(Settings.NameValueTable.VALUE);
+
             // If the value not cached, let us look it up.
             if (value == null) {
                 while (!cursor.isAfterLast()) {
-                    String cursorKey = cursor.getString(COLUMN_NAME);
-                    String cursorValue = cursor.getString(COLUMN_VALUE);
+                    String cursorKey = cursor.getString(nameColumnIndex);
+                    String cursorValue = cursor.getString(valueColumnIndex);
                     cursor.moveToNext();
                     if (key.equals(cursorKey)) {
                         value = cursorValue;