Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Fixes bug 976088 - add upsert for processed_crashes storage #1911

Merged
merged 2 commits into from

3 participants

@selenamarie
Owner
  • Add crashstorage upsert
  • Update tests
  • PEP8 fixes

Testing jenkins

Needs to be part of 76 to fix a gigantic logging problem we've got on postgres servers.

socorro/external/postgresql/crashstorage.py
((59 lines not shown))
)
- except self.config.database_class.IntegrityError:
- # report already exists
+ except:
@twobraids Owner

this code will prevent the app from responding to SIGTERM, because you're catching and suppressing all exceptions. You either need to catch just PG Exceptions or at the end of the block, re-raise anything that is not a PG exception.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
socorro/external/postgresql/crashstorage.py
((59 lines not shown))
)
- except self.config.database_class.IntegrityError:
- # report already exists
+ except:
+ # Silently rollback on any other failure
+ # Maybe this is a bad idea...
@peterbe Owner
peterbe added a note

:) indeed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
socorro/external/postgresql/crashstorage.py
((11 lines not shown))
)
- insert_sql = """insert into %s (uuid, processed_crash, date_processed) values
- (%%s, %%s, %%s)""" % processed_crashes_table_name
+ upsert_sql = """
@selenamarie Owner

This is where the upsert starts, and the most important thing to have a look at.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@selenamarie selenamarie commented on the diff
socorro/external/postgresql/crashstorage.py
((5 lines not shown))
+ def _save_processed_crash(self, connection, processed_crash):
@selenamarie Owner

This is where our primary change is occurring. I factored it out of save_processed() give its size and complexity.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
socorro/external/postgresql/crashstorage.py
((19 lines not shown))
+ date_processed = %%(date_processed)s
+ WHERE uuid = %%(uuid)s
+ RETURNING '%(uuid)s'
+ ),
+ insert_processed_crash (
+ INSERT INTO %(table)s (uuid, processed_crash, date_processed)
+ VALUES (%%(uuid)s, %%(processed_json)s, %%(date_processed)s)
+ WHERE NOT EXISTS (
+ SELECT uuid from %(table)s
+ WHERE
+ uuid = %%(uuid)s
+ LIMIT 1
+ )
+ RETURNING '%(uuid)s'
+ )
+ SELECT * from update_processed_crash
@selenamarie Owner

We could get the uuid out of this function if we wanted. Convention has been to use execute_no_result() for these inserts. Changing convention probably is best left to a future bug.

@twobraids Owner

I'm not sure why we'd want to get the uuid out of this query. The outer pythonic code that executes this sql already has the uuid in the variable crash_id on line 183.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
socorro/external/postgresql/crashstorage.py
((59 lines not shown))
)
- except self.config.database_class.IntegrityError:
- # report already exists
+ except self.config.database_class.Error:
@selenamarie Owner

We will no longer see IntegrityErrors because of the upsert, but we could see other kinds of Pg errors. I put in a generic catch-all for this and a rollback. We may wish to improve this in the future, and I've opened a new bug to get test coverage for this exception handling as well as some others in this same file.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
...rro/unittest/external/postgresql/test_crashstorage.py
((17 lines not shown))
(('savepoint MainThread', None),),
- (('insert into processed_crashes_20120402 (uuid, processed_crash, date_processed) values (%s, %s, %s)',
- ('936ce666-ff3b-4c7a-9674-367fe2120408', '{"startedDateTime": "2012-04-08 10:56:50.440752", "crashedThread": 8, "cpu_info": "None | 0", "PluginName": "wilma", "install_age": 22385, "topmost_filenames": [], "user_comments": null, "user_id": null, "uuid": "936ce666-ff3b-4c7a-9674-367fe2120408", "flash_version": "[blank]", "os_version": "0.0.0 Linux 2.6.35.7-perf-CL727859 #1 ", "PluginVersion": "69", "addons_checked": null, "completeddatetime": "2012-04-08 10:56:50.902884", "productid": "FA-888888", "success": true, "exploitability": "high", "client_crash_date": "2012-04-08 10:52:42.0", "PluginFilename": "dwight.txt", "dump": "...", "truncated": false, "product": "FennecAndroid", "distributor": null, "processor_notes": "SignatureTool: signature truncated due to length", "uptime": 170, "release_channel": "default", "distributor_version": null, "process_type": "plugin", "id": 361399767, "hangid": null, "version": "13.0a1", "build": "20120309050057", "ReleaseChannel": "default", "email": "bogus@bogus.com", "app_notes": "...", "os_name": "Linux", "last_crash": null, "date_processed": "2012-04-08 10:56:41.558922", "cpu_name": "arm", "reason": "SIGSEGV", "address": "0x1c", "url": "http://embarrassing.porn.com", "signature": "libxul.so@0x117441c", "addons": [["{1a5dabbd-0e74-41da-b532-a364bb552cab}", "1.0.4.1"]]}', '2012-04-08 10:56:41.558922')
+ (("""WITH update_processed_crash ( UPDATE processed_crashes_20120402 SET processed_crash = %(processed_json)s, date_processed = %(date_processed)s WHERE uuid = %(uuid)s RETURNING \'936ce666-ff3b-4c7a-9674-367fe2120408\'), insert_processed_crash ( INSERT INTO processed_crashes_20120402 (uuid, processed_crash, date_processed) VALUES (%(uuid)s, %(processed_json)s, %(date_processed)s) WHERE NOT EXISTS ( SELECT uuid from processed_crashes_20120402 WHERE uuid = %(uuid)s LIMIT 1) RETURNING \'936ce666-ff3b-4c7a-9674-367fe2120408\') SELECT * from update_processed_crash UNION ALL SELECT * from insert_processed_crash """,
@selenamarie Owner

this change is related to my patch and includes the SQL for the upsert.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
selenamarie added some commits
@selenamarie selenamarie Fixes bug 976088 - add upsert for processed_crashes storage
* Add crashstorage upsert
* Update tests
* PEP8 fixes
5d3f80e
@selenamarie selenamarie Fixes based on feedback r=twobraids
* Change RETURNING to 1 or 2 based on update or insert
* Remove savepoint and try/except block for processed_crashes insert
* Fix error in UPSERT SQL
5be75a5
@twobraids
Owner

an enthusiastic r+ from me

@selenamarie selenamarie merged commit f7d4fc4 into mozilla:master
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Feb 26, 2014
  1. @selenamarie

    Fixes bug 976088 - add upsert for processed_crashes storage

    selenamarie authored
    * Add crashstorage upsert
    * Update tests
    * PEP8 fixes
  2. @selenamarie

    Fixes based on feedback r=twobraids

    selenamarie authored
    * Change RETURNING to 1 or 2 based on update or insert
    * Remove savepoint and try/except block for processed_crashes insert
    * Fix error in UPSERT SQL
This page is out of date. Refresh to see the latest.
View
117 socorro/external/postgresql/crashstorage.py
@@ -36,7 +36,7 @@ class PostgreSQLCrashStorage(CrashStorageBase):
required_config.add_option(
'transaction_executor_class',
default="socorro.database.transaction_executor."
- "TransactionExecutorWithInfiniteBackoff",
+ "TransactionExecutorWithInfiniteBackoff",
doc='a class that will manage transactions',
from_string_converter=class_converter,
reference_value_from='resource.postgresql',
@@ -111,7 +111,7 @@ def save_raw_crash(self, raw_crash, dumps, crash_id):
#--------------------------------------------------------------------------
def _save_raw_crash_transaction(self, connection, raw_crash, crash_id):
raw_crash_table_name = (
- 'raw_crashes_%s' % self._table_suffix_for_crash_id(crash_id)
+ 'raw_crashes_%s' % self._table_suffix_for_crash_id(crash_id)
)
insert_sql = """insert into %s (uuid, raw_crash, date_processed) values
(%%s, %%s, %%s)""" % raw_crash_table_name
@@ -125,23 +125,23 @@ def _save_raw_crash_transaction(self, connection, raw_crash, crash_id):
try:
execute_no_results(connection, insert_sql, value_list)
execute_no_results(
- connection,
- "release savepoint %s" % savepoint_name
+ connection,
+ "release savepoint %s" % savepoint_name
)
except self.config.database_class.IntegrityError:
# report already exists
execute_no_results(
- connection,
- "rollback to savepoint %s" % savepoint_name
+ connection,
+ "rollback to savepoint %s" % savepoint_name
)
execute_no_results(
- connection,
- "release savepoint %s" % savepoint_name
+ connection,
+ "release savepoint %s" % savepoint_name
)
execute_no_results(
- connection,
- "delete from %s where uuid = %%s" % raw_crash_table_name,
- (crash_id,)
+ connection,
+ "delete from %s where uuid = %%s" % raw_crash_table_name,
+ (crash_id,)
)
execute_no_results(connection, insert_sql, value_list)
@@ -159,7 +159,7 @@ def get_raw_crash(self, crash_id):
#--------------------------------------------------------------------------
def _get_raw_crash_transaction(self, connection, crash_id):
raw_crash_table_name = (
- 'raw_crash_%s' % self._table_suffix_for_crash_id(crash_id)
+ 'raw_crash_%s' % self._table_suffix_for_crash_id(crash_id)
)
fetch_sql = 'select raw_crash from %s where uuid = %ss' % \
raw_crash_table_name
@@ -177,34 +177,48 @@ def _save_processed_transaction(self, connection, processed_crash):
report_id = self._save_processed_report(connection, processed_crash)
self._save_plugins(connection, processed_crash, report_id)
self._save_extensions(connection, processed_crash, report_id)
+ self._save_processed_crash(connection, processed_crash)
+ def _save_processed_crash(self, connection, processed_crash):
@selenamarie Owner

This is where our primary change is occurring. I factored it out of save_processed() give its size and complexity.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
crash_id = processed_crash['uuid']
processed_crashes_table_name = (
- 'processed_crashes_%s' % self._table_suffix_for_crash_id(crash_id)
+ 'processed_crashes_%s' % self._table_suffix_for_crash_id(crash_id)
)
- insert_sql = """insert into %s (uuid, processed_crash, date_processed) values
- (%%s, %%s, %%s)""" % processed_crashes_table_name
-
- savepoint_name = threading.currentThread().getName().replace('-', '')
- value_list = (
- crash_id,
- json.dumps(processed_crash, cls=JsonDTEncoder),
- processed_crash["date_processed"]
- )
- execute_no_results(connection, "savepoint %s" % savepoint_name)
- try:
- execute_no_results(connection, insert_sql, value_list)
- execute_no_results(
- connection,
- "release savepoint %s" % savepoint_name
- )
- except self.config.database_class.IntegrityError:
- # report already exists
- execute_no_results(
- connection,
- "rollback to savepoint %s" % savepoint_name
+ upsert_sql = """
+ WITH
+ update_processed_crash AS (
+ UPDATE %(table)s SET
+ processed_crash = %%(processed_json)s,
+ date_processed = %%(date_processed)s
+ WHERE uuid = %%(uuid)s
+ RETURNING 1
+ ),
+ insert_processed_crash AS (
+ INSERT INTO %(table)s (uuid, processed_crash, date_processed)
+ ( SELECT
+ %%(uuid)s as uuid,
+ %%(processed_json)s as processed_crash,
+ %%(date_processed)s as date_processed
+ WHERE NOT EXISTS (
+ SELECT uuid from %(table)s
+ WHERE
+ uuid = %%(uuid)s
+ LIMIT 1
+ )
)
+ RETURNING 2
+ )
+ SELECT * from update_processed_crash
+ UNION ALL
+ SELECT * from insert_processed_crash
+ """ % {'table': processed_crashes_table_name, 'uuid': crash_id}
+ values = {
+ 'processed_json': json.dumps(processed_crash, cls=JsonDTEncoder),
+ 'date_processed': processed_crash["date_processed"],
+ 'uuid': crash_id
+ }
+ execute_no_results(connection, upsert_sql, values)
#--------------------------------------------------------------------------
def _save_processed_report(self, connection, processed_crash):
@@ -217,7 +231,7 @@ def _save_processed_report(self, connection, processed_crash):
value_list.append(processed_crash[pro_crash_name])
crash_id = processed_crash['uuid']
reports_table_name = (
- 'reports_%s' % self._table_suffix_for_crash_id(crash_id)
+ 'reports_%s' % self._table_suffix_for_crash_id(crash_id)
)
insert_sql = "insert into %s (%s) values (%s) returning id" % (
reports_table_name,
@@ -236,23 +250,23 @@ def _save_processed_report(self, connection, processed_crash):
try:
report_id = single_value_sql(connection, insert_sql, value_list)
execute_no_results(
- connection,
- "release savepoint %s" % savepoint_name
+ connection,
+ "release savepoint %s" % savepoint_name
)
except self.config.database_class.IntegrityError:
# report already exists
execute_no_results(
- connection,
- "rollback to savepoint %s" % savepoint_name
+ connection,
+ "rollback to savepoint %s" % savepoint_name
)
execute_no_results(
- connection,
- "release savepoint %s" % savepoint_name
+ connection,
+ "release savepoint %s" % savepoint_name
)
execute_no_results(
- connection,
- "delete from %s where uuid = %%s" % reports_table_name,
- (processed_crash.uuid,)
+ connection,
+ "delete from %s where uuid = %%s" % reports_table_name,
+ (processed_crash.uuid,)
)
report_id = single_value_sql(connection, insert_sql, value_list)
return report_id
@@ -322,10 +336,10 @@ def _save_extensions(self, connection, processed_crash, report_id):
table_suffix = self._table_suffix_for_crash_id(crash_id)
extensions_table_name = 'extensions_%s' % table_suffix
extensions_insert_sql = (
- "insert into %s "
- " (report_id, date_processed, extension_key, extension_id, "
- " extension_version)"
- "values (%%s, %%s, %%s, %%s, %%s)" % extensions_table_name
+ "insert into %s "
+ " (report_id, date_processed, extension_key, extension_id, "
+ " extension_version)"
+ "values (%%s, %%s, %%s, %%s, %%s)" % extensions_table_name
)
for i, x in enumerate(extensions):
try:
@@ -337,8 +351,8 @@ def _save_extensions(self, connection, processed_crash, report_id):
x[1]))
except IndexError:
self.config.logger.warning(
- '"%s" is deficient as a name and version for an addon',
- str(x[0])
+ '"%s" is deficient as a name and version for an addon',
+ str(x[0])
)
#--------------------------------------------------------------------------
@@ -347,9 +361,8 @@ def _table_suffix_for_crash_id(crash_id):
"""given an crash_id, return the name of its storage table"""
crash_id_date = uuid_to_date(crash_id)
previous_monday_date = (
- crash_id_date + datetime.timedelta(days=-crash_id_date.weekday())
+ crash_id_date + datetime.timedelta(days=-crash_id_date.weekday())
)
return '%4d%02d%02d' % (previous_monday_date.year,
previous_monday_date.month,
previous_monday_date.day)
-
View
251 socorro/unittest/external/postgresql/test_crashstorage.py
@@ -16,7 +16,6 @@
TransactionExecutorWithLimitedBackoff
)
from socorro.external.postgresql.crashstorage import PostgreSQLCrashStorage
-from socorro.external.crashstorage_base import Redactor
empty_tuple = ()
@@ -75,6 +74,10 @@
}
+def remove_whitespace(string):
+ return string.replace('\n', '').replace(' ', '')
+
+
#class TestIntegrationPostgresSQLCrashStorage(unittest.TestCase):
class DontTestIntegrationPostgresSQLCrashStorage(object):
@@ -87,14 +90,14 @@ def _setup_config_manager(self, extra_value_source=None):
required_config.add_option('logger', default=mock_logging)
config_manager = ConfigurationManager(
- [required_config],
- app_name='testapp',
- app_version='1.0',
- app_description='app description',
+ [required_config],
+ app_name='testapp',
+ app_version='1.0',
+ app_description='app description',
values_source_list=[{
'logger': mock_logging,
}, extra_value_source],
- argv_source=[]
+ argv_source=[]
)
return config_manager
@@ -111,10 +114,12 @@ def setUp(self):
}
dsn = ('host=%(database_hostname)s dbname=%(database_name)s '
- 'user=%(database_username)s password=%(database_password)s' % DSN)
+ 'user=%(database_username)s password=%(database_password)s'
+ % DSN)
self.conn = psycopg2.connect(dsn)
cursor = self.conn.cursor()
- date_suffix = PostgreSQLCrashStorage._table_suffix_for_crash_id(a_processed_crash['uuid'])
+ date_suffix = PostgreSQLCrashStorage.\
+ _table_suffix_for_crash_id(a_processed_crash['uuid'])
self.reports_table_name = 'reports%s' % date_suffix
cursor.execute("""
DROP TABLE IF EXISTS %(table_name)s;
@@ -240,15 +245,15 @@ def test_basic_postgres_save_raw_crash(self):
required_config.add_option('logger', default=mock_logging)
config_manager = ConfigurationManager(
- [required_config],
- app_name='testapp',
- app_version='1.0',
- app_description='app description',
- values_source_list=[{
- 'logger': mock_logging,
- 'database_class': mock_postgres
- }],
- argv_source=[]
+ [required_config],
+ app_name='testapp',
+ app_version='1.0',
+ app_description='app description',
+ values_source_list=[{
+ 'logger': mock_logging,
+ 'database_class': mock_postgres
+ }],
+ argv_source=[]
)
with config_manager.context() as config:
@@ -271,12 +276,11 @@ def test_basic_postgres_save_raw_crash(self):
expected_execute_args = (
(('savepoint MainThread', None),),
- (('insert into raw_crashes_20120402 (uuid, raw_crash, date_processed) values (%s, %s, %s)',
- (
- '936ce666-ff3b-4c7a-9674-367fe2120408',
- '{"submitted_timestamp": "2012-04-08 10:52:42.0", "Version": "6.02E23", "ProductName": "Fennicky"}',
- "2012-04-08 10:52:42.0"
- )),),
+ (('insert into raw_crashes_20120402 (uuid, raw_crash, date_processed) values (%s, %s, %s)', (
+ '936ce666-ff3b-4c7a-9674-367fe2120408',
+ '{"submitted_timestamp": "2012-04-08 10:52:42.0", "Version": "6.02E23", "ProductName": "Fennicky"}',
+ "2012-04-08 10:52:42.0"
+ )),),
(('release savepoint MainThread', None),),
)
@@ -284,11 +288,9 @@ def test_basic_postgres_save_raw_crash(self):
for expected, actual in zip(expected_execute_args,
actual_execute_args):
expeceted_sql, expected_params = expected[0]
- expeceted_sql = expeceted_sql.replace('\n', '')
- expeceted_sql = expeceted_sql.replace(' ', '')
+ expeceted_sql = remove_whitespace(expeceted_sql)
actual_sql, actual_params = actual[0]
- actual_sql = actual_sql.replace('\n', '')
- actual_sql = actual_sql.replace(' ', '')
+ actual_sql = remove_whitespace(actual_sql)
self.assertEqual(expeceted_sql, actual_sql)
self.assertEqual(expected_params, actual_params)
@@ -300,15 +302,15 @@ def test_basic_key_error_on_save_processed(self):
required_config.add_option('logger', default=mock_logging)
config_manager = ConfigurationManager(
- [required_config],
- app_name='testapp',
- app_version='1.0',
- app_description='app description',
- values_source_list=[{
- 'logger': mock_logging,
- 'database_class': mock_postgres
- }],
- argv_source=[]
+ [required_config],
+ app_name='testapp',
+ app_version='1.0',
+ app_description='app description',
+ values_source_list=[{
+ 'logger': mock_logging,
+ 'database_class': mock_postgres
+ }],
+ argv_source=[]
)
with config_manager.context() as config:
@@ -327,7 +329,6 @@ def test_basic_key_error_on_save_processed(self):
crashstorage.save_processed,
broken_processed_crash)
-
def test_basic_postgres_save_processed_success(self):
mock_logging = mock.Mock()
@@ -336,15 +337,15 @@ def test_basic_postgres_save_processed_success(self):
required_config.add_option('logger', default=mock_logging)
config_manager = ConfigurationManager(
- [required_config],
- app_name='testapp',
- app_version='1.0',
- app_description='app description',
- values_source_list=[{
- 'logger': mock_logging,
- 'database_class': mock_postgres
- }],
- argv_source=[]
+ [required_config],
+ app_name='testapp',
+ app_version='1.0',
+ app_description='app description',
+ values_source_list=[{
+ 'logger': mock_logging,
+ 'database_class': mock_postgres
+ }],
+ argv_source=[]
)
with config_manager.context() as config:
@@ -355,6 +356,7 @@ def test_basic_postgres_save_processed_success(self):
crashstorage.save_processed(a_processed_crash)
fetch_all_returns = [((666,),), ((23,),), ]
+
def fetch_all_func(*args):
result = fetch_all_returns.pop(0)
return result
@@ -362,40 +364,36 @@ def fetch_all_func(*args):
m = mock.MagicMock()
m.__enter__.return_value = m
database = crashstorage.database.return_value = m
- m.cursor.return_value.fetchall.side_effect=fetch_all_func
+ m.cursor.return_value.fetchall.side_effect = fetch_all_func
crashstorage.save_processed(a_processed_crash)
- self.assertEqual(m.cursor.call_count, 9)
+ self.assertEqual(m.cursor.call_count, 7)
self.assertEqual(m.cursor().fetchall.call_count, 2)
- self.assertEqual(m.cursor().execute.call_count, 9)
+ self.assertEqual(m.cursor().execute.call_count, 7)
expected_execute_args = (
(('savepoint MainThread', None),),
(('insert into reports_20120402 (addons_checked, address, app_notes, build, client_crash_date, completed_datetime, cpu_info, cpu_name, date_processed, distributor, distributor_version, email, exploitability, flash_version, hangid, install_age, last_crash, os_name, os_version, processor_notes, process_type, product, productid, reason, release_channel, signature, started_datetime, success, topmost_filenames, truncated, uptime, user_comments, user_id, url, uuid, version) values (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s) returning id',
- [None, '0x1c', '...', '20120309050057', '2012-04-08 10:52:42.0', '2012-04-08 10:56:50.902884', 'None | 0', 'arm', '2012-04-08 10:56:41.558922', None, None, 'bogus@bogus.com', 'high', '[blank]', None, 22385, None, 'Linux', '0.0.0 Linux 2.6.35.7-perf-CL727859 #1 ', 'SignatureTool: signature truncated due to length', 'plugin', 'FennecAndroid', 'FA-888888', 'SIGSEGV', 'default', 'libxul.so@0x117441c', '2012-04-08 10:56:50.440752', True, [], False, 170, None, None, 'http://embarrassing.porn.com', '936ce666-ff3b-4c7a-9674-367fe2120408', '13.0a1'],),
+ [None, '0x1c', '...', '20120309050057', '2012-04-08 10:52:42.0', '2012-04-08 10:56:50.902884', 'None | 0', 'arm', '2012-04-08 10:56:41.558922', None, None, 'bogus@bogus.com', 'high', '[blank]', None, 22385, None, 'Linux', '0.0.0 Linux 2.6.35.7-perf-CL727859 #1 ', 'SignatureTool: signature truncated due to length', 'plugin', 'FennecAndroid', 'FA-888888', 'SIGSEGV', 'default', 'libxul.so@0x117441c', '2012-04-08 10:56:50.440752', True, [], False, 170, None, None, 'http://embarrassing.porn.com', '936ce666-ff3b-4c7a-9674-367fe2120408', '13.0a1'],),
),
(('release savepoint MainThread', None),),
(('select id from plugins where filename = %s and name = %s',
- ('dwight.txt', 'wilma')),),
+ ('dwight.txt', 'wilma')),),
(('insert into plugins_reports_20120402 (report_id, plugin_id, date_processed, version) values (%s, %s, %s, %s)',
- (666, 23, '2012-04-08 10:56:41.558922', '69')),),
+ (666, 23, '2012-04-08 10:56:41.558922', '69')),),
(('insert into extensions_20120402 (report_id, date_processed, extension_key, extension_id, extension_version)values (%s, %s, %s, %s, %s)',
- (666, '2012-04-08 10:56:41.558922', 0, '{1a5dabbd-0e74-41da-b532-a364bb552cab}', '1.0.4.1')),),
- (('savepoint MainThread', None),),
- (('insert into processed_crashes_20120402 (uuid, processed_crash, date_processed) values (%s, %s, %s)',
- ('936ce666-ff3b-4c7a-9674-367fe2120408', '{"startedDateTime": "2012-04-08 10:56:50.440752", "crashedThread": 8, "cpu_info": "None | 0", "PluginName": "wilma", "install_age": 22385, "topmost_filenames": [], "user_comments": null, "user_id": null, "uuid": "936ce666-ff3b-4c7a-9674-367fe2120408", "flash_version": "[blank]", "os_version": "0.0.0 Linux 2.6.35.7-perf-CL727859 #1 ", "PluginVersion": "69", "addons_checked": null, "completeddatetime": "2012-04-08 10:56:50.902884", "productid": "FA-888888", "success": true, "exploitability": "high", "client_crash_date": "2012-04-08 10:52:42.0", "PluginFilename": "dwight.txt", "dump": "...", "truncated": false, "product": "FennecAndroid", "distributor": null, "processor_notes": "SignatureTool: signature truncated due to length", "uptime": 170, "release_channel": "default", "distributor_version": null, "process_type": "plugin", "id": 361399767, "hangid": null, "version": "13.0a1", "build": "20120309050057", "ReleaseChannel": "default", "email": "bogus@bogus.com", "app_notes": "...", "os_name": "Linux", "last_crash": null, "date_processed": "2012-04-08 10:56:41.558922", "cpu_name": "arm", "reason": "SIGSEGV", "address": "0x1c", "url": "http://embarrassing.porn.com", "signature": "libxul.so@0x117441c", "addons": [["{1a5dabbd-0e74-41da-b532-a364bb552cab}", "1.0.4.1"]]}', '2012-04-08 10:56:41.558922')
+ (666, '2012-04-08 10:56:41.558922', 0, '{1a5dabbd-0e74-41da-b532-a364bb552cab}', '1.0.4.1')),),
+ (("""WITH update_processed_crash AS ( UPDATE processed_crashes_20120402 SET processed_crash = %(processed_json)s, date_processed = %(date_processed)s WHERE uuid = %(uuid)s RETURNING 1), insert_processed_crash AS ( INSERT INTO processed_crashes_20120402 (uuid, processed_crash, date_processed) ( SELECT %(uuid)s as uuid, %(processed_json)s as processed_crash, %(date_processed)s as date_processed WHERE NOT EXISTS ( SELECT uuid from processed_crashes_20120402 WHERE uuid = %(uuid)s LIMIT 1)) RETURNING 2) SELECT * from update_processed_crash UNION ALL SELECT * from insert_processed_crash """,
+ {'uuid': '936ce666-ff3b-4c7a-9674-367fe2120408', 'processed_json': '{"startedDateTime": "2012-04-08 10:56:50.440752", "crashedThread": 8, "cpu_info": "None | 0", "PluginName": "wilma", "install_age": 22385, "topmost_filenames": [], "user_comments": null, "user_id": null, "uuid": "936ce666-ff3b-4c7a-9674-367fe2120408", "flash_version": "[blank]", "os_version": "0.0.0 Linux 2.6.35.7-perf-CL727859 #1 ", "PluginVersion": "69", "addons_checked": null, "completeddatetime": "2012-04-08 10:56:50.902884", "productid": "FA-888888", "success": true, "exploitability": "high", "client_crash_date": "2012-04-08 10:52:42.0", "PluginFilename": "dwight.txt", "dump": "...", "truncated": false, "product": "FennecAndroid", "distributor": null, "processor_notes": "SignatureTool: signature truncated due to length", "uptime": 170, "release_channel": "default", "distributor_version": null, "process_type": "plugin", "id": 361399767, "hangid": null, "version": "13.0a1", "build": "20120309050057", "ReleaseChannel": "default", "email": "bogus@bogus.com", "app_notes": "...", "os_name": "Linux", "last_crash": null, "date_processed": "2012-04-08 10:56:41.558922", "cpu_name": "arm", "reason": "SIGSEGV", "address": "0x1c", "url": "http://embarrassing.porn.com", "signature": "libxul.so@0x117441c", "addons": [["{1a5dabbd-0e74-41da-b532-a364bb552cab}", "1.0.4.1"]]}', 'date_processed': '2012-04-08 10:56:41.558922'}
),),
- (('release savepoint MainThread', None),),
)
actual_execute_args = m.cursor().execute.call_args_list
for expected, actual in zip(expected_execute_args,
actual_execute_args):
expected_sql, expected_params = expected[0]
- expected_sql = expected_sql.replace('\n', '')
- expected_sql = expected_sql.replace(' ', '')
+ expected_sql = remove_whitespace(expected_sql)
actual_sql, actual_params = actual[0]
- actual_sql = actual_sql.replace('\n', '')
- actual_sql = actual_sql.replace(' ', '')
+ actual_sql = remove_whitespace(actual_sql)
self.assertEqual(expected_sql, actual_sql)
self.assertEqual(expected_params, actual_params)
@@ -407,15 +405,15 @@ def test_basic_postgres_save_processed_success2(self):
required_config.add_option('logger', default=mock_logging)
config_manager = ConfigurationManager(
- [required_config],
- app_name='testapp',
- app_version='1.0',
- app_description='app description',
- values_source_list=[{
- 'logger': mock_logging,
- 'database_class': mock_postgres
- }],
- argv_source=[]
+ [required_config],
+ app_name='testapp',
+ app_version='1.0',
+ app_description='app description',
+ values_source_list=[{
+ 'logger': mock_logging,
+ 'database_class': mock_postgres
+ }],
+ argv_source=[]
)
with config_manager.context() as config:
@@ -424,6 +422,7 @@ def test_basic_postgres_save_processed_success2(self):
self.assertTrue(isinstance(database, mock.Mock))
fetch_all_returns = [((666,),), None, ((23,),), ]
+
def fetch_all_func(*args):
result = fetch_all_returns.pop(0)
return result
@@ -431,41 +430,37 @@ def fetch_all_func(*args):
m = mock.MagicMock()
m.__enter__.return_value = m
database = crashstorage.database.return_value = m
- m.cursor.return_value.fetchall.side_effect=fetch_all_func
+ m.cursor.return_value.fetchall.side_effect = fetch_all_func
crashstorage.save_processed(a_processed_crash)
- self.assertEqual(m.cursor.call_count, 10)
+ self.assertEqual(m.cursor.call_count, 8)
self.assertEqual(m.cursor().fetchall.call_count, 3)
- self.assertEqual(m.cursor().execute.call_count, 10)
+ self.assertEqual(m.cursor().execute.call_count, 8)
expected_execute_args = (
(('savepoint MainThread', None),),
(('insert into reports_20120402 (addons_checked, address, app_notes, build, client_crash_date, completed_datetime, cpu_info, cpu_name, date_processed, distributor, distributor_version, email, exploitability, flash_version, hangid, install_age, last_crash, os_name, os_version, processor_notes, process_type, product, productid, reason, release_channel, signature, started_datetime, success, topmost_filenames, truncated, uptime, user_comments, user_id, url, uuid, version) values (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s) returning id',
- [None, '0x1c', '...', '20120309050057', '2012-04-08 10:52:42.0', '2012-04-08 10:56:50.902884', 'None | 0', 'arm', '2012-04-08 10:56:41.558922', None, None, 'bogus@bogus.com', 'high', '[blank]', None, 22385, None, 'Linux', '0.0.0 Linux 2.6.35.7-perf-CL727859 #1 ', 'SignatureTool: signature truncated due to length', 'plugin', 'FennecAndroid', 'FA-888888', 'SIGSEGV', 'default', 'libxul.so@0x117441c', '2012-04-08 10:56:50.440752', True, [], False, 170, None, None, 'http://embarrassing.porn.com', '936ce666-ff3b-4c7a-9674-367fe2120408', '13.0a1']),),
+ [None, '0x1c', '...', '20120309050057', '2012-04-08 10:52:42.0', '2012-04-08 10:56:50.902884', 'None | 0', 'arm', '2012-04-08 10:56:41.558922', None, None, 'bogus@bogus.com', 'high', '[blank]', None, 22385, None, 'Linux', '0.0.0 Linux 2.6.35.7-perf-CL727859 #1 ', 'SignatureTool: signature truncated due to length', 'plugin', 'FennecAndroid', 'FA-888888', 'SIGSEGV', 'default', 'libxul.so@0x117441c', '2012-04-08 10:56:50.440752', True, [], False, 170, None, None, 'http://embarrassing.porn.com', '936ce666-ff3b-4c7a-9674-367fe2120408', '13.0a1']),),
(('release savepoint MainThread', None),),
(('select id from plugins where filename = %s and name = %s',
- ('dwight.txt', 'wilma')),),
+ ('dwight.txt', 'wilma')),),
(('insert into plugins (filename, name) values (%s, %s) returning id',
- ('dwight.txt', 'wilma')),),
+ ('dwight.txt', 'wilma')),),
(('insert into plugins_reports_20120402 (report_id, plugin_id, date_processed, version) values (%s, %s, %s, %s)',
- (666, 23, '2012-04-08 10:56:41.558922', '69')),),
+ (666, 23, '2012-04-08 10:56:41.558922', '69')),),
(('insert into extensions_20120402 (report_id, date_processed, extension_key, extension_id, extension_version)values (%s, %s, %s, %s, %s)',
- (666, '2012-04-08 10:56:41.558922', 0, '{1a5dabbd-0e74-41da-b532-a364bb552cab}', '1.0.4.1')),),
- (('savepoint MainThread', None),),
- (('insert into processed_crashes_20120402 (uuid, processed_crash, date_processed) values (%s, %s, %s)',
- ('936ce666-ff3b-4c7a-9674-367fe2120408', '{"startedDateTime": "2012-04-08 10:56:50.440752", "crashedThread": 8, "cpu_info": "None | 0", "PluginName": "wilma", "install_age": 22385, "topmost_filenames": [], "user_comments": null, "user_id": null, "uuid": "936ce666-ff3b-4c7a-9674-367fe2120408", "flash_version": "[blank]", "os_version": "0.0.0 Linux 2.6.35.7-perf-CL727859 #1 ", "PluginVersion": "69", "addons_checked": null, "completeddatetime": "2012-04-08 10:56:50.902884", "productid": "FA-888888", "success": true, "exploitability": "high", "client_crash_date": "2012-04-08 10:52:42.0", "PluginFilename": "dwight.txt", "dump": "...", "truncated": false, "product": "FennecAndroid", "distributor": null, "processor_notes": "SignatureTool: signature truncated due to length", "uptime": 170, "release_channel": "default", "distributor_version": null, "process_type": "plugin", "id": 361399767, "hangid": null, "version": "13.0a1", "build": "20120309050057", "ReleaseChannel": "default", "email": "bogus@bogus.com", "app_notes": "...", "os_name": "Linux", "last_crash": null, "date_processed": "2012-04-08 10:56:41.558922", "cpu_name": "arm", "reason": "SIGSEGV", "address": "0x1c", "url": "http://embarrassing.porn.com", "signature": "libxul.so@0x117441c", "addons": [["{1a5dabbd-0e74-41da-b532-a364bb552cab}", "1.0.4.1"]]}', '2012-04-08 10:56:41.558922')
+ (666, '2012-04-08 10:56:41.558922', 0, '{1a5dabbd-0e74-41da-b532-a364bb552cab}', '1.0.4.1')),),
+ (("""WITH update_processed_crash AS ( UPDATE processed_crashes_20120402 SET processed_crash = %(processed_json)s, date_processed = %(date_processed)s WHERE uuid = %(uuid)s RETURNING 1), insert_processed_crash AS ( INSERT INTO processed_crashes_20120402 (uuid, processed_crash, date_processed) ( SELECT %(uuid)s as uuid, %(processed_json)s as processed_crash, %(date_processed)s as date_processed WHERE NOT EXISTS ( SELECT uuid from processed_crashes_20120402 WHERE uuid = %(uuid)s LIMIT 1)) RETURNING 2) SELECT * from update_processed_crash UNION ALL SELECT * from insert_processed_crash """,
+ {'uuid': '936ce666-ff3b-4c7a-9674-367fe2120408', 'processed_json': '{"startedDateTime": "2012-04-08 10:56:50.440752", "crashedThread": 8, "cpu_info": "None | 0", "PluginName": "wilma", "install_age": 22385, "topmost_filenames": [], "user_comments": null, "user_id": null, "uuid": "936ce666-ff3b-4c7a-9674-367fe2120408", "flash_version": "[blank]", "os_version": "0.0.0 Linux 2.6.35.7-perf-CL727859 #1 ", "PluginVersion": "69", "addons_checked": null, "completeddatetime": "2012-04-08 10:56:50.902884", "productid": "FA-888888", "success": true, "exploitability": "high", "client_crash_date": "2012-04-08 10:52:42.0", "PluginFilename": "dwight.txt", "dump": "...", "truncated": false, "product": "FennecAndroid", "distributor": null, "processor_notes": "SignatureTool: signature truncated due to length", "uptime": 170, "release_channel": "default", "distributor_version": null, "process_type": "plugin", "id": 361399767, "hangid": null, "version": "13.0a1", "build": "20120309050057", "ReleaseChannel": "default", "email": "bogus@bogus.com", "app_notes": "...", "os_name": "Linux", "last_crash": null, "date_processed": "2012-04-08 10:56:41.558922", "cpu_name": "arm", "reason": "SIGSEGV", "address": "0x1c", "url": "http://embarrassing.porn.com", "signature": "libxul.so@0x117441c", "addons": [["{1a5dabbd-0e74-41da-b532-a364bb552cab}", "1.0.4.1"]]}', 'date_processed': '2012-04-08 10:56:41.558922'}
),),
- (('release savepoint MainThread', None),),
)
actual_execute_args = m.cursor().execute.call_args_list
for expected, actual in zip(expected_execute_args,
actual_execute_args):
expected_sql, expected_params = expected[0]
- expected_sql = expected_sql.replace('\n', '')
- expected_sql = expected_sql.replace(' ', '')
+ expected_sql = remove_whitespace(expected_sql)
actual_sql, actual_params = actual[0]
- actual_sql = actual_sql.replace('\n', '')
- actual_sql = actual_sql.replace(' ', '')
+ actual_sql = remove_whitespace(actual_sql)
self.assertEqual(expected_sql, actual_sql)
self.assertEqual(expected_params, actual_params)
@@ -478,18 +473,18 @@ def test_basic_postgres_save_processed_operational_error(self):
required_config.add_option('logger', default=mock_logging)
config_manager = ConfigurationManager(
- [required_config],
- app_name='testapp',
- app_version='1.0',
- app_description='app description',
- values_source_list=[{
- 'logger': mock_logging,
- 'database_class': mock_postgres,
- 'transaction_executor_class':
- TransactionExecutorWithLimitedBackoff,
- 'backoff_delays': [0, 0, 0],
- }],
- argv_source=[]
+ [required_config],
+ app_name='testapp',
+ app_version='1.0',
+ app_description='app description',
+ values_source_list=[{
+ 'logger': mock_logging,
+ 'database_class': mock_postgres,
+ 'transaction_executor_class':
+ TransactionExecutorWithLimitedBackoff,
+ 'backoff_delays': [0, 0, 0],
+ }],
+ argv_source=[]
)
with config_manager.context() as config:
@@ -499,11 +494,6 @@ def test_basic_postgres_save_processed_operational_error(self):
database = crashstorage.database.return_value = mock.MagicMock()
self.assertTrue(isinstance(database, mock.Mock))
- fetch_all_returns = [((666,),), None, ((23,),), ]
- def fetch_all_func(*args):
- result = fetch_all_returns.pop(0)
- return result
-
m = mock.MagicMock()
m.__enter__.return_value = m
database = crashstorage.database.return_value = m
@@ -513,9 +503,7 @@ def fetch_all_func(*args):
a_processed_crash)
self.assertEqual(m.cursor.call_count, 3)
-
def test_basic_postgres_save_processed_succeed_after_failures(self):
-
mock_logging = mock.Mock()
mock_postgres = mock.Mock()
@@ -523,18 +511,18 @@ def test_basic_postgres_save_processed_succeed_after_failures(self):
required_config.add_option('logger', default=mock_logging)
config_manager = ConfigurationManager(
- [required_config],
- app_name='testapp',
- app_version='1.0',
- app_description='app description',
- values_source_list=[{
- 'logger': mock_logging,
- 'database_class': mock_postgres,
- 'transaction_executor_class':
- TransactionExecutorWithLimitedBackoff,
- 'backoff_delays': [0, 0, 0],
- }],
- argv_source=[]
+ [required_config],
+ app_name='testapp',
+ app_version='1.0',
+ app_description='app description',
+ values_source_list=[{
+ 'logger': mock_logging,
+ 'database_class': mock_postgres,
+ 'transaction_executor_class':
+ TransactionExecutorWithLimitedBackoff,
+ 'backoff_delays': [0, 0, 0],
+ }],
+ argv_source=[]
)
with config_manager.context() as config:
@@ -545,15 +533,18 @@ def test_basic_postgres_save_processed_succeed_after_failures(self):
self.assertTrue(isinstance(database, mock.Mock))
fetch_all_returns = [((666,),), None, ((23,),), ]
+
def fetch_all_func(*args):
result = fetch_all_returns.pop(0)
return result
+
fetch_mock = mock.Mock()
fetch_mock.fetchall.side_effect = fetch_all_func
connection_trouble = [OperationalError('bad'),
OperationalError('worse'),
]
+
def broken_connection(*args):
try:
result = connection_trouble.pop(0)
@@ -566,38 +557,34 @@ def broken_connection(*args):
database = crashstorage.database.return_value = m
m.cursor.side_effect = broken_connection
crashstorage.save_processed(a_processed_crash)
- self.assertEqual(m.cursor.call_count, 12)
+ self.assertEqual(m.cursor.call_count, 10)
self.assertEqual(m.cursor().fetchall.call_count, 3)
- self.assertEqual(m.cursor().execute.call_count, 10)
+ self.assertEqual(m.cursor().execute.call_count, 8)
expected_execute_args = (
(('savepoint MainThread', None),),
(('insert into reports_20120402 (addons_checked, address, app_notes, build, client_crash_date, completed_datetime, cpu_info, cpu_name, date_processed, distributor, distributor_version, email, exploitability, flash_version, hangid, install_age, last_crash, os_name, os_version, processor_notes, process_type, product, productid, reason, release_channel, signature, started_datetime, success, topmost_filenames, truncated, uptime, user_comments, user_id, url, uuid, version) values (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s) returning id',
- [None, '0x1c', '...', '20120309050057', '2012-04-08 10:52:42.0', '2012-04-08 10:56:50.902884', 'None | 0', 'arm', '2012-04-08 10:56:41.558922', None, None, 'bogus@bogus.com', 'high', '[blank]', None, 22385, None, 'Linux', '0.0.0 Linux 2.6.35.7-perf-CL727859 #1 ', 'SignatureTool: signature truncated due to length', 'plugin', 'FennecAndroid', 'FA-888888', 'SIGSEGV', 'default', 'libxul.so@0x117441c', '2012-04-08 10:56:50.440752', True, [], False, 170, None, None, 'http://embarrassing.porn.com', '936ce666-ff3b-4c7a-9674-367fe2120408', '13.0a1']),),
+ [None, '0x1c', '...', '20120309050057', '2012-04-08 10:52:42.0', '2012-04-08 10:56:50.902884', 'None | 0', 'arm', '2012-04-08 10:56:41.558922', None, None, 'bogus@bogus.com', 'high', '[blank]', None, 22385, None, 'Linux', '0.0.0 Linux 2.6.35.7-perf-CL727859 #1 ', 'SignatureTool: signature truncated due to length', 'plugin', 'FennecAndroid', 'FA-888888', 'SIGSEGV', 'default', 'libxul.so@0x117441c', '2012-04-08 10:56:50.440752', True, [], False, 170, None, None, 'http://embarrassing.porn.com', '936ce666-ff3b-4c7a-9674-367fe2120408', '13.0a1']),),
(('release savepoint MainThread', None),),
(('select id from plugins where filename = %s and name = %s',
- ('dwight.txt', 'wilma')),),
+ ('dwight.txt', 'wilma')),),
(('insert into plugins (filename, name) values (%s, %s) returning id',
- ('dwight.txt', 'wilma')),),
+ ('dwight.txt', 'wilma')),),
(('insert into plugins_reports_20120402 (report_id, plugin_id, date_processed, version) values (%s, %s, %s, %s)',
- (666, 23, '2012-04-08 10:56:41.558922', '69')),),
+ (666, 23, '2012-04-08 10:56:41.558922', '69')),),
(('insert into extensions_20120402 (report_id, date_processed, extension_key, extension_id, extension_version)values (%s, %s, %s, %s, %s)',
- (666, '2012-04-08 10:56:41.558922', 0, '{1a5dabbd-0e74-41da-b532-a364bb552cab}', '1.0.4.1')),),
- (('savepoint MainThread', None),),
- (('insert into processed_crashes_20120402 (uuid, processed_crash, date_processed) values (%s, %s, %s)',
- ('936ce666-ff3b-4c7a-9674-367fe2120408', '{"startedDateTime": "2012-04-08 10:56:50.440752", "crashedThread": 8, "cpu_info": "None | 0", "PluginName": "wilma", "install_age": 22385, "topmost_filenames": [], "user_comments": null, "user_id": null, "uuid": "936ce666-ff3b-4c7a-9674-367fe2120408", "flash_version": "[blank]", "os_version": "0.0.0 Linux 2.6.35.7-perf-CL727859 #1 ", "PluginVersion": "69", "addons_checked": null, "completeddatetime": "2012-04-08 10:56:50.902884", "productid": "FA-888888", "success": true, "exploitability": "high", "client_crash_date": "2012-04-08 10:52:42.0", "PluginFilename": "dwight.txt", "dump": "...", "truncated": false, "product": "FennecAndroid", "distributor": null, "processor_notes": "SignatureTool: signature truncated due to length", "uptime": 170, "release_channel": "default", "distributor_version": null, "process_type": "plugin", "id": 361399767, "hangid": null, "version": "13.0a1", "build": "20120309050057", "ReleaseChannel": "default", "email": "bogus@bogus.com", "app_notes": "...", "os_name": "Linux", "last_crash": null, "date_processed": "2012-04-08 10:56:41.558922", "cpu_name": "arm", "reason": "SIGSEGV", "address": "0x1c", "url": "http://embarrassing.porn.com", "signature": "libxul.so@0x117441c", "addons": [["{1a5dabbd-0e74-41da-b532-a364bb552cab}", "1.0.4.1"]]}', '2012-04-08 10:56:41.558922')
+ (666, '2012-04-08 10:56:41.558922', 0, '{1a5dabbd-0e74-41da-b532-a364bb552cab}', '1.0.4.1')),),
+ (("""WITH update_processed_crash AS ( UPDATE processed_crashes_20120402 SET processed_crash = %(processed_json)s, date_processed = %(date_processed)s WHERE uuid = %(uuid)s RETURNING 1), insert_processed_crash AS ( INSERT INTO processed_crashes_20120402 (uuid, processed_crash, date_processed) ( SELECT %(uuid)s as uuid, %(processed_json)s as processed_crash, %(date_processed)s as date_processed WHERE NOT EXISTS ( SELECT uuid from processed_crashes_20120402 WHERE uuid = %(uuid)s LIMIT 1)) RETURNING 2) SELECT * from update_processed_crash UNION ALL SELECT * from insert_processed_crash """,
+ {'uuid': '936ce666-ff3b-4c7a-9674-367fe2120408', 'processed_json': '{"startedDateTime": "2012-04-08 10:56:50.440752", "crashedThread": 8, "cpu_info": "None | 0", "PluginName": "wilma", "install_age": 22385, "topmost_filenames": [], "user_comments": null, "user_id": null, "uuid": "936ce666-ff3b-4c7a-9674-367fe2120408", "flash_version": "[blank]", "os_version": "0.0.0 Linux 2.6.35.7-perf-CL727859 #1 ", "PluginVersion": "69", "addons_checked": null, "completeddatetime": "2012-04-08 10:56:50.902884", "productid": "FA-888888", "success": true, "exploitability": "high", "client_crash_date": "2012-04-08 10:52:42.0", "PluginFilename": "dwight.txt", "dump": "...", "truncated": false, "product": "FennecAndroid", "distributor": null, "processor_notes": "SignatureTool: signature truncated due to length", "uptime": 170, "release_channel": "default", "distributor_version": null, "process_type": "plugin", "id": 361399767, "hangid": null, "version": "13.0a1", "build": "20120309050057", "ReleaseChannel": "default", "email": "bogus@bogus.com", "app_notes": "...", "os_name": "Linux", "last_crash": null, "date_processed": "2012-04-08 10:56:41.558922", "cpu_name": "arm", "reason": "SIGSEGV", "address": "0x1c", "url": "http://embarrassing.porn.com", "signature": "libxul.so@0x117441c", "addons": [["{1a5dabbd-0e74-41da-b532-a364bb552cab}", "1.0.4.1"]]}', 'date_processed': '2012-04-08 10:56:41.558922'}
),),
- (('release savepoint MainThread', None),),
)
actual_execute_args = m.cursor().execute.call_args_list
for expected, actual in zip(expected_execute_args,
actual_execute_args):
expected_sql, expected_params = expected[0]
- expected_sql = expected_sql.replace('\n', '')
- expected_sql = expected_sql.replace(' ', '')
+ expected_sql = remove_whitespace(expected_sql)
actual_sql, actual_params = actual[0]
- actual_sql = actual_sql.replace('\n', '')
- actual_sql = actual_sql.replace(' ', '')
+ actual_sql = remove_whitespace(actual_sql)
self.assertEqual(expected_sql, actual_sql)
self.assertEqual(expected_params, actual_params)
Something went wrong with that request. Please try again.