Skip to content
This repository

Base64 encode binary data before logging #177

Merged
merged 3 commits into from about 1 year ago

2 participants

Brian Feaver Jeremy Mikola
Brian Feaver

Both the logger and the data collector for the profiler would output raw
binary data. The logger would fail due to UTF-8 warnings when doing
json_encode and the profiler queries would fail in Twig when displaying
them in mongodb.html.twig.

Brian Feaver Base64 encode MongoBinData objects before logging.
Both the logger and the data collector for the profiler would output raw
binary data. The logger would fail due to UTF-8 warnings when doing
json_encode and the profiler queries would fail in Twig when displaying
them in mongodb.html.twig.
8e69509
Tests/DataCollector/PrettyDataCollectorTest.php
@@ -37,15 +37,20 @@ public function getQueries()
37 37
     {
38 38
         return array(
39 39
             // batchInsert
40  
-            array(
  40
+            'batch insert' => array(
3
Jeremy Mikola Owner
jmikola added a note March 15, 2013

Can you remove these array keys? This isn't used by PHPUnit and is redundant given the inline comments above.

Brian Feaver
bfeaver added a note March 15, 2013

The array keys are used by PHPUnit to name the data set being used. In debug output or when a test fails, instead of having "data set #1" failed, it'll put out "data set "batch insert" failed".

However, if you don't want named datasets, I can remove the keys.

Jeremy Mikola Owner
jmikola added a note March 15, 2013

Oh, I had no idea. That sounds great, then :+1:. If you like, we can drop the comments instead.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
DataCollector/PrettyDataCollector.php
@@ -216,7 +216,7 @@ private function bsonEncode($query, $array = true)
216 216
             } elseif ($value instanceof \MongoMaxKey) {
217 217
                 $formatted = 'new MaxKey()';
218 218
             } elseif ($value instanceof \MongoBinData) {
219  
-                $formatted = 'new BinData("'.$value->bin.'", "'.$value->type.'")';
  219
+                $formatted = 'new BinData("'.base64_encode($value->bin).'", "'.$value->type.'")';
3
Jeremy Mikola Owner
jmikola added a note March 15, 2013

For printing BinData, we should be consistent with the Mongo shell here. Could you wrap the type and value argument positions? We can continue printing the type as a decimal number, despite the docs below saying it prints it in hex (I believe they need to be updated).

See: http://docs.mongodb.org/manual/reference/mongodb-extended-json/#data_binary

Brian Feaver
bfeaver added a note March 15, 2013

Started this and realized I was a little confused. What did you want to wrap it with? It'll already currently print the decimal number, but if we want to output the hex value we could wrap it in dechex().

Jeremy Mikola Owner
jmikola added a note March 15, 2013

Sorry, I meant to say "swap", not "wrap". We can leave the type printed as-is. Moving it to the first argument would make this consistent with the Mongo shell.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Jeremy Mikola jmikola commented on the diff March 15, 2013
Logger/Logger.php
@@ -48,6 +48,12 @@ public function logQuery(array $query)
48 48
             $query['data'] = '**'.$query['num'].' item(s)**';
49 49
         }
50 50
 
  51
+        array_walk_recursive($query, function(&$value, $key) {
  52
+            if ($value instanceof \MongoBinData) {
  53
+                $value = base64_encode($value->bin);
4
Jeremy Mikola Owner
jmikola added a note March 15, 2013

Shouldn't this also be logged using BinData format above? Otherwise, the base64 string will be indistinguishable from a regular string.

Brian Feaver
bfeaver added a note March 15, 2013

I don't disagree. I was trying to remain consistent as no other types are currently treated special in this logger. I'll change it to use the same format as above.

Brian Feaver
bfeaver added a note March 15, 2013

While changing this, I realized the output will end up looking a little strange since it will be wrapped in quotes since it's just being json_encoded.

It'll end up looking like {"foo":{"bar":"new BinData(\"anVuayBkYXRh\", \"2\")"}}.

If that's fine, then I'll leave it as is. Otherwise, maybe just a tag like "BINARY: anVuayBkYXRh, TYPE: 2"

Jeremy Mikola Owner
jmikola added a note March 15, 2013

Just out of curiosity, do other objects that get logged simply get casted to strings?

It's been some time since I saw the logger output for this bundle, so your example helped jog my memory.

I suppose the most preferable option would be to serialize the driver classes in the "strict JSON" format defined in: http://docs.mongodb.org/manual/reference/mongodb-extended-json/#data_binary (the mongoexport commands use that notation), but that would be outside the scope of this PR. Let's leave it as-is for now, then.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Jeremy Mikola jmikola merged commit 00568cc into from March 18, 2013
Jeremy Mikola jmikola closed this March 18, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Showing 3 unique commits by 1 author.

Mar 14, 2013
Brian Feaver Base64 encode MongoBinData objects before logging.
Both the logger and the data collector for the profiler would output raw
binary data. The logger would fail due to UTF-8 warnings when doing
json_encode and the profiler queries would fail in Twig when displaying
them in mongodb.html.twig.
8e69509
Mar 15, 2013
Brian Feaver Swapped type and bin data when logging MongoBinData in the profiler. 951b133
Brian Feaver Fix tests using MongoBinData to pass BYTE_ARRAY constant. e3c9482
This page is out of date. Refresh to see the latest.
2  DataCollector/PrettyDataCollector.php
@@ -216,7 +216,7 @@ private function bsonEncode($query, $array = true)
216 216
             } elseif ($value instanceof \MongoMaxKey) {
217 217
                 $formatted = 'new MaxKey()';
218 218
             } elseif ($value instanceof \MongoBinData) {
219  
-                $formatted = 'new BinData("'.$value->bin.'", "'.$value->type.'")';
  219
+                $formatted = 'new BinData('.$value->type.', "'.base64_encode($value->bin).'")';
220 220
             } elseif ($value instanceof \MongoGridFSFile || $value instanceof GridFSFile) {
221 221
                 $formatted = 'new MongoGridFSFile("'.$value->getFilename().'")';
222 222
             } elseif ($value instanceof \stdClass) {
6  Logger/Logger.php
@@ -48,6 +48,12 @@ public function logQuery(array $query)
48 48
             $query['data'] = '**'.$query['num'].' item(s)**';
49 49
         }
50 50
 
  51
+        array_walk_recursive($query, function(&$value, $key) {
  52
+            if ($value instanceof \MongoBinData) {
  53
+                $value = base64_encode($value->bin);
  54
+            }
  55
+        });
  56
+
51 57
         $this->logger->info($this->prefix.json_encode($query));
52 58
     }
53 59
 }
10  Tests/DataCollector/PrettyDataCollectorTest.php
@@ -36,16 +36,18 @@ public function testCollect($query, $formatted)
36 36
     public function getQueries()
37 37
     {
38 38
         return array(
39  
-            // batchInsert
40  
-            array(
  39
+            'batch insert' => array(
41 40
                 array('db' => 'foo', 'collection' => 'bar', 'batchInsert' => true, 'num' => 1, 'data' => array('foo' => 'bar'), 'options' => array()),
42 41
                 array('use foo;', 'db.bar.insert({ "foo": "bar" });'),
43 42
             ),
44  
-            // find
45  
-            array(
  43
+            'find' => array(
46 44
                 array('db' => 'foo', 'collection' => 'bar', 'find' => true, 'query' => array('foo' => null), 'fields' => array()),
47 45
                 array('use foo;', 'db.bar.find({ "foo": null });'),
48 46
             ),
  47
+            'bin data' => array(
  48
+                array('db' => 'foo', 'collection' => 'bar', 'update' => true, 'query' => array('_id' => 'foo'), 'newObj' => array('foo' => new \MongoBinData('junk data', \MongoBinData::BYTE_ARRAY))),
  49
+                arraY('use foo;', 'db.bar.update({ "_id": "foo" }, { "foo": new BinData(2, "' . base64_encode('junk data') . '") });'),
  50
+            )
49 51
         );
50 52
     }
51 53
 }
28  Tests/Logger/LoggerTest.php
@@ -42,4 +42,32 @@ public function testLogQuery()
42 42
         $logger = new Logger($this->logger);
43 43
         $logger->logQuery(array('foo' => 'bar'));
44 44
     }
  45
+
  46
+    public function testMongoBinDataBase64Encoded()
  47
+    {
  48
+        $binData = new \MongoBinData('junk data', \MongoBinData::BYTE_ARRAY);
  49
+        $query = array('foo' => base64_encode($binData->bin));
  50
+        $log = json_encode($query);
  51
+
  52
+        $this->logger->expects($this->once())
  53
+            ->method('info')
  54
+            ->with('MongoDB query: '.$log);
  55
+
  56
+        $logger = new Logger($this->logger);
  57
+        $logger->logQuery(array('foo' => new \MongoBinData('junk data', \MongoBinData::BYTE_ARRAY)));
  58
+    }
  59
+
  60
+    public function testMongoBinDataBase64EncodedRecursively()
  61
+    {
  62
+        $binData = new \MongoBinData('junk data', \MongoBinData::BYTE_ARRAY);
  63
+        $query = array('foo' => array('bar' => base64_encode($binData->bin)));
  64
+        $log = json_encode($query);
  65
+
  66
+        $this->logger->expects($this->once())
  67
+            ->method('info')
  68
+            ->with('MongoDB query: '.$log);
  69
+
  70
+        $logger = new Logger($this->logger);
  71
+        $logger->logQuery(array('foo' => array('bar' => new \MongoBinData('junk data', \MongoBinData::BYTE_ARRAY))));
  72
+    }
45 73
 }
Commit_comment_tip

Tip: You can add notes to lines in a file. Hover to the left of a line to make a note

Something went wrong with that request. Please try again.