posts.post_text blank because facebook crawler only considers post->message #1480

leonatkinson opened this Issue Feb 2, 2013 · 1 comment


None yet
1 participant

It looks like issue #877 updated logic so that if a FB post had no "message" property, it used "name" instead. Looking at what's in the code now, though, that change has reverted. I'm looking at line 290 in class.FacebookCrawler.php:

$post_to_process = array(
  "in_reply_to_user_id"=> isset($p->to->data[0]->id) ? $p->to->data[0]->id : '', // assume only one recipient

I have many posts created by that seem to match the pattern: no message property but name and description properties. Here's what the JSON for the post looks like:

  "id": "658936572_10151226791686573", 
  "from": {
    "name": "Leon Atkinson", 
    "id": "658936572"
  "link": "", 
  "name": "Microsoft’s modern.IE", 
  "caption": "", 
  "description": "MS released a site dedicated to testing old versions of MSIE. In addition to a tester you can use right in the site, they offer up virtual machines for VirtualBox that cover all versions of MSIE, including MSIE 10. Home | Testing made easier in Internet Explorer | modern.IE modern.IE is a dev...", 
  "icon": "", 
  "actions": [
      "name": "Comment", 
      "link": ""
      "name": "Like", 
      "link": ""
  "privacy": {
    "description": "Networks, Friends", 
    "value": "CUSTOM", 
    "friends": "ALL_FRIENDS", 
    "networks": "1", 
    "allow": "", 
    "deny": ""
  "type": "link", 
  "status_type": "app_created_story", 
  "application": {
    "name": "", 
    "namespace": "dlvr_it", 
    "id": "232775914688"
  "created_time": "2013-02-02T16:45:09+0000", 
  "updated_time": "2013-02-02T16:45:09+0000", 
  "comments": {
    "count": 0

These posts with no text represent about 39% of my FB posts. Having name or description in there for the post_text will make the reports more readable. Thanks!

Perhaps the code should change to use



private static function getBestPostText($post) {
    $post_text = '';
    if(isset($post->message)) {
        $post_text = $post->message;
    if(!$post_text && isset($post->name)) {
        $post_text = $post->name;
    if(!$post_text && isset($post->description)) {
        $post_text = $post->description;
    if(!$post_text && isset($post->story)) {
        $post_text = $post->story;
    return $post_text;
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment