getThreadHistory parameters adjustments

[kieferlam]

Addressing #262

Since thread_info.php pretty much ignores the offset data, I thought it would be better to remove that option from the api as it may cause some confusion (as it did to me).

So I changed getThreadHistory to only require amount and timestamp to get the thread history. I also provided an example of how to load batches of messages in order (like scrolling up on a messenger client) without the need of offset functionality. However this means you cannot load messages in history without loading the messages before (more recent) it. Although you couldn't do this before anyhow.

[bsansouci]

Awesome! Thanks a lot for fixing this.
The example is a good choice.

[ravkr]

What? When it started ignoring end? I was able to pull all messages from thread using only start and end about a month ago :/

[kieferlam]

@ravkr Can you try it again with the old getThreadHistory? It's still ignoring offset for me.

api.getThreadHistory(threadID, 5, 10, undefined, (err, history)=>{
	history.forEach((msg, index)=>{
		console.log(msg.body);
	});
	app.quit();
});

The code above basically ignores the 5 and gets me the 5 most recent messages instead of the 5 before that.

[gamelaster]

If I remember good, the getThreadPictures have same behaviour

[ravkr]

yeah... offset seems not to work anymore :(

[bilyanhadzhi]

Unfortunately, this doesn't seem to work on my side.

timestamp is treated as the timestamp of the oldest message so it just loads all messages after it.
So I'm just stuck loading the same messages even though I'm updating timestamp.

[ravkr]

@bilyanhadzhi can you show us your code?

[bilyanhadzhi]

@ravkr Sure.

I'm trying to export all messages from a huge group chat (>32,000 messages) to a JSON file.

const login = require('facebook-chat-api');
const jsonfile = require('jsonfile');

const credentials = {
  email: process.env.FB_EMAIL,
  password: process.env.FB_PASSWORD,
};

const file = '/home/billy/Coding/projects/z-class-chat-analysis/src/messages.json';
const threadId = 814129402005312;

let timestamp = undefined;
let allMessages = [];

const writeNextHistory = api => {
  api.getThreadHistory(threadId, 3, timestamp, (err, history) => {
    if (err) {
      console.error(err);
    }

    if (timestamp !== undefined) {
      history.pop();
    }

    console.log('timestamp', timestamp);
    history.forEach(message => console.log(`${message.body}: timestamp is ${message.timestamp}`));
    console.log();

    allMessages = history.concat(allMessages);
    timestamp = history[0].timestamp;
  });
};

login(credentials, (err, api) => {
  if (err) {
    console.error(err);
  }

  writeNextHistory(api);
  writeNextHistory(api);
});

The result is a little something like this:

timestamp undefined
Thanks: timestamp is 1493019246820
You're welcome! Cya later: timestamp is 1493019392190
Cya: timestamp is 1493019397398

timestamp 1493019246820
Thanks: timestamp is 1493019246820
You're welcome! Cya later: timestamp is 1493019392190

As you can see, it's using the message with timestamp of timestamp as the oldest, not the most recent message.

[kieferlam]

@bilyanhadzhi The API works asynchronously. So the second writeNextHistory(api) is called before the first api call has even started (probably, but at least before the timestamp update). You will need to implement a callback in your writeNextHistory function to execute the callback after timestamp = history[0].timestamp.

In effect, you will have something like this:

login(credentials, (err, api) => {
  if (err) {
    console.error(err);
  }

  writeNextHistory(api, () => writeNextHistory(api));
});

[ravkr]

you are calling writeNextHistory twice, without waiting for result
if you try this:

// ...
const writeNextHistory = api => {
    console.log('timestamp', timestamp);
    api.getThreadHistory(threadId, 3, timestamp, (err, history) => {
// ...

you will see that both calls are made when timestamp is undefined

your second call to writeNextHistory should be made after receiving response from the first call

[bilyanhadzhi]

Ah, I see now.

So, I guess if I wanted to call this function again and again till I've loaded some arbitrary number of messages or reached the beginning of history, I guess I'd write some sort of recursive callback?

Thank you very much again, guys, I'll ask again if need be.