Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Question : What is the use of self._curReq.fetchCache #344

Closed
goelvivek opened this Issue · 1 comment

2 participants

@goelvivek

Hi,
you have variable called as self._curReq.fetchCache in lib/Connection.js. . From variable name I am assuming that you are catching response of fetch command.

In my one test program I am fetching From header from gmail server for 5,00,000 mails. Which is causing program to take 1GB ram.

Will it cause any problem If I delete from from that cache after readint attributes ?

 message.once('attributes', function (attribute) { 
        delete imap._curReq.fetchCache[seqno];

Will it cause any issue ? If yes,

What all :) ? or are you using that variable for some other purpose also ?

@mscdex
Owner

The cache should be garbage collected after the fetch() has completely finished.

The reason for the cache has to do with how IMAP is designed. Ideally there is only one untagged fetch server response for every message fetched. For example:

C: A0 FETCH 5 (UID FLAGS BODY[HEADER.FIELDS (To)])
S: * 5 FETCH (UID 511 FLAGS (\Seen) BODY[HEADER.FIELDS (To)] {24}
To: user@example.com
)
S: A0 OK Completed

However because of IMAP's design, servers are allowed to split up the requested message properties across multiple untagged fetch server responses (and these multiple responses do not have to be consecutive either, they can be mixed with responses for other emails). For example (worst case):

C: A0 FETCH 5 (UID FLAGS BODY[HEADER.FIELDS (To)])
S: * 5 FETCH (UID 511)
S: * 5 FETCH (FLAGS (\Seen))
S: * 5 FETCH (BODY[HEADER.FIELDS (To)] {24}
To: user@example.com
)
S: A0 OK Completed

So a change I made awhile back was to keep track of what message properties we've yet to see for a particular message so that we know when to emit 'end' on the message object.

EDIT: to answer your question, you CAN safely delete the particular message's cache (fetchCache[seqno]) when 'attributes' is emitted, but at that point you're only deleting an empty array. However using delete may cause noticeable performance degradation because it sends v8 into the slow mode for fetchCache. The other alternative would be to fetch smaller batches of messages instead of 5,000,000 in one go.

@mscdex mscdex closed this
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.