Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Attachment Display Failure #339

Open
RichardTaylor opened this issue Jan 11, 2012 · 4 comments

Comments

4 participants
@RichardTaylor
Copy link

commented Jan 11, 2012

An attachment, present in the raw message, isn't displaying at:

http://www.whatdotheyknow.com/request/cost_of_ofsted_report_into_home#incoming-100790

@sebbacon

This comment has been minimized.

Copy link
Contributor

commented Jan 12, 2012

This has come out fine on the staging server. And it's marked as parsed on the live server, but with no corresponding attachments in the foi_attachments table. So there must be some kind of a race condition. Thinking aloud, candidates for problem areas are:

  • incoming_message.mail returning nil. Could in turn be because the corresponding RawEmail had not yet been saved to disk (in this case, it must have been, I think). So unlikely.
  • the incoming message (with its last_parsed timestamp) was saved at the end of the parsing operation, but the corresponding foi_attachments were not, even though they happened earlier . Not sure how this would happen, but it would be safest to make all of that saving happen within a single transaction.
@sebbacon

This comment has been minimized.

Copy link
Contributor

commented Jan 12, 2012

I will make the parsing happen in a single transaction, force a FS sync when the attachment is written to file, and also reparse all the incoming_messages that claim to have been parsed but don't have any attachments.

@sebbacon sebbacon closed this in 9125f4c Jan 12, 2012

@sebbacon

This comment has been minimized.

Copy link
Contributor

commented Jan 12, 2012

Deployed. We need to check to see if this is still happening in a few days or weeks. The following query should return zero rows:

select id from incoming_messages where last_parsed is not null and not exists (select * from foi_attachments where foi_attachments.incoming_message_id = incoming_messages.id) order by id;

@sebbacon sebbacon reopened this Jan 12, 2012

sebbacon added a commit that referenced this issue Jan 13, 2012

To get around occasional race conditions, instead of forcing an fsync…
…, retry getting the body a few times before failing. May be related to issue #333 and #339
@garethrees

This comment has been minimized.

Copy link
Member

commented Nov 5, 2014

This returns 1 row, so still an issue?

foi=> select id from incoming_messages where last_parsed is not null and not exists (select * from foi_attachments where foi_attachments.incoming_message_id = incoming_messages.id) order by id;
   id
--------
 306557
(1 row)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.