tootctl media/account refresh Error processing 53521: Is a directory @ rb_sysopen
#14817
Comments
|
Could you run this again after editing rescue => e
progress.log pastel.red("Error processing #{item.id}: #{e}")This should raise an error with more details so we can more easily figure out what is happening. |
|
Removing those two lines resulted in nothing be printed (aside from progress bar) completing normally. It seems to silently fail. I do not understand how it doesnt throw but am not very familiar with Ruby |
|
hmpf… not what I expected… could you restore those lines and replace |
|
Ok it printed pretty gnarly as one huge line per, so here is one with new lines added after the comma, so indentation is probably not right: |
|
Some more background that might be useful: I did do I just ran media remove-orphans for the first time in a while and it appears to be removing every single piece of media in accounts/avatars/, headers, etc as orphans 😮 I did backup before running it, so I am going to let it finish on the instance and see if it indeed nuked everything. e: disregard above about the orphans, it was ripping through the remote media cache which was originally unclear, it appears to work normally. I will continue to investigate the original issue |
|
I was able to reproduce some kind of error trying accounts refresh on a single account: I went and looked at that so this is an example of an original dir becoming stuck I think. quite strange. something running during the command is stripping the execute bit on the directory. to reiterate steps:
while the fact the dir is missing in the first place is the first problem, hopefully this execute bit helps |
|
Hmm this is weird indeed. It seems like Paperclip is for some reason treating the What happens if you type, in a Rails console ( Account.find_by(username: 'neikocat', domain: 'blob.cat').header_file_name
Account.find_by(username: 'neikocat', domain: 'blob.cat').header.path |
|
Moving from S3 worked fine for a few years. Forgot to mention, front end is trying to serve the dir as complete URL, eg Something seems wrong w my rails console, i cant find that account or my own account but it seems to be loading production correctly: |
|
here it is in db: empty file name! made me realize name is case sensitive, here's your original request: |
|
ok, the issue is definitely that the file name is empty. That should never happen and I have no idea how it happened. I guess you could do something like Account.where(header_file_name: '').in_batches.update_all(header_file_name: nil)
Account.where(avatar_file_name: '').in_batches.update_all(avatar_file_name: nil)
MediaAttachment.where(file_file_name: '').in_batches.update_all(file_file_name: nil)Then things like |
|
Gave that a whirl, but the commands still blow up: Tried a few problematic domains, no luck. I checked in console that the values are still nil |
|
This is surprising. I suppose the following produces the same error? Account.find_by(username: 'NeikoCat', domain: 'blob.cat').download_header!What do the following commands result in? a = Account.find_by(username: 'NeikoCat', domain: 'blob.cat')
a.header..original_filename
a.header_remote_urlAlso, if the first thing errors out, could you edit def filename_from_content_disposition
disposition = @target.response.headers['content-disposition']
res = disposition&.match(/filename="([^"]*)"/)&.captures&.first
Rails.logger.warn "DEBUG: from content-disposition: #{res.inspect}"
Rails.logger.warn "DEBUG: content-disposition was: #{disposition}"
res
end
def filename_from_path
res = @target.response.uri.path.split('/').last
Rails.logger.warn "DEBUG: from path: #{res.inspect}"
Rails.logger.warn "DEBUG: path was: #{@target.response.uri.path}"
res
endand run |
|
First download header: properties: after editing I ran again but no change: frontend is proviting missing.png as header link, not sure when that changed. I went to the headers dir and deleted the |
|
Hm this sounds even weirder… could you retry with a Rails console launched this way? |
|
Sure, haha.. what the: |
|
It's funny because it is coming from the server like that .. snip.. |
|
Alright! Seems like a combination of a pleroma bug and a Mastodon bug to me. Investigating it. |
ClearlyClaire
added a commit
to ClearlyClaire/mastodon
that referenced
this issue
Sep 27, 2020
|
Could you try the |
Gargron
pushed a commit
that referenced
this issue
Sep 27, 2020
|
Great, thanks! after cherry pick, account refresh worked some more background info: while this particular instance is pleroma, its using external cdn, i believe MinIO. there was about 3k objects across a handful of other servers having same issue and at first glance it was all external cdns but I didnt have an easy way to see what kind |
thenameisnigel-old
pushed a commit
to ChatterlyOSE/Chatterly
that referenced
this issue
Sep 28, 2020
ClearlyClaire
added a commit
to ClearlyClaire/mastodon
that referenced
this issue
Dec 18, 2020
Gargron
added a commit
that referenced
this issue
Dec 18, 2020
* Fix 2FA/sign-in token sessions being valid after password change (#14802) If someone tries logging in to an account and is prompted for a 2FA code or sign-in token, even if the account's password or e-mail is updated in the meantime, the session will show the prompt and allow the login process to complete with a valid 2FA code or sign-in token * Fix Move handler not being triggered when failing to fetch target (#15107) When failing to fetch the target account, the ProcessingWorker fails as expected, but since it hasn't cleared the `move_in_progress` flag, the next attempt at processing skips the `Move` activity altogether. This commit changes it to clear the flag when encountering any unexpected error on fetching the target account. This is likely to occur because, of, e.g., a timeout, when many instances query the same actor at the same time. * Fix slow distinct queries where grouped queries are faster (#15287) About 2x speed-up on inboxes query * Fix possible inconsistencies in tag search (#14906) Do not downcase the queried tag before passing it to postgres when searching: - tags are not downcased on creation - `arel_table[:name].lower.matches(pattern)` generates an ILIKE anyway - if Postgres and Rails happen to use different case-folding rules, downcasing before query but not before insertion may mean that some tags with some casings are not searchable * Fix updating account counters when account_stat is not yet created (#15108) * Fix account processing failing because of large collections (#15027) Fixes #15025 * Fix downloading remote media files when server returns empty filename (#14867) Fixes #14817 * Fix webfinger redirect handling in ResolveAccountService (#15187) * Fix webfinger redirect handling in ResolveAccountService ResolveAccountService#process_webfinger! handled a one-step webfinger redirection, but only accepting the result if it matched the exact URI passed as input, defeating the point of a redirection check. Instead, use the same logic as in `ActivityPub::FetchRemoteAccountService`, updating the resulting `acct:` URI with the result of the first webfinger query. * Add tests * Remove dependency on unused and unmaintained http_parser.rb gem (#14574) It seems that years ago, the “http” gem dependend on the “http_parser.rb” gem (it now depends on the “http-parser” gem), and, still years ago, we pulled it from git in order to benefit from a bugfix that wasn't released yet (#7467). * Add tootctl maintenance fix-duplicates (#14860, #15201, #15264, #15349, #15359) * Fix old migration script not being able to run if it fails midway (#15361) * Fix old migration script not being able to run if it fails midway Improve the robustness of a migration script likely to fail because of database corruption so it can run again once database corruptions are fixed. * Display a specific error message in case of index corruption Co-authored-by: Eugen Rochko <eugen@zeonfederated.com> Co-authored-by: Claire <claire.github-309c@sitedethib.com> Co-authored-by: Eugen Rochko <eugen@zeonfederated.com> Co-authored-by: Claire <claire.github-309c@sitedethib.com>
shouo1987
pushed a commit
to CrossGate-Pawoo/mastodon
that referenced
this issue
Dec 7, 2022
* Fix 2FA/sign-in token sessions being valid after password change (mastodon#14802) If someone tries logging in to an account and is prompted for a 2FA code or sign-in token, even if the account's password or e-mail is updated in the meantime, the session will show the prompt and allow the login process to complete with a valid 2FA code or sign-in token * Fix Move handler not being triggered when failing to fetch target (mastodon#15107) When failing to fetch the target account, the ProcessingWorker fails as expected, but since it hasn't cleared the `move_in_progress` flag, the next attempt at processing skips the `Move` activity altogether. This commit changes it to clear the flag when encountering any unexpected error on fetching the target account. This is likely to occur because, of, e.g., a timeout, when many instances query the same actor at the same time. * Fix slow distinct queries where grouped queries are faster (mastodon#15287) About 2x speed-up on inboxes query * Fix possible inconsistencies in tag search (mastodon#14906) Do not downcase the queried tag before passing it to postgres when searching: - tags are not downcased on creation - `arel_table[:name].lower.matches(pattern)` generates an ILIKE anyway - if Postgres and Rails happen to use different case-folding rules, downcasing before query but not before insertion may mean that some tags with some casings are not searchable * Fix updating account counters when account_stat is not yet created (mastodon#15108) * Fix account processing failing because of large collections (mastodon#15027) Fixes mastodon#15025 * Fix downloading remote media files when server returns empty filename (mastodon#14867) Fixes mastodon#14817 * Fix webfinger redirect handling in ResolveAccountService (mastodon#15187) * Fix webfinger redirect handling in ResolveAccountService ResolveAccountService#process_webfinger! handled a one-step webfinger redirection, but only accepting the result if it matched the exact URI passed as input, defeating the point of a redirection check. Instead, use the same logic as in `ActivityPub::FetchRemoteAccountService`, updating the resulting `acct:` URI with the result of the first webfinger query. * Add tests * Remove dependency on unused and unmaintained http_parser.rb gem (mastodon#14574) It seems that years ago, the “http” gem dependend on the “http_parser.rb” gem (it now depends on the “http-parser” gem), and, still years ago, we pulled it from git in order to benefit from a bugfix that wasn't released yet (mastodon#7467). * Add tootctl maintenance fix-duplicates (mastodon#14860, mastodon#15201, mastodon#15264, mastodon#15349, mastodon#15359) * Fix old migration script not being able to run if it fails midway (mastodon#15361) * Fix old migration script not being able to run if it fails midway Improve the robustness of a migration script likely to fail because of database corruption so it can run again once database corruptions are fixed. * Display a specific error message in case of index corruption Co-authored-by: Eugen Rochko <eugen@zeonfederated.com> Co-authored-by: Claire <claire.github-309c@sitedethib.com> Co-authored-by: Eugen Rochko <eugen@zeonfederated.com> Co-authored-by: Claire <claire.github-309c@sitedethib.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
this is on v3.2.0 release, i havent touched anything on my instance in a while but a strange problem started cropping up.
noticed 404ing media links which previously worked (local media server), so tried running:
every single missing media (attachments, avatars, headers) prints this error. the directory exists and the file has a different file name than whats being served from the frontend. I've checked filehandle limits, and inode usage and they're all < 1% utilized. Any tips would be appreciated.
The text was updated successfully, but these errors were encountered: