Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix deduplicated filename not being persisted #2679

Merged
merged 1 commit into from
Aug 1, 2023

Conversation

mshibuya
Copy link
Member

I'm sorry that the deduplication feature introduced in the 3.0 release is completely broken, as reported by #2677. This PR is another attempt to fix it.

The approach is to move the deduplication logic from #store! to #write_identifier, which is invoked on before_save thus allowing the modified filename(identifier) to be persisted. To work around the limitation that the deduplication timing can't be made earlier because #store_dir relies on the model's id, the scope of deduplication was narrowed down only to filename, instead of whole #store_path. This sometimes can lead to unnecessary deduplication in a case like a user decided to use a custom #store_dir to handle deduplication by their own, but even if that happens it does not harm data consistency, so I think it's acceptable.

@krasnoukhov I want your opinion on this, what do you think?

Copy link

@krasnoukhov krasnoukhov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. I ran our app CI with this commit and there are no failures (previously there was a bunch because of missing file error)

@mshibuya mshibuya merged commit d6b522f into master Aug 1, 2023
26 checks passed
@mshibuya mshibuya deleted the fix-deduplicated-filename-not-persisted branch August 1, 2023 01:28
@Aiderlei
Copy link

Aiderlei commented Aug 1, 2023

Thanks for fixing this! When can I except this to be released?

@mshibuya
Copy link
Member Author

mshibuya commented Aug 1, 2023

Just released 3.0.2, which includes this fix.

@Aiderlei
Copy link

Aiderlei commented Aug 1, 2023

@mshibuya Thanks sir! That was fast 🚀

@mrudult
Copy link

mrudult commented Aug 2, 2023

@mshibuya Shouldn't this deduplicated filename feature be optional? We use S3 as our storage and S3 automatically replaces the file if the filename is already present in the directory. So #1855 was never really an issue on S3.

@ps231402
Copy link

ps231402 commented Aug 3, 2023

I still have the issue with 3.0.2.

@mshibuya
Copy link
Member Author

mshibuya commented Aug 4, 2023

@mrudult I'm afraid that you may be missing something.

automatically replaces the file if the filename is already present in the directory

This is the problem itself. As described in the issue, if a user uploads two different files with the same filename results in one overwrites the other, ending up with only 1 file persisted.
It is also problematic for a singular uploader (#mount_uploader, without 's'). If a user updates a record but the transaction rolls back, the existing file still be overwritten because the file is stored on #after_save (see #2546). That's why we need the deduplication feature to be in place.

If you don't like the default deduplicated filenames, you can choose to override #deduplicated_filename with your own way like

class YourUploader < CarrierWave::Uploader::Base
  ...

  def deduplicated_filename
    ...
  end
end

@mshibuya
Copy link
Member Author

mshibuya commented Aug 4, 2023

@ps231402 Just saying so isn't very helpful. Please show me what you did and what you got, like a Rails console output.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants