Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

inswapper_128.onnx in the wild - please verify the hashsums? #2315

Closed
chrisbward opened this issue May 30, 2023 · 20 comments
Closed

inswapper_128.onnx in the wild - please verify the hashsums? #2315

chrisbward opened this issue May 30, 2023 · 20 comments

Comments

@chrisbward
Copy link

Regardless if you guys have pulled the model for any which reason, it's probably best to verify the hashes below so that people don't download anything malicious?

Below is the version I discovered, matching another copy in the wild.

sha256sum inswapper_128.onnx            
e4a3f08c753cb72d04e10aa0f7dbe3deebbf39567d4ead6dce08e98aa49e16af  inswapper_128.onnx
md5sum ./inswapper_128.onnx
a3a155b90354160350efd66fed6b3d80  ./inswapper_128.onnx
@felixsanz
Copy link

so which one is the correct?

@chrisbward
Copy link
Author

@felixsanz it's a single file, but I printed out both the SHA256 and MD5 hashes so that the deepinsight devs could verify

@nttstar
Copy link
Collaborator

nttstar commented May 31, 2023

I will not officially support this model due to the substantial risks associated with video deepfakes.
See https://github.com/s0md3v/roop/issues/92

@felixsanz
Copy link

felixsanz commented May 31, 2023

I will not officially support this model due to the substantial risks associated with video deepfakes. See s0md3v/roop#92

Well, the risk of deepfakes is still there (model is public), you are just supporting the risk of malware and ransomware not saying what the correct hash it.

Also there is more deepfakes generation software like https://github.com/neuralchen/SimSwap, so deepfakes will exist anyway

And remember this is a tool. A knife can also kill a lot of people. It's not the tool but how you use it. There is nothing wrong or harmful on this

@felixsanz
Copy link

@felixsanz it's a single file, but I printed out both the SHA256 and MD5 hashes so that the deepinsight devs could verify

Yeah, i mean which correct hashes have the official model, either sha256 or md5sum

@mike-hearn
Copy link
Contributor

I will not officially support this model due to the substantial risks associated with video deepfakes. See s0md3v/roop#92

In the past few weeks you've said you won't release or support this model publicly due to a) the imminent release of a paper, b) because your Discord bot offers superior quality to the 128x128 model, and now c) the risk of video deepfakes. I'll admit I am somewhat skeptical of all of these excuses.

Anyway, I am commenting to confirm that the checksums in the original post by @chrisbward match the version of the model I previously downloaded from the official links before they were removed.

@nttstar
Copy link
Collaborator

nttstar commented May 31, 2023

I have too many versions of the model, making it impossible to confirm. There are over ten models at a resolution of 128 alone, and I simply chose one at random at the time.
Open sourcing for academic purposes is acceptable, but I will not release such high-precision models due to the significant risks associated with video deepfakes.

@felixsanz
Copy link

I have too many versions of the model, making it impossible to confirm. There are over ten models at a resolution of 128 alone, and I simply chose one at random at the time. Open sourcing for academic purposes is acceptable, but I will not release such high-precision models due to the significant risks associated with video deepfakes.

but your discord bot generate them, right?

@nttstar
Copy link
Collaborator

nttstar commented May 31, 2023

discord bot only generates images

@felixsanz
Copy link

felixsanz commented May 31, 2023

discord bot only generates images

Btw it's not impossible, it's just:

for file in ./*.onnx; do sha256sum "$file"; echo; done | grep e4a3f08c753cb72d04e10aa0f7dbe3deebbf39567d4ead6dce08e98aa49e16af

To confirm this model on the internet is malicious or not. People can use it anyway but you can avoid a lot of bad things

@kokozaurz
Copy link

I have too many versions of the model, making it impossible to confirm. There are over ten models at a resolution of 128 alone, and I simply chose one at random at the time. Open sourcing for academic purposes is acceptable, but I will not release such high-precision models due to the significant risks associated with video deepfakes.

could you perhaps provide instructions on how to train it from scratch then with some sample data to start?

@kormalev
Copy link

@nttstar
Could you please update us on the paper preparation?

@chrisbward
Copy link
Author

Hi @nttstar - it's too late, the model is out there and if you do not publish the hashes then the risk of malware being distributed is far greater!

We're not asking you to publish the model, just the hashes.

Thanks

@ivfloyd
Copy link

ivfloyd commented May 31, 2023

please @nttstar

@FurkanGozukara
Copy link

FurkanGozukara commented May 31, 2023

I will not officially support this model due to the substantial risks associated with video deepfakes. See s0md3v/roop#92

paid services are providing 100% quality sound cloning and face cloning

i think best way to combat is educating everyone

and i am 100% sure malicious actors have already much better models at their hands

by the way from this repo i feel like the aim was mass surveillance to help chinese government :d

@Inferencer
Copy link

"discord bot only generates images" hosently that's not a good excuse, people will create fake discord accounts to bybass the 50 images per-day per-account rules then join frames with ffmpeg & interpolate the frames with another free tool, they could also bypass any nsfw filters by masking the target video then overlaying results with the original footage.

So if you truley belive roop 'an easily detectable swapping tool' is bad then you might as well delete your discord for the saftey concerns you keep mentioning due to the supiror deepfake qaulity you provide free of charge to anybody with a computer, or at the very least limit the amount of image swaps per day to 6.

@nttstar
Copy link
Collaborator

nttstar commented Jun 1, 2023

I will no longer answer questions about this model, but this issue will not be closed for the time being. You are free to discuss here.

@chrisbward
Copy link
Author

WARNING - DO NOT USE THIS VERSION OF ROOP

I have jailed the binary inside a Docker container and will be forking/porting to Golang.

@FurkanGozukara
Copy link

WARNING - DO NOT USE THIS VERSION OF ROOP

I have jailed the binary inside a Docker container and will be forking/porting to Golang.

what does that mean?

@clearly-outsane
Copy link

But there's already more popular deepfake repos so i don't see the point

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

10 participants