Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix import of json file #421

Merged
merged 4 commits into from
Mar 6, 2023
Merged

Conversation

Flo4604
Copy link
Contributor

@Flo4604 Flo4604 commented Oct 9, 2022

This PR is for being able to use the import JSON button again, as its a feature that many ppl including myself would like to have back

@Flo4604
Copy link
Contributor Author

Flo4604 commented Oct 9, 2022

This feature is usually used when running the script for a longer period of time or overnight, and it would ask for confirmation about deleting every chat once it goes to the next one.

I would add a checkbox that would say auto-confirm, so you can run it overnight. Would that fit into this PR as well or should I create a new one for that?

EDIT: Same question with using GM_setValue and GM_getValue to store progress of the import file when reloading browser // shutting down pc

@TCNOco
Copy link

TCNOco commented Nov 18, 2022

This is what I was waiting and looking for! Happy Google led me to this. Will test now.

I had to compile this myself to test it, and in doing so, ran into some ESLint errors, which I've fixed in a small PR to your PR.
Very minor, but judging that this has no other errors, it's probably worth fixing.
Flo4604#1

@Flo4604
Copy link
Contributor Author

Flo4604 commented Nov 18, 2022

I will have a look regarding the eslint errors, also if you are interested give me feedback regarding my comment above

@TCNOco
Copy link

TCNOco commented Nov 18, 2022

I would... I wanted to use this to delete older messages, but it seems a lot of servers I'm not on, so things 403.
I also couldn't figure it (importing channels using the discord data package json) all out, but I see the Server ID is set to @me - this allows me to delete any channel ID without inputting a guild ID - which I never knew. So at least, I can manually start deletions of messages in channels.

Regarding servers I'm not in anymore, I saw someone comment in discussions that they contacted discord with a GDPR request to delete messages from specific channel IDs. Will likely have to resort to that.

@Flo4604
Copy link
Contributor Author

Flo4604 commented Nov 18, 2022

Alright, I will add a commit with the fixed eslint errors and support for auto confirm / keeping track of your already deleted messages in a few days

@Flo4604
Copy link
Contributor Author

Flo4604 commented Dec 3, 2022

@TcNobo Hey, I added the autoconfirm button + it now stores the progress and you are able to load it with a load last import button.

Please let me know if you think of anything that should be added

@victornpb
Copy link
Owner

@Flo4604 Thank you for submitting this PR. I haven't tested it but it looks fine.

I will try to merge this to main, but before I would like to refactor a few things. The funcional approach I had at the beginning was simple but it is not scaling with all the features being added.

I want to keep logic like this outside of the index.js where is mainly UI stuff.
Also I want to avoid using GM methods so undiscord still works even without the browser extension. I wanted to support BetterDiscord but I hit a roadblock there, but I didn't want to give up just yet.

@Flo4604
Copy link
Contributor Author

Flo4604 commented Feb 19, 2023

@victornpb all right, should I make a commit to remove the GM Methods?

I might investigate better discord as well and find a way to store progress that works in the browser as well

@victornpb
Copy link
Owner

victornpb commented Feb 25, 2023

@victornpb all right, should I make a commit to remove the GM Methods?

I refactored the entire codebase since last week, so it will be really hard to merge this PR. I could incorporate your changes myself. But if you would like to have your name as a contributor, I need you to open a new PR from the next branch back to next.

Let me know what you prefer, it should be more straightforward now, I implemented a runBatch that you can pass an Array (currently it is just an array of strings channelIds, but It can be changed to an array of objects containing channel sever etc).

About the GM method I think we should not have the restore button, and have the person to input the file every time, less convenient but it makes things simpler.

@victornpb
Copy link
Owner

I made it easier to test it locally, you install the extension from the dev server link
image

After installing the DEV extension you can click the "update" icon after making changes, and then refresh the discord tab.
image

It doesn't reload automatically, but at least no more copy pasting just to test stuff.

@Flo4604
Copy link
Contributor Author

Flo4604 commented Feb 26, 2023

I need you to open a new PR from the next branch back to next.

Let me know what you prefer, it should be more straightforward now, I implemented a runBatch that you can pass an Array (currently it is just an array of strings channelIds, but It can be changed to an array of objects containing channel sever etc).

I'll re-implement it at the new branch, should we just close this MR then?

About the GM method I think we should not have the restore button, and have the person to input the file every time, less convenient but it makes things simpler.

I understand the thought behind that, it is just more convenient since we don't have to scan DM channels which we already went through.

We could add this when undiscord is better discord compatible as well, I am sure there will be a way to store the needed data on both better discord and the browser using the same method.

I'll leave it out of the new pr

@victornpb
Copy link
Owner

Tbh I don't fully understand whats is inside the discord archive, I can assume looking at the code that it only works with DMs right? So the reason I thought the store was unnecessary is because I assumed we could wipe everything in one go, so no need to store anything.
I guess your reasoning behind it was if you had to stop it, you could resume where you left off to save some requests? is that it?

If so I don't think leaving it out would be much slower, because it wouldn't take too long to search a bunch of empty channels (dms) very quickly, and eventually catch where it was before.

Lets leave this open for now until we have a new PR.

I will think about a good way to have persistence in another moment, but for now I just want to have this tested and merged.

@Flo4604
Copy link
Contributor Author

Flo4604 commented Feb 26, 2023

I guess your reasoning behind it was if you had to stop it, you could resume where you left off to save some requests? is that it?

Yeah, that was my point

I can assume looking at the code that it only works with DMs right?

No, it works on all channels, the file you import from the discord package ( messages/index.json ) contains all channel's you sent messages in, either DM's or Server Channels / Threads.
But you can still use @me as the server id nonetheless

@victornpb
Copy link
Owner

victornpb commented Feb 26, 2023

But you can still use @me as the server id nonetheless

interesting, do you have the guildId as well? because I think I rather do that if I could, because I kept the requests as close as possible as the discord client, in an attempt to make it less detectable.

I have requested my data so I can test it, but I'm still waiting, they say it an take a couple days...
Do you have an example you can share? you can strip the data, I just want to see the schema/format of the file.

I kinda prepared the code to receive that feature, it think it is a nice improvement over the old codebase it was getting kinda hard to follow

https://github.com/victornpb/undiscord/blob/next/src/undiscord-ui.js#L284-L310

@Flo4604
Copy link
Contributor Author

Flo4604 commented Mar 1, 2023

Sorry for the late response

The issues is that you don't get the guildId in the same file the messages/index.json just contains an object with key values pairs the key being the channel id and the value the name

{
    "978290176805716019": "ticket-rona2470",
    "719600955141652493": null,
    "956217470832738314": "ticket-Charrito1196",
    "977372737842597920": "ticket-prkr6969",
    "991862570908667966": "ticket-lynx0108",
    "1000140341430714398": "ticket-________7814",
}

There is a secondary index.json servers/index,json containing the guild ids in the same format

{
    "478272966086492160": "Test123",
    "649346497216905216": "(\u25e3_\u25e2)",
    "955246629236121670": "griefing?!"
}

I also don't see the need for the server id as discord themselves don't use it to delete a message
image

Message loading is also done via channel id only
image

@victornpb victornpb merged commit ec3634d into victornpb:master Mar 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants