Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Please restore discussion on the Security threads #367

Closed
SRGOM opened this issue Apr 12, 2019 · 41 comments

Comments

Projects
None yet
@SRGOM
Copy link

commented Apr 12, 2019

It will be a good place for discussion of security. Locking down topics seems to indicate a lack of transparency and doesn't inspire confidence

@Sharparam

This comment has been minimized.

Copy link

commented Apr 12, 2019

What irks me is that several comments that were not offtopic were deleted for supposedly being off-topic. There was a lot of spam and offtopic but whoever did deletion seems to have gone overboard.

@rain-1

This comment has been minimized.

Copy link

commented Apr 12, 2019

agreed, bad form by matrix.

@ara4n

This comment has been minimized.

Copy link
Member

commented Apr 12, 2019

the threads were being filled with spam/abuse, which was making it an even worse place for discussion of security. we'll reopen them once the brigading has stopped. i'm not aware of any on-topic comments getting removed, but if so, it was by accident.

@Sharparam

This comment has been minimized.

Copy link

commented Apr 12, 2019

@ara4n I'm thinking mostly of these comments:

image

On #361.

@ghost

This comment has been minimized.

Copy link

commented Apr 12, 2019

Tons or relevant posts were deleted from the other thread, we have tons of archives of that thread but yes all of your data related to or federated with or in channels / rooms with matrix.org users can be out in the wild so change anything you've ever said in relation to them. (as there were several questions & answers about those things)

@ara4n

This comment has been minimized.

Copy link
Member

commented Apr 12, 2019

@Sharparam okay - i hadn't seen that; i think the moderation was overzealous here due to wanting to immediately remove the flood of spam on the issues in general.

@SRGOM

This comment has been minimized.

Copy link
Author

commented Apr 12, 2019

@ara4n Sure there were some trolls on both sides but people willing to learn were getting to learn.

Now is the best time for discussion and transparency. I am offering to help out moderation for the next two days if that helps you. I would urge you to restore the discussion.

In my humble opinion, this deletion was a net loss.

@pkramme

This comment has been minimized.

Copy link

commented Apr 12, 2019

I have made screenshots of all comments on all security issues 5 minutes before their deletion. @ara4n may i post the links to them here?

@tirkarthi

This comment has been minimized.

Copy link

commented Apr 12, 2019

Since this is #1 on Hackernews now it could potentially attract lot of offtopic comments : https://news.ycombinator.com/item?id=19642554

@RandomErrorMessage

This comment has been minimized.

Copy link

commented Apr 12, 2019

Funny, Hacker News spent most of the day deleting threads about it.

@ara4n

This comment has been minimized.

Copy link
Member

commented Apr 12, 2019

@paulkramme sure, if you think they are on-topic and constructive

@DylanMeeus

This comment has been minimized.

Copy link

commented Apr 12, 2019

The discussions could be informative. Nothing is 100% secure, but it'd be nice if out of the discussion other people learn what to do, and what not to do.

Furthermore, discussion can actually result in better security for matrix.org themselves :)

@ara4n

This comment has been minimized.

Copy link
Member

commented Apr 12, 2019

@DylanMeeus yup, agreed. just waiting for the flooding to subside first.

@Zenexer

This comment has been minimized.

Copy link

commented Apr 12, 2019

As it stands, if the issues were unlocked, discussion would be spread out over 9 (well, now 10) issues. If discussion is to be continued publicly at any point, it's probably worth creating a single issue for the matter.

@sjfbo

This comment has been minimized.

Copy link

commented Apr 12, 2019

It would be nice indeed to have a healthy and transparent discussion about it. Of course if there are many GH issues related to the security incident it's stupid to let all of them opened. But if we can at least centralize the discussion on one GH issue here it would be nice. Open sourcing might also requires at some point transparency with users, no? The team did it well I think via the blog. So if we can talk and improve the situation for some people who are worried despite the official blog post I think it's a good idea to keep the discussion about it open here.

@ekollof

This comment has been minimized.

Copy link

commented Apr 12, 2019

If people just ignore the shitposters and just focus on the discussion, the trolls will go away. As the old adage goes: don't feed the trolls.

@ghost

This comment has been minimized.

Copy link

commented Apr 12, 2019

https://news.ycombinator.com/item?id=19642554
I found a lot of this insightful. The first post is right up the alley of what is being discussed in this thread about openness.

dininski 1 hour ago [-]

I can see a lot of people trashing on Matrix.org or the "hacker" themselves (the hacker opened a series of issues, detailing how he managed to get in - https://github.com/matrix-org/matrix.org/issues/created_by/m...). However everyone seems to be missing the point - matrix seems like a pretty cool and open project. And someone taking over their infrastructure in such an open way is also great for the community. Even though a little dubious on the legal side of things, I believe it's great it was approached with transparency and a dose of humor.

Some might argue that this is harmful to matrix as a product and as a brand. But as long as there was no actual harm done and they react appropriately by taking infrastructure security seriously, it could play out well in the end for them. This whole ordeal could end up actually increase trust in the project, if they take swift steps to ensure that something like this does not happen again.

He's very correct, as I said in the last thread this could have been insanely worse. Especially since the guy was in the network for a month and nobody noticed.

@ekollof

This comment has been minimized.

Copy link

commented Apr 12, 2019

Maybe someone can aggregate all of the attacker's tickets into one ticket with all the relevant action points he left behind? That way, the discussion won't spread over 10+ tickets

@AlexanderBalson

This comment has been minimized.

Copy link

commented Apr 12, 2019

I don't know how keen the contributors would be, but having the hacker on board the dev team could have a big benefit on the security of the entire system.

@ghost

This comment has been minimized.

Copy link

commented Apr 12, 2019

After making them lose sleep and fix stuff the last 24 hours I don't think they'd ever not hate the guy lol

@Zenexer

This comment has been minimized.

Copy link

commented Apr 12, 2019

If I understand correctly, they got in a month ago and are only now just publishing info since they’ve been caught and cut off. I wouldn’t trust someone like that. They probably would’ve kept quiet indefinitely had they not been caught.

Also, blue team is a lot harder than red team. It’s easy to get in; it’s a lot harder to keep people from getting in.

@wioxjk

This comment has been minimized.

Copy link

commented Apr 12, 2019

That is unprofessional behavior from the matrix.org team. Alot of comments was related to the issues.
By removing them - you are also removing the trust you have gotten.

Shame

@pkramme

This comment has been minimized.

Copy link

commented Apr 12, 2019

@wioxjk No, "alot of comments" weren't related, constructive or helpful. Only the comments under #365 and #361 were constructive in any way. And ONE comment under #357 by linking to why SSH Agent Forwarding might be dangerous.

EDIT: Except for the one comment under #357, all the "good" comments have been restored. The comment there was written by @rain-1 and linked to https://heipei.io/2015/02/26/SSH-Agent-Forwarding-considered-harmful/
rain-1 comment

@pkramme

This comment has been minimized.

Copy link

commented Apr 12, 2019

@ara4n @neilisfragile Maybe one of you can restore the comment from @rain-1 the same way you did in #361 based on the screenshot.

@jryans

This comment has been minimized.

Copy link
Member

commented Apr 12, 2019

@paulkramme Thanks, I have restored that one using the screenshot.

@netspooky

This comment has been minimized.

Copy link

commented Apr 12, 2019

As frustrating and scary as it is to get owned, the person who was leaving these issues as matrixnotorg was actually giving solid security advice. I have run some Matrix infra and like the project a lot, and I hope that the Matrix dev teams took the core info they shared (2FA, signing keys etc.) to heart for the future. Good luck y'all.

@coldice

This comment has been minimized.

Copy link

commented Apr 12, 2019

Hey guys, I saw the original issues this morning and there was a lot of value in. I actually wanted to show them to our devops team as an example of how & what can happen.

I understand the problem of making sure the SNR keeps usable, but I think there is value in making those available again at least read only, which I hope will be possible soon.

@ara4n

This comment has been minimized.

Copy link
Member

commented Apr 12, 2019

we haven't removed these issues - it looks like the original author must have by deleting their account? (although I didn't realise github supported that)

@naj59

This comment has been minimized.

Copy link

commented Apr 12, 2019

@ara4n if an account is deleted the user will be displayed as @ghost

@pkramme

This comment has been minimized.

Copy link

commented Apr 12, 2019

@ara4n I still have the screenshots. Do you want me to reopen the issues with the original text?

@ara4n

This comment has been minimized.

Copy link
Member

commented Apr 12, 2019

please hold off whilst we investigate.

@pkramme

This comment has been minimized.

Copy link

commented Apr 12, 2019

@naj59 Maybe the user was removed by GitHub?

@ptman

This comment has been minimized.

Copy link
Contributor

commented Apr 12, 2019

I reported the repository while the DNS redir was in effect, but github only now got around to doing something about it.

@naj59

This comment has been minimized.

Copy link

commented Apr 12, 2019

@paulkramme Probably but also then it would be displayed as @ghost (as I think). Could also be made view-only for @matrix-org members.

@rain-1

This comment has been minimized.

Copy link

commented Apr 12, 2019

what's the point of banning him? he was helping us

@ara4n

This comment has been minimized.

Copy link
Member

commented Apr 12, 2019

i doubt that github would have looked that hard. to reiterate: we didn't ban them.

@rubo77

This comment has been minimized.

Copy link

commented Apr 18, 2019

Here someone pieced it together as one "story":

https://pastebin.com/3rzCqrFk

@SRGOM

This comment has been minimized.

Copy link
Author

commented Apr 28, 2019

@paulkramme Do you have the undeleted comments for this? #364

I missed the discussion some discussion there, and I have no trouble mentally ignoring trolling...

@SRGOM

This comment has been minimized.

Copy link
Author

commented Apr 28, 2019

Closing because no longer relevant.

@SRGOM SRGOM closed this Apr 28, 2019

@ara4n

This comment has been minimized.

Copy link
Member

commented Apr 28, 2019

@SRGOM: https://web.archive.org/web/20190412090012/https://github.com/matrix-org/matrix.org/issues/364 looks to have most of the undeleted comments if you really want it. The only on-topic things look to be confusion over whether Matrix has people working on it full time (it does), and whether it has professional sysadmins (it does, but they have been working exclusively on paid deployments like modular.im and the French government's one, hence the old core matrix.org infra not getting the security attention it needed).

Having got most of the infra rebuilt we're writing up the full postmortem which should be available at the end of the coming week.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.