New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Deprecate Gitter rooms, Download all data #8418
Comments
Rooms Notified
Left out the City specific rooms. @QuincyLarson I think that should be it. P.S I just realized that we have 540 rooms, with this audit. |
Added:
|
Nice work! |
@sludge256 @raisedadead actually, I think we need to do this in literally every room that isn't on our official rooms list (unless it happens to have significant activity within the past 3 days). |
Have checked on most rooms, all of the rooms which did not have any activity (in more than a month) have not been touched (simply just because it would have triggered discussion in them). The campsite rooms do not need this, they already have a message and the above also applies to all of them. Closing. |
@raisedadead We have discontinued the fcc wiki. Do the deprecated rooms point to the corresponding post in the forum? |
@BerkeleyTrue However, Ziplines and Bonfires sometimes get visitors, because Gitter shows them as suggested rooms. |
@raisedadead @BerkeleyTrue we may actually just want to delete the deprecated rooms. Few would be missed, and that way, Gitter's native discovery features would work properly. We wouldn't have to list our official rooms - we would only have official rooms (all the unofficial other rooms would be run by campers themselves and not under the freecodecamp prefix. Downside: we lose some history and some small amount of Google search results With the success of the forum, my goal is to shift a lot of communication that would have taken place intermittently on Gitter over to the forum, where the expectation is that it may take days to get a response. So many of the chat rooms are ghost towns - hence us frequently pruning or deprecating them. We would need to go through the official rooms and see which are active. |
Yeah, I agree. I think the upside is better in favor of deleting them. Will do the audit and post a list that can/must be deleted. |
many: > 100/day FreeCodeCamp - many |
Our community is primarily spread out across 3 places:
For what it's worth, here's my thinking on each of these mediums:
Of these three, chat rooms forge the tightest bonds. Talking with people in real time is exhilarating, and it can be hard to leave a chat room. This said, chat room suffer from the "ghost town" affect much more than forums or Facebook groups. Once things die down in a chat room, it can be hard to get the conversation started again. By concentrating as much of our chat room-going community as possible into a few key rooms, we mitigate the risk of ghost towns. The question is what rooms do we keep? Based on the above research, I humbly propose we:
I am excited to hear everyone's thoughts on this. |
I saw your proposed list and your invitation to discuss the future of a chatroom like the DataScience one. I don't think this room fits a "general room" format: the discussions that are occurring there are mostly about Data Science. Merging that room into another more general one could kill the current content and motivation of the room unless it is merged with rooms with similar intentions. A room like "contributors" doesn't looks like the best title for this one. I can also say that it is attracting people: there are currently 680 subscribers to the room, and some people are monitoring the activity. As it is now, I don't think the nature of this room and the activity involved would fit the forum format. I can only talk about this room because it is the room I have been managing since its foundation. |
I have also monitored the activity in the Python room: it is not 'few' but 'some'. However, this is a room that due to its nature could be moved somewhere else. |
The Spanish room is one of the most active ones I have seen too... The people there are really good in maintaining the room activity. This won't fit the forum format neither. If you think that the future is to delete those rooms, I think that the Spanish one will be affected by deleting related rooms like HelpDataViz, HelpJavaScript etc. The Spanish room is also a place to help to solve issues that Spanish-speakers found in those rooms. Once you delete the Help-related rooms and move activity to the forum, it is possible that the Spanish room won't survive or reduce its activity. If the activity in Gitter becomes less in general, the Spanish room could trend to disappear. |
I suggest the following: I think deleting rooms about Help- will simply reduce the whole activity at Gitter, as Gitter won't be a reference to the set of help resources any more. |
@evaristoc just came up with a great idea. Instead of making a big change at once, lets make these changes step-by-step. |
@QuincyLarson
At least, to prevent the users with enough time to "move their stuff somewhere else". My understanding is: This should be done only if you decide that those rooms are not contributing to FCC project directly. Otherwise I suggest you to keep them until you see what happens with Gitter activity after deleting other related rooms. |
I agree with Quincy. If there are very active city rooms that might be a consideration to keep (I know Cleveland, where I am from, is nothing since April.) I do think that pairing down the rooms that duplicate other rooms would make things easier. Like the old help rooms before the change over. Now, Spanish/Chinese we don't touch them. Chinese especially because of a lack of a good FB alternative. I think we should be able to get by with about 15-ish rooms using a flow of skill sets. Front, Data, Back, Code-review, Pairing, CodingJobs. Contributers, Core . I am pretty sure that there are other rooms that should be added, but these are the ones I feel most strongly on. Once we move to the new curriculum (Each segment of the certification being it's own certification might warrant its own room) Maybe keep off topic as a place to spend a pomodoro break away from coding. |
@QuincyLarson I've downloaded the Total Uncompressed Size: 588 MB (tab separated format Sample Format
Note. I've monitored the whole download and there is absolutely NO errors while downloading the messages. // cc: @evaristoc |
git-lfs cannot be used as it is chargeable. And github is not the right place to upload large files. https://github.com/ckolivas/lrzip gives the best compression. I was able to compress |
@abhisekp Awesome! I didn't realize Git-LFS costs money. Can't we just push these files to a repo? FreeCodeCamp/FreeCodeCamp is the largest Gitter room by an order of magnitude, so at that level of compression, the other rooms shouldn't be much of a problem. |
Can we dump this to Amazon S3? GitHub clearly isn't the right place for data storage. It comes with Free 5GB of storage and data retrievals cost $0.01 per GB. I think we already have an instance for this? |
@raisedadead yes - I can put them into our AWS S3 account for hosting. Once you have the files ready for all the rooms we're archiving, let's hop on a call and figure out how to get them from your computer to our S3 instance. |
@QuincyLarson, @abhisekp has the data currently. So maybe he can only guide the best. But, do let me know if I can help in any way. |
@raisedadead Thanks for the idea! I've created an S3 key especially for @abhisekp and sent it to him. |
Hi there PS: Here is a link for the downloaded rooms: https://we.tl/lcBU6Cs7E4 |
gitter-archive-cli Released 🎉Archive gitter communities worry-free 😃 Feature
How to use# install globally
$ npm i -g gitter-archive-cli Create a directory where you want to save the gitter community archives. Create a
Now simply start the archiving process using Example
|
@abhisekp Awesome! If this works well, can you go ahead and set it to run non-stop until all of our rooms are backed up? Then we can zip that up and push it to S3. |
@abhisekp Let the magic happen :) Cheers |
@QuincyLarson @ladybugtju Thanks. The magic is almost complete 😏 Github has a strict file size limit of 100 MB but gives a warning at 50 MB. I was able to push a 53 MB file ( The only remaining rooms are
These are the most largest rooms. Update Pushed |
@abhisekp Amazing work. So now that these rooms are in version control, do you think we're safe to start deleting these rooms? As far as the larger rooms, we might want to break them up into separate files by date. For example: FreeCodeCamp/FreeCodeCamp August 1 2015 - November 31 2015 This will make it easier for us to continue pushing updated archives to them. This will be a huge public dataset and I imagine a lot of people will be interested in it. We can publish this on Kaggle once it's ready :) |
@abhisekp how is this process going? Have you managed to do a full archive of our Gitter rooms? Can we start closing rooms that we don't plan on keeping now? |
Our plan is to:
Once all these messages are in the https://github.com/FreeCodeCamp/gitter-history repo, I will go through and delete all rooms except for those on our official rooms list. All of these rooms already have deprecation messages and aren't being used anyway. Campers can still create their own unofficial Gitter rooms, but freeCodeCamp's chatroom structure will be clean and simple, with minimal ambiguity about which rooms to go to. Then we will use https://gitter.im/FreeCodeCamp/home as our main entryway to our chat rooms. |
Sorry @QuincyLarson asking why FreeCodeCamp/FreeCodeCamp? |
@evaristoc sounds like the answer is
A room called just |
@QuincyLarson @erictleung for a small project I was planning to do I downloaded almost all messages from the main room. Data is until yesterday. @erictleung good point but I understand these are the list of official rooms. Freecodecamp room is still there: Is it a change in that list I am not aware of? |
@erictleung there's no way to rename a Gitter room. Otherwise we would indeed rename that chat room to "general" or "casual". |
@evaristoc yes - that would be awesome. If you pull all of freecodecamp/freecodecamp be sure to add it to https://github.com/FreeCodeCamp/gitter-history |
OK - all the rooms that were deprecated were backed up afterward. I haven't heard anything from @abhisekp recently so I've gone ahead and deleted the deprecated rooms. Thanks everyone! |
New rooms scheduled for archiving:
Currently exploring @abhisekp approach to archiving before initiating the process of downloading the data: |
I was trying to use the great package made by @abhisekp: https://www.npmjs.com/package/gitter-archive-cli but unfortunately it didn't work on my computer. It is giving a I will likely work this on Python. My current code seems to be outdated though. Apparently I am also affected by the rate limits with an If it works, I will make my Python code available. Hoping that will help to find a standard code to approach chatroom-archiving in the future. |
Another, simpler option by @ladybugtju : |
This is a previous message by @abhisekp to be kept here as reference: #8418 (comment) |
@evaristoc please continue on the new thread linked above. |
Many old rooms such as /helpBonfires are now deprecated, but campers still join them. We need a contributor who's earned write access to this repo to go through and update the topic of these rooms by running:
The text was updated successfully, but these errors were encountered: