Skip to content

Commit

Permalink
Resolve merge conflicts
Browse files Browse the repository at this point in the history
  • Loading branch information
kamranahmedse committed Jan 29, 2020
2 parents bb3260f + 3c5ea21 commit fc2eb36
Show file tree
Hide file tree
Showing 38 changed files with 408 additions and 177 deletions.
2 changes: 1 addition & 1 deletion components/about-header/index.js
Expand Up @@ -7,7 +7,7 @@ const AboutHeader = () => (
<div className="author-info">
<img className='author-img d-none d-sm-none d-md-block d-lg-block d-xl-block' src="/kamran.jpeg" />
<div className="author-msg">
<h2>Hello, I'm Kamran Ahmed.</h2>
<h1>Hello, I'm Kamran Ahmed.</h1>
<p>I created <span className='flow-black'>roadmap.sh</span> to help developers find their path if they are confused and help them grow in their career.</p>
<div className="author-links">
<a href={`https://twitter.com/${siteConfig.twitter}`} target="_blank">@kamranahmedse</a>
Expand Down
2 changes: 1 addition & 1 deletion components/about-header/style.js
Expand Up @@ -11,7 +11,7 @@ export const AboutHeaderWrap = styled.div`
flex-direction: row;
}
h2 {
h1 {
font-weight: 700;
}
Expand Down
17 changes: 12 additions & 5 deletions components/detailed-roadmap/index.js
Expand Up @@ -32,18 +32,24 @@ const DetailedRoadmap = ({ roadmap }) => {
author = {}
} = roadmap;

const roadmapPages = Object.keys(sidebar || {}).map(groupTitle => {
const roadmapPages = Object.keys(sidebar || {}).map((groupTitle, groupCounter) => {
if (groupTitle.startsWith('_')) {
return;
}

// @todo remove it after completing the frontend roadmap
const isInProgress = groupCounter !== 0;

return (
<div className='links-group' key={groupTitle}>
<h3>{ groupTitle }</h3>
<div className={`links-group ${isInProgress ? 'in-progress' : ''}`} key={groupTitle}>
<h3>
{ groupTitle }
{ isInProgress && <span className='badge badge-warning progress-badge'>In Progress</span> }
</h3>
<ul>
{ sidebar[groupTitle].map(page => {
const isActivePage = page.url === currentPage.url;
// e.g. /frontend should mark `/frontend/summary` as active
// e.g. /frontend should mark `/frontend/landscape` as active
const isSummaryPage = page.url === `${currentPage.url}/summary`;

return (
Expand Down Expand Up @@ -71,7 +77,8 @@ const DetailedRoadmap = ({ roadmap }) => {
<h3>{ roadmap.title }</h3>
<p>
Roadmap contributed by <a href={ author.url } target="_blank">{ author.name }</a>
{ roadmap.contributorsCount > 1 && ` and <a href="${roadmap.contributorsUrl}">${roadmap.contributorsCount} others</a>`}</p>
{ roadmap.contributorsCount > 1 && ` and <a href="${roadmap.contributorsUrl}">${roadmap.contributorsCount} others</a>`}
</p>
</RoadmapMeta>
<ShareRoadmap className="mt-2 mt-md-0">
<ShareIcon href={ siteConfig.url.repo } target="_blank">
Expand Down
16 changes: 16 additions & 0 deletions components/detailed-roadmap/style.js
Expand Up @@ -113,6 +113,16 @@ export const Sidebar = styled.div`
}
}
.progress-badge {
position: relative;
top: -2px;
margin-left: 5px;
}
.links-group.in-progress {
opacity: 0.3;
}
.links-group li {
list-style: none;
margin: 7px 0;
Expand Down Expand Up @@ -180,4 +190,10 @@ export const MobileSidebar = styled(Sidebar)`
.links-group {
width: auto;
}
.progress-badge {
position: relative;
top: -2px;
margin-left: 5px;
}
`;
5 changes: 2 additions & 3 deletions components/helmet/index.js
Expand Up @@ -2,8 +2,7 @@ import NextHead from 'next/head';
import siteConfig from 'content/site';

const prepareTitle = (givenTitle) => {
givenTitle = givenTitle || siteConfig.title;
return `${givenTitle} - ${siteConfig.name}`;
return givenTitle || siteConfig.title;
};

const prepareDescription = (givenDescription) => {
Expand All @@ -19,7 +18,7 @@ const Helmet = (props) => (
<meta name='description' content={ prepareDescription(props.description) } />

<meta name="author" content={ siteConfig.author } />
<meta name="keywords" content={ siteConfig.keywords.join(',') } />
<meta name="keywords" content={ props.keywords ? props.keywords.join(',') : siteConfig.keywords.join(',') } />

<meta name="viewport" content="width=device-width, user-scalable=yes, initial-scale=1.0, maximum-scale=3.0, minimum-scale=1.0" />
{ props.canonical && <link rel="canonical" href={ props.canonical } /> }
Expand Down
2 changes: 1 addition & 1 deletion content/guides/design-patterns-for-humans.md
Expand Up @@ -1595,7 +1595,7 @@ $stationList->removeStation(new RadioStation(89)); // Will remove station 89
```

👽 Mediator
========
--------

Real world example
> A general example would be when you talk to someone on your mobile phone, there is a network provider sitting between you and them and your conversation goes through it instead of being directly sent. In this case network provider is mediator.
Expand Down
32 changes: 16 additions & 16 deletions content/guides/torrent-client.md
Expand Up @@ -6,14 +6,14 @@ The protocol evolved organically over the past 20 years, and various people and

I'll be using a [Debian ISO](https://cdimage.debian.org/debian-cd/current/amd64/bt-cd/#indexlist) file as my guinea pig because it's big, but not huge, at 350MB. As a popular Linux distribution, there will be lots of fast and cooperative peers for us to connect to. And we'll avoid the legal and ethical issues related to downloading pirated content.

# Finding peers
## Finding peers
Here’s a problem: we want to download a file with BitTorrent, but it’s a peer-to-peer protocol and we have no idea where to find peers to download it from. This is a lot like moving to a new city and trying to make friends—maybe we’ll hit up a local pub or a meetup group! Centralized locations like these are the big idea behind trackers, which are central servers that introduce peers to each other. They’re just web servers running over HTTP, and you can find Debian’s at http://bttracker.debian.org:6969/

![illustration of a desktop computer and laptop sitting at a pub](/guides/torrent-client/trackers.png)

Of course, these central servers are liable to get raided by the feds if they facilitate peers exchanging illegal content. You may remember reading about trackers like TorrentSpy, Popcorn Time, and KickassTorrents getting seized and shut down. New methods cut out the middleman by making even **peer discovery** a distributed process. We won't be implementing them, but if you're interested, some terms you can research are **DHT**, **PEX**, and **magnet links**.

## Parsing a .torrent file
### Parsing a .torrent file
A .torrent file describes the contents of a torrentable file and information for connecting to a tracker. It's all we need in order to kickstart the process of downloading a torrent. Debian's .torrent file looks like this:

```markdown
Expand Down Expand Up @@ -106,7 +106,7 @@ func (bto *bencodeTorrent) toTorrentFile() (*TorrentFile, error) {
}
```

## Retrieving peers from the tracker
### Retrieving peers from the tracker
Now that we have information about the file and its tracker, let's talk to the tracker to **announce** our presence as a peer and to retrieve a list of other peers. We just need to make a GET request to the `announce` URL supplied in the .torrent file, with a few query parameters:

```go
Expand Down Expand Up @@ -136,7 +136,7 @@ The important ones:

![a file with a name tag saying 'info_hash' and a person with a name tag 'peer_id'](/guides/torrent-client/info-hash-peer-id.png)

## Parsing the tracker response
### Parsing the tracker response
We get back a bencoded response:

```markdown
Expand Down Expand Up @@ -179,7 +179,7 @@ func Unmarshal(peersBin []byte) ([]Peer, error) {
}
```

# Downloading from peers
## Downloading from peers
Now that we have a list of peers, it's time to connect with them and start downloading pieces! We can break down the process into a few steps. For each peer, we want to:

1. Start a TCP connection with the peer. This is like starting a phone call.
Expand All @@ -196,7 +196,7 @@ if err != nil {

I set a timeout so that I don't waste too much time on peers that aren't going to let me connect. For the most part, it's a pretty standard TCP connection.

## Complete the handshake
### Complete the handshake
We've just set up a connection with a peer, but we want do a handshake to validate our assumptions that the peer

* can communicate using the BitTorrent protocol
Expand Down Expand Up @@ -250,14 +250,14 @@ func Read(r io.Reader) (*Handshake, error) {
}
```

## Send and receive messages
### Send and receive messages
Once we've completed the initial handshake, we can send and receive **messages**. Well, not quite—if the other peer isn't ready to accept messages, we can't send any until they tell us they're ready. In this state, we're considered **choked** by the other peer. They'll send us an **unchoke** message to let us know that we can begin asking them for data. By default, we assume that we're choked until proven otherwise.

Once we've been unchoked, we can then begin sending **requests** for pieces, and they can send us messages back containing pieces.

!["A cartoon in which person 1 says 'hello I would like piece number—' and person 2 grabs him by the neck and says '00 00 00 01 00 (choke)'](/guides/torrent-client/choke.png)

### Interpreting messages
#### Interpreting messages
A message has a length, an **ID** and a **payload**. On the wire, it looks like:

![A message with 4 byte for the length, 1 byte for ID, and an optional payload](/guides/torrent-client/message.png)
Expand Down Expand Up @@ -333,7 +333,7 @@ func Read(r io.Reader) (*Message, error) {
}
```

### Bitfields
#### Bitfields
One of the most interesting types of message is the **bitfield**, which is a data structure that peers use to efficiently encode which pieces they are able to send us. A bitfield looks like a byte array, and to check which pieces they have, we just need to look at the positions of the *bits* set to 1. You can think of it like the digital equivalent of a coffee shop loyalty card. We start with a blank card of all `0`, and flip bits to `1` to mark their positions as "stamped."

![a coffee shop loyalty card with eight slots, with stamps on the first four slots and a stamp on the second to last slot, represented as 11110010](/guides/torrent-client/bitfield.png)
Expand All @@ -359,10 +359,10 @@ func (bf Bitfield) SetPiece(index int) {
}
```

## Putting it all together
### Putting it all together
We now have all the tools we need to download a torrent: we have a list of peers obtained from the tracker, and we can communicate with them by dialing a TCP connection, initiating a handshake, and sending and receiving messages. Our last big problems are handling the **concurrency** involved in talking to multiple peers at once, and managing the **state** of our peers as we interact with them. These are both classically Hard problems.

### Managing concurrency: channels as queues
#### Managing concurrency: channels as queues
In Go, we [share memory by communicating](https://blog.golang.org/share-memory-by-communicating), and we can think of a Go channel as a cheap thread-safe queue.

We'll set up two channels to synchronize our concurrent workers: one for dishing out work (pieces to download) between peers, and another for collecting downloaded pieces. As downloaded pieces come in through the results channel, we can copy them into a buffer to start assembling our complete file.
Expand Down Expand Up @@ -437,7 +437,7 @@ func (t *Torrent) startDownloadWorker(peer peers.Peer, workQueue chan *pieceWork
}
```

### Managing state
#### Managing state
We'll keep track of each peer in a struct, and modify that struct as we read messages. It'll include data like how much we've downloaded from the peer, how much we've requested from them, and whether we're choked. If we wanted to scale this further, we could formalize this as a finite state machine. But a struct and a switch are good enough for now.

```go
Expand Down Expand Up @@ -469,12 +469,12 @@ func (state *pieceProgress) readMessage() error {
}
```

### Time to make requests!
#### Time to make requests!
Files, pieces, and piece hashes aren't the full story—we can go further by breaking down pieces into **blocks**. A block is a part of a piece, and we can fully define a block by the **index** of the piece it's part of, its byte **offset** within the piece, and its **length**. When we make requests for data from peers, we are actually requesting *blocks*. A block is usually 16KB large, meaning that a single 256 KB piece might actually require 16 requests.

A peer is supposed to sever the connection if they receive a request for a block larger than 16KB. However, based on my experience, they're often perfectly happy to satisfy requests up to 128KB. I only got moderate gains in overall speed with larger block sizes, so it's probably better to stick with the spec.

### Pipelining
#### Pipelining
Network round-trips are expensive, and requesting each block one by one will absolutely tank the performance of our download. Therefore, it's important to **pipeline** our requests such that we keep up a constant pressure of some number of unfulfilled requests. This can increase the throughput of our connection by an order of magnitude.

![Two email threads simulating peer connections. The thread on the left shows a request followed by a reply, repeated three times. The thread on the left sends three requests, and receives three replies in quick succession.](/guides/torrent-client/pipelining.png)
Expand Down Expand Up @@ -529,7 +529,7 @@ func attemptDownloadPiece(c *client.Client, pw *pieceWork) ([]byte, error) {
}
```

### main.go
#### main.go
This is a short one. We're almost there.

```go
Expand Down Expand Up @@ -560,5 +560,5 @@ func main() {

<script id="asciicast-xqRSB0Jec8RN91Zt89rbb9PcL" src="https://asciinema.org/a/xqRSB0Jec8RN91Zt89rbb9PcL.js" async></script>

# This isn't the full story
## This isn't the full story
For brevity, I included only a few of the important snippets of code. Notably, I left out all the glue code, parsing, unit tests, and the boring parts that build character. View my [full implementation](https://github.com/veggiedefender/torrent-client) if you're interested.
14 changes: 7 additions & 7 deletions content/pages/about.md
@@ -1,22 +1,22 @@
#### What is roadmap.sh?
## What is roadmap.sh?
Roadmap.sh is the place containing community curated roadmaps, study plans, paths and resources for the budding developers. It started as a [set of charts to guide the developers](https://github.com/kamranahmedse/developer-roadmap) who are confused about what should they learn next but that alone wasn't enough so I expanded it into the website to get more contributors involved.

#### What are the plans for roadmap.sh?
## What are the plans for roadmap.sh?
The website started off as a [simple repository containing a few charts](https://github.com/kamranahmedse/developer-roadmap) for developers and based on my personal opinions but it could have been much more than that so I decided to expand it to a website where people can contribute to study plans with their areas of expertise as well, add more roadmaps, write guides etc.

We haven't opened up the sign ups for now but we will be doing. My long term plans for this website are to turn it into a goto place for the developers to seek guidance about their careers, help others, share their journeys, incentivize the learnings, get feedbacks on their projects etc.

#### How did you build roadmap.sh?
## How did you build roadmap.sh?
The basic version of the website has been built with [Next.js](https://github.com/zeit/next.js/), is opensource and can be found on [github](https://github.com/kamranahmedse/roadmap.sh). It was hastily done to get it out in front of the people and get people to start contributing so it might be rough on the edges, but that is where we need your help.

#### How does it make money?
## How does it make money?
It doesn't make any money. I have been using my personal time and budget to build it. I did not create this website with any intentions of monetization but as a good will, to help the people get out of the frustration that I was once in.

Having said that, I love teaching and my future plans are to be able to work full-time on roadmap.sh for which it has to make enough money to pay for my rent, groceries, bills, travel expenses, etc but even if it doesn't it's likely I'll continue growing the site however I can. My focus at the moment is not making money from it and just add content that creates value for the people.

> Sponsor the efforts by [paying as little as 3$ per month](http://gum.co/roadmap-sh) or with [one time payment via paypal](https://paypal.me/kamranahmedse). Alternatively, reach out to me at [kamran@roadmap.sh](mailto:kamran@roadmap.sh).
#### Can I contribute?
## Can I contribute?
You definitely can, infact you are encouraged to do that. Even your minor contributions such as typo fixes count. The source code of the website can be [found on Github](https://github.com/kamranahmedse/roadmap.sh). Your contributions can be:

* Adding a new roadmap
Expand All @@ -31,9 +31,9 @@ You definitely can, infact you are encouraged to do that. Even your minor contri

Just make sure to [follow the contribution guidelines](https://github.com/kamranahmedse/roadmap.sh/tree/master/contributing) when you decide to contribute.

#### Can I redistribute the content?
## Can I redistribute the content?
No, the license of the content on this website does not allow you to redistribute any of the content on this website anywhere. You can use it for personal use or share the link to the content if you have to but redistribution is not allowed.

#### What is the best way to contact you?
## What is the best way to contact you?
Tweet or send me a message [@kamranahmedse](https://twitter.com/kamranahmedse) or email me at [kamran@roadmap.sh](kamran@roadmap.sh). I get lots of messages so apologies in advance if you don't hear back from me soon but I do reply to everyone.

0 comments on commit fc2eb36

Please sign in to comment.