Skip to content
URL shortening service written in Go and React
Go TypeScript CSS Shell JavaScript HTML Dockerfile
Branch: master
Clone or download
2 authors and alldroll Refactor IdentityProvider for Facebook OAuth (#501)
* Refactor IdentityProvider for Facebook OAuth

* update

* add integration test

* address comment

* address comment

* address comment

Co-authored-by: Harry Liu <byliuyang11@gmail.com>
Latest commit 23de27c Jan 27, 2020

README.md

Short

Build Status codecov Maintainability Go Report Card Open Source Love Floobits Status

Demo

Preview

Demo

Get s/ Chrome extension

Install it from Chrome Web Store or build it from source

Dependent Projects

  • app: Reusable framework for Go apps & command line tools.
  • kgs: Offline unique key generation service.
  • toggle: Dynamic system behavior controller.

Table of Contents

  1. Getting Started
    1. Accessing the source code
    2. Prerequisites
    3. Create reCAPTCHA account
    4. Create Github OAuth application
    5. Create Facebook Application
    6. Create Google OAuth Client ID
    7. Backend
    8. Frontend
  2. System Design
    1. App Level Architecture
    2. Service Level Archtecture
    3. Object Oriented Design
    4. Dependency Injection
    5. Database Modeling
    6. Feature Toggle
    7. Search Engine Optimization
    8. Social Media Summary Card
  3. Testing
    1. The Importance Of Automation
    2. Testing Strategy
    3. Unit Testing
    4. Integration Testing
    5. Component Testing
    6. Contract Testing
    7. End To End Testing
    8. The Test Pyramid
  4. Deployment
    1. Kubernetes
    2. Staging
    3. Production
  5. Tools We Use
  6. Contributing
  7. Author
  8. License

Getting Started

Accessing the source code

git clone https://github.com/byliuyang/short.git

Prerequisites

  • Go v1.13.1
  • Node.js v12.12.0
  • Yarn v1.19.1
  • Postgresql v12.0 ( or use ElephantSQL instead )

Create reCAPTCHA account

  1. Sign up at ReCAPTCHA with the following configurations:

    Field Value
    Label Short
    reCAPTCHA type reCAPTCHAv3
    Domains localhost

    Register Site

  2. Open settings. Copy SITE KEY and SECRET KEY.

    Settings

    Credentials

  3. Replace the value of RECAPTCHA_SECRET in the backend/.env file with SECRET KEY.

  4. Replace the value of REACT_APP_RECAPTCHA_SITE_KEY in frontend/.env.development file with SITE KEY.

Create Github OAuth application

  1. Create a new OAuth app at Github Developers with the following configurations:

    Field Value
    Application Name Short
    Homepage URL http://localhost
    Application description URL shortening service written in Go and React
    Authorization callback URL http://localhost/oauth/github/sign-in/callback

    OAuth Apps

    New OAuth App

  2. Copy Client ID and Client Secret.

    Credentials

  3. Replace the value of GITHUB_CLIENT_ID in the backend/.env file with Client ID.

  4. Replace the value of GITHUB_CLIENT_SECRET in the backend/.env file with Client Secret.

Create Facebook Application

  1. Create a new app at Facebook Developers with the following configurations:

    Field Value
    Display Name Short Test
    Contact Email your_email
  2. Add Facebook Login to the app.

    Login

  3. Copy App ID and App Secret on Settings > Basic tab.

    Credentials

  4. Replace the value of FACEBOOK_CLIENT_ID in backend/.env file with App ID.

  5. Replace the value of FACEBOOK_CLIENT_SECRET in backend/.env file with App Secret.

Create Google OAuth Client ID

Create a new Client ID at Google API Credentials:

  1. Click on Create Credentials and select OAuth client ID.

    Credentials

  2. Select Web application for Application type.

  3. Fill in http://localhost/oauth/google/sign-in/callback for Authorized redirect URIs and click on Create

    OAuth

  4. Replace the value of GOOGLE_CLIENT_ID in backend/.env file with Your Client ID.

  5. Replace the value of GOOGLE_CLIENT_SECRET in backend/.env file with Your Client Secret.

Backend

  1. Copy backend/.env.dist file to backend/.env:

    cp backend/.env.dist backend/.env
  2. Update DB_HOST, DB_PORT, DB_USER, DB_PASSWORD, DB_NAME, RECAPTCHA_SECRET, GITHUB_CLIENT_ID, GITHUB_CLIENT_SECRET, FACEBOOK_CLIENT_ID, FACEBOOK_CLIENT_SECRET, FACEBOOK_REDIRECT_URI, JWT_SECRET, with your own configurations.

  3. Launch backend server

    cd backend
    ./scripts/dev
  4. Remember to install developers tools before start coding:

    ./scripts/tools

Frontend

Remember to update REACT_APP_RECAPTCHA_SITE_KEY in frontend/.env.development.

  1. Launch frontend server

    cd frontend
    ./scripts/dev
  2. Visit http://localhost:3000

System Design

App Level Architecture

Short backend is built on top of Uncle Bob's Clean Architecture, the central objective of which is separation of concerns.

Clean Architecture Boundary

It enables the developers to modify a single component of the system at a time while leaving the rest unchanged. This minizes the amount of changes have to be made in order to support new requirements as the system grows. Clean Architecture also improves the testability of system, which in turn saves precious time when creating automated tests.

Here is an exmample of finance app using clean architecture:

Finance App

Service Level Archtecture

Short adopts Microservices Architecture to organize dependent services around business capabilities and to enable independent deployment of each service.

Microservice Architecture

Object Oriented Design

Short leverages class design, package cohesion, and package coupling princiapls to manage logical dependency between internal components.

Class Design

Principal Description
Single Responsibility Principle A class should have one, and only one, reason to change.
Open Closed Principle You should be able to extend a classes behavior, without modifying it.
Liskov Substitution Principle Derived classes must be substitutable for their base classes.
Interface Segregation Principle Make fine grained interfaces that are client specific.
Dependency Inversion Principle Depend on abstractions, not on concretions.

Package Cohesion

Principal Description
Release Reuse Equivalency Principle The granule of reuse is the granule of release.
The Common Closure Principle Classes that change together are packaged together.
The Common Reuse Principle Classes that are used together are packaged together.

Package Coupling

Principal Description
Acyclic Dependencies Principle The dependency graph of packages must have no cycles.
Stable Dependencies Principle Depend in the direction of stability.
Stable Abstractions Principle Abstractness increases with stability.

Dependency Injection

Short produces flexible and loosely coupled code, by explicitly providing components with all of the dependencies they need.

type Authenticator struct {
  tokenizer          fw.CryptoTokenizer
  timer              fw.Timer
  tokenValidDuration time.Duration
}

func NewAuthenticator(
  tokenizer fw.CryptoTokenizer,
  timer fw.Timer,
  tokenValidDuration time.Duration,
) Authenticator {
  return Authenticator{
    tokenizer:          tokenizer,
    timer:              timer,
    tokenValidDuration: tokenValidDuration,
  }
}

Short also simplifies the management of the big block of order-dependent initialization code with Wire, a compile time depedency injection framework by Google.

func InjectGraphQlService(
  name string,
  sqlDB *sql.DB,
  graphqlPath provider.GraphQlPath,
  secret provider.ReCaptchaSecret,
  jwtSecret provider.JwtSecret,
  bufferSize provider.KeyGenBufferSize,
  kgsRPCConfig provider.KgsRPCConfig,
) (mdservice.Service, error) {
  wire.Build(
    wire.Bind(new(fw.GraphQlAPI), new(graphql.Short)),
    wire.Bind(new(url.Retriever), new(url.RetrieverPersist)),
    wire.Bind(new(url.Creator), new(url.CreatorPersist)),
    wire.Bind(new(repo.UserURLRelation), new(db.UserURLRelationSQL)),
    wire.Bind(new(repo.URL), new(*db.URLSql)),
    wire.Bind(new(keygen.KeyGenerator), new(keygen.Remote)),
    wire.Bind(new(service.KeyFetcher), new(kgs.RPC)),

    observabilitySet,
    authSet,

    mdservice.New,
    provider.NewGraphGophers,
    mdhttp.NewClient,
    mdrequest.NewHTTP,
    mdtimer.NewTimer,

    db.NewURLSql,
    db.NewUserURLRelationSQL,
    provider.NewRemote,
    url.NewRetrieverPersist,
    url.NewCreatorPersist,
    provider.NewKgsRPC,
    provider.NewReCaptchaService,
    requester.NewVerifier,
    graphql.NewShort,
  )
  return mdservice.Service{}, nil
}

Database Modeling

Entity Relation Diagram

Feature Toggle

Short employs feature toggles to modify system behavior without changing code. UI components controlled by the feature toggles are created inside a centralized UIFactory in order to avoid having nested if else statement across the code base:

// UIFactory.tsx
export class UIFactory {
  constructor(
    private featureDecisionService: IFeatureDecisionService
  ) {}

  public createGoogleSignInButton(): ReactElement {
    if (!this.featureDecisionService.includeGoogleSignButton()) {
      return <div />;
    }
    return (
      <GoogleSignInButton
        googleSignInLink={this.authService.googleSignInLink()}
      />
    );
  }

  public createGithubSignInButton(): ReactElement {
    if (!this.featureDecisionService.includeGithubSignButton()) {
      return <div />;
    }
    return (
      <GithubSignInButton
        githubSignInLink={this.authService.githubSignInLink()}
      />
    );
  }
}

Short also provides IFeatureDecisionService interface, allowing the developers to switch to dynamic feature toggle backend in the future by simply swapping the dependency injected.

// FeatureDecision.service.ts
export interface IFeatureDecisionService {
  includeGithubSignButton(): boolean;
  includeGoogleSignButton(): boolean;
  includeFacebookSignButton(): boolean;
}
// StaticConfigDecision.service.ts
import { IFeatureDecisionService } from './FeatureDecision.service';

export class StaticConfigDecisionService implements IFeatureDecisionService {
  includeGithubSignButton(): boolean {
    return false;
  }
  includeGoogleSignButton(): boolean {
    return false;
  }
  includeFacebookSignButton(): boolean {
    return true;
  }
}
// dep.ts
export function initUIFactory(
  ...
): UIFactory {
  ...
  const staticConfigDecision = new StaticConfigDecisionService();
  ...
  return new UIFactory(
    ...,
    staticConfigDecision
  );
}

You can read about the detailed feature toggle design on this article.

Search Engine Optimization

In order to improve the quality and quantity of the website's traffic, Short increases its visibility to web search engines through HTML meta tags.

<!-- ./frontend/public/index.html -->
<title>Short: Free online link shortening service</title>

<!-- Search Engine Optimization -->
<meta name="description"
      content="Short enables people to type less for their favorite web sites">
<meta name="robots" content="index, follow">
<link href="https://short-d.com" rel="canonical">

If you search short-d.com on Google, you should see Short shows up as the first result:

Google Search Result

Social Media Summary Card

Facebook & LinkedIn

Short leverages Open Graph tags to control what content shows up in the summary card when the website is shared on Facebook or LinkedIn:

<!-- ./frontend/public/index.html -->
<!-- Open Graph -->
<meta property="og:title" content="Short: Free link shortening service"/>
<meta property="og:description"
      content="Short enables people to type less for their favorite web sites"/>
<meta property="og:image"
      content="https://short-d.com/promo/small-tile.png"/>
<meta property="og:url" content="https://short-d.com"/>
<meta property="og:type" content="website"/>

Shared on Facebook:

Facebook Card

Shared on LinkedIn:

LinkedIn Card

Twitter

Twitter uses its own meta tags to determine what will show up when the website is mentioned in a Tweet:

<!-- Twitter -->
<meta name="twitter:card" content="summary_large_image"/>
<meta name="twitter:site" content="@byliuyang11"/>
<meta name="twitter:title" content="Short: Free link shortening service"/>
<meta name="twitter:description"
      content="Short enables people to type less for their favorite web sites"/>
<meta name="twitter:image" content="https://short-d.com/promo/twitter-card.png"/>

Twitter Card

Testing

The Importance Of Automation

Short is maintained by a small team of talented software engineers working at Google, Uber, and Vmware as a side project. The team wants to deliver new features faster without sacrificing its quality. Testing ever-increasing amount of features manually soon becomes impossible — unless we want to spend all our time with manual, repetitive work instead of delivering working features.

Test automation is the only way forward.

Testing Strategy

Test Strategy

Please read Testing Strategies in a Microservice Architecture for a detailed introduction on test strategies.

Unit Testing

A unit test exercises the smallest piece of testable software in the application to determine whether it behaves as expected.

Unit Test

Run unit tests for backend:

cd backend
./scripts/unit-test

Sociable And Solitary

Two Types of Unit Test

The FIRST Principal

  • [F]ast: Unit tests should be fast otherwise they will slow down development & deployment.
  • [I]ndependent: Never ever write tests which depend on other test cases.
  • [R]epeatable: A repeatable test is one that produces the same results each time you run it.
  • [S]elf-validating: There must be no manual interpretation of the results.
  • [T]imely/[T]horoughly: Unit tests must be included for every pull request of a new feature and cover edge cases, errors, and bad inputs.

Test Structure

A automated test method should be composed of 3As: Arrange, Act, and Assert.

  • [A]rrange: All the data needed for a test should be arranged as part of the test. The data used in a test should not depend on the environment in which the test is running.
  • [A]ct: Invoke the actual method under test.
  • [A]ssert: A test method should test for a single logical outcome.

Integration Testing

An integration test verifies the communication paths and interactions between components to detect interface defects.

Integration Test

Run integration tests for backend:

cd backend
./scripts/integration-test

Component Testing

A component test limits the scope of the exercised software to a portion of the system under test, manipulating the system through internal code interfaces and using test doubles to isolate the code under test from other components.

In Process

Component Test

Out Of Process

Component Test

Contract Testing

An integration contract test is a test at the boundary of an external service verifying that it meets the contract expected by a consuming service.

End To End Testing

An end-to-end test verifies that a system meets external requirements and achieves its goals, testing the entire system, from end to end.

The Test Pyramid

Test Pyramid

Deployment

Kubernetes

Short leverages Kubernetes to automate deployment, scaling, and management of containerized microservices.

Node overview

Short uses GitOps to manage Kubernetes cluster. GitOps

Staging

Merging pull request into master branch on Github will automatically deploy the changes to staging environment.

Production

Merging from master branch to production branch will automatically deploy the latest code to the production. This is called continuous delivery in the DevOps world.

Continuous Delivery

In the future, when there are enough automated tests, we may migrate to continuous deployment instead.

Continuous Deployment

Tools We Use

Contributing

Please read CONTRIBUTING.md for details on our code of conduct, the process for submitting pull requests to us, and our code review guideline.

Author

Harry Liu - Initial work - byliuyang

As the tech lead of Short, I am responsible for the overall planning, execution and success of complex software solutions to meet users' needs.

I deeply believe in and am striving to achieve the right column of the following diagram:

Manager vs Leader

License

This project is maintained under MIT license

You can’t perform that action at this time.