Permalink
Branch: master
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
222 lines (203 sloc) 16 KB

Top angular directives on github, including custom directives

Introduction

All github contents recently got query-able by the Google BigQuery. See the announcement from github. You can execute the BigQuery queries at the BigQuery console.

I used it to find top angular directives used on github. In first two sections I only list “ng-” directives. Even if parsing html with regexps is faulty, it is correct enough for frequency analysis.

Third section, Custom directives, uses a heuristic to find custom angular directives. You can use it to find popular directives you have not been aware of.

Revision history of this post is stored on github.

You may also look at my other posts analyzing github using BigQuery.

Unique usage per repository in top repositories with example usage

Counting unique usage is expensive and I am out of my free quota for this month. This query runs on a smaller table that only contains 10% random sample of files from top 130k top popular repositories.

It will calculate “how many unique repos use this directive” and add a link to an example usage.

SELECT
  REGEXP_EXTRACT(line, r".*[ <]+(ng-[a-zA-Z0-9-]+).*") AS line,
  COUNT(DISTINCT(sample_repo_name)) AS count_distinct_repos,
  CONCAT("https://github.com/",
      FIRST(sample_repo_name),
      "/blob/",
      REGEXP_EXTRACT(FIRST(sample_ref), r"refs/heads/(.*)$"),
      "/",
      FIRST(sample_path)) AS example_url,
FROM (
  SELECT
    SPLIT(content, '\n') line,
    sample_repo_name,
    sample_path,
    sample_ref
  FROM
    [bigquery-public-data:github_repos.sample_contents]
  WHERE
    (sample_path LIKE '%.html'
      OR sample_path LIKE '%.ng')
  HAVING
    line CONTAINS 'ng-')
GROUP BY
  1
ORDER BY
  count_distinct_repos DESC
LIMIT
  500;

Full results on google docs. Top 20 results:

linecount_distinct_reposexample_url
ng-click1040https://github.com/georchestra/georchestra/blob/15.12/ldapadmin/src/main/webapp/privateui/lib/angular/docs/partials/guide/dev_guide.services.injecting_controllers.html
ng-repeat827https://github.com/claroline/Distribution/blob/master/main/core/Resources/modules/facets/Partial/panel_roles_form.html
ng-model823https://github.com/georchestra/georchestra/blob/15.12/ldapadmin/src/main/webapp/privateui/lib/angular/docs/partials/guide/dev_guide.services.injecting_controllers.html
ng-show662https://github.com/BoxUpp/boxupp/blob/master/page/templates/vmConfigurations.html
ng-class560https://github.com/streamdataio/streamdataio-js/blob/master/stockmarket-angular/index.html
ng-controller559https://github.com/georchestra/georchestra/blob/15.12/ldapadmin/src/main/webapp/privateui/lib/angular/docs/partials/guide/dev_guide.services.injecting_controllers.html
ng-if536https://github.com/Groupmates-co/groupmates/blob/master/app/assets/javascripts/groupmates/mates/mates-tpl.html
ng-app421https://github.com/streamdataio/streamdataio-js/blob/master/stockmarket-angular/index.html
ng-hide290https://github.com/BoxUpp/boxupp/blob/master/page/templates/vmConfigurations.html
ng-disabled287https://github.com/JekyllWriter/JekyllWriter/blob/master/layout/proxy.html
ng-submit222https://github.com/Groupmates-co/groupmates/blob/master/app/assets/javascripts/groupmates/mates/mates-tpl.html
ng-src211https://github.com/Groupmates-co/groupmates/blob/master/app/assets/javascripts/groupmates/mates/mates-tpl.html
ng-include205https://github.com/pixelpark/ppnet/blob/master/app/views/map.html
ng-change198https://github.com/mmautner/github-email-thief/blob/master/app/views/search_codes.html
ng-href187https://github.com/asm-products/octobox/blob/master/public/views/content/file/modal.html
ng-bind173https://github.com/BoxUpp/boxupp/blob/master/page/templates/vmConfigurations.html
ng-bind-html150https://github.com/LeoLombardi/tos-laimas-compass/blob/master/tos-laimas-compass-win32-x64/resources/app/node_modules/ui-select/docs/examples/demo-select2-with-bootstrap.html
ng-init146https://github.com/asm-products/octobox/blob/master/public/views/content/file/modal.html
ng-view145https://github.com/TheWildHorse/MovieNight/blob/master/public/index.html
ng-options142https://github.com/mmautner/github-email-thief/blob/master/app/views/search_codes.html

Top usages in all repositories

Previous query was looking only at the sample of data. This query looks through all files accessible on github.

SELECT
  TOP(line, 500),
  COUNT(*) AS c
FROM (
  SELECT
    REGEXP_EXTRACT((SPLIT(contents.content, '\n')),
          r".*[^a-zA-Z](ng-[a-zA-Z0-9-]+).*") line,
    contents.id AS id
  FROM
    [bigquery-public-data:github_repos.contents] AS contents
  JOIN (
    SELECT
      path,
      id
    FROM
      [bigquery-public-data:github_repos.files]
    WHERE
      path LIKE '%.ng'
      OR path LIKE '%.html') AS files
  ON
    (contents.id == files.id)
  HAVING
    line CONTAINS "ng-");

Full results on google docs. Top 20 results:

directivecount
ng-click1572920
ng-model1355222
ng-show962245
ng-repeat697010
ng-if601903
ng-controller591669
ng-app460875
ng-class452863
ng-bind283218
ng-hide217121
ng-disabled168468
ng-include125913
ng-init125508
ng-submit118507
ng-switch-when111254
ng-href109513
ng-src108365
ng-template108241
ng-change101197
ng-bind-html89604

Custom directives

Methodology

I tried a heuristic for finding custom directives - extract all html tags and look at relative frequency in all html files vs “probably angular html” files.

“Probably angular html” is based on the assumption that “ng-” is ubiquitous in angular html, but not that frequent otherwise. Also some angular files use the .ng extension. This method is going to obviously have some false positives and negatives. Looking through results, 2.0 ratio was optimal. Here you can see top 50 results that were right past the edge of exclusion - ratio was between 2.0 and 2.5. Very few entries are legitimate.

I am again using the sampled sample_contents, since I ran out of free quota.

BigQuery query

SELECT
  tag,
  COUNT(1) / SUM(IF(probably_angular, 1, 0)) AS html_to_angular_ratio,
  COUNT(DISTINCT(sample_repo_name)) AS distinct_repository_count,
  CONCAT("https://github.com/",
      FIRST(sample_repo_name),
      "/blob/",
      REGEXP_EXTRACT(FIRST(sample_ref), r"refs/heads/(.*)$"),
      "/",
      FIRST(sample_path)) AS example_url
FROM (
  SELECT
    SPLIT(REGEXP_REPLACE(
        REGEXP_REPLACE(content, r"['\"\\\/\$]+[a-zA-Z-]*", ""), 
        r"[^a-zA-Z-]+", " "), " ") AS tag,
    (REGEXP_MATCH(content,
         r".*[ <]+ng-[a-zA-Z0-9-]+.*")
     OR (sample_path LIKE '%.ng')) AS probably_angular,
    sample_repo_name,
    sample_path,
    sample_ref
  FROM
    [bigquery-public-data:github_repos.sample_contents]
  WHERE
    (sample_path LIKE '%.html'
      OR sample_path LIKE '%.ng'))
GROUP BY
  1
HAVING
  html_to_angular_ratio < 2.0
ORDER BY
  distinct_repository_count DESC
LIMIT 1000;

Results

All results in google docs, including ng- entries.

For example, it found directives from ionic framework, bootstrap or ng-file-upload. Top 20 results excluding the entries that start with ng- or *angular*:

taghtml_to_angular_ratiodistinct_repository_countexample_url
translate1.9265236377444466729https://github.com/getlantern/lantern/blob/devel/lantern-ui/app/index.html
novalidate1.625251https://github.com/bonitasoft/bonita-ui-designer/blob/master/frontend/app/js/assets/asset-popup.html
ui-sref1.1605095541401274195https://github.com/tatool/tatool-web/blob/master/app/views/doc/dev-executable-additional.html
orderBy1.345821325648415194https://github.com/georchestra/georchestra/blob/15.12/ldapadmin/src/main/webapp/privateui/lib/angular/docs/partials/tutorial/step_06.html
ui-view1.2525252525252526146https://github.com/cityofasheville/simplicity-ui/blob/master/app/index.html
ion-content1.4666666666666666126https://github.com/gaplo917/hkepc-ionic-reader/blob/master/www/templates/features/mypost/my.post.html
endbuild1.6071428571428572105https://github.com/kwk/docker-registry-frontend/blob/v2/app/index.html
md-button1.124605678233438498https://github.com/deltaepsilon/quiver-cms/blob/master/app/views/address-dialog.html
ion-view1.16778523489932992https://github.com/gaplo917/hkepc-ionic-reader/blob/master/www/templates/features/mypost/my.post.html
glyphicon-edit1.883268482490272385https://github.com/uclouvain/osis/blob/dev/dissertation/templates/manager_dissertations_detail.html
data-angularjs1.73684210526315876https://github.com/georchestra/georchestra/blob/15.12/ldapadmin/src/main/webapp/privateui/lib/angular/docs/partials/guide/i18n.html
md-content1.163157894736842273https://github.com/nozelrosario/Dcare/blob/master/www/views/vitals/trend.html
limitTo1.193548387096774271https://github.com/kwk/docker-registry-frontend/blob/v2/app/index.html
layout-align1.06515580736543970https://github.com/nozelrosario/Dcare/blob/master/www/views/vitals/trend.html
ngIf1.921146953405017869https://github.com/johnpapa/angular2-go/blob/master/app/characters.component.html
md-icon1.227586206896551761https://github.com/radioit/radioit-desktop/blob/master/app/static/view/bangumi.detail.html
md-toolbar1.140186915887850460https://github.com/ctodmia/africaexpress/blob/master/client/views/shopView.html
ion-list1.786666666666666660https://github.com/gaplo917/hkepc-ionic-reader/blob/master/www/templates/features/mypost/my.post.html

My other posts analyzing github using BigQuery

You may also take a look at my other posts: