Dashboard: Document the list of all font project git repos that need to be automatically checked. #1241

felipesanches · 2017-03-09T18:47:30Z

After fixing #1240, I'd like to increase the list of font projects git repos being checked by the dashboard's worker container.

It seems to me that the canonical list of font projects to check against should be whatever we currently have at prod. I know it is 800-something families, but I don't know where to find an official list of them all other than automatically querying Google Fonts itself.

Is there a similar list for the git repos for each of those? I was able to compile a list of a bit more than 104 repos by scraping the google/fonts repo for URLs referencing other repos. (the list is here: #1158 (comment)) This would be roughtly only 13% of all projects. Does it mean 87% of the projects simply lack a git repo? Or the repos are there and I simply don't know exactly where?

davelab6 · 2017-03-10T22:36:03Z

They are scattered around github. Many don't have upstream repos set up yet but I have SRC data in the old hg repo or my emails archive lol A CSV or json or .py or whatever list of them needs to be maintained

…

On Mar 9, 2017 1:48 PM, "Felipe Corrêa da Silva Sanches" < ***@***.***> wrote: After fixing #1240 <#1240>, I'd like to increase the list of font projects git repos being checked by the dashboard's worker container. It seems to me that the canonical list of font projects to check against should be whatever we currently have at prod. I know it is 800-something families, but I don't know where to find an official list of them all other than automatically querying Google Fonts itself. Is there a similar list for the git repos for each of those? I was able to compile a list of a bit more than 40 repos by scraping the google/fonts repo for URLs referencing other repos. This would be roughtly only 5% of all projects. Does it mean 95% of the projects simply lack a git repo? Or the repos are there and I simply don't know exactly where? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1241>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAP9yyh9plMeSRJppH4Iy2E2OTPvMLRSks5rkElCgaJpZM4MYeoD> .

felipesanches · 2017-03-10T22:46:17Z

Sure! This kind of info is crucialy important. We definitely need to compile this list somewhere.

davelab6 · 2017-03-11T16:02:08Z

CSV should be fine

…

On Mar 10, 2017 5:46 PM, "Felipe Corrêa da Silva Sanches" < ***@***.***> wrote: Sure! This kind of info is crucialy important. We definitely need to compile this list somewhere. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1241 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAP9y0VxMxLQhhsY2TBM0xWbqFVK6rGdks5rkdK6gaJpZM4MYeoD> .

felipesanches · 2017-03-27T21:06:02Z

if by HG you mean Mercurial on Google Code, that's not readable anymore: https://code.google.com/archive/p/googlefontdirectory/source/default/source (I get a 404, actually!)

I also looked up this: https://github.com/blipvert/googlefontdirectory-hg
But I could not find the info in there as well.

@davelab6 Can you clarify where this kind of info could be found online? Or maybe it is only available in your personal data archives?

Extracting the list into a separate file: fontprojects.py (issue fonttools#1241)

…s, though. And some of these are binary only, so they still need to be inspected individually (issue fonttools#1241)

repos are enabled (not commented out) (issue fonttools#1241)

And annotation dozens of other repos with the issues found in each of them. (issue fonttools#1241)

so that we can generate things like this summary: ===================== Number of font repositories in this file: "TTX": 15 "OK": 41 "UFO": 3 "GH-PAGES": 2 "NOTE": 5 "GFONTS-REPO": 1 "OTF": 12 "?": 35 From the total of 114 repos, 46 are enabled. Enabled means: "OK" or "NOTE" status. (issue fonttools#1241)

Number of font repositories in this file: "TTX": 16 "OK": 47 "UFO": 18 "GH-PAGES": 2 "NOTE": 13 "GFONTS-REPO": 1 "OTF": 16 "404-ERROR": 1 "SOURCE-ONLY": 1 From the total of 115 repos, 60 are enabled. Enabled means: "OK" or "NOTE" status. (issue fonttools#1241)

(issue fonttools#1241)

font-family names, based on the foldernames in the google/fonts repo. We'll still have to fill the gaps in this table... Number of font repositories in this file: "TTX": 18 "OK": 46 "UFO": 17 "GH-PAGES": 2 "NOTE": 13 "GFONTS-REPO": 1 "OTF": 15 "404-ERROR": 1 "?": 802 "SOURCE-ONLY": 1 From the total of 916 repos, 59 are enabled. Enabled means: "OK" or "NOTE" status. (issue fonttools#1241)

Now we have 91 repos enabled. ==== Number of font repositories in this file: "TTX": 15 "OK": 76 "UFO": 18 "GH-PAGES": 2 "NOTE": 15 "GFONTS-REPO": 1 "OTF": 15 "404-ERROR": 1 "?": 777 "SOURCE-ONLY": 2 From the total of 922 repos, 91 are enabled. Enabled means: "OK" or "NOTE" status. (issue fonttools#1241)

davelab6 · 2017-04-03T16:07:21Z

@m4rc1e has made a Google Docs spreadsheet to collaborative live-edit this data, and there is a public CSV export URL to fetch the data here:

https://docs.google.com/spreadsheets/d/1ampzD9veEdrwUMkOAJkMNkftqtv1jEygiPR0wZ6eNl8/pub?gid=0&single=true&output=csv

I suggest setting up FB to pull from that URL :)

felipesanches · 2017-04-03T18:06:13Z

Sure! I've been chatting with @m4rc1e about that earlier today and that's our plan :-)

davelab6 · 2017-04-03T18:45:49Z

Regarding the hg repo, yes that's what I meant; lasse has a copy on his bitbucket.org/graphicore account iirc

…

On Apr 3, 2017 2:06 PM, "Felipe Corrêa da Silva Sanches" < ***@***.***> wrote: Sure! I've been chatting with @m4rc1e <https://github.com/m4rc1e> aobut that earlier today and that's our plan :-) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1241 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAP9yx9BQ8O5JkExOgfNGlXufNK2VJVaks5rsTUVgaJpZM4MYeoD> .

graphicore · 2017-04-03T18:48:51Z

it's here: https://bitbucket.org/lassefister/old-googlefontdirectory

felipesanches · 2017-04-03T19:37:02Z

@graphicore Is there any info about URLs of upstream font project repos in there ?

graphicore · 2017-04-03T19:39:14Z

@graphicore Is there any info about URLs of upstream font project repos in there ?

I don't know.

felipesanches · 2017-04-03T19:41:05Z

After a quick glance, I had the impression it does not have that kind of info in there. At least not consistently throughout the family folders. And it may also be outdated info, if any.

But thanks for pointing out the existence of the repo anyway! :-)

feed the container with data from a GoogleDocs spreadsheet. But we keep a cache file locally as well just in case (and for versioning purposes) (issue fonttools#1241)

(issue fonttools#1241)

otherwise it would re-run forever. (issue fonttools#1241)

(issue fonttools#1241)

Using git commit --depth={MAX_NUM_ITERATIONS} (issue fonttools#1241)

(issue fonttools#1241)

we'll later increase this value again, but now, for testing purposes, let's stick to the master of each font-repo. (issue fonttools#1241)

(issue fonttools#1241)

Cormorant SC, Amatica SC, etc... (issue fonttools#1241)

(issue fonttools#1241)

felipesanches · 2017-06-21T23:37:55Z

Migrating to fontbakery-dashboard issue tracker per #1383

felipesanches added the P0 Urgent label Mar 9, 2017

felipesanches self-assigned this Mar 9, 2017

felipesanches added a commit to felipesanches/fontbakery that referenced this issue Mar 29, 2017

working on the full list of font repos.

26a3bd6

Extracting the list into a separate file: fontprojects.py (issue fonttools#1241)

felipesanches added a commit to felipesanches/fontbakery that referenced this issue Mar 29, 2017

make the list of font repos tell us automatically how many

1ae6c2c

repos are enabled (not commented out) (issue fonttools#1241)

felipesanches added a commit to felipesanches/fontbakery that referenced this issue Mar 29, 2017

Raising active repos to a total of 46.

4ebdba5

And annotation dozens of other repos with the issues found in each of them. (issue fonttools#1241)

felipesanches mentioned this issue Mar 29, 2017

List of git repos for font projects #1262

Merged

felipesanches added a commit to felipesanches/fontbakery that referenced this issue Mar 29, 2017

comment naming the fields in the git repos list

d0f2423

(issue fonttools#1241)

felipesanches added a commit to felipesanches/fontbakery that referenced this issue Apr 4, 2017

updating cached list based on marc's spreadsheet

e6cde91

(issue fonttools#1241)

felipesanches added a commit to felipesanches/fontbakery that referenced this issue Apr 4, 2017

Update status/font-prefixes

fc4fa77

(issue fonttools#1241)

felipesanches added a commit to felipesanches/fontbakery that referenced this issue Apr 5, 2017

add fontprojects.csv to the dispatcher Dockerfile

b587cd5

(issue fonttools#1241)

felipesanches added a commit to felipesanches/fontbakery that referenced this issue Apr 5, 2017

add pylint cmd on Travis for the worker container script

3c9d5b5

(issue fonttools#1241)

felipesanches added a commit to felipesanches/fontbakery that referenced this issue Apr 5, 2017

fix font_prefix for Abhaya Libre repo

f8d28bb

(issue fonttools#1241)

felipesanches added a commit to felipesanches/fontbakery that referenced this issue Apr 5, 2017

fix anaheim & bevan font_prefixes

ab03407

(issue fonttools#1241)

felipesanches added a commit to felipesanches/fontbakery that referenced this issue Apr 5, 2017

fix cambay font_prefix

d65a2ec

(issue fonttools#1241)

felipesanches added a commit to felipesanches/fontbakery that referenced this issue Apr 5, 2017

exit value must be always zero in the worker container

41f382a

otherwise it would re-run forever. (issue fonttools#1241)

felipesanches added a commit to felipesanches/fontbakery that referenced this issue Apr 5, 2017

fix saving stats to database

6978aba

(issue fonttools#1241)

felipesanches added a commit to felipesanches/fontbakery that referenced this issue Apr 5, 2017

fix crash on the flask app

c8cc74b

(issue fonttools#1241)

felipesanches added a commit to felipesanches/fontbakery that referenced this issue Apr 5, 2017

A more efficient way of running a limited num of commits!

7cfb506

Using git commit --depth={MAX_NUM_ITERATIONS} (issue fonttools#1241)

felipesanches added a commit to felipesanches/fontbakery that referenced this issue Apr 5, 2017

fixing Inconsolata font_prefix

d8968da

(issue fonttools#1241)

felipesanches added a commit to felipesanches/fontbakery that referenced this issue Apr 5, 2017

reducing to a single commit per family for now

638258d

we'll later increase this value again, but now, for testing purposes, let's stick to the master of each font-repo. (issue fonttools#1241)

felipesanches added a commit to felipesanches/fontbakery that referenced this issue Apr 5, 2017

fixing Overpass font_prefix

3312d49

(issue fonttools#1241)

felipesanches added a commit to felipesanches/fontbakery that referenced this issue Apr 5, 2017

add a few more exceptions to name table rules, such as:

0e52a63

Cormorant SC, Amatica SC, etc... (issue fonttools#1241)

felipesanches added a commit to felipesanches/fontbakery that referenced this issue Apr 5, 2017

Added server logs page

1a6cd61

(issue fonttools#1241)

davelab6 modified the milestone: 0.2.0 Binary Checks Are Reliable Apr 18, 2017

felipesanches added P3 Soon and removed P0 Urgent labels Apr 25, 2017

felipesanches changed the title ~~Document the list of all font project git repos that need to be automatically checked.~~ Dashboard: Document the list of all font project git repos that need to be automatically checked. Apr 28, 2017

felipesanches modified the milestones: 0.3.0 Collection Dashboard Web UI, 0.2.0 Binary Checks Are Reliable Apr 28, 2017

davelab6 assigned graphicore and unassigned felipesanches May 8, 2017

felipesanches mentioned this issue Jun 21, 2017

Document the list of all font project git repos that need to be automatically checked. googlefonts/fontbakery-dashboard#15

Open

felipesanches closed this as completed Jun 21, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Dashboard: Document the list of all font project git repos that need to be automatically checked. #1241

Dashboard: Document the list of all font project git repos that need to be automatically checked. #1241

felipesanches commented Mar 9, 2017 •

edited

Loading

davelab6 commented Mar 10, 2017 via email

felipesanches commented Mar 10, 2017

davelab6 commented Mar 11, 2017 via email

felipesanches commented Mar 27, 2017

davelab6 commented Apr 3, 2017

felipesanches commented Apr 3, 2017 •

edited

Loading

davelab6 commented Apr 3, 2017 via email

graphicore commented Apr 3, 2017

felipesanches commented Apr 3, 2017

graphicore commented Apr 3, 2017

felipesanches commented Apr 3, 2017

felipesanches commented Jun 21, 2017

Dashboard: Document the list of all font project git repos that need to be automatically checked. #1241

Dashboard: Document the list of all font project git repos that need to be automatically checked. #1241

Comments

felipesanches commented Mar 9, 2017 • edited Loading

davelab6 commented Mar 10, 2017 via email

felipesanches commented Mar 10, 2017

davelab6 commented Mar 11, 2017 via email

felipesanches commented Mar 27, 2017

davelab6 commented Apr 3, 2017

felipesanches commented Apr 3, 2017 • edited Loading

davelab6 commented Apr 3, 2017 via email

graphicore commented Apr 3, 2017

felipesanches commented Apr 3, 2017

graphicore commented Apr 3, 2017

felipesanches commented Apr 3, 2017

felipesanches commented Jun 21, 2017

felipesanches commented Mar 9, 2017 •

edited

Loading

felipesanches commented Apr 3, 2017 •

edited

Loading