Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add console script entry point #1745

Merged
merged 11 commits into from
Apr 6, 2018
Merged

Add console script entry point #1745

merged 11 commits into from
Apr 6, 2018

Conversation

madig
Copy link
Contributor

@madig madig commented Mar 22, 2018

Instead of hand-rolling a main "fontbakery" script (c.f. #1738). This is a request for comments -- should I continue down this path?

The various Python bin/fontbakery-* scripts could be added as more entry points if you really want to keep e.g. fontbakery-check-* instead of having just one fontbakery script that serves as an entry point or everything else.

(Ignore the check-ufo-sources thing for now, will remove before merging if my other branch isn't merged first.)

@madig
Copy link
Contributor Author

madig commented Mar 22, 2018

Are the shell scripts needed (except bash_completion), do they need to stay in FB? Rewrite? Move somewhere else? What about the two test_* files?

@graphicore
Copy link
Collaborator

The various Python bin/fontbakery-* scripts could be added as more entry points

so we loose the fontbakery my-script interface. I'm opposed. I'd prefer a solution where we only have just a single command exported. We don't have that now, but that would be a real improvement.

Some random questions comming into mind:

  • What about the sh scripts?
  • Where do the new scripts live (hint https://github.com/googlefonts/fontbakery/tree/master/Lib/fontbakery/commands).
  • Will it be easier, harder or about the same effort to add new scripts?
  • Will the existing scripts be touched for this (needs review).

@graphicore
Copy link
Collaborator

Are the shell scripts needed

Well, they were needed at some point.

do they need to stay in FB?

Rather yes, it's good to have a place for things like this and when it is closely related to FB they are good here. We don't want to have people redo them all the time they need something similar. AFAIK this are tools @m4rc1e and @felipesanches are working with occasionally.

Rewrite?

Hard to justify, isn't it? Since they just work already. Sounds like an exercise. I'd prefer an idea how we can transition from old to new, without having to care too much about the old.

This is in bin, we could collect comments on these.

  • bash_completion
  • fb_summary_report.py
  • fontbakery
  • fontbakery-build-contributors.py
  • fontbakery-check-collection.sh
  • fontbakery-check-googlefonts.py
  • fontbakery-check-noto-version.sh
  • fontbakery-check-specification.py
  • fontbakery-generate-glyphdata.py
  • test_args.py
  • test_nametable-from-filename.py

Eventually I'd prefer a single fontbakery command, discovering available subcommands within fontbakery/Lib/fontbakery/commands/ and a convention how they are executed (e.g. they export a main function or they export a entrypoints dict or such).

With shell scripts (and others), I see the problem that the python module system is getting to its limits. Adding Non-Code Files could help (it's not python so it's Non-Code heh?)

@madig
Copy link
Contributor Author

madig commented Mar 22, 2018

so we loose the fontbakery my-script interface. I'm opposed.

Huh? We don't if we leave 'fontbakery=fontbakery.cli:main' in, but we can add more stuff if we wanted to. This is currently the only exported "entry point".

What about the sh scripts?

Nothing will happen to them I guess? The directory listing of my fontbakery venv on Windows looks like this:

Mode                LastWriteTime         Length Name
----                -------------         ------ ----
-a----       2018-03-20     12:57           2263 activate
-a----       2018-03-20     12:57            811 activate.bat
-a----       2018-03-20     12:57           8325 activate.ps1
-a----       2018-03-20     12:57           1137 activate_this.py
-a----       2018-03-20     12:57          89501 chardetect.exe
-a----       2018-03-20     12:57            508 deactivate.bat
-a----       2018-03-20     12:57          89510 easy_install.exe
-a----       2018-03-20     12:57          89510 easy_install-2.7.exe
-a----       2018-03-22     16:05            398 fontbakery
-a----       2018-03-22     16:05          65536 fontbakery.exe
-a----       2018-03-22     16:05            645 fontbakery.exe.manifest
-a----       2018-03-22     16:05            442 fontbakery-build-contributors.py
-a----       2018-03-22     16:05            763 fontbakery-check-collection.sh
-a----       2018-03-22     16:05            440 fontbakery-check-googlefonts.py
-a----       2018-03-22     16:05            279 fontbakery-check-noto-version.sh
-a----       2018-03-22     16:05            444 fontbakery-check-specification.py
-a----       2018-03-20     12:58            440 fontbakery-check-ufo-sources.py
-a----       2018-03-22     16:05            442 fontbakery-generate-glyphdata.py
-a----       2018-03-22     16:05            455 fontbakery-script.py
-a----       2018-03-20     12:57          89497 fonttools.exe
-a----       2018-03-20     12:58          89488 font-v.exe
-a----       2018-03-20     12:57          89482 pip.exe
-a----       2018-03-20     12:57          89482 pip2.7.exe
-a----       2018-03-20     12:57          89482 pip2.exe
-a----       2018-03-20     12:57          89496 pyftinspect.exe
-a----       2018-03-20     12:57          89494 pyftmerge.exe
-a----       2018-03-20     12:57          89495 pyftsubset.exe
-a----       2018-03-20     12:57          27136 python.exe
-a----       2018-03-20     12:57          27648 pythonw.exe
-a----       2018-03-20     12:57          89492 ttx.exe
-a----       2018-03-21     17:04          89490 ufodiff.exe
-a----       2018-03-20     12:57          89493 unidecode.exe
-a----       2018-03-20     12:57          89489 wheel.exe

All the current bin/* stuff is there, so you could ignore my work here even if it's merged and continue doing what you were doing before. If there should be only one script that relays everything, all shell scripts have to be rewritten or discarded.

Where do the new scripts live

Your hint is correct ;)

Will it be easier, harder or about the same effort to add new scripts?

As hard as making a new module in Lib/fontbakery/commands (and adding it in cli.py). The entry point stuff gets rid of the boilerplate seen in fontbakery-check-googlefonts.py, everything else stays the same.

Will the existing scripts be touched for this (needs review).

I didn't need to touch the specification checking scripts, not sure about the others... They'd have to be "ported" to commands, though.

@graphicore
Copy link
Collaborator

all shell scripts have to be rewritten or discarded.

How so? What about Popen?

@madig
Copy link
Contributor Author

madig commented Mar 22, 2018

Uhm, did I understand correctly that you would want fontbakery to export just one script that does everything? I guess you can install shell scripts somewhere in Lib and Popen them if you want to... would need a command in commands/ though. Might as well leave the script in bin/ then.

@graphicore
Copy link
Collaborator

Uhm, did I understand correctly that you would want fontbakery to export just one script that does everything?

Sure, why not? We already have it (last time I checked)

I guess you can install shell scripts somewhere in Lib

That's why I was referring to Adding Non-Code Files

would need a command in commands/ though

I'd build it into fontbakery directly. I don't see why we should settle for less than we have.

Might as well leave the script in bin/ then.

Yeah, there's that hack in https://github.com/googlefonts/fontbakery/blob/master/setup.py#L41 which installs all scripts in the current path. It's OK if you are in a virtualenv, but it's kind of ugly if you are not. So having all behind a single command is like a service to our users to keep their computers clean.

@madig
Copy link
Contributor Author

madig commented Mar 22, 2018

Alrighty then, will write something and push it here.

@madig
Copy link
Contributor Author

madig commented Mar 22, 2018

Quick check: does build-contributors work for you? I get a Unicode en/decode error upon writing doc. Looks like this is a one way of preparing Python 3 support ;)

@anthrotype
Copy link
Member

Adding Non-Code Files could help (it's not python so it's Non-Code heh?)

non-code as in data files. A shell script is code, should go with scripts setup keyword.

@graphicore
Copy link
Collaborator

Yeah, so I think the tricky part is to get pip into installing the scripts (Non-Code Files) and to figure out where to read them from. commands/scripts may be a place to put them, or even directly within commands, so we see which names are already used and can avoid collisions.

Name collisions may be a thing at some point in the future, i.e. to add power shell based scripts .ps1 and .sh commands of the same name could live side by side.

@madig
Copy link
Contributor Author

madig commented Mar 22, 2018

I think that would be more elegantly handled by installing these scripts in the path. Powershell and cmd.exe have a list of suffixes specified that describe executables and that the user can leave out. Executing "venv/scripts/activate" in Powershell automatically chooses the file with the ".ps1" suffix

@graphicore
Copy link
Collaborator

graphicore commented Mar 22, 2018

A shell script is code, should go with scripts setup keyword.

It's not code for pip I think. And I'm not sure if shell scripts are scripts for pip.

sys.argv[0] += " " + sys.argv[1]
del sys.argv[1] # Make this indirection less visible for subcommands.
cmd = importlib.import_module(subcommand)
sys.exit(cmd.main())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's basically like doing python -m some_module

@anthrotype
Copy link
Member

"scripts" can be anything. setuptools simply chmod +x and copy them to the bin or Scripts folder

@madig
Copy link
Contributor Author

madig commented Mar 22, 2018

I can "port" the Python scripts over, the shell scripts could stay in "scripts" until they're deprecated or replaced. The path will be less cluttered either way.



def main(args=None):
subcommands = {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we (maybe later) add an auto discovery mechanism to this. It would spare us from having to register each command. Also we have that kind of thing in bin/fontbakery so this would be a regression.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. I just need to look into discovering the submodules of a module :)

@graphicore
Copy link
Collaborator

I can "port" the Python scripts over, the shell scripts could stay in "scripts" until they're deprecated or replaced. The path will be less cluttered either way.

In essence, is having fontbakery my-shellscript as an interface to no value for you?

@anthrotype
Copy link
Member

anthrotype commented Mar 22, 2018

"scripts" can be anything.

ehm, well almost anything, but they must be text, not binary, see:

pypa/setuptools#210

(even though the afdko hacks its way to it somehow and installs binary native executables as scripts, but I wouldn't recommend it)

@anthrotype
Copy link
Member

having fontbakery my-shellscript as an interface to no value for you?

(stupid keyboard, I keep hitting "Comment" before finishing to write my comments..)

it's certainly cool, but doesn't work on a system where bash or powershell or whatever shell is not present. Maybe you could check if the shell is in the PATH and if not print a nice error message.

@madig
Copy link
Contributor Author

madig commented Mar 22, 2018

In essence, is having fontbakery my-shellscript as an interface to no value for you?

Ehm, it is? Just a thought on the way forward because you seem to be wary of additional work with little ROI.

@madig
Copy link
Contributor Author

madig commented Mar 22, 2018

What's this fb_summary_report thing for? Should I carry it over, too? Looks like a one-off script.

@graphicore
Copy link
Collaborator

Just trying to figure out what features to leave behind and for what reason. Me being wary about ROI was discussing doing ln -s fontbakery fontbakery.py vs. doing this PR. Since we're on this PR I want it to be done right.

What's this fb_summary_report thing for? Should I carry it over, too? Looks like a one-off script.

Looks old. Does it still work?

@graphicore
Copy link
Collaborator

it's certainly cool, but doesn't work on a system where bash or powershell or whatever shell is not present.

So we can either dump the feature or just issue the warning you suggested and use that occasion to replace it with a better compatible python version. After all, that would be a case where we know it's justified (because someone wants to use it).

I like the shell scripts, because I believe it's sometimes just very fast to make them. And if there's a low entry barrier to extending Font Bakery, it sounds good to me at first.

We can also over engineer a bit and if there are two commands my-script.sh and my-script.ps1 (.ps1 is powershell?), which would be a namespace collision anyways, we pick the appropriate version. Better than having multiple implementations would be having one made in python of course.

@madig
Copy link
Contributor Author

madig commented Mar 22, 2018

Looks old. Does it still work?

No idea. It expects a certain directory with certain files in it but has no documentation.

@felipesanches
Copy link
Collaborator

felipesanches commented Mar 22, 2018

I wrote that summary report in order to get a grasp of the overall quality of the gfonts collection by running check-googlefonts locally on the full collection with the --json output mode saving the individual reports in that directory. Then I used this tiny python script to compute the totals, displaying which files had problems and how many errors were reported.

I think this might still be useful, but it is also pretty quick to draft from scratch if anyone really needs that kind of summary report. This is the kind of thing that will be much better done with the fontbakery-dashboard, so I would actually be happy to see this deleted now.

@graphicore
Copy link
Collaborator

graphicore commented Mar 22, 2018

@felipesanches we could put it in a Snippets dir as fonttools has one. But your explanation from above is crucial.

Also, for Google Fonts, the dashboard will do it. But not everyone will set up the dashbooard for their own collection, at first at least unless it becomes easy and cheap.

@felipesanches
Copy link
Collaborator

yep! My explanation from above fills the "we lack documentation" gap ;-)
Sorry that I did not persist the info from my brain into a computer file earlier... :-P

@graphicore
Copy link
Collaborator

Sorry that I did not persist the info from my brain into a computer file earlier... :-P

🤔 Documentation is sometimes like a good wine, it needs to ripen. LOL

@graphicore
Copy link
Collaborator

This is what @madig would expect with (nargs='+')

(BTW, should we remove the There is a total of {n} checks on <Section ... output, it's at the bottom of specifications/googlefonts.py?)

$ fontbakery check-googlefonts 
There is a total of 76 checks on <Section: Google Fonts>.
There is a total of 15 checks on <Section: fontbakery.specifications.general>.
There is a total of 4 checks on <Section: fontbakery.specifications.cmap>.
There is a total of 3 checks on <Section: fontbakery.specifications.head>.
There is a total of 6 checks on <Section: fontbakery.specifications.os2>.
There is a total of 2 checks on <Section: fontbakery.specifications.post>.
There is a total of 8 checks on <Section: fontbakery.specifications.name>.
There is a total of 3 checks on <Section: fontbakery.specifications.hhea>.
There is a total of 1 checks on <Section: fontbakery.specifications.dsig>.
There is a total of 1 checks on <Section: fontbakery.specifications.hmtx>.
There is a total of 2 checks on <Section: fontbakery.specifications.gpos>.
There is a total of 1 checks on <Section: fontbakery.specifications.gdef>.
There is a total of 1 checks on <Section: fontbakery.specifications.kern>.
There is a total of 2 checks on <Section: fontbakery.specifications.glyf>.
There is a total of 1 checks on <Section: fontbakery.specifications.prep>.
There is a total of 6 checks on <Section: fontbakery.specifications.fvar>.
usage: fontbakery-check-googlefonts.py [-h] [-c CHECKID] [-v] [-l LOGLEVEL]
                                       [-m LOGLEVEL_MESSAGES] [-n] [-C] [-L]
                                       [--json JSON_FILE] [-g ITERATED_ARG]
                                       [-o ORDER]
                                       fonts [fonts ...]
fontbakery-check-googlefonts.py: error: too few arguments

@felipesanches
Copy link
Collaborator

I would simply remove "There is a total of " but keep the rest of the info...

@felipesanches
Copy link
Collaborator

Regarding the --list-checks problem, yes I agree. I also had noticed that problem in the past. Isn't there a way to declare multiple choices of cmdline argument rules ?

@graphicore
Copy link
Collaborator

Some of them actually pass. 😁

I had a quick look, e.g.

in cmap.py

@check(id='com.google.fonts/check/013')
def com_google_fonts_check_013(ttFonts):
  """Fonts have equal unicode encodings?"""

Will also run with an empty list of ttFonts. Could be cured with:

@check(
    id='com.google.fonts/check/013',
    conditions=['ttFonts'])
def com_google_fonts_check_013(ttFonts):
  """Fonts have equal unicode encodings?"""

@graphicore graphicore mentioned this pull request Apr 5, 2018
@davelab6
Copy link
Contributor

davelab6 commented Apr 5, 2018 via email

@madig
Copy link
Contributor Author

madig commented Apr 6, 2018

Hrm. check-googlefonts won't return an error code with the current cure. I also don't like how all the test numbers are printed before the usage is displayed; ideally, argument checking comes first and the script aborts.

@graphicore
Copy link
Collaborator

Hrm. check-googlefonts won't return an error code with the current cure.

In what case, without fonts, so you ask for a sys.exec(1) (easy!)? That should also be true for all other specs that use FontSpec, i.e. also for check-specification.

I also don't like how all the test numbers are printed before the usage is displayed;

That looks to me like it was always just a cheap hack in googlefonts.py. We could easily put it into the check-specification command. As it is now it's printed when the module is loaded, clearly not ideal.

@madig
Copy link
Contributor Author

madig commented Apr 6, 2018

In what case, without fonts, so you ask for a sys.exec(1) (easy!)?

Basically, argparse should explode when it doesn't get what it wants. That will exit nicely.

That looks to me like it was always just a cheap hack in googlefonts.py. We could easily put it into the check-specification command.

check-specification and check-ufo-sources exit as expected, so, uh, would have to look into it.

@graphicore
Copy link
Collaborator

Basically, argparse should explode when it doesn't get what it wants. That will exit nicely.

But argparse is getting what it wants. Read the source, Luke.

check-specification and

check-ufo-sources work as expected, so, uh, would have to look into it.

check-ufo-sources fails via argparse because it needs a specification argument

$ fontbakery check-specification fontbakery.specifications.fvar && echo 'GOOD'
Validation of expected values failed:
fonts: Value is empty. (value: ())
usage: fontbakery-check-specification.py [-h] [-c CHECKID] [-v] [-l LOGLEVEL]
                                         [-m LOGLEVEL_MESSAGES] [-n] [-C] [-L]
                                         [--json JSON_FILE] [-g ITERATED_ARG]
                                         [-o ORDER]
                                         specification [fonts [fonts ...]]
GOOD

check-ufo-sources (UFOSpec) doesn't use nargs="*" for the fonts argument

@madig
Copy link
Contributor Author

madig commented Apr 6, 2018

Oops, overlooked your message above. I think '+' should definitely stay. We could introduce a new command fontbakery list-checks <specification> instead. Then we can also convert --list-commands to a proper list-commands

@graphicore
Copy link
Collaborator

In any case, we should use sys.exit(1) when user input validation fails.

We could introduce a new command fontbakery list-checks instead.

Or hack the argument parser in a way that it knows when to expect fonts. A new command must share some thngs with check-specification, but check-googlefonts already imports from check-specification so that would fine.

@madig
Copy link
Contributor Author

madig commented Apr 6, 2018

Mergy merge? 👷🔧

@felipesanches felipesanches merged commit 18c3e88 into fonttools:master Apr 6, 2018
@felipesanches
Copy link
Collaborator

thanks, @madig and @graphicore !

@@ -192,6 +192,9 @@ def main(specification=None, values=None):
values_[key] = getattr(args, key)

try:
if not args.fonts:
raise ValueValidationError, "No files to check specified."
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I dislike this for two reasons:

  1. now a spec has to have a font's argument, check-specification was clean of that before and specifications could define their dependencies much more freely.
  2. There's a mechanism that raises exactly this error and ends the script (with this PR also with sys.exit(1) So why do it two times?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed in #1777

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants