Generic, extensible package metadata file

[IainNZ]

I think the time may have come to expand from just REQUIRE and to move to a more rich file that contains various properties of a package.

A short list:

• Keywords
• Short description
• License
• Package precompilation options (ref WIP: static compile part 4 (user-interface) #8745 (comment))
• Coverage options (e.g. skip these files/lines, ref Mechanism for excluding files and lines. JuliaCI/Coverage.jl#55)
• More controversially: REQUIRE and test/REQUIRE's current contents

This is fairly simple data, just lists and strings, so ideally we can settle on a simple format. An option is to have the file be a Julia dict + array literal. Other thing to be bikeshed is the filename, and finally whether the file is reflected into METADATA.jl as well (like the current REQUIRE/requires situation).

[ScottPJones]

JSON maybe? Good ideas here

[JeffBezanson]

All else being equal, JSON would be a good choice. That would probably require putting a JSON parser in Base. Not necessarily a bad thing: it's arguably as important (or more) than csv or delimited files these days. .INI or java .properties are other candidates; very easy to parse.

[IainNZ]

We could bring JSON.jl into Base I suppose, the package is fairly stable.

[tkelman]

I think a declarative metadata format, a la conda, is a better install-time option for packages than what we have now with executing an arbitrary deps/build.jl script. Not thrilled about bringing a json parser into base, but json would be the natural format choice here. Perhaps by the time this kind of thing gets fleshed out we will have done more experimentation with installed-by-default packages.

[dcjones]

👍 to putting everything (including REQUIRE) in one package.json (or similar) file. With separate files for every piece of metadata we'll eventually want, and separate mechanisms for consuming that metadata, making packages is going to become complex and aggravating.

TOML is another option that's arguably nicer to write than JSON without being as bonkers as YAML.

[IainNZ]

TOML was definitely more what I was thinking of, instead of JSON, but there is utility I guess in that JSON is so widely recognized.

[kmsquire]

One benefit of TOML is that much of it could be directly parsed in Julia. In, e.g., Python, JSON would probably make more sense, since it's structure matches Python's syntax for dictionaries. In Julia, that takes more work. (Of course, there may not be much of a need to parse TOML as Julia statements, since the data might be stored in a dictionary anyway? JSON would be fine.)

[carnaval]

What about a julia literal you would just eval ?

[carnaval]

Oups sorry didn't read the whole issue

[StefanKarpinski]

What about just using files with simple formats – i.e. one line per keyword for keywords, a file that happens to contain markdown text for description, the license can be a file with a license indicator from a known set, etc.

[IainNZ]

Is anyone other than @carnaval and I into the Julia literal? It seems so extremely simple and lightweight.

[malmaud]

I'm more into it than I've ever been into anything
On Tue, Jun 30, 2015 at 5:21 PM Iain Dunning notifications@github.com
wrote:

Is anyone other than @carnaval https://github.com/carnaval and I into
the Julia literal? It seems so extremely simple and lightweight.

—
Reply to this email directly or view it on GitHub
#11955 (comment).

[dcjones]

It it possible/practical for it to be a julia literal without it becoming arbitrary julia code that needs to be executed to read a package's metadata?

[StefanKarpinski]

It it possible/practical for it to be a julia literal without it becoming arbitrary julia code that needs to be executed to read a package's metadata?

It is not – there is no such thing as JLSON (yet). This would require a modified / restricted parser that only handles the "safe" part of Julia that only constructs objects.

[JeffBezanson]

-1 for something turing complete. I suspect we'd regret that.

[carnaval]

JLSON could reuse the base parser, essentially doing a one pass over the AST constructing only safe literals (list,dict,symbol,numbers,strings). The function "parsing" the AST to the nested structure is probably 10 lines of code so it's probably lighter than having JSON in base.

[wildart]

I would like to see something similar to SBT syntax with Julia specifics.

[StefanKarpinski]

Completely agree that we'd very likely regret choosing a turing complete format for providing pure data.

[StefanKarpinski]

JLSON could reuse the base parser

Fair point, although there's some sketchy stuff like emitting macros for large numbers. Of course, if you can't define a macro, then it's not obvious how you can take advantage of that.

[tkelman]

For version number constraints specifically (#7763), a scheme like https://github.com/Masterminds/semver#basic-comparisons might be more intuitive and more flexible (in that it allows disjunctive constraints) than what we have right now.

Another thing we might need to consider is conditional dependencies (#6884) - some dependency package is only needed when running in a given Julia version, that kind of thing. Right now this requires maintaining separate branches in the dependency package for different Julia versions.

[IainNZ]

Tidying up old issues/PRs.