Allow to depend on simple F# or C# code files

[forki]

In F# land we often want to share one or two F# files between multiple repositories. It would be cool to have support for that.

Not sure if nuget is the best way for that.

[agross]

Some people call these minimods. They use the normal nuget infrastructure but install source files and add them to the repo.

[isaacabraham]

I would prefer to see a more lightweight mechanism - if it's just a couple of files on e.g. github, why not just reference them directly in the paket configuration file?

[forki]

We would somehow still want to reference specific versions or version
ranges.
On Aug 31, 2014 4:31 PM, "Isaac Abraham" notifications@github.com wrote:

I would prefer to see a more lightweight mechanism - if it's just a couple
of files on e.g. github, why not just reference them directly in the paket
configuration file?

—
Reply to this email directly or view it on GitHub
#9 (comment).

[agross]

Actually, bundler has something like that. You can specify a git source for a gem and even tell it to track master or a specific git commit.

[forki]

Yes. That's what we should implement. Any links?
On Aug 31, 2014 4:43 PM, "Alexander Groß" notifications@github.com wrote:

Actually, bundler has something like that. You can specify a git source
for a gem and even tell it to track master or a specific git commit.

—
Reply to this email directly or view it on GitHub
#9 (comment).

[agross]

Bundler docs ;-)

It also support paths:
gem 'foo', :path => '/somewhere'

Git looks very similar.

[agross]

http://bundler.io/git.html

[isaacabraham]

Only thing that I'm not clear is that bundler still looks for a .gemspec file in the repo. I would also like to see the ability to simply reference particular files (or folders) against a particular git commit etc.

[forki]

Do we need a git implementation for this? Or is http download enough?

[isaacabraham]

Bundler talks about avoiding http re: man in middle. But I'm of the opinion to start simple, just use http to start with - can always move to something else if needed?

[agross]

Bundler requires git in path and just calls the executable.

[isaacabraham]

@agross would you need to provide auth then?

[agross]

@isaacabraham I never used bundler with authenticated git repos. I think the git defaults would work if you have the keypairs for SSH access available. It seem's there's a solution for authenticated HTTP repos as well: https://gist.github.com/masonforest/4048732 Other than that, you can still embed username and password in the HTTP URL (if you use Gitlab, for example).

[forki]

git in path is OK for me. We do the same in FAKE.
On Aug 31, 2014 6:16 PM, "Alexander Groß" notifications@github.com wrote:

@isaacabraham https://github.com/isaacabraham I never used bundler with
authenticated git repos. I think the git defaults would work if you have
the keypairs for SSH access available. It seem's there's a solution for
authenticated HTTP repos as well:
https://gist.github.com/masonforest/4048732 Other than that, you can
still embed username and password in the HTTP URL (if you use Gitlab, for
example).

—
Reply to this email directly or view it on GitHub
#9 (comment).

[ilkerde]

I second this proposal and would love to see this feature. As for the source, I think git (and using it with git in path as prereq) is fair enough. AFAIK bundler, bower and composer do all the same. One nicety to note here actually is the support of semver-style git tags (as bower does, see docs).

[forki]

who wants to work on this?

[isaacabraham]

I’d be happy to have a crack at this but might not be able to start for a couple of days. Given that I’m not as intimately familiar with the entire Paket codebase, maybe a good way would be for me to just do something decoupled like the code that can retrieve the source files from git e.g.

(repo:string, filePath:string) -> seq<async>

or something like that?

From: Steffen Forkmann [mailto:notifications@github.com]
Sent: 10 September 2014 19:05
To: fsprojects/Paket
Cc: Isaac Abraham
Subject: Re: [Paket] Allow to depend on simple F# or C# code files (#9)

who wants to work on this?

—
Reply to this email directly or view it on GitHub #9 (comment) . https://github.com/notifications/beacon/1781813__eyJzY29wZSI6Ik5ld3NpZXM6QmVhY29uIiwiZXhwaXJlcyI6MTcyNTk5MTQ4MiwiZGF0YSI6eyJpZCI6NDExNTgyMzJ9fQ==--3616df338c2757faabf76510964edc114375b224.gif

[forki]

sounds good.

As already discussed you can assume that git is in the path if you need it. Also take a look at https://github.com/fsharp/FAKE/tree/master/src/app/FakeLib/Git and copy functions over if you need.

[bartelink]

I'd have a look at NuGet source code packages; here's a good writeup.

A .nuspec to the project emitting the code and a NuGet pack to the build allows one to carry semantic info along with files. Then they can be first class and e.g. local edits can be detected (see #65 - I mention source file substitutions in there too ).

Having said all that, I'd love to abuse the scheme envisaged above too. But I'd feel very dirty if I did :(

[forki]

Very cool, but please create a separate issue so that we can keep track of
this.
On Sep 10, 2014 8:52 PM, "bartelink" notifications@github.com wrote:

I'd have a look at NuGet source code packages; here's a good writeup
http://nikcodes.com/2013/10/23/packaging-source-code-with-nuget/.

A .nuspec to the project emitting the code and a NuGet pack to the build
allows one to carry semantic info along with files. Then they can be first
class and e.g. local edits can be detected (see #65
#65 - I mention source file
substitutions in there too
#65 (comment) ).

Having said all that, I'd love to abuse the currently described version.
But I'd feel very dirty if I did :(

—
Reply to this email directly or view it on GitHub
#9 (comment).

[bartelink]

@forki Not sure what you mean. Paket doesn't need to do anything new (and the post I cite is conventions for authors which tie things up nicely but ultimately don't affect consumption).

#65 deals with installing source packages - specifically expanding Content. While a naive/minimal impl might not cover the substitutions aspect (and I could log an issue re that), it will need to consider what to do about overwriting files that have been changed (not sure how native NuGet does this but pretty sure it involves a cross check against the current version as part of uninstallation).

... can you explain what you'd like an issue for? happy to do it once that's clear :)

(Logged some notes on #65 about stuff around this)

[isaacabraham]

Confused. What we're aiming for I thought is a lightweight binding to individual (or paths) of files in github. What's the relationship between that and nuget packages?

[bartelink]

@isaacabraham @forki Still awaiting clarification from Steffen; My last comment was a reaction to "please create a separate issue so that we can keep track of
this."

As I see it:

1. source packages exist and are not as heavy as you think
   • They have manifests
   • They have versions
2. Paket already needs to handle install/update of source packages
3. They are not hard to generate
   • All you need to do to generate them is a .nuspec and run NuGet pack *.nuspec at the end of a build
   • one could use NuGet libs to compose them or one could do it in code oneself without too much work (it'd just be a zip with a nuspec and the files AFAIK)

Therefore, in designing this feature its important to consider overlaps with source packages

1. definitely don't build too much into this feature that can be handled via source packages if it complicates the offering as a whole in any way
2. at a minimum, perhaps there's terminology to share
3. perhaps this feature could download the files and synthesize a package into the package storage area
   • downstream its packages all the way down
   • metadata could be embedded into the faux-package which allows conditional gets for the retrieval of updates etc.
   • perhaps such a wrapper service should exist / already exists on the net (I'm thinking of @maartenba 's nuget signed version of a (package?DLL?) generator (have read 2 tweets but nothing else about it to date))

[isaacabraham]

The point of dependencies on source code is that there's no requirement for the owner of the code to make a source package. Anyone can be dependent on any code in github - the versioning can be achieve based on a git commit tag.

As @forki says - this sounds like a separate feature to what we've been proposing here.

[forki]

I see two different features:

1. including plain old C# or F# files from website like github (this issue
   is meant to represent this)
2. source code packages which rely on nuget infrastructure (new issue)
   On Sep 11, 2014 12:26 PM, "bartelink" notifications@github.com wrote:

@isaacabraham https://github.com/isaacabraham @forki
https://github.com/forki Still awaiting clarification from Steffen.

My last comment was a reaction to "please create a separate issue so that
we can keep track of
this."

As I see it:

1. source packages exist and are not as heavy as you think

   • They have manifests
   • They have versions
   • All you need to do to generate them is a .nuspec and run NuGet
     pack *.nuspec at the end of a build
     1. They are not hard to generate - one could use NuGet libs to
        compose them or one could do it in code oneself without too much work (it'd
        just be a zip with a nuspec and the files AFAIK)

2. Paket already needs to handle source packages

3. In designing this feature, consider overlaps with source packages

   • definitely don't build too much into this feature that can be
     handled via source packages
   • perhaps there's terminology to share
   • perhaps this feature could download the files and yield a package
     into the package storage area

    - downstream its packages all the way down
    - metadata could be embedded into the faux-package which allows
    conditional gets for the downloads etc.
    - perhaps such a wrapper service should exist / aready exists on
    the net (I'm thinking of @maartenba
    <https://github.com/maartenba> 's nuget signing generator (have
    read 2 tweets but nothing else about it to date))
   

—
Reply to this email directly or view it on GitHub
#9 (comment).

[ilkerde]

+1 @forki

One thing to note for this issue: We should enable having a dependency on any file, even those with no metadata (i.e. no git tag).

[bartelink]

@forki @isaacabraham I dont disagree but

1. I think its it's important that the potential impl approach of generating faux packages to not force any downstream complexity/unecessary bifurcation of code paths be considerd
2. I think its important that to keep this feature minimal, one should consider that there's a point where source packages are the right tool for the job

Based on the above all I'm saying is that an impl and design of this feature should not be blinkered to the presence of source packages

[bartelink]

@ilkerde maybe on the local file system / unc path too ?

(Maybe impl can share a general location specification with package sources to some degree?)

[ilkerde]

@bartelink I think it's enough for the time being just to have one concrete impl (in this case sth. like fetching a file from a github repo).

Taking additional sources into consideration feels like premature generalization to me.

[isaacabraham]

I had a quick play with the GitHub website (not the rate-limited GitHub API), and it's easy enough just to download a single file - not a folder - with a straight HTTP GET. You can specify any commit, branch or tag in the URI.

If this is enough to start with then I'll go ahead and do it.

[bartelink]

@ilkerde Fair enough, makes sense.

On reflection, while source needs to deal with a http or a local or a unc path, what it then in turn does is pretty different to what this needs to do.

The main point I'd make is that, at the point where this would support other sources, where possible the syntax for specifiyng them should be shared insofar as that principle makes sense.

[forki]

Yes depending on a single F# file would be a good start and actually help
the F# community a lot. That's for sure
On Sep 11, 2014 10:22 PM, "bartelink" notifications@github.com wrote:

@ilkerde https://github.com/ilkerde Fair enough, makes sense.

On reflection, while source needs to deal with a http or a local or a unc
path, what it then in turn does is pretty different to what this needs to
do.

The main point I'd make is that, at the point where this would support
other sources, where possible the syntax for specifiyng them should be
shared insofar as that principle makes sense.

—
Reply to this email directly or view it on GitHub
#9 (comment).

[agross]

Just for your reference regarding source: bundler does not use source when pulling from git. It rather uses some parameters to the gemmethod, probably adding them to an internal list of sources (haven't checked).

gem 'nokogiri', :git => 'https://github.com/tenderlove/nokogiri.git'
gem 'rails', :git => 'https://github.com/rails/rails.git', :ref => '4aded'
gem 'rails', :git => 'https://github.com/rails/rails.git', :branch => '2-3-stable'
gem 'rails', :git => 'https://github.com/rails/rails.git', :tag => 'v2.3.5'

There are also some nice built-in git sources for GitHub, Gist and Bitbucket:

gem 'rails', :github => 'rails'

And you can define new ones as well:

git_source(:stash){ |repo_name| "https://stash.corp.acme.pl/#{repo_name}.git" }
gem 'rails', :stash => 'forks/rails'

http://bundler.io/git.html

[isaacabraham]

@forki before I go any further could you review https://github.com/isaacabraham/Paket/tree/github-files?

• I've modified the dependencies file format to allow capturing single files as dependencies.
• There's no change to the lockfile because there's no indirect dependencies on a source file (I'm not parsing the file to see what other files are required!).
• The files are downloaded and copied into a path underneath the location of the dependencies file.

I still have to do the .proj modification stuff as well as removing dependencies - but first just make sure you're happy with how it looks so far. I've modified the sample dependency file in the C# test project to illustrate how it works.

[forki]

I can't see any diff. Can you please create a pull request and mark it as
WIP? Thanks.

Btw: I think we have to add it also to the lockfile since this is the only
thing we read during a normal install.
On Sep 12, 2014 12:37 AM, "Isaac Abraham" notifications@github.com wrote:

@forki https://github.com/forki before I go any further could you
review https://github.com/isaacabraham/Paket/tree/github-files?

• I've modified the dependencies file format to allow capturing single
  files as dependencies.
• There's no change to the lockfile because there's no indirect
  dependencies on a source file (I'm not parsing the file to see what other
  files are required!).
• The files are downloaded and copied into a path underneath the
  location of the dependencies file.

I still have to do the .proj modification stuff as well as removing
dependencies - but first just make sure you're happy with how it looks so
far.

—
Reply to this email directly or view it on GitHub
#9 (comment).

[isaacabraham]

K will do.

---

From: Steffen Forkmannmailto:notifications@github.com
Sent: ‎12/‎09/‎2014 07:19
To: fsprojects/Paketmailto:Paket@noreply.github.com
Cc: Isaac Abrahammailto:isaac.abraham@gmail.com
Subject: Re: [Paket] Allow to depend on simple F# or C# code files (#9)

I can't see any diff. Can you please create a pull request and mark it as
WIP? Thanks.

Btw: I think we have to add it also to the lockfile since this is the only
thing we read during a normal install.
On Sep 12, 2014 12:37 AM, "Isaac Abraham" notifications@github.com wrote:

@forki https://github.com/forki before I go any further could you
review https://github.com/isaacabraham/Paket/tree/github-files?

• I've modified the dependencies file format to allow capturing single
  files as dependencies.
• There's no change to the lockfile because there's no indirect
  dependencies on a source file (I'm not parsing the file to see what other
  files are required!).
• The files are downloaded and copied into a path underneath the
  location of the dependencies file.

I still have to do the .proj modification stuff as well as removing
dependencies - but first just make sure you're happy with how it looks so
far.

—
Reply to this email directly or view it on GitHub
#9 (comment).

---

Reply to this email directly or view it on GitHub:
#9 (comment)

[isaacabraham]

Here's the changes BTW for the moment (without the lockfile stuff):

https://github.com/isaacabraham/Paket/compare/github-files

[isaacabraham]

So currently we can import single files from GitHub into projects. Is anyone looking at taking this forward - I think we need to still look at removal of files (probably a Packet=true tag will be needed on individual files) as well as how files interoperate with the references file / individual projects.

[forki]

yes I'm currently dogfooding this and will open new issues.