Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Packages with !include in them don't work with Pantry. #4906

Closed
myShoggoth opened this issue Jun 25, 2019 · 8 comments
Closed

Packages with !include in them don't work with Pantry. #4906

myShoggoth opened this issue Jun 25, 2019 · 8 comments

Comments

@myShoggoth
Copy link

@myShoggoth myShoggoth commented Jun 25, 2019

With the new Pantry system taking package dependencies that don't have a top level .cabal or package.yaml file fail.

Example:

- git: git@github.com:kim/opentracing.git
  commit: b6e12de671f51f39ea2a9230e43cef2badd993f4
  subdirs:
    - opentracing
    - opentracing-zipkin-common
    - opentracing-zipkin-v2

This project has a stack.yaml that references packages, but doesn't have a .cabal or package.yaml file for the top directory.

The error looks like:

/private/var/folders/6y/_mvyr6tj4px_52pkvychlw1r0000gp/T/hpack-pkg-dir23692/package.yaml: Yaml file not found: /private/var/folders/6y/_mvyr6tj4px_52pkvychlw1r0000gp/T/share/package.yaml

@qrilka

This comment has been minimized.

Copy link
Contributor

@qrilka qrilka commented Jun 26, 2019

@myShoggoth the problem is not about top-level directory missing package.yaml it's about shared yaml file, see the following line in opentracin/package.yaml:

_include: !include '../share/package.yaml'

@snoyberg it looks like Pantry broke this feature of yaml sharing, does it make sense to bring it back or maybe it contradicts Pantry logic in some way?

@snoyberg

This comment has been minimized.

Copy link
Contributor

@snoyberg snoyberg commented Jun 26, 2019

I'd shy away from such a change to how Pantry works. In order to make this use case work, we would need to store every file in an archive or Git repo on the off chance that something is using an !include or similar feature.

@myShoggoth myShoggoth changed the title Package dependencies with subdirs fails looking for package.yaml Packages with !include in them don't work with Pantry. Jul 3, 2019
@myShoggoth

This comment has been minimized.

Copy link
Author

@myShoggoth myShoggoth commented Jul 3, 2019

Is there a path forward for this? A pre-processor step on the yaml, as gross as that is?

@snoyberg

This comment has been minimized.

Copy link
Contributor

@snoyberg snoyberg commented Jul 4, 2019

I don't see a path forward like that. The closest I can imagine is a setting to Pantry which says "cache the entire repository, not just the subdirectory in question." I don't intend to work on that, but if someone is interested in approaching it I can provide guidance. For now, I'd say generating a tarball with a cabal file instead would be a decent workaround.

chrisdone added a commit to chrisdone/yaml-repro-4906 that referenced this issue Aug 14, 2019
@asivitz

This comment has been minimized.

Copy link

@asivitz asivitz commented Sep 5, 2019

I have no real knowledge of Pantry's implementation, so I apologize if this is completely off base, but would it be reasonable to generate the cabal files after the clone but before it is cached? Unless there is value in caching the subdirectory exactly as it is cloned. I guess I'm suggesting something similar to @myShoggoth but it might not be gross because it's just moving the hpack generation step earlier. Maybe there's even a performance benefit to be gained here. 🤷‍♂

I'm assuming of course that the full repo is cloned, and not just the specified subdirs.

@maksbotan

This comment has been minimized.

Copy link

@maksbotan maksbotan commented Mar 2, 2020

Hello, @snoyberg.
To clarify -- would you accept a patch to Pantry that would cache the generated *.cabal files instead of source package.yamls?

@snoyberg

This comment has been minimized.

Copy link
Contributor

@snoyberg snoyberg commented Mar 3, 2020

I'm not really in favor of that, it will still break the reproducibility guarantees of pantry. Overall, I think the problem is that we shouldn't have allowed package.yaml files in Git repos to be used as they're being used. The original design of pantry required a .cabal file to be present in the repo, and that overall is the correct solution. My advice remains to refer to tarballs or Git repos that include the cabal file.

I'm strongly considering deprecating support in pantry in the future for using package.yaml files as they are supported today.

@snoyberg

This comment has been minimized.

Copy link
Contributor

@snoyberg snoyberg commented Mar 8, 2020

Since we'll be deprecating this workflow (see #5210), I'm closing this issue as wontfix.

@snoyberg snoyberg closed this Mar 8, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
6 participants
You can’t perform that action at this time.