Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new-show-build-info command #5954

Open
wants to merge 12 commits into
base: master
from

Conversation

Projects
None yet
8 participants
@fendor
Copy link

fendor commented Mar 20, 2019


Please include the following checklist in your PR:

  • Patches conform to the coding conventions.
  • Any changes that could be relevant to users have been recorded in the changelog.
  • The documentation has been updated, if necessary.
  • If the change is docs-only, [ci skip] is used to avoid triggering the build bots.

Please also shortly describe how you tested your change. Bonus points for added tests!

Based on the pull request #2771 and the work of @cfraz89.
Rebases on cfraz89's work to work on master.

Introduces the command new-show-build-info.
Example output for my local cabal-install package:

Example output

[
    {
        "cabal-version": "3.0.0.0",
        "compiler": {
            "flavour": "ghc",
            "compiler-id": "ghc-8.4.4",
            "path": "/nix/store/lb5bicfrbx2h2sgvcb6nhxffijwdk7fp-ghc-8.4.4-with-packages/bin/ghc-8.4.4"
        },
        "components": [
            {
                "type": "exe",
                "name": "exe:cabal",
                "compiler-args": [
                    "-fbuilding-cabal-package",
                    "-O",
                    "-outputdir",
                    "/home/baldr/Documents/haskell/cabal/dist-newstyle/build/x86_64-linux/ghc-8.4.4/cabal-install-3.0.0.0/build",
                    "-odir",
                    "/home/baldr/Documents/haskell/cabal/dist-newstyle/build/x86_64-linux/ghc-8.4.4/cabal-install-3.0.0.0/build",
                    "-hidir",
                    "/home/baldr/Documents/haskell/cabal/dist-newstyle/build/x86_64-linux/ghc-8.4.4/cabal-install-3.0.0.0/build",
                    "-stubdir",
                    "/home/baldr/Documents/haskell/cabal/dist-newstyle/build/x86_64-linux/ghc-8.4.4/cabal-install-3.0.0.0/build",
                    "-i",
                    "-i/home/baldr/Documents/haskell/cabal/dist-newstyle/build/x86_64-linux/ghc-8.4.4/cabal-install-3.0.0.0/build",
                    "-imain",
                    "-i.",
                    "-i/home/baldr/Documents/haskell/cabal/dist-newstyle/build/x86_64-linux/ghc-8.4.4/cabal-install-3.0.0.0/build/cabal/autogen",
                    "-i/home/baldr/Documents/haskell/cabal/dist-newstyle/build/x86_64-linux/ghc-8.4.4/cabal-install-3.0.0.0/build/global-autogen",
                    "-I/home/baldr/Documents/haskell/cabal/dist-newstyle/build/x86_64-linux/ghc-8.4.4/cabal-install-3.0.0.0/build/cabal/autogen",
                    "-I/home/baldr/Documents/haskell/cabal/dist-newstyle/build/x86_64-linux/ghc-8.4.4/cabal-install-3.0.0.0/build/global-autogen",
                    "-I/home/baldr/Documents/haskell/cabal/dist-newstyle/build/x86_64-linux/ghc-8.4.4/cabal-install-3.0.0.0/build",
                    "-optP-include",
                    "-optP/home/baldr/Documents/haskell/cabal/dist-newstyle/build/x86_64-linux/ghc-8.4.4/cabal-install-3.0.0.0/build/cabal/autogen/cabal_macros.h",
                    "-hide-all-packages",
                    "-Wmissing-home-modules",
                    "-no-user-package-db",
                    "-package-db",
                    "/home/baldr/.cabal/store/ghc-8.4.4/package.db",
                    "-package-db",
                    "/home/baldr/Documents/haskell/cabal/dist-newstyle/packagedb/ghc-8.4.4",
                    "-package-id",
                    "Cabal-3.0.0.0-inplace",
                    "-package-id",
                    "HTTP-4000.3.12-7VzzTYjmS00JEuV6LhSlps",
                    "-package-id",
                    "array-0.5.2.0",
                    "-package-id",
                    "async-2.2.1-1CuEIvGS3OJLsN4gbNyjc9",
                    "-package-id",
                    "base-4.11.1.0",
                    "-package-id",
                    "base16-bytestring-0.1.1.6-I0igvRcEwRNBMqqPC2yQBh",
                    "-package-id",
                    "binary-0.8.5.1",
                    "-package-id",
                    "bytestring-0.10.8.2",
                    "-package-id",
                    "containers-0.5.11.0",
                    "-package-id",
                    "cryptohash-sha256-0.11.101.0-5CO9AFPvemHGgff3tAvBC6",
                    "-package-id",
                    "deepseq-1.4.3.0",
                    "-package-id",
                    "directory-1.3.1.5",
                    "-package-id",
                    "echo-0.1.3-G7cuM0pWhoYJlNCMI5dtUe",
                    "-package-id",
                    "edit-distance-0.2.2.1-2UTOm3vhgl8C2Izvm4wSjl",
                    "-package-id",
                    "filepath-1.4.2",
                    "-package-id",
                    "hackage-security-0.5.3.0-inplace",
                    "-package-id",
                    "hashable-1.2.7.0-LHfSposRBGi1iCUd6KrSLc",
                    "-package-id",
                    "mtl-2.2.2",
                    "-package-id",
                    "network-2.6.3.6-2g6VId0Xlc85XRtUcfQj0T",
                    "-package-id",
                    "network-uri-2.6.1.0-KQiMjxVJTcx5MsYuPchm84",
                    "-package-id",
                    "parsec-3.1.13.0",
                    "-package-id",
                    "pretty-1.1.3.6",
                    "-package-id",
                    "process-1.6.3.0",
                    "-package-id",
                    "random-1.1-9LLJAJa4iQFLJiLXBOBXBV",
                    "-package-id",
                    "resolv-0.1.1.1-JE0F05VqIRvJXa0qbVDgHe",
                    "-package-id",
                    "stm-2.4.5.1",
                    "-package-id",
                    "tar-0.5.1.0-5HrFyZlpKeAI6XvXje12oT",
                    "-package-id",
                    "text-1.2.3.1",
                    "-package-id",
                    "time-1.8.0.2",
                    "-package-id",
                    "unix-2.7.2.2",
                    "-package-id",
                    "zlib-0.6.2-FP80mWgJNoyCiVcPtw6kKj",
                    "-XHaskell2010",
                    "-Wall",
                    "-fwarn-tabs",
                    "-rtsopts",
                    "-threaded",
                    "-Wcompat",
                    "-Wnoncanonical-monad-instances",
                    "-Wnoncanonical-monadfail-instances"
                ],
                "modules": [
                    "Distribution.Deprecated.ParseUtils",
                    "Distribution.Deprecated.ReadP",
                    "Distribution.Deprecated.Text",
                    "Distribution.Deprecated.ViewAsFieldDescr",
                    "Distribution.Client.BuildReports.Anonymous",
                    "Distribution.Client.BuildReports.Storage",
                    "Distribution.Client.BuildReports.Types",
                    "Distribution.Client.BuildReports.Upload",
                    "Distribution.Client.Check",
                    "Distribution.Client.CmdBench",
                    "Distribution.Client.CmdBuild",
                    "Distribution.Client.CmdClean",
                    "Distribution.Client.CmdShowBuildInfo",
                    "Distribution.Client.CmdConfigure",
                    "Distribution.Client.CmdUpdate",
                    "Distribution.Client.CmdErrorMessages",
                    "Distribution.Client.CmdExec",
                    "Distribution.Client.CmdFreeze",
                    "Distribution.Client.CmdHaddock",
                    "Distribution.Client.CmdInstall",
                    "Distribution.Client.CmdInstall.ClientInstallFlags",
                    "Distribution.Client.CmdRepl",
                    "Distribution.Client.CmdRun",
                    "Distribution.Client.CmdTest",
                    "Distribution.Client.CmdLegacy",
                    "Distribution.Client.CmdSdist",
                    "Distribution.Client.Compat.Directory",
                    "Distribution.Client.Compat.ExecutablePath",
                    "Distribution.Client.Compat.FileLock",
                    "Distribution.Client.Compat.FilePerms",
                    "Distribution.Client.Compat.Prelude",
                    "Distribution.Client.Compat.Process",
                    "Distribution.Client.Compat.Semaphore",
                    "Distribution.Client.Config",
                    "Distribution.Client.Configure",
                    "Distribution.Client.Dependency",
                    "Distribution.Client.Dependency.Types",
                    "Distribution.Client.DistDirLayout",
                    "Distribution.Client.Exec",
                    "Distribution.Client.Fetch",
                    "Distribution.Client.FetchUtils",
                    "Distribution.Client.FileMonitor",
                    "Distribution.Client.Freeze",
                    "Distribution.Client.GZipUtils",
                    "Distribution.Client.GenBounds",
                    "Distribution.Client.Get",
                    "Distribution.Client.Glob",
                    "Distribution.Client.GlobalFlags",
                    "Distribution.Client.Haddock",
                    "Distribution.Client.HttpUtils",
                    "Distribution.Client.IndexUtils",
                    "Distribution.Client.IndexUtils.Timestamp",
                    "Distribution.Client.Init",
                    "Distribution.Client.Init.Heuristics",
                    "Distribution.Client.Init.Licenses",
                    "Distribution.Client.Init.Types",
                    "Distribution.Client.Install",
                    "Distribution.Client.InstallPlan",
                    "Distribution.Client.InstallSymlink",
                    "Distribution.Client.JobControl",
                    "Distribution.Client.List",
                    "Distribution.Client.Manpage",
                    "Distribution.Client.Nix",
                    "Distribution.Client.Outdated",
                    "Distribution.Client.PackageHash",
                    "Distribution.Client.PackageUtils",
                    "Distribution.Client.ParseUtils",
                    "Distribution.Client.ProjectBuilding",
                    "Distribution.Client.ProjectBuilding.Types",
                    "Distribution.Client.ProjectConfig",
                    "Distribution.Client.ProjectConfig.Legacy",
                    "Distribution.Client.ProjectConfig.Types",
                    "Distribution.Client.ProjectOrchestration",
                    "Distribution.Client.ProjectPlanOutput",
                    "Distribution.Client.ProjectPlanning",
                    "Distribution.Client.ProjectPlanning.Types",
                    "Distribution.Client.RebuildMonad",
                    "Distribution.Client.Reconfigure",
                    "Distribution.Client.Run",
                    "Distribution.Client.Sandbox",
                    "Distribution.Client.Sandbox.Index",
                    "Distribution.Client.Sandbox.PackageEnvironment",
                    "Distribution.Client.Sandbox.Timestamp",
                    "Distribution.Client.Sandbox.Types",
                    "Distribution.Client.SavedFlags",
                    "Distribution.Client.Security.DNS",
                    "Distribution.Client.Security.HTTP",
                    "Distribution.Client.Setup",
                    "Distribution.Client.SetupWrapper",
                    "Distribution.Client.SolverInstallPlan",
                    "Distribution.Client.SourceFiles",
                    "Distribution.Client.SourceRepoParse",
                    "Distribution.Client.SrcDist",
                    "Distribution.Client.Store",
                    "Distribution.Client.Tar",
                    "Distribution.Client.TargetSelector",
                    "Distribution.Client.Targets",
                    "Distribution.Client.Types",
                    "Distribution.Client.Update",
                    "Distribution.Client.Upload",
                    "Distribution.Client.Utils",
                    "Distribution.Client.Utils.Assertion",
                    "Distribution.Client.Utils.Json",
                    "Distribution.Client.Utils.Parsec",
                    "Distribution.Client.VCS",
                    "Distribution.Client.Win32SelfUpgrade",
                    "Distribution.Client.World",
                    "Distribution.Solver.Compat.Prelude",
                    "Distribution.Solver.Modular",
                    "Distribution.Solver.Modular.Assignment",
                    "Distribution.Solver.Modular.Builder",
                    "Distribution.Solver.Modular.Configured",
                    "Distribution.Solver.Modular.ConfiguredConversion",
                    "Distribution.Solver.Modular.ConflictSet",
                    "Distribution.Solver.Modular.Cycles",
                    "Distribution.Solver.Modular.Dependency",
                    "Distribution.Solver.Modular.Explore",
                    "Distribution.Solver.Modular.Flag",
                    "Distribution.Solver.Modular.Index",
                    "Distribution.Solver.Modular.IndexConversion",
                    "Distribution.Solver.Modular.LabeledGraph",
                    "Distribution.Solver.Modular.Linking",
                    "Distribution.Solver.Modular.Log",
                    "Distribution.Solver.Modular.Message",
                    "Distribution.Solver.Modular.PSQ",
                    "Distribution.Solver.Modular.Package",
                    "Distribution.Solver.Modular.Preference",
                    "Distribution.Solver.Modular.RetryLog",
                    "Distribution.Solver.Modular.Solver",
                    "Distribution.Solver.Modular.Tree",
                    "Distribution.Solver.Modular.Validate",
                    "Distribution.Solver.Modular.Var",
                    "Distribution.Solver.Modular.Version",
                    "Distribution.Solver.Modular.WeightedPSQ",
                    "Distribution.Solver.Types.ComponentDeps",
                    "Distribution.Solver.Types.ConstraintSource",
                    "Distribution.Solver.Types.DependencyResolver",
                    "Distribution.Solver.Types.Flag",
                    "Distribution.Solver.Types.InstSolverPackage",
                    "Distribution.Solver.Types.InstalledPreference",
                    "Distribution.Solver.Types.LabeledPackageConstraint",
                    "Distribution.Solver.Types.OptionalStanza",
                    "Distribution.Solver.Types.PackageConstraint",
                    "Distribution.Solver.Types.PackageFixedDeps",
                    "Distribution.Solver.Types.PackageIndex",
                    "Distribution.Solver.Types.PackagePath",
                    "Distribution.Solver.Types.PackagePreferences",
                    "Distribution.Solver.Types.PkgConfigDb",
                    "Distribution.Solver.Types.Progress",
                    "Distribution.Solver.Types.ResolverPackage",
                    "Distribution.Solver.Types.Settings",
                    "Distribution.Solver.Types.SolverId",
                    "Distribution.Solver.Types.SolverPackage",
                    "Distribution.Solver.Types.SourcePackage",
                    "Distribution.Solver.Types.Variable",
                    "Paths_cabal_install"
                ],
                "src-files": [
                    "Main.hs"
                ],
                "src-dirs": [
                    "main",
                    "."
                ]
            }
        ]
    }
]

@fendor fendor changed the title Rebase new show builf info Add new-show-build-info command Mar 20, 2019

@@ -0,0 +1,34 @@
module Distribution.Simple.Utils.Json

This comment has been minimized.

Copy link
@fendor

fendor Mar 20, 2019

Author

I think this file should be removed, is that still true?

This comment has been minimized.

Copy link
@hvr

hvr Apr 3, 2019

Member

Yes, definitely! We already have a JSON library (which is more or less a derivative of http://hackage.haskell.org/package/microaeson-0.1.0.0/docs/Data-Aeson-Micro.html) inside exe:cabal.

By moving show-build-info into exe:cabal we'd kill two birds with one stone...

This comment has been minimized.

Copy link
@hvr

hvr Apr 8, 2019

Member

At this point I don't care whether we address this as part of this PR, or at a later stage; so feel free to consider this issue of the JSON library duplication punted to a follow-up PR

It may be worth leaving a TODO source-comment at the top of this module

@fendor

This comment has been minimized.

Copy link
Author

fendor commented Mar 20, 2019

Request review from @hvr, to validate that the output is relevant and correct.

@DanielG
Copy link
Collaborator

DanielG left a comment

Awesome, it builds and isn't a mess of merges. Well done, @fendor. I don't even want to know the git vodoo rituals you had to perform to get this :)

Show resolved Hide resolved Cabal/Distribution/Simple/Setup.hs Outdated
Show resolved Hide resolved cabal-install/Distribution/Client/Setup.hs Outdated
Show resolved Hide resolved cabal-install/Distribution/Client/Setup.hs Outdated
Show resolved Hide resolved cabal-install/main/Main.hs
Show resolved Hide resolved cabal-install/main/Main.hs Outdated
Show resolved Hide resolved cabal-install/Distribution/Client/CmdShowBuildInfo.hs Outdated
Show resolved Hide resolved cabal-install/Distribution/Client/CmdShowBuildInfo.hs Outdated
Show resolved Hide resolved cabal-install/Distribution/Client/CmdShowBuildInfo.hs Outdated
Show resolved Hide resolved cabal-install/Distribution/Client/CmdShowBuildInfo.hs Outdated
Show resolved Hide resolved Cabal/Distribution/Simple/Utils/Json.hs
@sboosali

This comment has been minimized.

Copy link
Collaborator

sboosali commented Mar 25, 2019

@fendor fendor force-pushed the fendor:rebase-new-show-builf-info branch from 255df31 to 056c810 Mar 27, 2019

@bgamari

This comment has been minimized.

Copy link
Contributor

bgamari commented Apr 2, 2019

@fendor, as the original author of #2771 thank you for pushing this over the finish line. This will make Haskell's tooling story significantly simpler for us all.

@hvr
Copy link
Member

hvr left a comment

Sorry it took me longer to review than I promised. Thanks for doing this; this is a first good step!

So to move forward from here, the biggest concern for me is to

  1. Separate the autogen-files concern into a separate PR
  2. Simplify and avoid duplication by moving code into the frontend
  3. Align JSON schema with plan.json prior-art

UPDATE: #5954 (comment)

Show resolved Hide resolved Cabal/Distribution/Simple.hs Outdated
Show resolved Hide resolved Cabal/Distribution/Simple.hs Outdated
Show resolved Hide resolved Cabal/Distribution/Simple.hs Outdated
Show resolved Hide resolved Cabal/Distribution/Simple/Build.hs
Show resolved Hide resolved Cabal/Distribution/Simple/Setup.hs Outdated
@@ -0,0 +1,34 @@
module Distribution.Simple.Utils.Json

This comment has been minimized.

Copy link
@hvr

hvr Apr 3, 2019

Member

Yes, definitely! We already have a JSON library (which is more or less a derivative of http://hackage.haskell.org/package/microaeson-0.1.0.0/docs/Data-Aeson-Micro.html) inside exe:cabal.

By moving show-build-info into exe:cabal we'd kill two birds with one stone...

Show resolved Hide resolved cabal-install/Distribution/Client/CmdWriteAutogenFiles.hs Outdated
Show resolved Hide resolved cabal-install/Distribution/Client/CmdWriteAutogenFiles.hs Outdated
verbosity
flags
-- Default silent verbosity so as not to pollute json output
where verbosity = fromFlagOrDefault silent (buildVerbosity buildFlags)

This comment has been minimized.

Copy link
@hvr

hvr Apr 3, 2019

Member

The stdout/stderr issue is a general issue (iirc we have a separate ticket for that) in cabal we don't need to solve in this PR :-)

Btw, I don't see a flag to redirect the output to a file. If there isn't we should have one (maybe --buildinfo-json-output=FILE)

Show resolved Hide resolved cabal-install/main/Main.hs Outdated

bgamari and others added some commits Aug 16, 2015

Add show-build-info command
This allows users to get a JSON representation of various information
about how Cabal would go about building a package. The output of this
command is intended for external tools and therefore the format should
remain stable.
Rebase work of cfraz89 and bgamari
Add (currently nonfunctional) new-show-build-info
Fix compile error
Make new-show-build-info functional
Use silent verbosity by default on showBuildInfo commands to keep output json clean
Make show-build-info commands hidden
Implement write-autogen-files
Make new-write-autogen-files work
Make new-write-autogen-files configure if necessary
Use target selectors for new-show-build-info
Don't prune plan for new-show-build-info
Only configure in new-show-build-info and new-write-autogen-files if no persist build info file is found
Wrap multiple target output of new-show-build-info in json list

@fendor fendor force-pushed the fendor:rebase-new-show-builf-info branch from 4252120 to f711572 Apr 3, 2019

@DanielG

This comment has been minimized.

Copy link
Collaborator

DanielG commented Apr 3, 2019

@fendor hold off on trying to move show-build-info into exe:cabal, I'm disscussing that point with @hvr ATM. I think it's a bad idea.

@fendor fendor force-pushed the fendor:rebase-new-show-builf-info branch from 13feaec to 64c59ff Apr 3, 2019

@fendor fendor force-pushed the fendor:rebase-new-show-builf-info branch from ef3090d to b0fd93a Apr 3, 2019

let lbi' = lbi { withPrograms = progs }
pkg_descr0 = localPkgDescr lbi'
pkg_descr = updatePackageDescription pbi pkg_descr0
-- TODO: Somehow don't ignore build hook?

This comment has been minimized.

Copy link
@hvr

hvr Apr 8, 2019

Member

what about this TODO? What is the consequence of ignoring build hooks, i.e. what's the worst that can happen?

This comment has been minimized.

Copy link
@fendor

fendor Apr 11, 2019

Author

Why are even build hooks being ignored?
How could they influence the result?

Show resolved Hide resolved Cabal/Distribution/Simple/Build.hs Outdated
@@ -0,0 +1,34 @@
module Distribution.Simple.Utils.Json

This comment has been minimized.

Copy link
@hvr

hvr Apr 8, 2019

Member

At this point I don't care whether we address this as part of this PR, or at a later stage; so feel free to consider this issue of the JSON library duplication punted to a follow-up PR

It may be worth leaving a TODO source-comment at the top of this module

-- It selects the 'AvailableTarget's that the 'TargetSelector' refers to,
-- or otherwise classifies the problem.
--
-- For the @write-autogen-files@ command select all components except non-buildable and disabled

This comment has been minimized.

Copy link
@hvr

hvr Apr 8, 2019

Member

Shouldn't this be moved to a separate PR as well if it concerns the write-autogen-files command?

This comment has been minimized.

Copy link
@fendor

fendor Apr 11, 2019

Author

In my opiniion the comment might be outdated, since this function is neither exported nor used by write-autogen-files

import Distribution.Simple.Configure (tryGetPersistBuildConfig)
import Data.List (find)

showBuildInfoCommand :: CommandUI (ConfigFlags, ConfigExFlags, InstallFlags, HaddockFlags, TestFlags)

This comment has been minimized.

Copy link
@hvr

hvr Apr 8, 2019

Member

What needs to happen either here or in a subsequent PR is to implement the ability to refer to targets by their unique "unit id". So far this command appears to only support the less accurate symbolic target specifiers which aren't able to address all elements of a build-plan.

This comment has been minimized.

Copy link
@fendor

fendor Apr 12, 2019

Author

Unit Id as in TargetInfo.ComponentLocalBuildInfo.componentUnitId? Or whose unit id?
Are we missing components?

liftOptions snd setSnd (buildExOptions showOrParseArgs)
}
where
setFst a (_,b) = (a,b)

This comment has been minimized.

Copy link
@hvr

hvr Apr 8, 2019

Member

@phadej don't we have _1/_2 lenses for this somewhere? :-)

This comment has been minimized.

Copy link
@fendor

fendor Apr 12, 2019

Author

Such custom setters are used in all Cmd* files. So, I either doubt it, or I can do a subsequent pr to replace that for every Cmd.

@fendor

This comment has been minimized.

Copy link
Author

fendor commented Apr 12, 2019

Now the output of new-show-build-info can be written into a File. Flag is called: --buildinfo-json-output

Next is the flag for filtering unit-ids.
Would appreciate more reviews!
@hvr can you take a look at it again?

@DanielG
Copy link
Collaborator

DanielG left a comment

I think we really need some tests for this. Have a look at cabal-testsuite. If you need any help with that give me a shout.

Show resolved Hide resolved Cabal/unit.json Outdated
Show resolved Hide resolved cabal-install/shell.nix Outdated
Show resolved Hide resolved Cabal/Distribution/Simple/Build.hs Outdated

@fendor fendor force-pushed the fendor:rebase-new-show-builf-info branch from c6eb3fd to a4271dc Apr 12, 2019

Add flags for new-show-build-info
Add functioning flag `--buildinfo-json-output=FILE`
Add custom flags to parse more flags.
Dont write always to stdout but to a file if specified.

@fendor fendor force-pushed the fendor:rebase-new-show-builf-info branch from a4271dc to d54726c Apr 12, 2019

@fendor

This comment has been minimized.

Copy link
Author

fendor commented Apr 13, 2019

I think we really need some tests for this. Have a look at cabal-testsuite. If you need any help with that give me a shout.
@DanielG

I'd need some help with how I should start with that.
Should I create a new file and model it after Plan.hs?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.