Skip to content

Commit

Permalink
Orig. marconi migration from plutus-apps
Browse files Browse the repository at this point in the history 
Migrating marconi related modules and corresponding git history from
plutus-apps repository.
This effort does not address CI related tasks.
Additionally adding:
- docs related files
- nix standard related files/directories
- remove legacy nix folder
  • Loading branch information
@kayvank
kayvank committed Mar 27, 2023
1 parent 6dd43b7 commit 6e0c7fe
Show file tree
Hide file tree
Showing 48 changed files with 4,850 additions and 959 deletions.
62 changes: 10 additions & 52 deletions CONTRIBUTING.adoc
View file
@@ -1,4 +1,4 @@
= Contributing to Plutus Apps
= Contributing to Marconi
:toc: left
:reproducible:

Expand All @@ -10,7 +10,7 @@ Thanks to Nix we ensure that everyone has consistent versions of the tools that

Please use Nix since problems due to mismatched versions of tools are particularly annoying to fix!

If you _really_ cannot use Nix and still want to contribute to Plutus then xref:develop-without-nix[skip to the relevant section].
If you _really_ cannot use Nix and still want to contribute to Marconi then xref:develop-without-nix[skip to the relevant section].

=== Installing and setting up Nix

Expand Down Expand Up @@ -98,7 +98,7 @@ On NixOS, add the user/group name to the list under [`nix.settings.trusted-users

=== What to do once inside the `nix develop` shell

Your prompt will change to `[plutus-shell]` and you will be presented with a menu of available commands.
Your prompt will change to `[marconi]` and you will be presented with a menu of available commands.

Please read that menu carefully.

Expand All @@ -109,7 +109,7 @@ From here you can build the project packages directly with `cabal`.

NOTE: You may need to run `cabal update` so that `cabal` knows about the index state xref:update-index-state[we have pinned].

Run `cabal build plutus-core` from the root to build the Plutus Core library.
Run `cabal build all` from the root to build all marconi modules.

See the link:./cabal.project[cabal project file] for a list of other packages that you can build with `cabal`.

Expand All @@ -126,9 +126,9 @@ We don't have a `hie.yaml`, the implicit cradle support in HLS seems to work fin

Haskell components are provisioned by Nix via link:https://github.com/input-output-hk/haskell.nix[Haskell.nix]

In general you can run `nix build .#SYSTEM.plutus.library.plutus-project.hsPkgs.PACKAGE.components.COMPONENT`
In general you can run `nix build .#SYSTEM.marconi.library.marconi-project.hsPkgs.PACKAGE.components.COMPONENT`

For example `nix build .#x86_64-linux.plutus.library.plutus-project.hsPkgs.plutus-core.components.library`
For example `nix build .#x86_64-linux.marconi.library.marconi-project.hsPkgs.marconi-sidechain.components.library` to build marconi-sidechain.

For full documentation about the Nix code see the link:nix/README.md[Nix README].

Expand Down Expand Up @@ -196,7 +196,7 @@ In particular, we should not release our packages while we depend on a `source-r

If we are stuck in a situation where we need a long-running fork of a package, we should release it to CHaP instead (see the https://github.com/input-output-hk/cardano-haskell-packages[CHaP README] for more).

If you do add a `source-repository-package`, you need to update the `sha256` mapping in `nix/cells/plutus/library/make-plutus-project.nix`.
If you do add a `source-repository-package`, you need to update the `sha256` mapping in `nix/cells/marconi/library/make-marconi-project.nix`.
For the moment you have to do this by hand, using the following command to get the sha: `nix-prefetch-git --quiet <repo-url> <rev> | jq .sha256`, or by just getting it wrong and trying to build it, in which case Nix will give you the right value.

[[update-nix-pins]]
Expand All @@ -215,7 +215,6 @@ Do *not* use `nix flake update`, as that will update all the inputs, which we ty

TODO: Currently not available, coming soon

If you launch `nix develop .#profiled-plutus-apps` you will get a shell where all the dependencies have been built with profiling.

[WARNING]
====
Expand Down Expand Up @@ -308,51 +307,10 @@ allows us to remove the ``source-repository-package`` stanza.

=== How to work with a local copy of source dependencies

Sometimes you may want to make a change that spans both `plutus-apps` *and* some of its dependencies (most commonly, the packages in `plutus`).
The obvious workflow is to make changes in the `plutus` repository, update the pin in `plutus-apps` to point to the new commit, test, repeat.
Sometimes you may want to make a change that spans both `marconi` *and* some of its dependencies (most commonly, the packages in `plutus-apps`).
The obvious workflow is to make changes in the `plutus-apps` repository, update the pin in `marconi` to point to the new commit, test, repeat.
But this is very tedious and it's much nicer to work with a local checkout where cabal can incrementally rebuild the whole thing.

Here's an example of doing this for some packages in `plutus`.
You may need to adapt this example depending on what exactly you're trying to do.

First of all, we add some lines to `cabal.project.local` (we do this to avoid modifying `cabal.project`, which is tracked by git, as much as possible).
```
-- Add the packages that we want to pull in locally as "local" packages
-- (assuming a `plutus-core` checkout in `../plutus`)
packages:
../plutus/plutus-core
../plutus/plutus-tx
../plutus/plutus-tx-plugin
../plutus/plutus-ledger-api

-- Sometimes cabal may get upset about dependencies, you can make its life easier
-- by turning off unnecessary features for the altered packages.
package plutus-core
benchmarks: false
tests: false
package plutus-tx
benchmarks: false
tests: false
package plutus-tx-plugin
benchmarks: false
tests: false
```

Then we unfortunately do need to modify the main `cabal.project`, to comment out the packages that we are replacing with local ones.
```
source-repository-package
...
subdir:
--plutus-core
--plutus-ledger-api
--plutus-tx
--plutus-tx-plugin
word-array
prettyprinter-configurable
stubs/plutus-ghc-stub
```

After this, a `cabal build` should build with the local checkouts of `plutus` for the packages you specified.
https://github.com/input-output-hk/plutus-apps/blob/main/CONTRIBUTING.adoc#how-to-work-with-a-local-copy-of-source-dependencies[Plutus-apps repository] has a nice example of such workflow.

== Working conventions

Expand Down
77 changes: 18 additions & 59 deletions README.adoc
View file
@@ -1,26 +1,23 @@
= https://github.com/input-output-hk/plutus-apps[The Plutus Application Framework]
= https://github.com/input-output-hk/marconi[Cardano Blockchain Indexer Framework]
:email: plutus@iohk.io
:author: Input Output HK Limited
:toc: left
:reproducible:

image:https://img.shields.io/badge/policy-Cardano%20Engineering%20Handbook-informational[link=https://input-output-hk.github.io/cardano-engineering-handbook]

The Plutus Application Framework, part of the Plutus Platform, is a framework for developing distributed applications using the Cardano blockchain.
For more information about the projects, see the <<user-documentation>>.
## Introduction

This repository contains:
Marconi is A lightweight customizable solution for indexing and querying the Cardano blockchain.

* Plutus Platform
** Libraries which implement the Plutus Application Framework, a framework for writing applications that work with Cardano.
** A selection of end-to-end usecases written with the Plutus Application Framework
[[Architecture]]
== Architecture

[IMPORTANT]
====
The rest of this README is focussed on people who want to develop or contribute to the Framework.
Marconi consists of a set of compomentes:

For people who want to *use* the Framework, please consult the <<user-documentation>>.
====
- link:./marconi-core[marconi-core]
- link:./marconi-chain-index[marconi-chain-indexer]
- link:./marconi-sidechain[marconi-sidechain]

== Development

Expand All @@ -35,26 +32,14 @@ Run `nix develop` to enter the development shell and you will be presented with

=== User documentation

The main documentation is located https://plutus-apps.readthedocs.io/en/latest/[here].
User documentations are work in progress. You may generate Haskell API documenations (haddocks) directly from the `nix shell` for each of marconi compoments:

The generated Haskell API documentation (haddocks) are here:
<https://playground.plutus.iohkdev.io/doc/haddock>.
```
cabal haddock marconi-core
cabal haddock marconi-chain-index
cabal haddock marconi-sidechain


=== Talks

- https://www.youtube.com/watch?v=MpWeg6Fg0t8[Functional Smart Contracts on Cardano]
- https://www.youtube.com/watch?v=usMPt8KpBeI[The Plutus Platform]

=== Specifications and design

- https://ci.zw3rk.com/job/input-output-hk-plutus/master/x86_64-linux.plutus-report/latest/download/1[Plutus Technical Report (draft)]

== Branching, Versioning and Releases

There are two protected development branches in `plutus-apps`: `main` and `next-node`.
We adopt the PVP versioning scheme.
Check out link:doc/BRANCHING-AND-RELEASE.adoc[Branching Policy and Release Process] to learn more.
```

=== Dependency update

Expand All @@ -76,47 +61,21 @@ Independently of the protected branch:
* It should not update `cardano-node` to a new major-version. In other words, it should use a `cardano-node` version which is compatible with the current Cardano mainnet
* It should use a `cardano-wallet` version which is compatible with the current `cardano-node` version

`next-node` branch:

* It may update the `cardano-node` to a new major-version. In other words, it may use a `cardano-node` version which is incompatible with the current Cardano mainnet
* It may use a `cardano-wallet` version which is incompatible with the current `cardano-node` version

=== Version ranges

Packages which depend on `plutus-apps` packages should use version ranges to control which version of those packages they build against.

* Packages in `plutus-apps` which are used downstream should pin the major-version of each other (e.g. `plutus-pab-1.0.1` should depend on `plutus-contract ^>= 1.0`).
* Downstream packages should pin at least the first-major-version of `plutus-apps` packages.
** Upgrading to a new second-major-version should always be safe for working on the current mainnet, with at most code breakage (following the PVP). Users may of course want to pin this version as well to avoid such breakage.
* Downstream packages pulling in `plutus-apps` packages via `source-repository-package` stanzas should always take tagged commits.

== Working with the project

=== How to submit an issue

Issues can be filed in the https://github.com/input-output-hk/plutus-apps/issues[GitHub Issue tracker].
Issues can be filed in the https://github.com/input-output-hk/marconi/issues[GitHub Issue tracker].

However, note that this is pre-release software, so we will not usually be providing support.

[[how-to-develop]]
=== How to develop and contribute to the project

See link:CONTRIBUTING{outfilesuffix}[CONTRIBUTING], which describes our processes in more detail including development environments; and link:ARCHITECTURE{outfilesuffix}[ARCHITECTURE], which describes the structure of the repository.

=== How to depend on the project from another Haskell project

None of our libraries are on Hackage, unfortunately (many of our dependencies aren't either).
So for the time being, you need to:

. Add `plutus-apps` as a `source-repository-package` to your `cabal.project`.
. Copy the `source-repository-package` stanzas from our `cabal.project` to yours.
. Copy additional stanzas from our `cabal.project` as you need, e.g. you may need some of the `allow-newer` stanzas.

The https://github.com/input-output-hk/plutus-starter[plutus-starter] project
(deprecated) provides an example.
See link:CONTRIBUTING{outfilesuffix}[CONTRIBUTING], which describes our processes in more detail including development environments; and <<Architecture>>, which describes the structure of the repository.

== Licensing

You are free to copy, modify, and distribute the Plutus Platform with
You are free to copy, modify, and distribute the Marconi Platform with
under the terms of the Apache 2.0 license. See the link:./LICENSE[LICENSE]
and link:./NOTICE[NOTICE] files for details.
42 changes: 21 additions & 21 deletions __std__/README.md
View file
Expand Up @@ -28,9 +28,9 @@ A description of the arguments follows:
Remember that a cell is named after its folder.\
The full format is: `inputs.cells.<cell>.<cell-block>.value`.\
Examples:
- `inputs.cells.plutus-apps.packages.read-the-docs-site`
- `inputs.cells.plutus-apps.devshells.plutus-apps-shell`
- `inputs.cells.plutus-apps.library.cardano-node`
- `inputs.cells.marconi.packages.read-the-docs-site`
- `inputs.cells.marconi.devshells.marconi-shell`
- `inputs.cells.marconi.library.cardano-node`

- `inputs.<flake-input>`\
The flake inputs proper.\
Expand All @@ -40,10 +40,10 @@ A description of the arguments follows:
Provides access to the cell's blocks.\
This is a shorthand for `inputs.cells.<cell>.<cell-block>`, where `<cell>` evaluates to the cell housing this nix file.\
Examples:
- `cell.library.cardano-node` only works for code in `/cells/plutus-apps`\
Alternatively `inputs.cells.plutus-apps.library.agda-cardano-node` works everywhere
- `cell.packages.hlint` only works for code in `/cells/plutus-apps`\
Alternatively `inputs.cells.plutus-apps.packages.hlint` works everywhere
- `cell.library.cardano-node` only works for code in `/cells/marconi`\
Alternatively `inputs.cells.marconi.library.agda-cardano-node` works everywhere
- `cell.packages.hlint` only works for code in `/cells/marconi`\
Alternatively `inputs.cells.marconi.packages.hlint` works everywhere

## One derivation per nix file

Expand Down Expand Up @@ -75,33 +75,33 @@ While these blocks are not exposed directly to the flake (they are not "harveste
`nix (develop|build|run) .#<system>.<cell>.<cell-block>.<valid.attr.path>`

For example:
`nix build .#x86_64-linux.plutus-apps.library.plutus-apps-project.hsPkgs.marconi-sidechain.components.exes.marconi-sidechain`
`nix build .#x86_64-linux.marconi.library.marconi-project.hsPkgs.marconi-sidechain.components.exes.marconi-sidechain`

## Reference example

As an example, consider the file `./__std__/cells/plutus-apps/packages/marconi-chain-index.nix`:
As an example, consider the file `./__std__/cells/marconi/packages/marconi-chain-index.nix`:

- `./__std__/cells` is the `cellsFrom` value in `flake.nix`
- `/plutus-apps` is the cell name
- `/plutus-apps/*` are accessible via `cell.*` from `{ inputs, cell }` (while inside `cells/plutus-apps`)
- `/plutus-apps/*` are accessible via `inputs.cells.plutus-apps.*` (everywhere)
- `/marconi` is the cell name
- `/marconi/*` are accessible via `cell.*` from `{ inputs, cell }` (while inside `cells/marconi`)
- `/marconi/*` are accessible via `inputs.cells.marconi.*` (everywhere)
- `/packages` is the cell block name
- `/packages/*` are accessible via `cell.packages.*` (while inside `cells/plutus-apps`)
- `/packages/*` are accessible via `inputs.cells.plutus-apps.packages.*` (everywhere)
- `/packages/*` are accessible via `cell.packages.*` (while inside `cells/marconi`)
- `/packages/*` are accessible via `inputs.cells.marconi.packages.*` (everywhere)
- `/marconi-chain-index.nix` contains a *single derivation*
- `marconi-chain-index` is the name of the flake fragment
- A derivation named `marconi-chain-index` is accessible via `cell.packages.marconi-chain-index` (while inside `cells/plutus-apps`)
- And also accessible via `inputs.cells.plutus-apps.packages.marconi-chain-index` (everywhere)
- A derivation named `marconi-chain-index` is accessible via `cell.packages.marconi-chain-index` (while inside `cells/marconi`)
- And also accessible via `inputs.cells.marconi.packages.marconi-chain-index` (everywhere)
- And also buildable via `nix build .#marconi-chain-index`

As another example, consider the file `./__std__/cells/plutus-apps/library/default.nix`
As another example, consider the file `./__std__/cells/marconi/library/default.nix`

- `./__std__/cells` is the `cellsFrom` value in `flake.nix`
- `/plutus-apps` is the cell name
- `/plutus-apps/*` are accessible via `cell.*` from `{ inputs, cell }` (while inside `cells/plutus-apps`)
- `/plutus-apps/*` are accessible via `inputs.cells.plutus-apps.*` (everywhere)
- `/marconi` is the cell name
- `/marconi/*` are accessible via `cell.*` from `{ inputs, cell }` (while inside `cells/marconi`)
- `/marconi/*` are accessible via `inputs.cells.marconi.*` (everywhere)
- `/library` is the cell block name
- `/library/*` are accessible via `cell.library.*` (while inside `cells/plutus-apps`)
- `/library/*` are accessible via `cell.library.*` (while inside `cells/marconi`)
- `/library/*` are accessible via `inputs.cells.library.*` (everywhere)
- `/default.nix` imports every file in its directory
- `/default.nix` contains a derivation for each file in its directory
Expand Down
14 changes: 7 additions & 7 deletions __std__/cells/automation/hydra-jobs.nix
View file
Expand Up @@ -2,7 +2,7 @@

let

inherit (inputs.cells.plutus-apps.library) plutus-apps-project pkgs;
inherit (inputs.cells.marconi.library) marconi-project pkgs;
inherit (pkgs.stdenv) system;

make-haskell-jobs = project:
Expand All @@ -19,18 +19,18 @@ let
plan-nix = project.plan-nix;
};

native-plutus-apps-jobs = make-haskell-jobs plutus-apps-project;
native-marconi-jobs = make-haskell-jobs marconi-project;

windows-plutus-apps-jobs = make-haskell-jobs plutus-apps-project.projectCross.mingwW64;
windows-marconi-jobs = make-haskell-jobs marconi-project.projectCross.mingwW64;

other-jobs = inputs.cells.plutus-apps.devshells // inputs.cells.plutus-apps.packages;
other-jobs = inputs.cells.marconi.devshells // inputs.cells.marconi.packages;

jobs =
{ native = native-plutus-apps-jobs; } //
{ native = native-marconi-jobs; } //
# Only cross-compile to windows from linux
pkgs.lib.optionalAttrs (system == "x86_64-linux") { mingwW64 = windows-plutus-apps-jobs; } //
pkgs.lib.optionalAttrs (system == "x86_64-linux") { mingwW64 = windows-marconi-jobs; } //
# Devcontainer is only available on linux
pkgs.lib.optionalAttrs (system == "x86_64-linux") inputs.cells.plutus-apps.devcontainer //
pkgs.lib.optionalAttrs (system == "x86_64-linux") inputs.cells.marconi.devcontainer //
other-jobs;

# Hydra doesn't like these attributes hanging around in "jobsets": it thinks they're jobs!
Expand Down
26 changes: 26 additions & 0 deletions __std__/cells/marconi/devcontainer/README.md
View file
@@ -0,0 +1,26 @@
Skeleton of a devcontainer for working on Plutus projects, based on https://github.com/hamishmack/docker-nixpkgs/blob/hkm/nix-devcontainer/images/devcontainer with some tweaks.

Usage:
1. `docker load < $(nix build .#devcontainer-docker-image)`
2. Create `.devcontainer/devcontainer.json` in your project as below, the "image" property is most important
3. Install the Remote Development extension pack in VSCode
4. Open the folder "in the container"

Example `devcontainer.json`:
```
{
"name": "My Plutus Project",
"image": "plutus-devcontainer:latest",
// Use 'settings' to set *default* container specific settings.json values on container create.
// You can edit these settings after create using File > Preferences > Settings > Remote.
"settings": {
"terminal.integrated.shell.linux": "/bin/bash"
},
// IDs of extensions inside container.
"extensions": [
"haskell.haskell"
],
}
```
6 changes: 6 additions & 0 deletions __std__/cells/marconi/devcontainer/default.nix
View file
@@ -0,0 +1,6 @@
{ inputs, cell }@block:
{
devcontainer-docker-image = import ./devcontainer-docker-image.nix block;

devcontainer-push-docker-image = import ./devcontainer-push-docker-image.nix block;
}

0 comments on commit 6e0c7fe

Please sign in to comment.