Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multi protobuf module emission and consumption #2344

Merged
merged 25 commits into from
Sep 8, 2021

Conversation

jared-barocsi
Copy link
Contributor

@jared-barocsi jared-barocsi commented Aug 31, 2021

Contributor Checklist

  • Did you add Scaladoc to every public function/method?
  • Did you update the FIRRTL spec to include every new feature/behavior?
  • Did you add at least one test demonstrating the PR?
  • Did you delete any extraneous printlns/debugging code?
  • Did you specify the type of improvement?
  • Did you state the API impact?
  • Did you specify the code generation impact?
  • Did you request a desired merge strategy?
  • Did you add text to be included in the Release Notes for this change?

Type of Improvement

  • new feature/API

API Impact

No API impact

Backend Code Generation Impact

Can change ordering of modules. MultiInfos do not serialize properly because of placeholder code in the protobuf emitter.

Desired Merge Strategy

  • Squash and merge

Release Notes

Implement multiple protobuf emission and consumption in the FIRRTL compiler

  • -p option exports all modules into individual protobuf files, similar to -e.
  • -I option takes a directory of FIRRTL files, deserializes them and combines them back into a single circuit. Currently supports only the protobuf files emitted by -p

Reviewer Checklist (only modified by reviewer)

  • Did you add the appropriate labels?
  • Did you mark the proper milestone (1.2.x, 1.3.0, 1.4.0) ?
  • Did you review?
  • Did you check whether all relevant Contributor checkboxes have been checked?
  • Did you mark as Please Merge?

Copy link
Contributor

@jackkoenig jackkoenig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally looks good although a few minor issues. Also obviously needs tests but it is a Draft for now 🙂

src/main/scala/firrtl/Emitter.scala Outdated Show resolved Hide resolved
src/main/scala/firrtl/Utils.scala Outdated Show resolved Hide resolved
src/main/scala/firrtl/proto/FromProto.scala Outdated Show resolved Hide resolved
src/main/scala/firrtl/stage/FirrtlAnnotations.scala Outdated Show resolved Hide resolved
new ShellOption[String](
longOption = "input-directory",
toAnnotationSeq = a => Seq(FirrtlDirectoryAnnotation(a)),
helpText = "A directory of FIRRTL files",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can these be either .fir or .pb files? A mixture?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should figure that out. I think to start we should make it be only .fir or only .pb files. Note that -i actually doesn't care about file extensions, it just tries to deserialize as protobuf and if it fails it deserializes as .fir but I think it's okay to require correct file extensions here.

@jared-barocsi jared-barocsi marked this pull request as ready for review September 7, 2021 23:01
@jackkoenig jackkoenig added this to the 1.4.x milestone Sep 7, 2021
Copy link
Contributor

@jackkoenig jackkoenig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good and a nice suite of tests! Just some suggestions for improved error messages then let's get this merged.

src/main/scala/firrtl/proto/FromProto.scala Outdated Show resolved Hide resolved
src/main/scala/firrtl/proto/FromProto.scala Outdated Show resolved Hide resolved
src/main/scala/firrtl/Utils.scala Outdated Show resolved Hide resolved
src/main/scala/firrtl/Utils.scala Outdated Show resolved Hide resolved
src/test/scala/firrtlTests/UtilsSpec.scala Outdated Show resolved Hide resolved
@jackkoenig jackkoenig merged commit 0c1ca58 into chipsalliance:master Sep 8, 2021
mergify bot pushed a commit that referenced this pull request Sep 8, 2021
* Add compiler option (`-p`) to emit individual module protobufs
* Implement multi module combination when reading directory of protobufs

Co-authored-by: Jack Koenig <koenig@sifive.com>
(cherry picked from commit 0c1ca58)
@mergify mergify bot added the Backported This PR has been backported to marked stable branch label Sep 8, 2021
mergify bot added a commit that referenced this pull request Sep 8, 2021
* Multi protobuf module emission and consumption (#2344)

* Add compiler option (`-p`) to emit individual module protobufs
* Implement multi module combination when reading directory of protobufs

Co-authored-by: Jack Koenig <koenig@sifive.com>
(cherry picked from commit 0c1ca58)

* Multi PB backport fixes (#2353)

Co-authored-by: Jared Barocsi <82000041+jared-barocsi@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Backported This PR has been backported to marked stable branch
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants