Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New flatten buildpacks/builder implementation - Part 2 - Implementing RFC-0123 #1985

Merged
merged 11 commits into from
Jan 22, 2024

Conversation

jjbustamante
Copy link
Member

@jjbustamante jjbustamante commented Nov 20, 2023

Summary

This PR implements RFC-0123.

The main idea can be summarize with the following class diagram:

classDiagram

    ManagedCollection <|-- managedCollection
    managedCollection <|-- managedCollectionV1
    managedCollection <|-- managedCollectionV2

    class ManagedCollection {
        <<interface>>
    }

     ModuleInfo --* ModuleInfos
     ModuleInfos --*  FlattenModuleInfos
     managedCollectionV2 -- FlattenModuleInfos

    class ModuleInfo {
        FullName() string
    } 

    class ModuleInfos {
        <<interface>>
        BuildModule() []dist.ModuleInfo
    }

    class FlattenModuleInfos {
	    <<interface>>
        FlattenModules() []ModuleInfos
    }

    class managedCollection {
             +AllModules()
	    +ExplodedModules() 
	    +FlattenedModules()
	    +ShouldFlatten()
    }

    class managedCollectionV1 {
        +AddModules()
    }

    class managedCollectionV2 {
        +AddModules()

    }

Our first attempt to implement the flatten feature added a ManagedCollection structure, I extracted the methods to a new interface. managedCollection is a base implementation with all the common code for each implementation. The idea was to keep our initial implementation on managedCollectionV1 and implement the new behavior with managedCollectionV2.

According to the RFC, the user is responsible of defining which buildpacks they want to flatten. In order to represent this we added the FlattenModuleInfos interface. It is a two dimensional array with the group of modules we want to put together.

managedCollectionV2 uses FlattenModuleInfos to determine how to flatten the modules when they are added.

Output

Before

After

Documentation

  • Should this change be documented?
    • Yes, see #___
    • No

Related

Resolves #1880

During builder creation, end-users can provide the flag `--flatten` with
the buildpacks they want to put in one layer.

Signed-off-by: Juan Bustamante <jbustamante@vmware.com>
@github-actions github-actions bot added the type/enhancement Issue that requests a new feature or improvement. label Nov 20, 2023
@github-actions github-actions bot added this to the 0.33.0 milestone Nov 20, 2023
Copy link
Member

@natalieparellano natalieparellano left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jjbustamante apologies for being so slow to review this one. In general it looks good to me - I made some suggestions to potentially help readability but otherwise it looks like it does what it's supposed to do. It might make sense to add an acceptance test if we don't have one already.

builder/buildpack_identifier.go Outdated Show resolved Hide resolved
internal/builder/builder.go Outdated Show resolved Hide resolved
pkg/buildpack/package.go Outdated Show resolved Hide resolved
pkg/buildpack/managed_collection.go Outdated Show resolved Hide resolved
pkg/buildpack/managed_collection.go Outdated Show resolved Hide resolved
pkg/client/create_builder.go Outdated Show resolved Hide resolved
pkg/client/create_builder.go Outdated Show resolved Hide resolved
Comment on lines +11 to +17
type ModuleInfos interface {
BuildModule() []dist.ModuleInfo
}

type FlattenModuleInfos interface {
FlattenModules() []ModuleInfos
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any way to combine these two interfaces? Is there a reason for them to be different?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is actually a way of having a two dimensional array.
FlattenModules() -> return an array of ModuleInfos where each element is an array of dist.ModuleInfo

pkg/buildpack/managed_collection.go Outdated Show resolved Hide resolved
pkg/buildpack/managed_collection.go Outdated Show resolved Hide resolved
Base automatically changed from enhancement/jjbustamante/flatten-1880/part-1 to main December 20, 2023 20:17
jjbustamante and others added 4 commits December 20, 2023 17:41
Signed-off-by: Juan Bustamante <jbustamante@vmware.com>
Signed-off-by: Juan Bustamante <jbustamante@vmware.com>
Signed-off-by: Juan Bustamante <juan.bustamante@broadcom.com>
Signed-off-by: Juan Bustamante <jbustamante@vmware.com>
@jjbustamante jjbustamante marked this pull request as ready for review January 17, 2024 17:07
@jjbustamante jjbustamante requested review from a team as code owners January 17, 2024 17:07
Copy link

codecov bot commented Jan 17, 2024

Codecov Report

Attention: 2 lines in your changes are missing coverage. Please review.

Comparison is base (a51140e) 79.49% compared to head (c044cb7) 79.63%.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1985      +/-   ##
==========================================
+ Coverage   79.49%   79.63%   +0.15%     
==========================================
  Files         175      176       +1     
  Lines       13135    13199      +64     
==========================================
+ Hits        10440    10510      +70     
+ Misses       2026     2020       -6     
  Partials      669      669              
Flag Coverage Δ
os_linux 78.53% <98.43%> (+0.13%) ⬆️
os_macos 76.39% <98.43%> (?)
os_windows 79.02% <98.43%> (+0.15%) ⬆️
unit 79.63% <98.43%> (+0.15%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

@@ -166,17 +163,9 @@ func constructBuilder(img imgutil.Image, newName string, errOnMissingLabel bool,
return bldr, nil
}

func FlattenAll() BuilderOption {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • FlattenAll came from my first pull request (see Part 1)
  • Because the RFC is changing the pack builder create interface, now the end-user must define which buildpacks must be flatten together, that's why I found more readable to use WithFlatten as name to the option method

}
return &buildpack.ManagedCollection{}
return nil
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I should return an managedCollectionV1 instance here.

Publish bool
BuilderTomlPath string
Registry string
Policy string
FlattenExclude []string
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The --flaten-exclude flag is being removed according to the RFC

Copy link
Member

@natalieparellano natalieparellano left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work @jjbustamante. I left a few more nits - I think it could be helpful to add some more tests, but overall this is looking good.

internal/builder/builder.go Outdated Show resolved Hide resolved
internal/commands/builder_create_test.go Outdated Show resolved Hide resolved
internal/commands/builder_create_test.go Outdated Show resolved Hide resolved
internal/commands/builder_create_test.go Outdated Show resolved Hide resolved
pkg/client/create_builder_test.go Outdated Show resolved Hide resolved
pkg/buildpack/managed_collection.go Outdated Show resolved Hide resolved
pkg/buildpack/managed_collection.go Outdated Show resolved Hide resolved
pkg/buildpack/managed_collection.go Show resolved Hide resolved
pkg/buildpack/managed_collection.go Show resolved Hide resolved
pkg/buildpack/build_module_info.go Show resolved Hide resolved
jjbustamante and others added 5 commits January 19, 2024 15:02
Signed-off-by: Juan Bustamante <jbustamante@vmware.com>
Signed-off-by: Juan Bustamante <jbustamante@vmware.com>
Signed-off-by: Juan Bustamante <jbustamante@vmware.com>
Signed-off-by: Juan Bustamante <jbustamante@vmware.com>
@jjbustamante
Copy link
Member Author

@natalieparellano .. if you can take one more look will be awesome

Copy link
Member

@natalieparellano natalieparellano left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me :)

pkg/buildpack/managed_collection.go Outdated Show resolved Hide resolved
pkg/buildpack/managed_collection_test.go Outdated Show resolved Hide resolved
pkg/buildpack/managed_collection_test.go Outdated Show resolved Hide resolved
pkg/buildpack/managed_collection_test.go Outdated Show resolved Hide resolved
pkg/buildpack/managed_collection_test.go Outdated Show resolved Hide resolved
Signed-off-by: Juan Bustamante <jbustamante@vmware.com>
@jjbustamante jjbustamante merged commit 85e3e48 into main Jan 22, 2024
18 checks passed
@jjbustamante jjbustamante deleted the enhancement/jjbustamante/flatten-1880/part-2 branch January 22, 2024 21:10
@edmorley
Copy link
Contributor

edmorley commented Feb 5, 2024

I couldn't get this feature to work? I filed:
#2050

Also I filed a usability feature request:
#2051

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/enhancement Issue that requests a new feature or improvement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Flatten Feature
3 participants