Skip to content

chore: remove buildkit dependency #2864

Merged
mergify[bot] merged 6 commits intoaws:mainlinefrom
efekarakus:dockerfile-lexer
Sep 23, 2021
Merged

chore: remove buildkit dependency #2864
mergify[bot] merged 6 commits intoaws:mainlinefrom
efekarakus:dockerfile-lexer

Conversation

@efekarakus
Copy link
Copy Markdown
Contributor

Removing the dependency saves us 3.2MB of space and brings us down to 42MB.
The bulk of the lexer is heavily inspired by https://cs.opensource.google/go/go/+/refs/tags/go1.17.1:src/text/template/parse/lex.go

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License.

@efekarakus efekarakus requested a review from a team as a code owner September 22, 2021 18:46
@efekarakus efekarakus requested review from iamhopaul123 and removed request for a team September 22, 2021 18:46
Copy link
Copy Markdown
Contributor

@iamhopaul123 iamhopaul123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a really mind-blowing PR! Thank you for sharing this idea 🎉


// parse parses the contents of a Dockerfile into a Dockerfile struct.
func parse(content string) (*Dockerfile, error) {
func parse(name, content string) (*Dockerfile, error) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe

Suggested change
func parse(name, content string) (*Dockerfile, error) {
func parse(lexer *lexer) (*Dockerfile, error) {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kept it as is because i removed name from the lexer following Wanxian's comment, so now I'm writing the name information in this function!


const (
exposeInstrMarker = "expose " // start of an EXPOSE instruction.
heathCheckInstrMarker = "healthcheck " // start of a HEALTHCHECK instruction.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit:

Suggested change
heathCheckInstrMarker = "healthcheck " // start of a HEALTHCHECK instruction.
healthCheckInstrMarker = "healthcheck " // start of a HEALTHCHECK instruction.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lol oops, changed the const names to markerExposeInstr and markerHealthCheckInstr

Comment on lines +223 to +224
idx := strings.Index(normalized, instrMarker) + len(instrMarker)
return line[idx:]
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe trimPrefix?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TrimPrefix as is wouldn't work because it's possible that the instruction has leading spaces, for example:

            EXPOSE 8080

But good point! to be safe with the idx I added the following guard-statement:

if !strings.Contains(normalized, instrMarker) {
	return line
}

func trimContinuationLineMarker(line string) string {
for _, marker := range lineContinuationMarkers {
if strings.HasSuffix(line, marker) {
return line[:len(line)-1-len(marker)]
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe trimSuffix?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!


// trimLeadingWhitespaces removes any leading space characters.
func trimLeadingWhitespaces(line string) string {
return strings.TrimLeftFunc(line, unicode.IsSpace)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe https://pkg.go.dev/strings#TrimSpace to trim both leading and trailing spaces?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mostly just followed the logic from buildkit, so I'm hesitant to change it in case it results in poor parsing

)

const (
exposeInstrMarker = "expose " // start of an EXPOSE instruction.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or instrMarkerExpose and instrMarkerHealthcheck?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

renamed to markerExposeInstr and markerHealthCheckInstr!

}

// stateFn represents a state machine transition of the scanner going from one INSTRUCTION to the next.
type stateFn func(*lexer) stateFn
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤯 👏🏼

func (lex *lexer) readLine() (isEOF bool, err error) {
if ok := lex.scanner.Scan(); !ok {
if err := lex.scanner.Err(); err != nil {
return false, fmt.Errorf("scan Dockerfile %s: %w", lex.name, err)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if the lex's error cares about the file name information 🤔 If lex is just scanning and processing the content, maybe the file name information would be better suited in the error that wraps this error, that is in the caller error? Just a thought 💭

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point! removed

func trimContinuationLineMarker(line string) string {
for _, marker := range lineContinuationMarkers {
if strings.HasSuffix(line, marker) {
return line[:len(line)-1-len(marker)]
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the extra -1 for space " "?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

omg that was a bug! The test cases passed because all my line-wraps had an extra space abc \ instead of abc\.
I've added a new test case and removed the -1

}

// stateFn represents a state machine transition of the scanner going from one INSTRUCTION to the next.
type stateFn func(*lexer) stateFn
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

computer science working 💻 🔮

Copy link
Copy Markdown
Contributor

@Lou1415926 Lou1415926 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit: !!!

@mergify mergify bot merged commit cbf4349 into aws:mainline Sep 23, 2021
@efekarakus efekarakus deleted the dockerfile-lexer branch September 23, 2021 17:11
thrau pushed a commit to localstack/copilot-cli-local that referenced this pull request Dec 9, 2022
Removing the dependency saves us 3.2MB of space and brings us down to 42MB.
The bulk of the lexer is heavily inspired by https://cs.opensource.google/go/go/+/refs/tags/go1.17.1:src/text/template/parse/lex.go

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the Apache 2.0 License.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants