Skip to content

Use pty for running procs#62

Merged
rprtr258 merged 2 commits intomasterfrom
use-pty
Nov 27, 2024
Merged

Use pty for running procs#62
rprtr258 merged 2 commits intomasterfrom
use-pty

Conversation

@rprtr258
Copy link
Owner

@rprtr258 rprtr258 commented Nov 27, 2024

first, it is required to run js shit which waits for tty and does nothing until gets it
second, it is required for future attach feature to connect to proc stdin/stdout

Summary by CodeRabbit

  • New Features

    • Enhanced process handling and I/O management through the integration of pseudo-terminal (PTY) operations.
    • Improved error handling and logging for PTY operations.
  • Bug Fixes

    • Updated error message for command execution failures to improve clarity.
  • Refactor

    • Changed method visibility from public to private for better encapsulation.
    • Adjusted expected output format in tests to align with server response changes.

@coderabbitai
Copy link

coderabbitai bot commented Nov 27, 2024

Caution

Review failed

The pull request is closed.

Walkthrough

The changes in this pull request involve updates to several files, primarily focusing on dependency management and error handling. A new dependency, github.com/creack/pty, has been added to the go.mod file to facilitate pseudo-terminal operations. Modifications were made to the startShimImpl function's error message, and enhancements were introduced to the implShim function for improved process handling using the new PTY functionality. Additionally, the visibility of the NoFilters method in the filter struct was changed to private.

Changes

File Change Summary
go.mod Added new dependency: github.com/creack/pty v1.1.24.
internal/cli/impl_start.go Modified error message in startShimImpl function from "running failed: %v" to "run command: %v".
internal/cli/shim.go Added PTY handling with github.com/creack/pty, updated implShim for PTY integration, improved error handling, and added debug logging.
internal/core/filter.go Changed visibility of NoFilters method to private, renaming it to noFilters and updating its usage in FilterFunc.
e2e/e2e_test.go Updated Test_ClientServerNetcat to check for "123\r\n" instead of "123\n" in server output, with additional logging for debugging.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant CLI
    participant PTY

    User->>CLI: Execute command
    CLI->>PTY: Open PTY
    PTY-->>CLI: PTY created
    CLI->>PTY: Set Stdin and Stdout
    CLI->>CLI: Run command with PTY
    CLI->>User: Return command output
Loading

🐰 "In the code where rabbits hop,
New dependencies help us swap.
With PTYs now, we dance and play,
Error messages brightening the way.
Filters hidden, out of sight,
Code's a garden, blooming bright!" 🌼

Warning

Rate limit exceeded

@rprtr258 has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 10 minutes and 26 seconds before requesting another review.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

📥 Commits

Reviewing files that changed from the base of the PR and between 5af96a5 and fea39eb.


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Outside diff range and nitpick comments (5)
internal/cli/impl_start.go (2)

Line range hint 71-89: Critical: Missing PTY implementation for process execution

The current implementation using exec.Cmd with direct file descriptors won't support JavaScript processes requiring a TTY. According to the PR objectives, we need to implement PTY support.

Consider refactoring the command execution to use PTY:

 import (
+	"github.com/creack/pty"
 	// ... other imports
 )

 // ... in startShimImpl
-	cmd := exec.Cmd{
+	cmd := &exec.Cmd{
 		Path: pmExecutable,
 		Args: []string{pmExecutable, _cmdShim.Name(), string(procDesc)},
 		Dir:  proc.Cwd,
 		Env: slices.Collect(func(yield func(string) bool) {
 			for k, v := range maps.All(proc.Env) {
 				if !yield(fmt.Sprintf("%s=%s", k, v)) {
 					break
 				}
 			}
 		}),
-		Stdin:  os.Stdin,
-		Stdout: stdoutLogFile,
-		Stderr: stderrLogFile,
 		SysProcAttr: &syscall.SysProcAttr{
 			Setpgid: true,
 		},
 	}
 	log.Debug().Str("cmd", cmd.String()).Msg("starting")
-	if err := cmd.Start(); err != nil {
+	ptmx, err := pty.Start(cmd)
+	if err != nil {
 		return errors.Wrapf(err, "run command: %v", proc)
 	}
+	defer ptmx.Close()
+
+	// Handle I/O copying in separate goroutines
+	go func() {
+		_, _ = io.Copy(stdoutLogFile, ptmx)
+	}()
+	go func() {
+		_, _ = io.Copy(stderrLogFile, ptmx)
+	}()

This implementation:

  1. Uses pty.Start to create a pseudo-terminal
  2. Handles I/O copying from PTY to log files
  3. Supports processes requiring TTY
  4. Prepares for future stdin/stdout attachment feature

PTY I/O operations are properly handled, but some improvements could be made

The PTY implementation in internal/cli/shim.go already includes:

  • Error handling for PTY opening with proper error wrapping
  • Deferred cleanup of TTY
  • Goroutines for I/O copying between PTY and stdin/stdout
  • Proper process group and session management via SysProcAttr

However, a few minor improvements could be made:

  • The stdin/stdout copy goroutines silently discard errors (_, _ = io.Copy())
  • No explicit cleanup of PTY master (ptmx) file descriptor
🔗 Analysis chain

Line range hint 71-89: Consider error handling for PTY I/O operations

After implementing PTY support, ensure proper error handling for I/O operations.

Let's verify the PTY implementation in other parts of the codebase:

Consider implementing:

  1. Graceful shutdown of PTY and associated goroutines
  2. Error channels for I/O operation failures
  3. Context-based cancellation for cleanup

Let me check the actual PTY-related code to better understand the implementation.


Let me check the complete implementation of the shim to understand the full PTY handling context.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Check for other PTY-related code and potential integration points
rg -l "pty\." .
ast-grep --pattern 'pty.Start($_)'

Length of output: 115


Script:

#!/bin/bash
# Check the content of PTY-related files and their context
rg -A 10 "pty\." ./internal/cli/shim.go ./internal/logrotation/logrotation.go

Length of output: 2443


Script:

#!/bin/bash
# Get the complete shim implementation
cat ./internal/cli/shim.go

Length of output: 7809

go.mod (1)

Line range hint 3-3: Fix incorrect Go version

The specified Go version 1.23.0 does not exist. The latest stable version of Go is 1.22.x.

Apply this diff to fix the version:

-go 1.23.0
+go 1.22
internal/cli/shim.go (2)

200-201: Handle errors from io.Copy operations

The errors returned from io.Copy are currently being ignored. Consider handling these errors to improve error detection and debugging.

Suggested code change:

-	go func() { _, _ = io.Copy(ptmx, os.Stdin) }()
-	go func() { _, _ = io.Copy(outw, ptmx) }()
+	go func() {
+		if _, err := io.Copy(ptmx, os.Stdin); err != nil {
+			log.Error().Err(err).Msg("Error copying from os.Stdin to ptmx")
+		}
+	}()
+	go func() {
+		if _, err := io.Copy(outw, ptmx); err != nil {
+			log.Error().Err(err).Msg("Error copying from ptmx to outw")
+		}
+	}()

211-211: Address TODO: Decide whether Stderr should be directed to tty

The TODO comment indicates uncertainty about where Stderr should be directed. Since both Stdin and Stdout are using the PTY (tty), you might want to direct Stderr to tty for consistent I/O handling. Alternatively, if you need to capture Stderr separately for logging purposes, ensure this aligns with your intended behavior.

Would you like assistance in modifying the code to redirect Stderr to tty, or do you prefer retaining the current setup with Stderr directed to errw?

🧰 Tools
🪛 golangci-lint (1.62.2)

211-211: internal/cli/shim.go:211: Line contains TODO/BUG/FIXME: "TODO: should it go into tty???"

(godox)

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between c6d0f5f and 8cc97c3.

⛔ Files ignored due to path filters (1)
  • go.sum is excluded by !**/*.sum
📒 Files selected for processing (4)
  • go.mod (1 hunks)
  • internal/cli/impl_start.go (1 hunks)
  • internal/cli/shim.go (3 hunks)
  • internal/core/filter.go (2 hunks)
🧰 Additional context used
🪛 golangci-lint (1.62.2)
internal/cli/shim.go

211-211: internal/cli/shim.go:211: Line contains TODO/BUG/FIXME: "TODO: should it go into tty???"

(godox)

🔇 Additional comments (5)
internal/core/filter.go (2)

Line range hint 17-21: LGTM! Good encapsulation practice.

Converting NoFilters to noFilters improves encapsulation by making the method private, which is appropriate since it's only used internally within the package.


74-74: LGTM! Consistent method call update.

The method call has been correctly updated to use the renamed private method.

internal/cli/impl_start.go (1)

88-88: Minor: Error message improvement

The error message change improves clarity, but the underlying implementation needs attention.

go.mod (1)

8-8: LGTM on the pty dependency!

The addition of github.com/creack/pty is appropriate for implementing PTY support. This is a well-maintained library commonly used for pseudo-terminal operations in Go.

internal/cli/shim.go (1)

199-199: 🛠️ Refactor suggestion

Ensure both PTY file descriptors are properly closed

Currently, only tty is being closed in the deferred function. To prevent potential resource leaks, consider closing ptmx as well.

Suggested code change:

-	defer func() { _ = tty.Close() }() // Best effort.
+	defer func() {
+		_ = tty.Close()
+		_ = ptmx.Close()
+	}() // Best effort.

Likely invalid or redundant comment.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Outside diff range and nitpick comments (2)
e2e/e2e_test.go (1)

212-213: Improve debug logging format and consider test output verbosity.

The debug logging is helpful for PTY-related issues, but consider these improvements:

  1. Use a more descriptive message prefix
  2. Consider making the debug output conditional on test verbosity
-t.Logf("server logs:\n%q", string(d))
+t.Logf("PTY server output (quoted):\n%q", string(d))

Additionally, consider using t.Helper() for this test helper function to improve test output when failures occur.

internal/cli/shim.go (1)

195-215: Document PTY implementation details

The PTY implementation successfully addresses the requirements for running JavaScript processes requiring a TTY and prepares for future stdin/stdout attachment features. However, the code would benefit from documentation explaining:

  1. The PTY setup and its purpose
  2. The rationale behind stderr handling
  3. Process group management decisions

Consider adding a comment block explaining these implementation details:

+// implShim implements process management with PTY support.
+// PTY (pseudo-terminal) is required for:
+// 1. Running processes that require a TTY (e.g., certain JavaScript tools)
+// 2. Supporting future features for attaching to process stdin/stdout
+// The implementation maintains separate stderr logging while providing
+// full terminal emulation for stdin/stdout.
 func implShim(proc core.Proc) error {
🧰 Tools
🪛 golangci-lint (1.62.2)

211-211: internal/cli/shim.go:211: Line contains TODO/BUG/FIXME: "TODO: should it go into tty???"

(godox)

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between 8cc97c3 and 5af96a5.

⛔ Files ignored due to path filters (1)
  • go.sum is excluded by !**/*.sum
📒 Files selected for processing (4)
  • e2e/e2e_test.go (1 hunks)
  • go.mod (1 hunks)
  • internal/cli/impl_start.go (1 hunks)
  • internal/cli/shim.go (3 hunks)
🚧 Files skipped from review as they are similar to previous changes (2)
  • go.mod
  • internal/cli/impl_start.go
🧰 Additional context used
🪛 golangci-lint (1.62.2)
internal/cli/shim.go

211-211: internal/cli/shim.go:211: Line contains TODO/BUG/FIXME: "TODO: should it go into tty???"

(godox)

🔇 Additional comments (1)
e2e/e2e_test.go (1)

213-213: LGTM: Line ending change aligns with PTY behavior.

The change from "123\n" to "123\r\n" is correct as PTY emulates a real terminal which converts line endings to CRLF.

Let's verify this is the only place that needs line ending adjustment:

✅ Verification successful

Let me check the specific occurrences in the e2e_test.go file to verify if there are other instances that need CRLF adjustment.


Line ending change is isolated and correctly handled

The search results confirm that this is the only instance of a line ending assertion in the test files, and it's already correctly using CRLF (\r\n). No other instances need adjustment.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Search for other test assertions that might need CRLF adjustment
rg -l '== ".*\\n"' --type go

Length of output: 46


Script:

#!/bin/bash
# Show all test assertions with line endings in e2e_test.go
rg '== ".*\\n"' -A 2 -B 2 e2e/e2e_test.go

Length of output: 214

Comment on lines +195 to +202
ptmx, tty, err := pty.Open()
if err != nil {
return errors.Wrap(err, "open pty")
}
defer func() { _ = tty.Close() }() // Best effort.
go func() { _, _ = io.Copy(ptmx, os.Stdin) }()
go func() { _, _ = io.Copy(outw, ptmx) }()
log.Debug().Any("pty", ptmx.Fd()).Any("tty", tty.Fd()).Msg("pty created")
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Improve PTY setup error handling and resource cleanup

The PTY setup has potential resource leaks and error handling issues:

  1. The master PTY (ptmx) is never explicitly closed
  2. The goroutines copying data could leak on early returns
  3. Copy errors are silently ignored

Consider applying this improvement:

 ptmx, tty, err := pty.Open()
 if err != nil {
   return errors.Wrap(err, "open pty")
 }
-defer func() { _ = tty.Close() }() // Best effort.
+defer func() {
+    if err := tty.Close(); err != nil {
+        log.Error().Err(err).Msg("failed to close tty")
+    }
+    if err := ptmx.Close(); err != nil {
+        log.Error().Err(err).Msg("failed to close ptmx")
+    }
+}()
+
+ctx, cancel := context.WithCancel(context.Background())
+defer cancel()
+
+go func() {
+    if _, err := io.Copy(ptmx, os.Stdin); err != nil {
+        log.Error().Err(err).Msg("failed to copy stdin to ptmx")
+        cancel()
+    }
+}()
+
+go func() {
+    if _, err := io.Copy(outw, ptmx); err != nil {
+        log.Error().Err(err).Msg("failed to copy ptmx to stdout")
+        cancel()
+    }
+}()
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
ptmx, tty, err := pty.Open()
if err != nil {
return errors.Wrap(err, "open pty")
}
defer func() { _ = tty.Close() }() // Best effort.
go func() { _, _ = io.Copy(ptmx, os.Stdin) }()
go func() { _, _ = io.Copy(outw, ptmx) }()
log.Debug().Any("pty", ptmx.Fd()).Any("tty", tty.Fd()).Msg("pty created")
ptmx, tty, err := pty.Open()
if err != nil {
return errors.Wrap(err, "open pty")
}
defer func() {
if err := tty.Close(); err != nil {
log.Error().Err(err).Msg("failed to close tty")
}
if err := ptmx.Close(); err != nil {
log.Error().Err(err).Msg("failed to close ptmx")
}
}()
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
go func() {
if _, err := io.Copy(ptmx, os.Stdin); err != nil {
log.Error().Err(err).Msg("failed to copy stdin to ptmx")
cancel()
}
}()
go func() {
if _, err := io.Copy(outw, ptmx); err != nil {
log.Error().Err(err).Msg("failed to copy ptmx to stdout")
cancel()
}
}()
log.Debug().Any("pty", ptmx.Fd()).Any("tty", tty.Fd()).Msg("pty created")

Comment on lines +209 to +215
Stdin: tty,
Stdout: tty,
Stderr: errw, // TODO: should it go into tty???
SysProcAttr: &syscall.SysProcAttr{
Setpgid: true,
// Setpgid: true,
Setsid: true,
Setctty: true,
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codebase verification

Keep stderr separate from TTY and restore Setpgid

The current implementation correctly handles stderr through log rotation (errw), which is the right approach for maintaining separate log files. The TODO can be resolved by keeping the current design. Additionally, Setpgid should be restored as it's important for proper process group management in PTY sessions.

  • internal/cli/shim.go:
    • Remove the TODO comment as the current stderr handling is correct
    • Uncomment Setpgid: true for proper process group management

The rationale:

  1. The code shows stderr is intentionally handled through log rotation (errw := logrotation.New), which is the correct pattern for maintaining separate log files
  2. Setpgid should be enabled along with Setsid and Setctty for complete PTY session management
🔗 Analysis chain

Review stderr handling and process group settings

A few points need attention:

  1. There's an unresolved TODO about stderr handling
  2. The commented out Setpgid might affect process group management

Let's verify the impact of these settings:

Consider these recommendations:

  1. For stderr handling, you have two options:
    • Redirect to TTY for full terminal emulation
    • Keep separate for better log segregation (current approach)
  2. Document the rationale for commenting out Setpgid

Let me gather more context about the stderr handling patterns in the codebase.


Let me check one more aspect to understand if there are any specific error handling patterns or requirements in the codebase.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Check for potential issues with process group handling
# and stderr redirection in similar Go projects

# Search for patterns of PTY usage with stderr handling
rg -A 5 'pty\.Open\(\)' --type go

# Look for Setpgid usage patterns with PTY
ast-grep --pattern 'SysProcAttr.*{.*Setpgid.*}.*'

Length of output: 401


Script:

#!/bin/bash
# Search for similar command execution patterns with stderr handling
rg -B 3 -A 3 'Stderr:.*tty' --type go

# Look for any comments or patterns related to process groups
rg -B 2 -A 2 'Setpgid.*=.*true' --type go

# Check for any documentation about PTY handling
fd README.md | xargs rg -i "pty|terminal"

Length of output: 2056


Script:

#!/bin/bash
# Check the context around the PTY usage to understand error handling patterns
rg -B 10 -A 10 'pty\.Open\(\)' --type go

# Look for any logging patterns related to stderr
rg 'errw.*:=' --type go

Length of output: 1126

🧰 Tools
🪛 golangci-lint (1.62.2)

211-211: internal/cli/shim.go:211: Line contains TODO/BUG/FIXME: "TODO: should it go into tty???"

(godox)

@rprtr258 rprtr258 merged commit 52a0c71 into master Nov 27, 2024
@rprtr258 rprtr258 deleted the use-pty branch November 27, 2024 01:34
@coderabbitai coderabbitai bot mentioned this pull request Nov 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant