feat(output): add csv and ndjson output modes for list commands#278
feat(output): add csv and ndjson output modes for list commands#278ISTHA1607 wants to merge 1 commit into
Conversation
Extends --output to accept csv/ndjson in addition to json/text, for project list, test list, and test result --history. Enables piping CLI output directly into spreadsheets/log pipelines without custom parsing.
|
✅ This PR is linked to an issue assigned to @ISTHA1607 — thanks! The |
github-actions
Bot
added
the
needs-issue
WalkthroughCSV and NDJSON output modes are added for supported list commands. Shared renderers validate modes, escape CSV fields, emit newline-delimited JSON, and keep pagination metadata on stderr. Project and test commands now use command-specific column mappings and expanded validation tests. ChangesCSV and NDJSON list output
Estimated code review effort: 4 (Complex) | ~45 minutes Possibly related issues
Possibly related PRs
Suggested reviewers: Sequence Diagram(s)sequenceDiagram
participant Command as project list or test list
participant Output
participant stdout
participant stderr
Command->>Output: render CSV or NDJSON rows
Output->>stdout: write parseable row data
Command->>stderr: write pagination cursor and metadata
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
github-actions
Bot
removed
the
needs-issue
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/lib/output.ts`:
- Around line 50-61: Update the NDJSON rendering flow around renderNdjson and
printNdjson to use the same selected ListColumn projections as CSV, emitting
only each column’s header/value fields per row. If the intended contract is
raw-row JSON instead, explicitly document that behavior in the relevant comments
and documentation rather than implying column alignment.
- Around line 69-76: Update csvEscapeField to neutralize spreadsheet
formula-triggering values by prefixing the required safe character when the
rendered field begins with =, +, -, @, tab, or CR, then apply CSV
quoting/escaping to the resulting literal value. Preserve null/undefined
handling and existing RFC 4180 behavior for all other fields.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: 4ac9c48e-c169-4bde-ad63-9a13d7055f67
⛔ Files ignored due to path filters (2)
package-lock.jsonis excluded by!**/package-lock.json,!package-lock.jsontest/__snapshots__/help.snapshot.test.ts.snapis excluded by!**/*.snap,!**/*.snap
📒 Files selected for processing (6)
src/commands/doctor.test.tssrc/commands/project.test.tssrc/commands/project.tssrc/commands/test.tssrc/index.tssrc/lib/output.ts
| /** | ||
| * One column of a CSV/NDJSON list rendering, derived straight from a | ||
| * source-of-truth wire type (e.g. `CliProject`, `CliTest`, | ||
| * `RunHistoryItem`) rather than the (possibly reordered/truncated) text | ||
| * table columns used for `--output text`. | ||
| */ | ||
| export interface ListColumn<T> { | ||
| /** CSV header cell / JSON-ish field name for this column. */ | ||
| header: string; | ||
| /** Extract the raw value for this column from one row. */ | ||
| value: (row: T) => unknown; | ||
| } |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Compare CliTest's field set against TEST_CSV_COLUMNS headers.
ast-grep run --pattern 'interface CliTest { $$$ }' --lang typescript src/commands/test.ts
rg -n 'TEST_CSV_COLUMNS' -A 20 src/commands/test.ts | head -30Repository: TestSprite/testsprite-cli
Length of output: 4298
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== output.ts outline =="
ast-grep outline src/lib/output.ts --view expanded || true
echo "== relevant output.ts sections =="
sed -n '1,80p;150,225p' src/lib/output.ts | cat -n
echo "== TEST_CSV_COLUMNS full =="
sed -n '10245,10275p' src/commands/test.ts | cat -n
echo "== CSV table constants =="
rg -n ': ReadonlyArray<ListColumn<.*>>(?:\n[ \t]+\$|=)' src -g '*.ts' | head -20 || true
echo "== print/select implementations =="
rg -n 'function print|const print|renderNdjson|printNdjson|printCsv|renderCsv' src/lib/output.ts src/commands -g '*.ts' -A 25 -B 5
echo "== tests around NDJSON =="
rg -n 'ndjson|printNdjson|renderNdjson|ndjson|PROJECT_CSV_COLUMNS|TEST_CSV_COLUMNS' src -g '*test*.ts' -A 8 -B 4Repository: TestSprite/testsprite-cli
Length of output: 50382
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== Ndjson list tests =="
rg -n 'output: '\''ndjson'\''|output: "\'ndjson\'"|ndjson|printNdjson|renderNdjson' src/commands src/lib -g '*.ts' -A 20 -B 8 | head -220
echo "== CSV constants and list fixtures =="
rg -n '(PROJECT|TEST|RUN_HISTORY)_CSV_COLUMNS' src/commands/src/lib -g '*.ts' -A 35
echo "== CliProject full =="
rg -n 'interface CliProject|export interface CliProject' src/commands -g '*.ts' -A 120 | head -180
echo "== RunHistoryItem =="
rg -n 'interface RunHistoryItem|export interface RunHistoryItem' src/commands src/lib -g '*.ts' -A 120 | head -160Repository: TestSprite/testsprite-cli
Length of output: 248
Keep NDJSON output aligned with selected columns
renderNdjson writes JSON.stringify(row), and printNdjson receives no ListColumn selector, so list NDJSON renders every serializable field even though CSV uses selected column projections. If NDJSON is meant to mirror the documented CSV field set, pass/select columns into the NDJSON path; otherwise, make NDJSON raw-row output explicit in docs/comments.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/lib/output.ts` around lines 50 - 61, Update the NDJSON rendering flow
around renderNdjson and printNdjson to use the same selected ListColumn
projections as CSV, emitting only each column’s header/value fields per row. If
the intended contract is raw-row JSON instead, explicitly document that behavior
in the relevant comments and documentation rather than implying column
alignment.
| export function csvEscapeField(raw: unknown): string { | ||
| if (raw === null || raw === undefined) return ''; | ||
| const s = String(raw); | ||
| if (/[",\r\n]/.test(s)) { | ||
| return `"${s.replace(/"/g, '""')}"`; | ||
| } | ||
| return s; | ||
| } |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "## locate output.ts and relevant files"
fd -a 'output\.ts$|samples\.ts$|DOCUMENTATION\.md$' . | sed 's#^\./##' | sort | head -80
echo
echo "## git diff stat"
git diff --stat || true
echo
echo "## inspect src/lib/output.ts around csvEscapeField"
if [ -f src/lib/output.ts ]; then
wc -l src/lib/output.ts
sed -n '1,180p' src/lib/output.ts | cat -n
fi
echo
echo "## inspect usage of csvEscapeField/printCsv"
rg -n "csvEscapeField|printCsv|output mode|csv|ndjson" src lib . --glob '!node_modules' --glob '!dist' --glob '!build' | head -200
echo
echo "## inspect relevant sample/dry-run list files"
if [ -f src/lib/dry-run/samples.ts ]; then
wc -l src/lib/dry-run/samples.ts
sed -n '1,220p' src/lib/dry-run/samples.ts | cat -n
fiRepository: TestSprite/testsprite-cli
Length of output: 22427
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "## inspect CSV column definitions in project.ts and test.ts"
sed -n '90,1125p' src/commands/project.ts | cat -n
echo
sed -n '4645,4820p' src/commands/test.ts | cat -n
echo
sed -n '10200,10340p' src/commands/test.ts | cat -n
echo
echo "## inspect dry-run sample CSV values for formula-like triggers"
python3 - <<'PY'
from pathlib import Path
p = Path('src/lib/dry-run/samples.ts')
s = p.read_text()
for i, line in enumerate(s.splitlines(), 1):
if any(c in line for c in '=+-@') or 'PROJECT_CSV_COLUMNS' in line or 'TEST_CSV_COLUMNS' in line or 'RUN_HISTORY_CSV_COLUMNS' in line:
print(f'{i}: {line}')
PY
echo
echo "## read-only behavioral probe of current csvEscapeField semantics"
python3 - <<'PY'
import re
def csv_escape_field(raw):
if raw is None or raw is None:
return ''
s = str(raw)
if re.search(r'["\r\n]', s):
# This simple model doubles only existing double quotes and wraps.
return '"' + s.replace('"', '""') + '"'
return s
for raw in ['=cmd|\'/c calc\'!A1', '+123', '-123', '`@cell`', '\ttab', '\rcr', 'normal,quote"quote', 'cmd|cmd|A1']:
print(repr(raw), '=>', repr(csv_escape_field(raw)))
PYRepository: TestSprite/testsprite-cli
Length of output: 50381
🌐 Web query:
OWASP CSV Injection spreadsheet application formula characters = + - @ leading tab CR mitigation prefix apostrophe
💡 Result:
CSV Injection (or Formula Injection) occurs when untrusted user input is embedded in CSV files, causing spreadsheet applications (like Microsoft Excel, LibreOffice Calc, or Google Sheets) to interpret cell content as executable formulas [1][2]. Attackers can use this to perform actions ranging from data exfiltration to potential remote code execution via DDE or other "gadget" features [2][3]. Mitigation is difficult because there is no single universal standard for CSV parsing across different spreadsheet applications [1][4]. Security best practices involve the following approaches: Core Mitigation Techniques 1. Single Quote Prefix: Prepending a single quote (') to a field that begins with a formula-triggering character (=, +, -, @, |, \t, or \r) is a commonly cited mitigation intended to force the spreadsheet to treat the cell as literal text [2][5][6]. However, its effectiveness varies; some applications may display the quote or treat it inconsistently [7]. 2. Tab Character Prefix: To more reliably prevent formula execution in Microsoft Excel, it is recommended to prefix cells starting with formula-triggering characters with a tab character (0x09) inside a quoted field (e.g., "\t=1+1") [1][2]. Note that this makes the tab character part of the data, which may interfere with downstream programmatic processing [1][4]. 3. Structural Protection: Always encapsulate every cell field in double quotes and escape internal double quotes by doubling them (e.g., "") to prevent attackers from breaking out of the cell context [1][4][7]. Important Considerations - Microsoft Excel behavior: Excel may remove quotes or escape characters when a file is saved and re-opened, potentially re-activating formulas that were previously neutralized [1][2][3]. - Defense-in-depth: Relying solely on CSV output sanitization is insufficient. Implement input validation at the point of data entry to reject or sanitize inputs that begin with dangerous characters [8]. - Unicode Variants: Be aware that full-width or double-byte variants of formula characters (e.g., ＝, ＋, －, ＠) may also be interpreted as formulas in some locales and should be included in sanitization logic [1][4]. There is no perfect, universal solution. Organizations should assess the specific spreadsheet software used by their users and consider the trade-offs between security and data integrity [1][2][6].
Citations:
- 1: https://owasp.org/www-community/attacks/CSV_Injection
- 2: https://owasp.org/www-project-web-security-testing-guide/latest/4-Web_Application_Security_Testing/07-Input_Validation_Testing/21-Testing_for_CSV_Injection
- 3: https://github.com/OWASP/www-project-web-security-testing-guide/blob/master/latest/4-Web_Application_Security_Testing/07-Input_Validation_Testing/21-Testing_for_CSV_Injection.md
- 4: https://github.com/OWASP/www-community/blob/master/pages/attacks/CSV_Injection.md
- 5: https://security.stackexchange.com/questions/190846/preventing-dangerous-csv-formula-injection
- 6: https://cwe.mitre.org/data/definitions/1236.html
- 7: https://www.usenix.org/system/files/woot25-karl.pdf
- 8: https://www.securecodinghub.com/guides/csv-injection
Neutralize CSV formula-trigger values before rendering --output csv.
csvEscapeField only handles RFC 4180 delimiters/quotes, while project list, test list, and test result --history may render project/test names or metadata through CSV. Values starting with formula characters such as =, +, -, @, or leading tab/CR can be evaluated by spreadsheets; prefix neutralization should be paired with proper CSV quoting for that literal field.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/lib/output.ts` around lines 69 - 76, Update csvEscapeField to neutralize
spreadsheet formula-triggering values by prefixing the required safe character
when the rendered field begins with =, +, -, @, tab, or CR, then apply CSV
quoting/escaping to the resulting literal value. Preserve null/undefined
handling and existing RFC 4180 behavior for all other fields.
|
Maintainer review: accepted — one nod pending before merge. Verified: no new dependency (the lockfile hunk is just the stale version-field sync), RFC 4180 escaping with doubled quotes and CR/LF handling, columns derived from the wire types rather than the text-table layout, non-list commands reject csv/ndjson with a typed VALIDATION_ERROR at print time, and CI is green on all three OSes — the local |
2 participants
Summary
--outputto acceptcsv/ndjsonin addition tojson/text, forproject list,test list, andtest result --history.Closes #164
Test plan
npm test: 2019 passed, 3 skipped, 7 failed all 7 failures are insrc/lib/ticker.spec.tsand reproduce identically on a cleanmaincheckout (unrelated pre-existing flakiness, confirmed before opening this PR).npx eslinton changed files: clean.