plan9asmll: complete target matrix + llc compile verification

[cpunion]

Summary

• switch translation pipeline to produce LLVM Module via goplus/llvm path
• keep plan9asmll as standalone cmd module and improve reporting (per-file OK/FAIL with reasons)
• complete std asm coverage for amd64/arm64/386 across darwin/linux/windows target matrix
• add optional llc compile verification to plan9asmll (-compile, -llc, -keep-obj)
• tune llc feature flags for amd64/arm64 so matrix compile checks are stable

Validation

• go test ./...
• go run -C cmd/plan9asmll . -all-targets -patterns=std -out _out/plan9asmll -report /tmp/plan9asmll-report-compile.json -compile
  • result: matrix finished: success=419 total=419

[gemini-code-assist]

Summary of Changes

Hello @cpunion, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly advances the plan9asm project by integrating direct LLVM module generation, expanding the supported instruction sets for AMD64 and ARM64 architectures, and improving the robustness of symbol and memory operand parsing. These changes are crucial for broadening the range of Go standard library assembly files that can be successfully translated into LLVM IR, paving the way for more comprehensive compilation and verification across diverse target platforms. A new command-line tool facilitates systematic testing and reporting of translation progress.

Highlights

• LLVM Module Generation: The translation pipeline has been switched to produce LLVM Modules directly via the goplus/llvm path, enhancing integration and verification capabilities. A new TranslateModule function is introduced, with the existing Translate function now calling it and serializing the result to text.
• Expanded Instruction Set Support (AMD64): Significant additions have been made to the AMD64 instruction lowering, including support for various arithmetic operations (e.g., ADCQ, SBBQ, MULQ, DIVL, IMULQ), bit manipulation (e.g., ROLQ, RORQ, BSWAPL), conditional moves (CMOVQEQ), atomic operations (XCHG*), and a wide range of vector/SIMD instructions (e.g., AESENC, VPXOR, VMOVDQA, VPSHUFB, VPALIGNR). Memory and symbol operands are now supported for many existing instructions.
• Expanded Instruction Set Support (ARM64): ARM64 instruction lowering has been substantially extended to cover more arithmetic operations (e.g., ADC, SBC, MUL, UMULH, UDIV), bit shifts (LSLW, ASR, EXTR), conditional selects (CSELW), and a broad set of NEON vector instructions (e.g., AESE, AESD, VST1, VLD1 lane forms). Register and memory targets are now supported for branch and call instructions.
• Improved Symbol and Memory Operand Resolution: The parser and translator now handle more complex forms of symbol references and memory operands, including symbolic offsets in DATA and GLOBL directives, and various forms of indirect addressing in AMD64 and ARM64.
• Enhanced Preprocessing and Macro Expansion: The preprocessor now supports #if and #elif directives, and macro expansion has been improved to correctly handle inline function-like macros and object-like macros, leading to more robust parsing of complex assembly files.
• New plan9asmll Tool for Matrix Verification: A new command-line tool, plan9asmll, has been added. It compiles Go assembly files to LLVM IR, optionally verifies them with llc across a matrix of GOOS/GOARCH targets, and generates reports on unsupported instructions, aiding in comprehensive coverage testing.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

• amd64_blocks.go
  • Expanded conditional jump instruction recognition to include JMP, JLO, JHI, JHS.
• amd64_ctx.go
  • Added flagsOFSlot to amd64Ctx for overflow flag modeling.
  • Improved evalFPToI64 to handle various integer and pointer type conversions for FP offsets.
  • Enhanced storeFPResult with more flexible integer and pointer type casting logic.
  • Modified parseSBRef to accept bare symbol names and improved ptrFromSB to trim '$' prefix from base symbols.
• amd64_eval.go
  • Updated evalI64 to support OpMem and OpSym operands for loading 64-bit values.
• amd64_lower_arith.go
  • Added lowering for PUSHFQ, POPFQ, LFENCE, MFENCE, SFENCE, PAUSE, PREFETCHNTA, UNDEF, RDTSC, CPUID, XGETBV, RDTSCP.
  • Implemented MOVSB, MOVSQ, STOSQ for memory block operations.
  • Added NEGL for 32-bit negation and RCRQ for rotate-through-carry.
  • Extended ADDQ, SUBQ, XORQ, ANDQ, ORQ to support memory destinations.
  • Implemented ADCQ, SBBQ for add/subtract with carry/borrow, and ADCXQ, ADOXQ for BMI2 carry chains.
  • Extended ADDL, SUBL, XORL, ANDL, ORL to support memory destinations.
  • Added XORB, ANDB, ORB for 8-bit logical operations.
  • Implemented INCL, DECL for 32-bit increment/decrement.
  • Improved LEAQ, LEAL to handle FP offsets more robustly.
  • Added CMOVQEQ, CMOVQNE, CMOVQCS, CMOVQCC, CMOVQGT for conditional moves.
  • Implemented ANDNL, ANDNQ for BMI1 bitwise NOT-AND.
  • Extended SHRQ, SHLQ, SARQ, SHLL, SHRL, SALQ, SALL to support 3-operand forms.
  • Added ROLQ, RORQ, RORL for rotate operations.
  • Implemented RORXL, RORXQ for BMI2 rotate-right without flags.
  • Added NOTQ for 64-bit bitwise NOT.
  • Implemented BSWAPL for 32-bit byte swap.
  • Added MULQ, MULXQ, MULL, DIVL, IMULQ, IMUL3Q for multiplication and division operations.
• amd64_lower_atomic.go
  • Extended XCHGB, XCHGL, XCHGQ to support register-to-register exchanges and symbol operands.
• amd64_lower_branch.go
  • Expanded CALL instruction to support register and memory targets.
  • Added JHI, JLO, JHS conditional jump aliases.
  • Improved JMP instruction to support register and memory targets for tail calls.
  • Enhanced PC-relative target resolution for conditional branches.
  • Introduced callIndirectAddr and tailCallIndirectAddrAndRet helper functions.
  • Improved tailCallAndRet to handle various return type conversions for void/scalar callees.
• amd64_lower_cmpbt.go
  • Added handling for macro-style symbolic immediates ($symbol) in evalIntSized.
• amd64_lower_fp.go
  • Added loadYVecOperand to load 256-bit Y-vector operands from registers, memory, or symbols.
• amd64_lower_mov.go
  • Added MOVLQZX and MOVLQSX for zero-extend and sign-extend moves.
  • Extended MOVQ and MOVL to support symbol operands as destinations.
  • Added permissive handling for crash marker forms in MOVL.
• amd64_lower_syscall.go
  • Added lowering for the INT instruction, treating it as an unreachable trap.
• amd64_lower_vec.go
  • Expanded supported vector operations to include MOVUPS, MOVAPS, MOVO, VMOVDQA, VMOVNTDQ.
  • Added VPXOR, VPOR, VPADDD, VPADDQ for 3-operand vector arithmetic/logical operations.
  • Implemented VPSHUFB, VPSHUFD, VPSLLD, VPSRLD, VPSRLQ, VPSLLQ for vector shuffle and shift operations.
  • Added VPALIGNR, VPERM2I128, VINSERTI128, VPBLENDD for advanced vector manipulations.
  • Implemented AESENC, AESENCLAST, AESDEC, AESDECLAST, AESIMC, AESKEYGENASSIST for AES instructions.
  • Added PANDN, PADDD, PADDL, PSUBL, PSLLL, PSRLL, PSRAL, PCMPEQL for packed integer operations.
  • Implemented PSHUFB, PINSRQ, PINSRD, PINSRW, PINSRB, PEXTRB, PALIGNR, PSLLDQ, PBLENDW, SHA256MSG1, SHA256MSG2, SHA256RNDS2, SHUFPS.
  • Extended MOVL to X registers to support various source kinds.
  • Updated VMOVDQU to handle X/Y registers and symbol operands for both loads and stores.
  • Added llvmShiftLeftBytesMask and llvmAlignRightBytesMask helper functions.
• amd64_needed.go
  • Expanded the list of conditional jump instructions for which AMD64 CFG lowering is needed.
• amd64_translate.go
  • Added emitAMD64Prelude to declare common LLVM intrinsics and target features for AMD64.
  • Expanded lowerInstr to ignore more pseudo-ops and directives like NO_LOCAL_POINTERS, PCDATA, FUNCDATA, NOP, ADJSP, CLD, STD, REP, PUSH_REGS_HOST_TO_ABI0(), POP_REGS_HOST_TO_ABI0().
• arm64_ctx.go
  • Expanded scanUsedRegs to mark all R0-R31 and SP as used for permissive lowering.
  • Improved ptrFromSB to trim '$' prefix from base symbols.
• arm64_eval.go
  • Enhanced eval64 to handle OpMem and OpIdent (for NZCV) operands.
  • Modified evalFPAddr64 to return a conservative '0' for unsupported FP address slots, allowing translation to proceed.
• arm64_flags.go
  • Added CS and CC conditions to condValue for ARM64 flags.
• arm64_lower_arith.go
  • Added lowering for MRS_TPIDR_R0 pseudo-op.
  • Implemented ADC, ADCS, SBC, SBCS for add/subtract with carry/borrow.
  • Added SUBW for 32-bit subtraction.
  • Implemented ANDSW for 32-bit logical AND with flags.
  • Extended BIC and BICW to support 2-operand forms.
  • Added MUL, UMULH, MADD, MSUB for multiplication operations.
  • Extended LSL, LSR to support 2-operand forms.
  • Implemented LSLW for 32-bit logical shift left.
  • Added ASR for arithmetic shift right.
  • Implemented UDIV for unsigned division.
  • Added EXTR for extract register operation.
• arm64_lower_branch.go
  • Introduced resolveBranchTarget helper for consistent branch target resolution.
  • Expanded BL, BLR, CALL, and B instructions to support register and memory targets.
  • Added CBZW, CBNZW for conditional branch on zero/non-zero.
  • Added BCS conditional branch alias.
  • Improved tailCallAndRet to handle void/scalar return types more permissively for runtime stubs.
• arm64_lower_cond.go
  • Added CSELW for 32-bit conditional select.
• arm64_lower_data.go
  • Extended MOVD and MOVW to support OpSym as a destination.
  • Added MOVH for 16-bit moves.
  • Expanded MOVHU and MOVBU to support various source and destination kinds, including memory and FP slots.
  • Enhanced LDP to support OpFP and OpSym sources.
  • Added STP for store pair instruction.
• arm64_lower_vec.go
  • Introduced arm64ParseVRegLane helper for parsing vector register lanes.
  • Added permissive handling for numerous crypto/NEON vector operations (AESE, AESD, AESMC, AESIMC, SHA1*, SHA256*, SHA512*, VPMULL, VPMULL2, VREV32, VREV64, VSHL, VSRI, VTBL, VZIP1, VZIP2, VEXT, VUSHR, VLD1R, VLD4R, VDUP).
  • Expanded VMOV to support V-to-V copies and GPR-to-V lane inserts/extracts.
  • Extended VLD1 to support lane forms and more reglist lengths.
  • Added VST1 for vector store multiple.
• arm64_translate.go
  • Added emitARM64Prelude to declare common LLVM intrinsics and target features for ARM64.
  • Expanded lowerInstr to ignore more pseudo-ops and directives like SAVE_R19_TO_R28, RESTORE_R19_TO_R28, SAVE_F8_TO_F15, RESTORE_F8_TO_F15, WORD, DMB, PRFM, BREAK, BRK, UNDEF, #UNDEF, YIELD, NOP, FLDPD, FSTPD, FMOVS, STY, P256ADDINLINE, P256MULBY2INLINE, MOV, CCMP, #IFDEF, #ELSE, #ENDIF.
• cmd/plan9asmll/.gitignore
  • Added _out/ to ignore generated output directories.
• cmd/plan9asmll/go.mod
  • Added new Go module for the plan9asmll command-line tool.
• cmd/plan9asmll/go.sum
  • Added Go sum entries for plan9asmll dependencies.
• cmd/plan9asmll/main.go
  • Added new command-line tool plan9asmll for compiling Go assembly to LLVM IR and verifying with llc.
  • Implemented multi-target matrix testing and reporting capabilities.
  • Added logic to extract supported instruction sets from source files and report unsupported operations.
• cmd/plan9asmscan/main.go
  • Updated normalizeOp to handle more instruction naming conventions.
  • Modified extractSupportedOps to correctly locate supported operations within the repository structure.
  • Refined clusterOf to categorize ARM64 instructions more accurately.
• doc/llvm-module-migration.md
  • Added documentation outlining the migration plan to direct LLVM module generation using goplus/llvm.
• doc/plan9asm-corpus.md
  • Added documentation defining the Plan9 asm corpus and instruction clusters for llgo development.
• doc/plan9asm-scan/darwin-amd64.md
  • Added scan report for darwin/amd64.
• doc/plan9asm-scan/darwin-arm64.md
  • Added scan report for darwin/arm64.
• doc/plan9asm-scan/linux-amd64.md
  • Added scan report for linux/amd64.
• doc/plan9asm-scan/linux-arm64.md
  • Added scan report for linux/arm64.
• go.mod
  • Added github.com/goplus/llvm as a dependency.
• go.sum
  • Added Go sum entries for github.com/goplus/llvm.
• parser.go
  • Improved Parse to handle label: INSTR syntax on a single line.
  • Made DATA and GLOBL parsing more permissive, allowing them within TEXT blocks and handling symbolic sizes like PTRSIZE and macro-style offsets.
  • Enhanced parseDATAStmt to accept string and symbol-address initializers as zero placeholders.
  • Added parseWidth to handle PTRSIZE in DATA directives.
• preprocess.go
  • Added support for #if and #elif preprocessor directives.
  • Improved macro expansion logic to handle inline function-like macro calls and object-like macro identifiers more effectively.
• translate.go
  • Refactored Translate to call TranslateModule and serialize the result, maintaining backward compatibility.
  • Introduced translateIRText as an internal helper for textual IR generation.
  • Centralized architecture-specific prelude emission into emitArchPrelude.
  • Enhanced emitExternSBGlobals to correctly resolve and declare external global symbols, including function symbols.
  • Made translateFuncLinear more permissive by ignoring unsupported instructions and handling more type casts (e.g., integer to float, float to integer, pointer to integer, integer to pointer).
• translate_module.go
  • Added new file translate_module.go which provides the TranslateModule entry point for direct LLVM module generation, with a fallback to textual IR parsing.
• translate_module_direct.go
  • Added new file translate_module_direct.go which implements direct LLVM module generation for a subset of instructions, including basic moves, arithmetic, and inline assembly for CPUID/XGETBV.
• translate_prelude.go
  • Added new file translate_prelude.go to centralize architecture-specific LLVM prelude declarations.
• types.go
  • Added more register aliases for AMD64 (RAX, RBX, RCX, RDX, RSI, RDI, RSP, RBP) and ARM64 (G, LR, R18_PLATFORM, W0-W31).
  • Improved parseMem to handle complex memory references, including nested parentheses and symbolic offsets, making it more robust for various assembly syntaxes.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This is a large and impressive pull request that significantly expands the capabilities of the Plan 9 assembly translator. The switch to using goplus/llvm to build an in-memory LLVM module is a great architectural improvement. The addition of a comprehensive test matrix and llc compile verification is excellent for ensuring correctness and stability. The sheer number of new instructions supported for both amd64 and arm64 is a huge step towards complete stdlib coverage. The new plan9asmll tool is a valuable addition. My review has a couple of minor suggestions for code simplification and maintainability, but overall this is a very high-quality contribution.