Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add LoongArch support #47

Closed
wants to merge 52 commits into from
Closed

Conversation

jiegec
Copy link
Contributor

@jiegec jiegec commented May 3, 2024

  • Comment out unsupported code
  • Fill in operands

See capstone-engine/capstone#2349

The MCInstDesc table changed. Bsides this only minor changes were done
and some additional code is emitted now for LLVM.

This commit is the combination of all previous Auto-Sync commits.
The list of commit messages follows:

-----------

Combination of all commits of the refactored tablegen backends.

These are the changes made for LLVM 16.

Refactor Capstone relevant TableGen Emitter backends.

This commit extracts the code which emits generated tables into two printer classes.
The Printer is called whenever actual code is written to a file.
There is the PrinterLLVM which emits tht code as before and
PrinterCapstone which is tailored to or needs (emitting C and generates
more info).

Additionally missing memory access properties were added to ARMs td
files.

Emit a single header for all files.

Captialize Target name for enums.

Add lay metric to emit enum value for Banked and system regs.

Malloc substr

Sort instructions in ascending order.

Free substr after use

Add vanished constrainsts

Fix `regInfoEmitEnums()` and indent

Fix `GenDisassemblerTables.inc#checkDecoderPredicate()`

Fix `TriCoreGenRegisterInfo.inc` | `PrinterCapstone::regInfoEmitRegClasses`

revert changes to NEON instructions

Add instructions with duplicate operands as Matchables.

Add memory load and store info

Correct memory access and out operand info

Set register lists again as read ops due to llvm/llvm-project#62455

Make printAliasInstr and getMnemonic static.

Generate CS instruction enums from actual mnemonic. Not via the flawed AsmMatcher.

Fix typo in InstrInfoEmitter.cpp

Add deprecated QPX feature

Replace + and - with p and m

Add AssemblerPredicates to PPC

Generate RegEncodingTable

Define functions which are called by the Mapper as static.

Necessary because these functions are present in each arch'

Remove set_mem_access().

The cases where this is used to mark access to actual memory operands are
either very rare, or those are neon lane indicies.

Generate correct op type for absolute addresses.

Check for RegisterPointer operands first to prevent mis-categorization.

Add missing Operand types

Generate Instruction formats for PPC.

Add Paired Single instructions.

Partly revert 94e41ce (introduces accidentially removed code.)

Set correct operand types for PS operands

Add memory read/write attributes

Add missing operand types

Add mayLoad and mayStore information.

Add documentation.

Handle special AArch64 operand

Replace C++ with C code.

Check for duplicate enum instr. names

Check for duplicate defintions of system registers.

Add note about missing target names.

Resolve templates in a single static method and add docs about it.

Revert printing target name in upper case.

Revert partially C++ syntax fixes in .td files.

They break the TemplateCOllector since it searches for exactly those references but can't find any'

Add all SubtargetFeatures to feature enum.

Not just the one used by CGIs.

Pass Decoder

Enable to check specific table fields to determine if reg enum must be emitted.

Allow to add namespace to type name/

Formatting

Rework emitting of tables.

The system operands are now emitted in reg, imm and aliass groups.
Also a bug was fixed which emitted incorrect code..

Check for rename IMPLICIT_IMM operand types

Pass DecodeComplete as pointer not as reference

Print undef when it needs to be printed.

Add namespace ids to all types and functions.

Rework C translation.

Pass MCOp as pointer not as ref

Add missing SysImm type

Fix syntax mistakes

Generate additonal sys immediates and op groups.

Handle edge case for printSVERegOp

Handle default arguments of template functions.

Add two missing op groups

Generate a static RecEncodingTable

Set enum values to encodings of the sys ops

Generate a single Enum value file for system operands.

Replace System operand groups with their operand types

Fix missing braces warning

Emit MCOperand validator.

Emit lookupByName functions for sys operands

Add namespaces for ARM.

Check for Target if default arguments of template functions are resolved.

auto-sync opcode & operand encoding info generation (capstone-engine#14)

* Added operand and opcode info generation

* Wrapped deprecated macro under an intellisense check

Basically intellisense fails, causing multiple errors in other files,

so when intellisense parses the code it will use the different version of the macro

* Fixed a small bug

Used double braces to prevent an old bug

Removed extra new line and fixed a bug regarding move semantics
@jiegec jiegec marked this pull request as draft May 3, 2024 07:38
Copy link
Collaborator

@Rot127 Rot127 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please, before you continue with this rebase it on top of the LLVM 18 branch from #45. There are many bug fixes and it is based on LLVM-18.

Now to the changes:

  • Before you proceed, please make sure you have read the architecture about the auto-sync updater. I would advice you to read this blog entry first and afterward check out the docs about: the general architecture, the auto-sync updater and lastly the cpp-translator. Last but not least I highly recommend to step with a debugger through a decoding of an ARM, PPC or AArch64 instruction. This way you can see in practice how it is done.
  • Please do not make any changes to the LLVM files here. Exceptions are TableGen related files. But any target related source code should stay the same. The reason is, that it is a lot and very very annoying work to rebase our patched LLVM code onto the newest release. We need to have as little changes in the original LLVM source code as possible. This is why all our editing of the LoongArchDisassembler etc files, all happen in the auto-sync scripts and not here. Because the scritps automate a lot of them and keep the changes in this repo small.

You are one of the first users of the auto-sync updater. So I would love to improve the first steps for people like you. If you could tell me where you got confused I would like to know it, so I can make the "new-user" steps easier.

@jiegec jiegec changed the base branch from auto-sync to auto-sync-18 May 3, 2024 08:12
@jiegec
Copy link
Contributor Author

jiegec commented May 3, 2024

I have rebased on top of auto-sync-18 branch and dropped LoongArch-specific changes under llvm/lib.

@jiegec
Copy link
Contributor Author

jiegec commented May 3, 2024

Using capstone next branch and llvm-capstone auto-sync-18 branch:

INFO:root:Generating Disassembler tables...
DEBUG:root:Generating CCS tables...
DEBUG:root:/home/jiegec/capstone/suite/auto-sync/vendor/llvm_root/build/bin/llvm-tblgen --printerLang=CCS --gen-disassembler -I /home/jiegec/capstone/suite/auto-sync/vendor/llvm_root/llvm/include -I /home/jiegec/capstone/suite/auto-sync/vendor/llvm_root/llvm/lib/Target/LoongArch -o /home/jiegec/capstone/suite/auto-sync/build/llvm_c_inc/LoongArchGenDisassemblerTables.inc /home/jiegec/capstone/suite/auto-sync/vendor/llvm_root/llvm/lib/Target/LoongArch/LoongArch.td
llvm-tblgen: Unknown command line argument '--printerLang=CCS'.  Try: '/home/jiegec/capstone/suite/auto-sync/vendor/llvm_root/build/bin/llvm-tblgen --help'
llvm-tblgen: Did you mean '--print-options=CCS'?
CRITICAL:root:Generation failed
Traceback (most recent call last):
  File "/home/jiegec/capstone/suite/auto-sync/./src/autosync/ASUpdater.py", line 263, in <module>
    Updater.update()
  File "/home/jiegec/capstone/suite/auto-sync/./src/autosync/ASUpdater.py", line 141, in update
    self.inc_generator.generate()
  File "/home/jiegec/capstone/suite/auto-sync/src/autosync/IncGenerator.py", line 101, in generate
    self.gen_incs()
  File "/home/jiegec/capstone/suite/auto-sync/src/autosync/IncGenerator.py", line 177, in gen_incs
    raise e
  File "/home/jiegec/capstone/suite/auto-sync/src/autosync/IncGenerator.py", line 171, in gen_incs
    subprocess.run(
  File "/usr/lib/python3.11/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/home/jiegec/capstone/suite/auto-sync/vendor/llvm_root/build/bin/llvm-tblgen', '--printerLang=CCS', '--gen-disassembler', '-I', '/home/jiegec/capstone/suite/auto-sync/vendor/llvm_root/llvm/include', '-I', '/home/jiegec/capstone/suite/auto-sync/vendor/llvm_root/llvm/lib/Target/LoongArch', '-o', '/home/jiegec/capstone/suite/auto-sync/build/llvm_c_inc/LoongArchGenDisassemblerTables.inc', '/home/jiegec/capstone/suite/auto-sync/vendor/llvm_root/llvm/lib/Target/LoongArch/LoongArch.td']' returned non-zero exit status 1.

@jiegec
Copy link
Contributor Author

jiegec commented May 3, 2024

Using capstone next branch and llvm-capstone auto-sync-18 branch:

INFO:root:Generating Disassembler tables...
DEBUG:root:Generating CCS tables...
DEBUG:root:/home/jiegec/capstone/suite/auto-sync/vendor/llvm_root/build/bin/llvm-tblgen --printerLang=CCS --gen-disassembler -I /home/jiegec/capstone/suite/auto-sync/vendor/llvm_root/llvm/include -I /home/jiegec/capstone/suite/auto-sync/vendor/llvm_root/llvm/lib/Target/LoongArch -o /home/jiegec/capstone/suite/auto-sync/build/llvm_c_inc/LoongArchGenDisassemblerTables.inc /home/jiegec/capstone/suite/auto-sync/vendor/llvm_root/llvm/lib/Target/LoongArch/LoongArch.td
llvm-tblgen: Unknown command line argument '--printerLang=CCS'.  Try: '/home/jiegec/capstone/suite/auto-sync/vendor/llvm_root/build/bin/llvm-tblgen --help'
llvm-tblgen: Did you mean '--print-options=CCS'?
CRITICAL:root:Generation failed
Traceback (most recent call last):
  File "/home/jiegec/capstone/suite/auto-sync/./src/autosync/ASUpdater.py", line 263, in <module>
    Updater.update()
  File "/home/jiegec/capstone/suite/auto-sync/./src/autosync/ASUpdater.py", line 141, in update
    self.inc_generator.generate()
  File "/home/jiegec/capstone/suite/auto-sync/src/autosync/IncGenerator.py", line 101, in generate
    self.gen_incs()
  File "/home/jiegec/capstone/suite/auto-sync/src/autosync/IncGenerator.py", line 177, in gen_incs
    raise e
  File "/home/jiegec/capstone/suite/auto-sync/src/autosync/IncGenerator.py", line 171, in gen_incs
    subprocess.run(
  File "/usr/lib/python3.11/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/home/jiegec/capstone/suite/auto-sync/vendor/llvm_root/build/bin/llvm-tblgen', '--printerLang=CCS', '--gen-disassembler', '-I', '/home/jiegec/capstone/suite/auto-sync/vendor/llvm_root/llvm/include', '-I', '/home/jiegec/capstone/suite/auto-sync/vendor/llvm_root/llvm/lib/Target/LoongArch', '-o', '/home/jiegec/capstone/suite/auto-sync/build/llvm_c_inc/LoongArchGenDisassemblerTables.inc', '/home/jiegec/capstone/suite/auto-sync/vendor/llvm_root/llvm/lib/Target/LoongArch/LoongArch.td']' returned non-zero exit status 1.

Update: oh, I need to use auto-sync-18 from your repo, not capstone-engine.

llvm/utils/TableGen/PrinterCapstone.cpp Outdated Show resolved Hide resolved
Rot127 and others added 12 commits May 15, 2024 03:11
- Determine memory operands by there membership of patterns.

Operands can be defined via patterns. The pattern has a type itself.
Patterns are commonly used to define memory operands
(as a collection of multiple other operands).

Here we check, if the operand is part of a pattern with type iPTR.
Type iPTR indicates a memory operand.

- Search for unspecified instruction patterns and extract memory op info from them.

AArch64 has the case of patterns which are not set in the CGI class.
Instead those patterns are free floating in the record keeper.

With this commit we first collect all such unassigned patterns and save them in a map
(indexed by instruction name they belong to).

When instruction operand details are generated, the operand type is searched in the pattern of the instruction.
If there is an operand with the same type in the pattern, which is also of a iPTR pattern, we treat it as memory operand.

Note that we can not compare the operands of the instruction and the pattern operands by name.
Because they simply are not named the same way.

This is a short coming in the td files and should be addressed in LLVM.
Because it is enormiously hard to get a 1 to 1 mapping from a pattern to an instruction
and its operands.
@Rot127
Copy link
Collaborator

Rot127 commented May 29, 2024

Merged with #45

@Rot127 Rot127 closed this May 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants