Add Tensilica Xtensa processor support #5442

austinc3030 · 2023-06-11T02:16:07Z

This PR is aimed at adding support for Tensillica's Xtensa processor/architecture to ghidra. This PR is an updated duplicate of #1407. I am creating this PR as the existing PR's submitter appears to no longer maintain their branch. I intend to continue maintaining this PR until:

merged
the original submitter reappears
advised otherwise

An updated sample elf can be found at https://dl.espressif.com/dl/esp32_rom.elf. Additional samples will be provided promptly upon request.

…ng/blob/master/1407-Add-Tensilica-Xtensa-processor-support.patch)

mumbel · 2023-06-11T15:08:36Z

Ghidra/Processors/Xtensa/data/languages/xtensaInstructions.sinc

+    at = nsa(as);
+}
+
+# NSAU - Normalization Shift Amount Unsigned, pg. 462. (Count leading zeros)


lzcount is a new built-in pcode that might simplify this assuming it is Count leading zeros

Addressed in 29bf647.

This definitely cleans up the de-compiler output for the NSAU instruction. I was unaware of this new pcode.

Also looks like you can replace the NSA implementation nsa() with lzcount(~as) (and delete pcodeop nsa in the other file)

Addressed in afa0ff6. I saw this but wasn't sure myself whether this would be applicable. Thank you!

…ith in-tree PR repo

mumbel · 2023-06-16T03:42:06Z

Ghidra/Processors/Xtensa/data/languages/xtensaInstructions.sinc

+
+# RFDD - Return from Debug and Dispatch, pg. 484.
+:rfdd is op2 = 0b1111 & op1 = 0b0001 & ar = 0b1110 & (as = 0b0000 | as = 0b0001) & at = 0b0001 & op0 = 0 {
+    rfdd();


I think all these return from might help disassembler and produce better decompilation with return something. right now they'd just fall through, but they should have some sort of flow

might try something like this for all of these:

- rfdd(); + local tmp:4 = rfdd(); + return [tmp];

Addressed in c5e8f01. Thank you!

mumbel · 2023-06-16T03:47:01Z

I honestly cant remember the state of this file, but I have a pattern file

this would be data/patterns/xtensa_patterns.xml
https://gist.github.com/mumbel/f7a81b2f416a8cb4bb489ace1f458a97

and then data/patterns/patternconstraints.xml

<patternconstraints>
  <language id="Xtensa:LE:32:default">
  	<compiler id="default">
  	  <patternfile>xtensa_patterns.xml</patternfile>
  	</compiler>
  </language>
</patternconstraints>

…ls plugin

austinc3030 · 2023-06-19T19:48:45Z

I honestly cant remember the state of this file, but I have a pattern file

this would be data/patterns/xtensa_patterns.xml https://gist.github.com/mumbel/f7a81b2f416a8cb4bb489ace1f458a97

and then data/patterns/patternconstraints.xml
<patternconstraints>
  <language id="Xtensa:LE:32:default">
  	<compiler id="default">
  	  <patternfile>xtensa_patterns.xml</patternfile>
  	</compiler>
  </language>
</patternconstraints>

Added in 563efe9.

…ough

…cleanup.

…cy#1407

redfast00 · 2023-07-12T12:18:42Z

@austinc3030 Aside from the relocations (which are not needed for embedded targets, like the ESP32), is there any work that still needs to happen on this PR? I'd really like to see this as a part of Ghidra, so I'll try to help out if needed.

austinc3030 · 2023-07-12T16:57:11Z

@austinc3030 Aside from the relocations (which are not needed for embedded targets, like the ESP32), is there any work that still needs to happen on this PR? I'd really like to see this as a part of Ghidra, so I'll try to help out if needed.

I don't think so? I have been using this pretty successfully against some esp32 firmware images. I appreciate any help or suggestions you may have!

k0rd · 2023-07-12T23:02:47Z

thank you for this work - it works well for me!

redfast00 · 2023-07-13T11:57:08Z

@austinc3030 Aside from the relocations (which are not needed for embedded targets, like the ESP32), is there any work that still needs to happen on this PR? I'd really like to see this as a part of Ghidra, so I'll try to help out if needed.

I don't think so? I have been using this pretty successfully against some esp32 firmware images. I appreciate any help or suggestions you may have!

I've been using this for a large reversing project, it does indeed work pretty well :) only blocker to getting this mainlined is then waiting for @emteere to review/merge this

emteere · 2023-09-01T02:50:38Z

We're working on doing a de-conflict with the orignal PR, some internal further work on the original PR that was almost ready to merge into main, and this PR. My fault for missing this and not communicating what was going on, or maybe your fault for working so fast :-)

If you could hold off doing any changes until then, we'd greatly appreciate it!

There were good things from both forks of the PR, but I think we are almost done. We'd also like to give commit props to both PR submissions which I think we have done.

If all goes well, we'll be pushing it into main in the next week or so. Then you can give it a spin and make any fixes where it doesn't work for you before the next major release.

emteere · 2023-09-01T02:54:20Z

Also, if you have a good sample with windowed calling convention that is legal to share, it would help with the final deconflict/review.
Both had an implementation of windowing and I'd like to make sure it is still working well. The sample I have is Call0.

austinc3030 · 2023-09-02T05:48:25Z

We're working on doing a de-conflict with the orignal PR, some internal further work on the original PR that was almost ready to merge into main, and this PR. My fault for missing this and not communicating what was going on, or maybe your fault for working so fast :-)

If you could hold off doing any changes until then, we'd greatly appreciate it!

There were good things from both forks of the PR, but I think we are almost done. We'd also like to give commit props to both PR submissions which I think we have done.

If all goes well, we'll be pushing it into main in the next week or so. Then you can give it a spin and make any fixes where it doesn't work for you before the next major release.

Great News!! I will hold off on changes until merged to main. Thanks for the update!

mumbel · 2023-09-16T03:12:31Z

@emteere don't know if still relevant, but you'll want an esp32 toolchain to get windowed calls

If you apt-get, it'll probably be esp8266 and non-windowed

https://docs.espressif.com/projects/esp-idf/en/release-v3.0/get-started/linux-setup.html

I made these from an old MIPS build and switched out for xtensa's gcc (has an O3 and O0 in the zip)

XTENSA_PCODETEST.zip

austinc3030 · 2023-10-11T14:59:19Z

We're working on doing a de-conflict with the orignal PR, some internal further work on the original PR that was almost ready to merge into main, and this PR. My fault for missing this and not communicating what was going on, or maybe your fault for working so fast :-)

If you could hold off doing any changes until then, we'd greatly appreciate it!

There were good things from both forks of the PR, but I think we are almost done. We'd also like to give commit props to both PR submissions which I think we have done.

If all goes well, we'll be pushing it into main in the next week or so. Then you can give it a spin and make any fixes where it doesn't work for you before the next major release.

Hey @emteere , just curious if there has been any progress on getting this merged and if I should do anything more to this PR such as squashing commits or rebasing on master? Just wanna help out where possible and excited to see this merged!

Thanks!

ryanmkurtz · 2023-10-12T14:51:55Z

Progress is still moving in the right direction, despite the slow pace. There is no need to alter your PR...the squashing/rebasing as well as getting the commit authorship correct will be handled on our end. Thank you.

…tensa-support

emteere · 2023-11-22T13:20:58Z

@mumbel btw, thanks for the samples. Helped to work test the windowing.

There is still some work to do on windowing as I think emulation of windowed binaries will be an issue as implemented.

Apply patch from jobermayr (https://github.com/jobermayr/ghidra-stagi…

c4a2080

…ng/blob/master/1407-Add-Tensilica-Xtensa-processor-support.patch)

mumbel reviewed Jun 11, 2023

View reviewed changes

ryanmkurtz assigned emteere Jun 12, 2023

ryanmkurtz added Feature: Processor/Unsupported Status: Triage Information is being gathered labels Jun 12, 2023

austinc3030 added 3 commits June 14, 2023 11:54

Use lzcount pcode for NSAU instruction

29bf647

Bring out-of-tree processor repo (austinc3030/ghidra-xtensa) inline w…

f83c629

…ith in-tree PR repo

Remove nsa pcode op and replace with lzcount

afa0ff6

mumbel reviewed Jun 16, 2023

View reviewed changes

austinc3030 added 2 commits June 19, 2023 15:47

Fix some errors/warnings present when loading in eclipse SleighDevToo…

56ff99f

…ls plugin

Add patterns and constraints per PR feedback

563efe9

austinc3030 and others added 9 commits June 20, 2023 08:25

Return something from "Return From" operations instead of falling thr…

c5e8f01

…ough

Merge branch 'master' into xtensa-support

6ef6c38

Merge branch 'NationalSecurityAgency:master' into xtensa-support

004adc4

Additions from forked repos, refactoring, deduplication, and general …

020cca2

…cleanup.

Apply changes from some feedback in pull request NationalSecurityAgen…

0a0f286

…cy#1407

Merge branch 'NationalSecurityAgency:master' into xtensa-support

269c46d

Merge branch 'NationalSecurityAgency:master' into xtensa-support

2fa7620

Merge branch 'NationalSecurityAgency:master' into xtensa-support

8244c9a

Merge branch 'NationalSecurityAgency:master' into xtensa-support

a34bfc0

Merge branch 'NationalSecurityAgency:master' into xtensa-support

e30bca9

austinc3030 added 3 commits July 17, 2023 08:58

Merge branch 'NationalSecurityAgency:master' into xtensa-support

70233b9

Merge branch 'NationalSecurityAgency:master' into xtensa-support

297f514

Merge branch 'NationalSecurityAgency:master' into xtensa-support

b9677f1

austinc3030 added 6 commits July 24, 2023 08:12

Merge branch 'NationalSecurityAgency:master' into xtensa-support

492a6ab

Merge branch 'NationalSecurityAgency:master' into xtensa-support

d9f3013

Merge branch 'NationalSecurityAgency:master' into xtensa-support

be6da80

Merge branch 'NationalSecurityAgency:master' into xtensa-support

5a4a3fa

Merge branch 'NationalSecurityAgency:master' into xtensa-support

8d26f0c

Merge branch 'NationalSecurityAgency:master' into xtensa-support

b527811

emteere added Status: Internal This is being tracked internally by the Ghidra team and removed Status: Triage Information is being gathered labels Sep 1, 2023

Merge branch 'master' into xtensa-support

82e758f

austinc3030 and others added 5 commits October 12, 2023 22:52

Merge branch 'NationalSecurityAgency:master' into xtensa-support

9fc7b5c

Merge branch 'xtensa-support' of github.com:austinc3030/ghidra into x…

8e24b8d

…tensa-support

Merge branch 'NationalSecurityAgency:master' into xtensa-support

ceb242d

Merge branch 'NationalSecurityAgency:master' into xtensa-support

95ce776

Merge branch 'NationalSecurityAgency:master' into xtensa-support

a5559f4

ryanmkurtz added Feature: Processor/Xtensa and removed Feature: Processor/Unsupported labels Nov 20, 2023

ryanmkurtz added this to the 10.5 milestone Nov 20, 2023

ryanmkurtz closed this in 7df3fd3 Nov 20, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Tensilica Xtensa processor support #5442

Add Tensilica Xtensa processor support #5442

austinc3030 commented Jun 11, 2023 •

edited

mumbel Jun 11, 2023

austinc3030 Jun 14, 2023

mumbel Jun 14, 2023

austinc3030 Jun 14, 2023

mumbel Jun 16, 2023

austinc3030 Jun 20, 2023

mumbel commented Jun 16, 2023

austinc3030 commented Jun 19, 2023

redfast00 commented Jul 12, 2023

austinc3030 commented Jul 12, 2023 •

edited

k0rd commented Jul 12, 2023

redfast00 commented Jul 13, 2023

emteere commented Sep 1, 2023

emteere commented Sep 1, 2023 •

edited

austinc3030 commented Sep 2, 2023 •

edited

mumbel commented Sep 16, 2023

austinc3030 commented Oct 11, 2023

ryanmkurtz commented Oct 12, 2023

emteere commented Nov 22, 2023

Add Tensilica Xtensa processor support #5442

Add Tensilica Xtensa processor support #5442

Conversation

austinc3030 commented Jun 11, 2023 • edited

mumbel Jun 11, 2023

Choose a reason for hiding this comment

austinc3030 Jun 14, 2023

Choose a reason for hiding this comment

mumbel Jun 14, 2023

Choose a reason for hiding this comment

austinc3030 Jun 14, 2023

Choose a reason for hiding this comment

mumbel Jun 16, 2023

Choose a reason for hiding this comment

austinc3030 Jun 20, 2023

Choose a reason for hiding this comment

mumbel commented Jun 16, 2023

austinc3030 commented Jun 19, 2023

redfast00 commented Jul 12, 2023

austinc3030 commented Jul 12, 2023 • edited

k0rd commented Jul 12, 2023

redfast00 commented Jul 13, 2023

emteere commented Sep 1, 2023

emteere commented Sep 1, 2023 • edited

austinc3030 commented Sep 2, 2023 • edited

mumbel commented Sep 16, 2023

austinc3030 commented Oct 11, 2023

ryanmkurtz commented Oct 12, 2023

emteere commented Nov 22, 2023

austinc3030 commented Jun 11, 2023 •

edited

austinc3030 commented Jul 12, 2023 •

edited

emteere commented Sep 1, 2023 •

edited

austinc3030 commented Sep 2, 2023 •

edited