Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tracking Issue for merging to upstream (LLVM-57) #4

Open
polarathene opened this issue Jul 9, 2019 · 78 comments
Open

Tracking Issue for merging to upstream (LLVM-57) #4

polarathene opened this issue Jul 9, 2019 · 78 comments

Comments

@polarathene
Copy link

I understand it will take quite some time for this backend to be merged upstream into LLVM.

This issue is for notifying anyone that subscribes to it when the merge to upstream has completed closing this issue.

@DarkWiiPlayer
Copy link

Is there a rough ETA or roadmap for this?

@igrr
Copy link
Member

igrr commented Jul 19, 2019

The patches are in review (see links in https://esp32.com/viewtopic.php?t=11412#p46446) and further timeline will depend on the review process. We will keep updating the patch set to address the feedback!

@breckwagner
Copy link

breckwagner commented Dec 24, 2019

@igrr igrr transferred this issue from espressif/llvm-xtensa Jan 15, 2020
@igrr
Copy link
Member

igrr commented Jan 15, 2020

We are moving this and other issues to https://github.com/espressif/llvm-project, which will host the Xtensa patches for LLVM based on the new monorepo layout. This new repository will be updated with Xtensa patches based on LLVM 9 soon.

@espressif-bot espressif-bot changed the title Tracking Issue for merging to upstream Tracking Issue for merging to upstream (LLVM-57) Jan 29, 2020
@andreisfr
Copy link
Collaborator

@polarathene, @DarkWiiPlayer , @breckwagner, we updated current patches (1-10), which are in LLVM review, according to changes in upstream version and reviewers requests for changes.

@lexxvir
Copy link

lexxvir commented Jul 22, 2020

Hi @andreisfr,

Is there any plan/way to merge this fork into the upstream?

I've skimmed some pull requests on the reviews.llvm.org, it seems the process is stalled.

@andreisfr
Copy link
Collaborator

Hi @lexxvir,

Yes, we plan to continue integration process of the Xtensa backend to the upstream.

@windoze
Copy link

windoze commented Sep 8, 2020

Is there any progress for this?
I'm dying to see it merging into upstream, it's really a PITA to build the whole LLVM/Clang project from the scratch in a CI pipeline.

@shymega
Copy link

shymega commented May 19, 2021

Hey, hope you don't mind me chasing this up 😄 I've been getting ready to start work on Rust code for an ESP32, but without official LLVM and Rust support, I fear I'll have to use C. It does take ages to compile as well. I might be able to influence a different chip, but I think the ESP32 is a solid choice.

Thanks.

@torpak
Copy link

torpak commented Jun 25, 2021

It seems nobody at LLVM is even answering since may. Does anyone know a reason for this?

@clouds56
Copy link

@torpak according to this https://reviews.llvm.org/D64830#2615149 comment, I think they are waiting the author to rewrite a RFC since things changed a lot.
Correct me if wrong.

@Hao123br
Copy link

Hao123br commented Jun 26, 2021

@clouds56 the author already sent another RFC but had no response. https://lists.llvm.org/pipermail/llvm-dev/2021-March/149090.html

@clouds56
Copy link

@Hao123br thanks for the link, shall we also provide the link in the thread and ping aykevl & tstellar, maybe they are too busy and missing the RFC?

igrr pushed a commit that referenced this issue Dec 3, 2021
This patch re-introduces the fix in the commit llvm@66b0cebf7f736 by @yrnkrn

> In DwarfEHPrepare, after all passes are run, RewindFunction may be a dangling
>
> pointer to a dead function. To make sure it's valid, doFinalization nullptrs
> RewindFunction just like the constructor and so it will be found on next run.
>
> llvm-svn: 217737

It seems that the fix was not migrated to `DwarfEHPrepareLegacyPass`.

This patch also updates `llvm/test/CodeGen/X86/dwarf-eh-prepare.ll` to include `-run-twice` to exercise the cleanup. Without this patch `llvm-lit -v llvm/test/CodeGen/X86/dwarf-eh-prepare.ll` fails with

```
-- Testing: 1 tests, 1 workers --
FAIL: LLVM :: CodeGen/X86/dwarf-eh-prepare.ll (1 of 1)
******************** TEST 'LLVM :: CodeGen/X86/dwarf-eh-prepare.ll' FAILED ********************
Script:
--
: 'RUN: at line 1';   /home/arakaki/build/llvm-project/main/bin/opt -mtriple=x86_64-linux-gnu -dwarfehprepare -simplifycfg-require-and-preserve-domtree=1 -run-twice < /home/arakaki/repos/watch/llvm-project/llvm/test/CodeGen/X86/dwarf-eh-prepare.ll -S | /home/arakaki/build/llvm-project/main/bin/FileCheck /home/arakaki/repos/watch/llvm-project/llvm/test/CodeGen/X86/dwarf-eh-prepare.ll
--
Exit Code: 2

Command Output (stderr):
--
Referencing function in another module!
  call void @_Unwind_Resume(i8* %ehptr) #1
; ModuleID = '<stdin>'
void (i8*)* @_Unwind_Resume
; ModuleID = '<stdin>'
in function simple_cleanup_catch
LLVM ERROR: Broken function found, compilation aborted!
PLEASE submit a bug report to https://bugs.llvm.org/ and include the crash backtrace.
Stack dump:
0.      Program arguments: /home/arakaki/build/llvm-project/main/bin/opt -mtriple=x86_64-linux-gnu -dwarfehprepare -simplifycfg-require-and-preserve-domtree=1 -run-twice -S
1.      Running pass 'Function Pass Manager' on module '<stdin>'.
2.      Running pass 'Module Verifier' on function '@simple_cleanup_catch'
 #0 0x000056121b570a2c llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) /home/arakaki/repos/watch/llvm-project/llvm/lib/Support/Unix/Signals.inc:569:0
 #1 0x000056121b56eb64 llvm::sys::RunSignalHandlers() /home/arakaki/repos/watch/llvm-project/llvm/lib/Support/Signals.cpp:97:0
 #2 0x000056121b56f28e SignalHandler(int) /home/arakaki/repos/watch/llvm-project/llvm/lib/Support/Unix/Signals.inc:397:0
 #3 0x00007fc7e9b22980 __restore_rt (/lib/x86_64-linux-gnu/libpthread.so.0+0x12980)
 #4 0x00007fc7e87d3fb7 raise /build/glibc-S7xCS9/glibc-2.27/signal/../sysdeps/unix/sysv/linux/raise.c:51:0
 #5 0x00007fc7e87d5921 abort /build/glibc-S7xCS9/glibc-2.27/stdlib/abort.c:81:0
 #6 0x000056121b4e1386 llvm::raw_svector_ostream::raw_svector_ostream(llvm::SmallVectorImpl<char>&) /home/arakaki/repos/watch/llvm-project/llvm/include/llvm/Support/raw_ostream.h:674:0
 #7 0x000056121b4e1386 llvm::report_fatal_error(llvm::Twine const&, bool) /home/arakaki/repos/watch/llvm-project/llvm/lib/Support/ErrorHandling.cpp:114:0
 #8 0x000056121b4e1528 (/home/arakaki/build/llvm-project/main/bin/opt+0x29e3528)
 #9 0x000056121adfd03f llvm::raw_ostream::operator<<(llvm::StringRef) /home/arakaki/repos/watch/llvm-project/llvm/include/llvm/Support/raw_ostream.h:218:0
FileCheck error: '<stdin>' is empty.
FileCheck command line:  /home/arakaki/build/llvm-project/main/bin/FileCheck /home/arakaki/repos/watch/llvm-project/llvm/test/CodeGen/X86/dwarf-eh-prepare.ll

--

********************
********************
Failed Tests (1):
  LLVM :: CodeGen/X86/dwarf-eh-prepare.ll

Testing Time: 0.22s
  Failed: 1
```

Reviewed By: loladiro

Differential Revision: https://reviews.llvm.org/D110979

(cherry picked from commit e8806d7)
@arnauorriols
Copy link

FYI, just noticed patch 2 is also accepted since March 21 2021

igrr pushed a commit that referenced this issue May 2, 2022
…ified offset and its parents or children with spcified depth."

This reverts commit a3b7cb0.

symbol-offset.test fails under MSAN:

[  1] ; RUN: llvm-pdbutil yaml2pdb %p/Inputs/symbol-offset.yaml --pdb=%t.pdb [FAIL]
llvm-pdbutil yaml2pdb <REDACTED>/llvm/test/tools/llvm-pdbutil/Inputs/symbol-offset.yaml --pdb=<REDACTED>/tmp/symbol-offset.test/symbol-offset.test.tmp.pdb
==9283==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x55f975e5eb91 in __libcpp_tls_set <REDACTED>/include/c++/v1/__threading_support:428:12
    #1 0x55f975e5eb91 in set_pointer <REDACTED>/include/c++/v1/thread:196:5
    #2 0x55f975e5eb91 in void* std::__msan::__thread_proxy<std::__msan::tuple<std::__msan::unique_ptr<std::__msan::__thread_struct, std::__msan::default_delete<std::__msan::__thread_struct> >, llvm::parallel::detail::(anonymous namespace)::ThreadPoolExecutor::ThreadPoolExecutor(llvm::ThreadPoolStrategy)::'lambda'()::operator()() const::'lambda'()> >(void*) <REDACTED>/include/c++/v1/thread:285:27
    #3 0x7f74a1e55b54 in start_thread (<REDACTED>/libpthread.so.0+0xbb54) (BuildId: 64752de50ebd1a108f4b3f8d0d7e1a13)
    #4 0x7f74a1dc9f7e in clone (<REDACTED>/libc.so.6+0x13cf7e) (BuildId: 7cfed7708e5ab7fcb286b373de21ee76)
@saeidakbari
Copy link

Is any new news on reviewing process?

@flaminisx
Copy link

just noticed that patches 4, 5, 6, 8 and 10 are also accepted

@igrr
Copy link
Member

igrr commented Sep 26, 2022

For future reference, the new RFC is at https://discourse.llvm.org/t/rfc-request-for-upstream-tensilica-xtensa-esp32-backend/65355.

@SergioGasquez
Copy link

The patches are in review (see links in https://esp32.com/viewtopic.php?t=11412#p46446) and further timeline will depend on the review process. We will keep updating the patch set to address the feedback!

From Espressif Forum post Re: Status of LLVM support?

Status as of 2019-12-24 [x] = Accepted [ ] = Needs Review

Patches 7 and 9 have now been accepted!

@shish
Copy link

shish commented Dec 31, 2022

Looks like these are all merged into the main branch now, if I'm reading it right? Merry Christmas all :D

@aq1018
Copy link

aq1018 commented Mar 17, 2023

Should this be closed? LLVM has has accepted all patches according to this article - https://www.phoronix.com/news/LLVM-Xtensa-Backend

@Skgland
Copy link

Skgland commented Mar 18, 2023

Unless esp-rs/rust#89 (comment) is outdated, there are still some parts missing.

@MihaelIsaev
Copy link

I'm subscribed to all the activity in this repository because waiting for Xtensa support to start using ESP32 with Embedded Swift. Would love to support a thread about that on Swift Forums 😊 there are already at least one thread and on reddit as well

@teburd
Copy link

teburd commented Jun 29, 2024

Are the hifi patches also upstreamed?

@gerekon
Copy link
Collaborator

gerekon commented Jun 29, 2024

@teburd

Are the hifi patches also upstreamed?

Do you mean ones from this PR?
Not yet I think. They need Boolean and Floating point features as far as I see.
We will do this when Boolean and Float are upstreamed.

@bugadani
Copy link

Considering how long it takes to upstream few-hundered-LOC PRs, I am somewhat worried about throwing 80k+ LOC patches in 😅

@gerekon
Copy link
Collaborator

gerekon commented Jun 29, 2024

Considering how long it takes to upstream few-hundered-LOC PRs, I am somewhat worried about throwing 80k+ LOC patches in 😅

I expect that after the next 10-15 patches we will start doing this in parallel. Maybe we can try to keep 2-3 active patches in upstream review right now and see if it will speed up the process.

@andreisfr what do you think?

@andreisfr
Copy link
Collaborator

@gerekon , yes, I think we need to upstream about 10-15 patches and probably we will be able to create parallel PRs

@ProfFan
Copy link

ProfFan commented Jul 31, 2024

Is there an update or target timeline?

@Skgland
Copy link

Skgland commented Jul 31, 2024

Have you seen the estimate approximately a month ago #4 (comment)

@andreisfr
Copy link
Collaborator

andreisfr commented Jul 31, 2024

Current status. The last PR actually contains 2 base patches.

@johnnyasantoss
Copy link

Apparently llvm#99981 just got merged

@JetForMe
Copy link

IIUC everything is merged? I wonder how long before it makes it into a Swift toolchain.

@Skgland
Copy link

Skgland commented Aug 22, 2024

IIUC everything is merged?

No, #4 (comment) only includes the seven patches already submitted to upstream. Based on #4 (comment) that makes it 7 out of 10-15 patches that need to be submitted in series after which more patches will follow in parallel.

@JetForMe
Copy link

JetForMe commented Aug 22, 2024

This comment listed the following items remain. I was unable to easily correlate them to PRs in later comments.

@SergioGasquez There are 13 patches left:

  1. Implement lowering SELECT_CC, SETCC - In Progress
  2. Add support of the Xtensa function calls
  3. Lower stack operations.
  4. Support for a variety of additional LLVM IR constructs.
  5. Lower SHIFT PARTS and shift operations.
  6. Implement load pseudo operations and patterns.
  7. Support for variable arguments
  8. Implement lowering BR_JT operation
  9. Support for address intrinsics.
  10. Add basic support for inline asm constraints
  11. Implement volatile load/store.
  12. Implement branch analysis
  13. Implement support for the BranchRelaxation

@andreisfr
Copy link
Collaborator

andreisfr commented Aug 26, 2024

Current status.

@TurkeyMan
Copy link

🎉

What happens next now that these all seem to be done?

@bugadani
Copy link

Now the next patch will be submitted, then the next, and the next... :)

@TurkeyMan
Copy link

Lol, but as someone following closely and with much excitement... it seems to be the end of the list of outstanding patches that I was aware of. Wasn't clear if there's another whole bunch, or if we're approaching branch synchronisation?

@bugadani
Copy link

bugadani commented Aug 28, 2024

Considering Andrei has estimated 10-15 patches two months ago to be submitted one by one, and since then 2 got merged, I wouldn't hold my breath. And as I understand even that isn't the end of this story.

@andreisfr
Copy link
Collaborator

The llvm#106053 PR is merged.
We preparing next PR now.

@Skgland
Copy link

Skgland commented Aug 28, 2024

Maybe the Current status Comment could include an additional bullet point at the and for further patches to avoid this confusion.

@andreisfr
Copy link
Collaborator

Maybe the Current status Comment could include an additional bullet point at the and for further patches to avoid this confusion.

Ok, I will add this information in next update

@andreisfr
Copy link
Collaborator

andreisfr commented Sep 11, 2024

Hi all!

I prepared a plan which describes nearest steps of upstreaming Xtensa backend. The goal is to implement most of functionality to be able generate Rust code and compile ESP-IDF examples.

  1. We have 6 prepared patches with base functionality not upstreamed. This patches are not very large, so I think optimistic estimation is to finish this step till middle of October.
  1. Implement other Xtensa features like Window, Boolean, Float, etc. At this step we didn't prepare exactly patch list, because it depends on previous step, it will be about 8-12 patches. Also in optimistic case will take another 2-3 month and probably be finished in December. At this stage we probably can make some parallel PRs.

Currently we have feature list to implement:

  • Implement code density feature
  • Implement Windowed feature
  • Implement FP feature
  • Implement Boolean feature
  • Implement Mac16 feature and operations.
  • Implement Debug, DFPAccel, S32C1I, THREADPTR, Extended L32R, ATOMCTL, MEMCTL features.
  • Implement Exception, HighPriInterrupts, Coprocessor, Interrupt, RelocatableVector, TimerInt, PRID, RegionProtection and
  • MiscSR features. Implement instructions for Exception, Interrupt and RegionProtection features.

We also have a patches for this features, but number of patches and order maybe changed. I will prepare more precise patch set when first step will be finished.

  1. Implement Xtensa targets in Clang esp32,esp8266, etc" and add support for multilib and linker. It is about 8-12 patches. Some of these patches will be complex, so in optimistic case will take another 2-3 month and will be finished in Febraury/March. At this stage we probably can make some parallel PRs.
    After that we can probably build esp-idf and Rust examples.

  2. Implement minor features, like hwloop, etc.

@andreisfr
Copy link
Collaborator

andreisfr commented Sep 17, 2024

The Xtensa backend upstreaming status update:

@andreisfr
Copy link
Collaborator

andreisfr commented Oct 3, 2024

The Xtensa backend upstreaming status update:

@andreisfr
Copy link
Collaborator

andreisfr commented Nov 7, 2024

The Xtensa backend upstreaming status update. Currently we preparing PR with the last patch from step 1 #4 (comment) . In a few days I will an update with a list of the next patch set for upstream.

@gerekon
Copy link
Collaborator

gerekon commented Nov 28, 2024

Hi guys one more PR is pending which implements generic Xtensa target support in Clang. It is related to the next (2nd) set of patches mentioned here.

Backend base implementation

Xtensa features and toolchain implementation

LLVM:

  • Implement Xtensa density option
  • Implement Xtensa windowed option
  • Implement Xtensa FP option and operations
  • Implement Loop, SEXT and NSA features.
  • Implement Mul32, Mul32High and Div32 features.
  • Implement Mac16 feature and operations.
  • Implement Debug, DFPAccel, S32C1I, THREADPTR, Extended L32R, ATOMCTL, MEMCTL features.
  • Implement Exception, HighPriInterrupts, Coprocessor, Interrupt, RelocatableVector, TimerInt, PRID, RegionProtection and MiscSR features. Implement instructions for Exception, Interrupt and RegionProtection features with tests.
  • Implement Atomic operations support.

Clang:

  • Add the Xtensa target.
  • Implement Xtensa ABI lowering.
  • Add subtargets ESP32. ESP8266 and ESP32-S2.
  • Implement ESP Baremetal toolchain

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests