Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[regression/8.0.0-rc.1.9171] Vanilla app crashing on Windows in release mode #17368

Closed
1 of 4 tasks
DDHSchmidt opened this issue Sep 14, 2023 · 9 comments
Closed
1 of 4 tasks
Labels
area-publishing Issues with the app packaging/publishing process (ipk/apk/msix/trimming) external high It doesn't work at all, crashes or has a big impact. partner/winui WinUI / Project Reunion platform/windows 🪟 potential-regression This issue described a possible regression on a currently supported version., verification pending s/needs-attention Issue has more information and needs another look s/triaged Issue has been reviewed t/bug Something isn't working

Comments

@DDHSchmidt
Copy link

DDHSchmidt commented Sep 14, 2023

Description

I used to be able to fire up our build pipeline and install/start the *.msix file from dotnet publish -c Release ... on Windows.
Starting with the latest RC1, the msix files/app generated from that command crashes right on starting.

The output in EventViewer is:

<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
	<System>
		<Provider Name="Application Error" /> 
		<EventID Qualifiers="0">1000</EventID> 
		<Version>0</Version> 
		<Level>2</Level> 
		<Task>100</Task> 
		<Opcode>0</Opcode> 
		<Keywords>0x80000000000000</Keywords> 
		<TimeCreated SystemTime="2023-09-14T09:46:15.8101143Z" /> 
		<EventRecordID>1385623</EventRecordID> 
		<Correlation /> 
		<Execution ProcessID="0" ThreadID="0" /> 
		<Channel>Application</Channel> 
		<Computer>nb-hschmidt</Computer> 
		<Security /> 
	</System>
	<EventData>
		<Data>CrashTestWin.exe</Data> 
		<Data>1.0.0.0</Data> 
		<Data>64e00000</Data> 
		<Data>KERNELBASE.dll</Data> 
		<Data>10.0.19041.3324</Data> 
		<Data>6967c799</Data> 
		<Data>c000027b</Data> 
		<Data>000000000012d9b2</Data> 
		<Data>3bb4</Data> 
		<Data>01d9e6f04ab860e5</Data> 
		<Data>C:\Program Files\WindowsApps\com.companyname.crashtestwin_1.0.0.1_x64__w8rw78cr7ghbw\CrashTestWin.exe</Data> 
		<Data>C:\WINDOWS\System32\KERNELBASE.dll</Data> 
		<Data>fb356e76-23cd-4510-a16b-38a3c493e7b8</Data> 
		<Data>com.companyname.crashtestwin_1.0.0.1_x64__w8rw78cr7ghbw</Data> 
		<Data>App</Data> 
	</EventData>
</Event>

With earlier regressions it was usually enough to revert the <MauiVersion>-Tag in the csproj file to the previous version but that doesn't seem to have any effect here. Setting that back to '8.0.0-preview.7.8842' still resulted in an app from dotnet publish that crashes on startup.
I guess this might have to do with the installed workloads on the build machine, where everything is set to maui 8.0.0-rc.1.9171/8.0.100-rc.1 ??
I googled on how to install previous workloads and was greeted with tips like "Use different rollback file" but I have no idea where to point that? So I couldn't test that idea.

I linked a public repository with a vanilla app on RC1 that easily exhibits the problem.

Please tell me that I did something terribly wrong. I just don't want to believe that no one ever tested a release build of the vanilla app on Windows before greenlighting RC1 ...

Steps to Reproduce

  1. Checkout reproduction poject
  2. dotnet restore CrashTestWin.csproj -p:PublishReadyToRun=false -r win-x64
  3. dotnet publish CrashTestWin.csproj -f net8.0-windows10.0.19041.0 -r win-x64 -c Release --no-restore --self-contained /p:RuntimeIdentifierOverride=win-x64 /p:PublishReadyToRun=false -p:WindowsAppSDKSelfContained=true /p:Platform=x64 /p:GenerateAppxPackageOnBuild=true /p:AppxPackageSigningEnabled=true /p:PackageCertificateThumbprint=<YourCertificateThumbPrintHere>
  4. Install generated msix
  5. Crash on startup

Link to public reproduction project repository

https://github.com/DDHSchmidt/Maui8RC1CrashWinRelease

Version with bug

8.0.0-rc.1.9171

Is this a regression from previous behavior?

Yes, this used to work in .NET MAUI

Last version that worked well

8.0.0-preview.7.8842

Affected platforms

Windows

Affected platform versions

Windows 10 (22H2)

Did you find any workaround?

None yet, but I suspect the WindowsAppSDK to be involved again, since that was the source of numerous problems in the past.
I'm testing inclusion of different WindowsAppSDK versions but since building release mode takes time that is still ongoing

Relevant log output

I specifically installed WinDBG with PDE extension according to this guide (https://github.com/microsoft/microsoft-ui-xaml/blob/main/docs/debugging_crashes.md#stowed-exception-crashes-exception-code-0xc000027b) in order to analyze the CrashDump but i have no friggin idea how to read the output :/

Stowed Exception Array @ 0x000002712a38aad0

Stowed Exception #1 @ 0x0000027127cd42d8
	0x80131534 (FACILITY_URT - .NET CLR): Uncaught exception during type initialization.

	Stack	 : 0x2712a3876b0
		7ffa82ef6b64 combase!RoOriginateLanguageException+0x54
		7ff9534a4ed7

Stowed Exception #2 @ 0x0000027127dd59f8
	0x80131534 (FACILITY_URT - .NET CLR): Uncaught exception during type initialization.

	Stack	 : 0x27127dd49e0
		7ff986c9a81b Microsoft_ui_xaml!DirectUI::CRoutedEventSourceBase<DirectUI::IUntypedEventSource,ABI::Microsoft::UI::Xaml::IRoutedEventHandler,IInspectable,ABI::Microsoft::UI::Xaml::IRoutedEventArgs>::Raise+0x18c4b3
		7ff986b0e312 Microsoft_ui_xaml!DirectUI::CRoutedEventSourceBase<DirectUI::IUntypedEventSource,ABI::Microsoft::UI::Xaml::IRoutedEventHandler,IInspectable,ABI::Microsoft::UI::Xaml::IRoutedEventArgs>::UntypedRaise+0x92
		7ff986a574fa Microsoft_ui_xaml!CCoreServices::CLR_FireEvent+0x3da
		7ff986a5710d Microsoft_ui_xaml!CommonBrowserHost::CLR_FireEvent+0x1d
		7ff986ab9851 Microsoft_ui_xaml!CControlBase::ScriptCallback+0x151
		7ff9869b5309 Microsoft_ui_xaml!CXcpDispatcher::OnScriptCallback+0x179
		7ff9869b4dbe Microsoft_ui_xaml!CXcpDispatcher::OnWindowMessage+0x3e
		7ff9869b4d47 Microsoft_ui_xaml!CXcpDispatcher::WindowProc+0xb7
		7ffa8349e858 user32!UserCallWinProcCheckWow+0x2f8
		7ffa8349de1b user32!SendMessageWorker+0x70b
		7ffa8349d68a user32!SendMessageW+0xda
		7ff986aa2138 Microsoft_ui_xaml!CXcpBrowserHost::SyncScriptCallbackRequest+0xf8
		7ff9869fb361 Microsoft_ui_xaml!CEventManager::RaiseHelper+0x201
		7ff9869cd6be Microsoft_ui_xaml!CEventManager::RaiseLoadedEventForObject+0xca
		7ff9869cd303 Microsoft_ui_xaml!CEventManager::RaiseLoadedEvent+0xeb
		7ff98698b571 Microsoft_ui_xaml!CCoreServices::NWDrawTree+0x3f1
		7ff9869895aa Microsoft_ui_xaml!CCoreServices::NWDrawMainTree+0x14a
		7ff98698941d Microsoft_ui_xaml!CWindowRenderTarget::Draw+0x6d
		7ff986989359 Microsoft_ui_xaml!CXcpBrowserHost::OnTick+0x59
		7ff9869b5a4a Microsoft_ui_xaml!CXcpDispatcher::Tick+0x8a
		7ff9869b5973 Microsoft_ui_xaml!CXcpDispatcher::OnReentrancyProtectedWindowMessage+0x223
		7ff9869b4d32 Microsoft_ui_xaml!CXcpDispatcher::WindowProc+0xa2
		7ff986ab8235 Microsoft_ui_xaml!CDeferredInvoke::DispatchQueuedMessage+0xd5
		7ff986ab8109 Microsoft_ui_xaml!Microsoft::WRL::Details::DelegateArgTraits<long (__cdecl ABI::Windows::Foundation::ITypedEventHandler_impl<ABI::Windows::Foundation::Internal::AggregateType<ABI::Microsoft::UI::Dispatching::DispatcherQueueTimer *,ABI::Microsoft::UI::Dispatching::IDispatcherQueueTimer *>,IInspectable *>::*)(ABI::Microsoft::UI::Dispatching::IDispatcherQueueTimer *,IInspectable *)>::DelegateInvokeHelper<Microsoft::WRL::Implements<Microsoft::WRL::RuntimeClassFlags<2>,ABI::Windows::Foundation::ITypedEventHandler<ABI::Microsoft::UI::Dispatching::DispatcherQueueTimer *,IInspectable *>,Microsoft::WRL::FtmBase>,`CXcpDispatcher::Init'::`55'::<lambda_1> &,1,ABI::Microsoft::UI::Dispatching::IDispatcherQueueTimer *,IInspectable *>::Invoke+0x79
		7ffa4e1187e8 CoreMessagingXP!Microsoft::WRL::Details::DelegateArgTraits<long (__cdecl Windows::Foundation::ITypedEventHandler_impl<Windows::Foundation::Internal::AggregateType<Microsoft::UI::Dispatching::DispatcherQueueTimer * __ptr64,Microsoft::UI::Dispatching::IDispatcherQueueTimer * __ptr64>,IInspectable * __ptr64>::*)(Microsoft::UI::Dispatching::IDispatcherQueueTimer * __ptr64,IInspectable * __ptr64) __ptr64>::DelegateInvokeHelper<Microsoft::WRL::Implements<Microsoft::WRL::RuntimeClassFlags<2>,Windows::Foundation::ITypedEventHandler<Microsoft::UI::Dispatching::DispatcherQueueTimer * __ptr64,IInspectable * __ptr64>,Microsoft::WRL::FtmBase>,<lambda_82cf8073f4f042d1a68771c460cb9f49>,-1,Microsoft::UI::Dispatching::IDispatcherQueueTimer * __ptr64,IInspectable * __ptr64>::Invoke+0xa8
		7ffa4e1189ee CoreMessagingXP!Microsoft::WRL::InvokeTraits<-2>::InvokeDelegates<<lambda_1e854da9c9ccd42f6138c3b007a32877>,Windows::Foundation::ITypedEventHandler<Microsoft::UI::Dispatching::DispatcherQueueTimer * __ptr64,IInspectable * __ptr64> >+0x8e
		7ffa4e1185bb CoreMessagingXP!Microsoft::UI::Dispatching::DispatcherQueueTimer::TimerCallback+0xdb
		7ffa4e0f1b68 CoreMessagingXP!CFlat::SehSafe::Execute<<lambda_654db17c35df07198786f0867aa10de6> >+0x2c
		7ffa4e0f1a3b CoreMessagingXP!Microsoft::CoreUI::Dispatch::TimeoutHandler::ImportAdapter$+0x5b
		7ffa4e0ca247 CoreMessagingXP!Microsoft::CoreUI::Dispatch::TimeoutManager::Callback_OnDispatch+0x1a7
		7ffa4e096ec4 CoreMessagingXP!Microsoft::CoreUI::Dispatch::Dispatcher::Callback_DispatchNextItem+0x1e4
		7ffa4e096725 CoreMessagingXP!Microsoft::CoreUI::Dispatch::Dispatcher::Callback_DispatchLoop+0x165
		7ffa4e08ed33 CoreMessagingXP!Microsoft::CoreUI::Dispatch::EventLoop::Callback_RunCoreLoop+0x16b
		7ffa4e0913a8 CoreMessagingXP!Microsoft::CoreUI::Dispatch::UserAdapter::DrainCoreMessagingQueue+0x138
		7ffa4e09186b CoreMessagingXP!Microsoft::CoreUI::Dispatch::UserAdapter::OnUserDispatch+0x97
		7ffa4e091774 CoreMessagingXP!Microsoft::CoreUI::Dispatch::UserAdapter::OnUserDispatchRaw+0x80
		7ffa4e0d8b8d CoreMessagingXP!Microsoft::CoreUI::Dispatch::UserAdapter::DoWork+0x59
		7ffa4e0d8b12 CoreMessagingXP!Microsoft::CoreUI::Dispatch::UserAdapter::HandleDispatchNotifyMessage+0x13e
		7ffa4e0d89c3 CoreMessagingXP!Microsoft::CoreUI::Dispatch::UserAdapter::WindowProc+0x73
		7ffa8349e858 user32!UserCallWinProcCheckWow+0x2f8
		7ffa8349e3dc user32!DispatchClientMessage+0x9c
		7ffa834b0c33 user32!_fnDWORD+0x33
		7ffa84790e94 ntdll!KiUserCallbackDispatcherContinue+0x0
		7ffa822d1104 win32u!NtUserGetMessage+0x14
		7ffa834b1bae user32!GetMessageW+0x2e
		7ff986b5e758 Microsoft_ui_xaml!DirectUI::FrameworkApplication::RunDesktopWindowMessageLoop+0x3c
		7ff986ac3856 Microsoft_ui_xaml!DirectUI::FrameworkApplication::StartDesktop+0x346
		7ff986ae1bfe Microsoft_ui_xaml!DirectUI::FrameworkApplicationFactory::Start+0x6e
		7ff9516f13b5

Depends on

VS bug #1887772

@DDHSchmidt DDHSchmidt added the t/bug Something isn't working label Sep 14, 2023
@DDHSchmidt
Copy link
Author

Update:

I'm testing inclusion of different WindowsAppSDK versions but since building release mode takes time that is still ongoing

That proved to be fruitless. I wasted hours manually including different NuGet versions of the WinAppSdk, but none of them produced a build that could be started.
I also played around with dotnet publish options, like omitting -p:WindowsAppSDKSelfContained=true , --self-contained, etc. but every msix that was output contained an app that crashed after start.

I can't stress enough how that has only started to happen after I installed the newest maui workload on my build machine and how I seem unable to revert that now :(

@samhouts samhouts added platform/windows 🪟 partner/winui WinUI / Project Reunion area-publishing Issues with the app packaging/publishing process (ipk/apk/msix/trimming) high It doesn't work at all, crashes or has a big impact. potential-regression This issue described a possible regression on a currently supported version., verification pending labels Sep 14, 2023
@samhouts samhouts added this to the .NET 8 GA milestone Sep 14, 2023
@mattleibow
Copy link
Member

mattleibow commented Sep 14, 2023

Thanks for reporting this and I think the issue is in the combination of the args and it seems sometimes the dotnet CLI is doing "interesting" things.

You have 2 commands, and I will break down why some of them cause issues.

However, I believe the main/core issue is that your runtime identifier is win-x64 when in fact the WASDK does not support that and is instead win10-x64 (with a 10). Ironically/sadly, this suddenly changed in RC 1 very near the end of the dev so actually it is win, but because no one knew and could react in time it is still win10 for now. Hopefully RC 2 all the players are ready. For thgis specifically, we have #17142 to track that but actually are waiting on WindowsAppSDK and Win2D.

The reason this fails is that when you build for win, the required WASDK assets are not added because they are win10. This means your app is missing the WASDK.

Now if you think to use the win10-x64 in the restore, it will fail on iOS targets. Which is true since iOS is not win10. Adding -f net8.0-windows10.0.19041.0 also seems to cause issues because for some reason the dornet restore CLI is not designed to handle TFMs. You can work around this with -p:TargetFramework=net8.0-windows10.0.19041.0. (tracked here dotnet/sdk#35408)

This leaves your restore command looking like:

dotnet restore CrashTestWin.csproj -p:PublishReadyToRun=false -r win10-x64 -p:TargetFramework=net8.0-windows10.0.19041.0

Next, your build command also needs to be updated to use win10:

dotnet publish CrashTestWin.csproj -f net8.0-windows10.0.19041.0 -r win10-x64 -c Release --no-restore --self-contained /p:RuntimeIdentifierOverride=win10-x64 /p:PublishReadyToRun=false -p:WindowsAppSDKSelfContained=true /p:Platform=x64 /p:GenerateAppxPackageOnBuild=true /p:AppxPackageSigningEnabled=true /p:PackageCertificateThumbprint=<YourCertificateThumbPrintHere>

This will work, but actually some of the things are no longer needed:

  • --self-contained - All Windows apps are self contained - unless you specifically are building an unpackaged app.
  • -p:WindowsAppSDKSelfContained=true - All Windows apps include the WASDK so this is not needed. You can choose NOT to have this, but true is the default.
  • /p:Platform=x64 - This is typically not needed since the Runtime Identifier already has this information.
  • /p:GenerateAppxPackageOnBuild=true - This is not needed since a publish will produce the msix.

I also have a question about the -p:PublishReadyToRun=false. Are you specifically trying to build a windows app that is not pre-compiled / AOT? Setting this to true usually speeds up the app a fair bit - mainly the launch.

If you remove the args I mention, then you can build and publish with:

dotnet restore CrashTestWin.csproj -r win10-x64 -p:TargetFramework=net8.0-windows10.0.19041.0
dotnet publish CrashTestWin.csproj -f net8.0-windows10.0.19041.0 -r win10-x64 -c Release --no-restore /p:AppxPackageSigningEnabled=true /p:PackageCertificateThumbprint=<YourCertificateThumbPrintHere>

Please let me know if this helps or if you are still seeing crashes.

@mattleibow mattleibow added s/needs-info Issue needs more info from the author not-regression labels Sep 14, 2023
@ghost
Copy link

ghost commented Sep 14, 2023

Hi @DDHSchmidt. We have added the "s/needs-info" label to this issue, which indicates that we have an open question for you before we can take further action. This issue will be closed automatically in 7 days if we do not hear back from you by then - please feel free to re-open it if you come back to this issue after that time.

@DDHSchmidt
Copy link
Author

Thank you for support, @mattleibow !

Ironically/sadly, this suddenly changed in RC 1 very near the end of the dev so actually it is win, but because no one knew and could react in time it is still win10 for now.

The irony of RC1 with the "go live license" to render our build pipeline useless wasn't lost on us... I do hope that changes of that magnitude will stay out of future RCs or at least handled way more carefully :/

But on with the more important news: Your command lines do produce a working, starting app... for the provided reproduction project.
When i tried to port that over to our real app, dotnet restore failed already with NETSDK1083: The specified RuntimeIdentifier 'win10-x64' is not recognized

Our app includes a project reference to another .Net project, a class library, and that seems to upset the restore command :/
I updated the reproduction project to also include a reference to a class library and it now displays the aforementioned error.

I followed the recommended actions but putting the RuntimeHostConfigurationOption into either or both of the projects had no discernible effect.

I also have a question about the -p:PublishReadyToRun=false. Are you specifically trying to build a windows app that is not pre-compiled / AOT? Setting this to true usually speeds up the app a fair bit - mainly the launch.

To be honest: We've been trying to port our app to MAUI since .Net 6 and most of the parameters (including that one) are probably remnants of our desperate attempts to get a working build somehow.
That's why your tips and in-depth explanations about them are very much appreciated 👍

@ghost ghost added s/needs-attention Issue has more information and needs another look and removed s/needs-info Issue needs more info from the author labels Sep 15, 2023
@DDHSchmidt
Copy link
Author

After a lot of trial & error (again...)
I can now conclude that for the reproduction project only, the following command will work:
dotnet publish CrashTestWin.csproj -r win10-x64 -f net8.0-windows10.0.19041.0 -c Release /p:TargetFramework=net8.0-windows10.0.19041.0 /p:AppxPackageSigningEnabled=true /p:PackageCertificateThumbprint=YourCertificateThumbPrintHere

Most obvious changes:

  1. Omit dotnet restore, By letting dotnet publish restore, the internal cogs seem to click together
  2. I had to dismiss the official recommended actions and use a csproj-property that was well hidden in a github issue comment.

The catch is: This still doesn't work for our real app...

Our app uses a private nuget repository for certain, internal artifacts, spread over different feeds.

  • I can add a nuget.config file via the --config parameter to dotnet restore which works fine.
  • But I cannot get dotnet publish --no-restore to run successfully if it was preceded by dotnet restore. It will always crash with 'NETSDK1005'. It's like those two were never meant to be run together -_-
  • I also cannot get dotnet publish to accept the nuget.config file. It only has a --source parameter which appears to accept only Uris to a single feed, but not multiple.

In summary:

  • dotnet restore && dotnet publish --no-restore are unable to play together
  • dotnet publish alone would probalby work but is unable to pick up the feeds for our internal artifacts
  • I'm forced to wait another month for RC2, when the tooling finally fits together?

I'm also curious about the "not-regression" tag? Clearly this was all working for us, before the introduction of RC1 and its associated changes.

@mattleibow
Copy link
Member

Just dropping this for reference, but the WASDK has a bug and does not like the RIDs: https://learn.microsoft.com/en-us/dotnet/maui/windows/deployment/publish-cli

We probably need to add this to the template for now, but you need to add this to the csproj:

<PropertyGroup Condition="$([MSBuild]::GetTargetPlatformIdentifier('$(TargetFramework)')) == 'windows' and '$(RuntimeIdentifierOverride)' != ''">
    <RuntimeIdentifier>$(RuntimeIdentifierOverride)</RuntimeIdentifier>
</PropertyGroup>

If you use the -r win10-x86 on the CLI, then this causes issues later for projects that have class libraries. We have work around this and add a property that sets it specifically for the windows app project. This means that instead of -r win10-x86, you need to /p:RuntimeIdentifierOverride=win10-x86

If you use the RID for iOS, then maybe you can get around the issue by setting that special hidden property, but I have not tested that.

And to work around the SDK issue, you need to use /p:TargetFramework=net8.0-windows10.0.19041.0 to pick the TFM for the restore.

With regards to the --source args, you can actually pass multiple --source XXX --source YYY.

I am unsure if either of the issues will get fixed for .NET 8, unfortunately. But I think the issue is with passing the RID, you need to trick the SDK into accepting it by using the snippet of code in your app project only. Libraries that have a RID will get yet another WASDK issue: microsoft/WindowsAppSDK#3337

All this is very unfortunate.

@jsuarezruiz jsuarezruiz added this to the Backlog milestone Sep 25, 2023
@ghost
Copy link

ghost commented Sep 25, 2023

We've added this issue to our backlog, and we will work to address it as time and resources allow. If you have any additional information or questions about this issue, please leave a comment. For additional info about issue management, please read our Triage Process.

@samhouts samhouts modified the milestones: Backlog, .NET 8 + Servicing Sep 25, 2023
@DDHSchmidt
Copy link
Author

Sorry for the late reply. Had some health issues and needed to take some unplanned away time.

If you use the -r win10-x86 on the CLI, ...

Was this a typo? Do we have to use x86 or did you mean to write x64 ?

As for your other suggestions /p:RuntimeIdentifierOverride & /p:TargetFramework -> They didn't work.
It produced NETSDK1005 again.

I got a working pipeline again by fiddling a ton with the parameters. To my surprise I got it working with -r win10-x64 on both restore and publish.
I might have screwed up other parameters for this again, but honestly: I don't want to rage code myself into hospital again and I'm becoming too tired to even care anymore.

With regards to the --source args, you can actually pass multiple --source XXX --source YYY.

While that appears possible, our internal feeds are behind password protection and I don't see that working without providing the whole nuget.config?
Really seems like an oversight that you can pass a configuration file to restore but when you use publish you can't ...

@QianaJiao QianaJiao added the s/triaged Issue has been reviewed label Feb 27, 2024
@jfversluis
Copy link
Member

While this all is very unfortunate, it seems like this is not something that originates in the .NET MAUI SDK. There are linked issues on the WinUI repo which should track work to make this better. Please add comments and details there in order for them to prioritize this correctly.

It seems for now that you are in a good place with the pipeline for building your app which I'm happy to see. Closing this here!

@github-actions github-actions bot locked and limited conversation to collaborators Mar 29, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-publishing Issues with the app packaging/publishing process (ipk/apk/msix/trimming) external high It doesn't work at all, crashes or has a big impact. partner/winui WinUI / Project Reunion platform/windows 🪟 potential-regression This issue described a possible regression on a currently supported version., verification pending s/needs-attention Issue has more information and needs another look s/triaged Issue has been reviewed t/bug Something isn't working
Projects
None yet
Development

No branches or pull requests

7 participants