This repository demonstrates what seems to be a bug where if you nest a dialog trigger within a menu you it doesn't work.
Run the following:
yarn
yarn dev
This will run the app and you will see a page with who buttons. One is to toggle a menu and the other is to open a dialog. The dialog is the same in both cases but the dialog will not open when inside the Menu.