-
-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Upside-down image sent to AI translation service when using HW renderer #16275
Comments
Tagging the original PR #15640 in hope of more attention - maybe this will ring a bell to someone. |
Did the translation service work with HW-rendered cores before this was merged? I was thinking it didn't, though I don't know if this is the reason why. |
orque fechada se não resolveu? |
I do actually. I will not be able to look to much into it right away but off the top of my head I remember that the frame was returned upside down by the rendered, so we had to manually invert it before sending it to the AI service. It would seem that this particular renderer configuration causes the frame to be capture in the correct position from the start, if we can detect the renderer being used we can simply ignore this step. |
@xunkar here also another case is described for GNU/Linux, when retroarch crashes when using the hw-rendered core. RetroArch/tasks/task_translation.c Line 1250 in ac69414
Env
|
Looks like it can't even be reverted anymore, too much stuff has changed. So it looks like 1.18.0 will have this regression in it until @xunkar fixes this. |
Hey, I'm sorry about this and replying so late on this thread. I can try to take a look at this too since I did some work on that PR |
That'd be awesome yes if this issue can be fixed. |
Wait, something's toubling me here. OP showed that Ikaruga was flipped vertically, but with Citra it's flipped horizontally? I'm confused. I've found the part of the code I mentioned earlier. It's legacy code from Beaker that I left as is. He said as a comment: "Returned output from the png processor is an upside down RGBA image, so we have to change that to RGB first. This should probably be replaced with a scaler call." RetroArch/tasks/task_translation.c Line 768 in 06fa532
As it stands, a simple conditional statement on this block to filter out the problematic renderers should be enough. But the question about the difference in cores is most important, it might require much more in-depth testing. |
@xunkar Could it be that some cores use 32-bit color and others 16bit color? For 32bit it's XRGB8888 and for 16bit RGB565 |
Firstly, is it verified that screenshots themselves are not flipped upside down in these cases? If that’s the case then we should not fix it at the ai service level. @xunkar The op stated that the image the server gets is upside down. The code for decoding pngs is used in the callback after getting the response back from the server since pngs are naturally bottom to top encoded and have to be flipped for rendering. |
Then what we need is a scaler call in RetroArch/tasks/task_translation.c Line 1363 in 06fa532
|
Sending BMPs of hardware-accelerated cores could end up with some very large files, right? |
The frames are grabbed as native core resolution, so for the most part it makes little difference. Before the refactoring only BMP was supported and there wasn't much complain, I still added PNG support to be safe. |
This is still not fixed isn't it? I am planning a release of 1.18.0 sometime next week. We will see if by then someone will have submitted a PR to fix this issue or not. |
Description
When using the AI translation hotkey with a HW rendered core, the screenshot is sent upside-down to the translation server, making the translation impossible.
For example this is my RetroArch window:
and this is what the translation server received:
AFAICT it doesn't happen with software rendered cores.
Expected behavior
Send screenshot in the correct position.
Actual behavior
Send screenshot upside-down.
Steps to reproduce the bug
Bisect Results
Bisected to 274d47f
Version/Commit
Environment information
The text was updated successfully, but these errors were encountered: