Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added support for reading from stdin. #6

Merged
merged 3 commits into from
May 29, 2020
Merged

Conversation

leoleoasd
Copy link
Contributor

Added support for reading from stdin.

As described in this issue.

I only added functionality to read, didn't add an interface in HTML to specify the input. You can change the default MemFS.stdinStr to test this patch.

@leoleoasd
Copy link
Contributor Author

Why the memfs binary is smaller than original 🤔

Copy link
Owner

@binji binji left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few comments, otherwise looks pretty good!

Also, can you remove memfs changes here from the PR?

shared.js Outdated Show resolved Hide resolved
shared.js Outdated Show resolved Hide resolved
// write buf back
size += lenToWrite;
this.stdinStrPos += lenToWrite;
if(lenToWrite !== len){
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if this is correct behavior; I think the rest of the reads should still go through, even if the previous one fails.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when the lenToWrite is not equivalent with the len, it means that there is nothing more to read, we reached the EOF of the stdin file. I wrote break to speed things up, but when reading from regular files as you can see here, it will return some random data when reached EOF? I'm not sure about this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked the memfs.c again, I still can't figure out how the procedure of reading from a regular file works. These lines seems will write the whole file to the buf at once even the buf is smaller than the file? Have you tested this on use cases like reading large files?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No you're right. These functions are meant to work like linux readv. The confusion here is the difference between readv and preadv I think. ReadIovec assumes that the offset is fixed (like preadv), so it won't work properly if there are multiple buffers on a readv call.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So should we break here? In my opinion, we should, because we reached the end of the input string, continue reading seems useless.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think so, at least for now. Ultimately I think we may want a separate call, so preadv and readv can be handled separately when interacting with JS. But if most applications are treating stdin as stream-like, then updating the offset and stopping when there's no more input seems like the right choice.

shared.js Outdated Show resolved Hide resolved
@leoleoasd
Copy link
Contributor Author

leoleoasd commented May 28, 2020

Also, can you remove memfs changes here from the PR?

Should I make a git revert commit or git reset --soft and commit again to avoid the modified memfs file left in the git history?

Copy link
Owner

@binji binji left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just realized you will need to update memfs anyway; host_read isn't defined there, so you'll need to call out from memfs to the host in the same way host_write is implemented.

shared.js Outdated Show resolved Hide resolved
// write buf back
size += lenToWrite;
this.stdinStrPos += lenToWrite;
if(lenToWrite !== len){
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No you're right. These functions are meant to work like linux readv. The confusion here is the difference between readv and preadv I think. ReadIovec assumes that the offset is fixed (like preadv), so it won't work properly if there are multiple buffers on a readv call.

@leoleoasd
Copy link
Contributor Author

I just realized you will need to update memfs anyway; host_read isn't defined there, so you'll need to call out from memfs to the host in the same way host_write is implemented.

Also created a PR for your fork of llvm contains the changes of memfs.c.

@leoleoasd
Copy link
Contributor Author

Another question is that the js file (shared.js) requires the binary mapping of the iovs, such as the first 4 bytes is the buf. So, this can only be used with a specified wasi-sdk version, (10.0 wont work on my side), should we migrate it to use some helper functions from memfs.c such as getBufOfIovs to get the pointer of buf?

@binji
Copy link
Owner

binji commented May 28, 2020

So, this can only be used with a specified wasi-sdk version, (10.0 wont work on my side), should we migrate it to use some helper functions from memfs.c such as getBufOfIovs to get the pointer of buf?

I'd guess that there would be a lot more changes required for a different WASI version, so I'm not sure it's worth abstracting it for just this one case.

@leoleoasd
Copy link
Contributor Author

Ready for final review and merge.

Copy link
Owner

@binji binji left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, lgtm!

@kkoomen
Copy link

kkoomen commented Feb 10, 2024

How can I make this work in the current version without the use of any server? Because the example C code below inside the editor component doesn't prompt xterm for some stdin:

#include <stdio.h>

int main() {
    int num;
    printf("Enter a number: ");
    scanf("%d", &num);
    printf("The number you entered is %d\n", num);
    return 0;
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants