New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AvalonEdit issues with large files #788

Open
zaksmolen opened this Issue Mar 23, 2017 · 2 comments

Comments

Projects
None yet
2 participants
@zaksmolen

zaksmolen commented Mar 23, 2017

I am trying to use AvalonEdit integrated in my application. We have the potential for some obscenely large text files (115MB, 1 million+ lines). That is too large to put into a string, so the normal TextEditor.Load(string filename) function fails. After researching and finding a few somewhat-related threads I put this together:

private static IEnumerable<char> getFileAsIEnumerable(string fname)
{
	using (StreamReader file = new StreamReader(fname))
	{
		while (file.Peek() >= 0)
		{
			yield return (char)file.Read();
		}
	}
}		
		
// in other part of the code
EditorBox.Document = new TextDocument(getFileAsIEnumerable(fname));

I tried creating a char Rope and passing that into a text source and such, but that was actually consistently a tiny bit slower. As a benchmark I also tried loading this file in SharpDevelop and it loads in half the time that it takes to run the above code (<=5 seconds instead of 10). Does anyone have any ideas as to what could help speed it up to match the performance seen in SharpDevelop?

A few seconds doesn't sound too bad, but a. it's causing the application to hang and b. there are also a few seconds for other calculations, such as pre-loading the syntax coloring (otherwise scrolling the 1MM lines is jumpy) and other parsing we have to do with the file contents. So, now the application hangs for at least 15 seconds total (10 for loading and 5 for parsing and such). And that is on a better than average machine. Hoping to reduce loading time as much as possible to help make things run smoother.

@jogibear9988

This comment has been minimized.

Show comment
Hide comment
@jogibear9988

jogibear9988 Mar 23, 2017

Member

Maybe look at https://www.codeproject.com/articles/161871/fast-colored-textbox-for-syntax-highlighting It has a lazy load mode directly build in. You can use it in WPF with WinFormsHost Control

Member

jogibear9988 commented Mar 23, 2017

Maybe look at https://www.codeproject.com/articles/161871/fast-colored-textbox-for-syntax-highlighting It has a lazy load mode directly build in. You can use it in WPF with WinFormsHost Control

@zaksmolen

This comment has been minimized.

Show comment
Hide comment
@zaksmolen

zaksmolen Mar 23, 2017

Thanks for the fast feedback. That is a pretty awesome control, and even has some things I had to manually implement myself in my application using AvalonEdit. Unfortunately we already put a bunch of time and effort into porting from a previous control to AvalonEdit and aren't going to ditch it and redo everything with a new control again.

I like the idea of lazy loading, but it may have some trouble with our usage. In the case where people use million-line files, they will be searching and jumping around quite a lot. That would be tough on a lazy-load system.

I'm more curious why just the file loading on my application (not much else going on in the background) takes consistently 10 seconds while SharpDevelop loading the same file took 5 or less. Both tests were done 10+ times with consistent results so it wasn't just some fluke. Ideally, I would like to do whatever is done in SD. I have been reading through the source but nothing is jumping out at me. It almost even looks like they read the whole file in as a string, but that can't work because the file contents are too long for a string (again, the guiding motivation for writing my own loading functions).

zaksmolen commented Mar 23, 2017

Thanks for the fast feedback. That is a pretty awesome control, and even has some things I had to manually implement myself in my application using AvalonEdit. Unfortunately we already put a bunch of time and effort into porting from a previous control to AvalonEdit and aren't going to ditch it and redo everything with a new control again.

I like the idea of lazy loading, but it may have some trouble with our usage. In the case where people use million-line files, they will be searching and jumping around quite a lot. That would be tough on a lazy-load system.

I'm more curious why just the file loading on my application (not much else going on in the background) takes consistently 10 seconds while SharpDevelop loading the same file took 5 or less. Both tests were done 10+ times with consistent results so it wasn't just some fluke. Ideally, I would like to do whatever is done in SD. I have been reading through the source but nothing is jumping out at me. It almost even looks like they read the whole file in as a string, but that can't work because the file contents are too long for a string (again, the guiding motivation for writing my own loading functions).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment