Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ZnBufferedReadWriteStream should handle better the buffer. #4726

Merged

Conversation

tesonep
Copy link
Collaborator

@tesonep tesonep commented Sep 25, 2019

Creating a custom Buffered Read/Write Stream that shares the buffer for writing and reading.

In Windows, this is really important when using disks with physical segments of 4k.

@tesonep
Copy link
Collaborator Author

tesonep commented Sep 25, 2019

This fix improves the speed of loading code in around 20 times.

@Ducasse
Copy link
Member

Ducasse commented Sep 25, 2019

Pablo you got some SDL 20 mixed inside (just a couple of methods).

@Ducasse Ducasse closed this Sep 25, 2019
@Ducasse Ducasse reopened this Sep 25, 2019
@Ducasse
Copy link
Member

Ducasse commented Sep 25, 2019

I wonder why we have code from MemoryFileStream in this PR.

MemoryFileWriteStream >> next: anInteger putAll: aCollection startingAt: anInteger3 [

stream next: anInteger putAll: aCollection startingAt: anInteger3 

]

@tesonep
Copy link
Collaborator Author

tesonep commented Sep 26, 2019

I wonder why we have code from MemoryFileStream in this PR.

MemoryFileWriteStream >> next: anInteger putAll: aCollection startingAt: anInteger3 [

stream next: anInteger putAll: aCollection startingAt: anInteger3 

]

The API of MemoryFileWriteStream was missing this method to be polymorphic with FileWriteStream

@tesonep
Copy link
Collaborator Author

tesonep commented Sep 26, 2019

More insight about the change.
Basically in new machines with 4kb physical sectors solid hard drives, Windows formats the drives using 4kb sectors and emulating 512 bytes sectors. (https://docs.microsoft.com/es-es/windows/win32/w8cookbook/advanced-format--4k--disk-compatibility-update?redirectedfrom=MSDN)

Basically it handles the writing and reading of small sectors accessing the disk.
This is really bad in the changes file as we are reading and writing the same file all the time.
Moreover, the changes file was using buffered streams for reading and writing, but the integration of them is through the disk, so any combination of reading and writing operations flush the buffers forcing accesses to the disk, therefore, slowing the process.

In a stock Pharo 8 image, the time to compile methods in a machine using emulated 512 bytes segments takes a lot of time:

Object subclass: #Prb
instanceVariableNames: ''
classVariableNames: ''
package: 'Prueba'.

[20 timesRepeat: [ Prb compile: 'm2
^ 42']] timeToRun.

"0:00:00:23.194"

This problem is really clear when loading big projects, as compiling any class or method access the changes file. Moreover, this also improves the times in a machine without 4kb physical sector disks.

When using the new solution that shares the buffer and improves the flushing handle the times get better:

[20 timesRepeat: [ Prb compile: 'm2
^ 42']] timeToRun.
"0:00:00:00.323"

Advanced format (4K) disk compatibility update

@jecisc
Copy link
Member

jecisc commented Sep 26, 2019

The failing test seems unrelated.

@Ducasse Ducasse merged commit 82d893e into pharo-project:Pharo8.0 Sep 26, 2019
@tesonep tesonep deleted the improving-speed-of-source-file branch March 31, 2021 09:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants