Skip to content
This repository was archived by the owner on Sep 30, 2022. It is now read-only.

Pr/performance tuning #475

Closed
wants to merge 17 commits into from
Closed

Pr/performance tuning #475

wants to merge 17 commits into from

Conversation

@edgargabriel
Copy link
Member

@edgargabriel edgargabriel commented Aug 7, 2015

No description provided.

@edgargabriel
Performance tuning: make sure the individual component is selected fo…
b0d975f 
…r 1 and 2 process communicators (important for some benchmarks)
@edgargabriel
free the datatypes that were created using type_dup during file_set_view
78c901d
@edgargabriel
Performance tuning. make sure we catch if the user wants to set the d…
4711106 
…efault fileview and replace it with our optimized default file view. Otherwise, performance will suffer. file_get_view should still return the correct filetype, not our optimized default file view. This is the correct version compared to ffa67b9, which unfortunately broke

some test cases in mpi_test_suite. Thanks for @ggouaillardet for reporting this!

Conflicts:
	ompi/mca/io/ompio/io_ompio.c
	ompi/mca/io/ompio/io_ompio_file_set_view.c
@edgargabriel
Performance tuning: change the default behavior of ompio to *not* seg…
1ab0545 
…ment individual read/write operations.

In most cases, performance seems to be better if not segmented.
@edgargabriel
free memory correctly in case of an error. Fixes CID 131540 and CID 1…
f12995a 
…315419
@edgargabriel
make sure the final number of aggregators is recorded correctly when …
fcaa733 
…not using

our aggregator selection logic.
@edgargabriel
Performance tuning: increase the priority of the sm sharedfp componen…
23ad3d0 
…t to ensure that it is selected if it can run.
@hppritcha hppritcha added this to the v2.0.0 milestone Aug 8, 2015
@hppritcha
pml/crcpw - remote crcpw pml
2192530 
First in a series of commits to remove unmaintained
checkpoint/restart related code from the v2.x
release branch.

Signed-off-by: Howard Pritchard <howardp@lanl.gov>
@hppritcha hppritcha self-assigned this Aug 11, 2015
@hppritcha
Copy link
Member

hppritcha commented Aug 11, 2015

I'll check this at nersc.

@hppritcha
Copy link
Member

hppritcha commented Aug 11, 2015

I'm having problems running filetest. If I run on a lustre fs, I get a coredump almost immediately with a trace back of:

0 0x00002aaaab921885 in raise () from /lib64/libc.so.6
(gdb) where
#0 0x00002aaaab921885 in raise () from /lib64/libc.so.6
#1 0x00002aaaab922df5 in abort () from /lib64/libc.so.6
#2 0x00002aaaab96030f in __libc_message () from /lib64/libc.so.6
#3 0x00002aaaab965b18 in malloc_printerr () from /lib64/libc.so.6
#4 0x00002aaaab96ab5c in free () from /lib64/libc.so.6
#5 0x00002aaaab3aa26f in opal_free (addr=0x792560, file=0x2aab080fd969 "io_ompio_file_open.c", line=369) at malloc.c:184
#6 0x00002aab080f7b1d in ompio_io_ompio_file_close (ompio_fh=0x7b5330) at io_ompio_file_open.c:369
#7 0x00002aab080f7797 in mca_io_ompio_file_close (fh=0x784940) at io_ompio_file_open.c:279
#8 0x00002aaaaad0f0cc in file_destructor (file=0x784940) at file/file.c:277
#9 0x00002aaaaad0e0d0 in opal_obj_run_destructors (object=0x784940) at ../opal/class/opal_object.h:446
#10 0x00002aaaaad0ea47 in ompi_file_close (file=0x7fffffff8288) at file/file.c:150
#11 0x00002aaaaad7a74b in PMPI_File_close (fh=0x7fffffff8288) at pfile_close.c:59
#12 0x0000000000404232 in open_test (comm=0x60b280 <ompi_mpi_comm_world>, root=0) at open.c:82
#13 0x00000000004022cb in main (argc=1, argv=0x7fffffff83b8) at filetest.c:56

@hppritcha
Copy link
Member

hppritcha commented Aug 11, 2015

verified that current v2.x HEAD (@5a80ea1f) can run filetest without incident on lustre and nfs fs's.

@edgargabriel
Copy link
Member Author

edgargabriel commented Aug 12, 2015

ok, will double check, probably a missing patch or similar. Thanks!

@hppritcha
Copy link
Member

hppritcha commented Aug 12, 2015

i will check further for missing prs fro. master. maybe didnt get the
lustre. malloc patch.

sent from my smart phonr so no good type.

Howard

ok, will double check, probably a missing patch or similar. Thanks!

—
Reply to this email directly or view it on GitHub
#475 (comment).

@edgargabriel
Performance tuning: make sure the individual component is selected fo…
78f85e0 
…r 1 and 2 process communicators (important for some benchmarks)
@edgargabriel
free the datatypes that were created using type_dup during file_set_view
6f50ca1
@edgargabriel
Performance tuning. make sure we catch if the user wants to set the d…
5318454 
…efault fileview and replace it with our optimized default file view. Otherwise, performance will suffer. file_get_view should still return the correct filetype, not our optimized default file view. This is the correct version compared to ffa67b9, which unfortunately broke

some test cases in mpi_test_suite. Thanks for @ggouaillardet for reporting this!

Conflicts:
	ompi/mca/io/ompio/io_ompio.c
	ompi/mca/io/ompio/io_ompio_file_set_view.c
@edgargabriel
Performance tuning: change the default behavior of ompio to *not* seg…
c53d974 
…ment individual read/write operations.

In most cases, performance seems to be better if not segmented.
@edgargabriel
free memory correctly in case of an error. Fixes CID 131540 and CID 1…
3fba27b 
…315419
@edgargabriel
Performance tuning: increase the priority of the sm sharedfp componen…
59e07e5 
…t to ensure that it is selected if it can run.
@edgargabriel
Copy link
Member Author

edgargabriel commented Aug 12, 2015

I see the problem, the v2.x branch that I used to create this pr did not contain the lustre fix yet. Updating right now the pr, give me a couple of minutes.

@hppritcha
Copy link
Member

hppritcha commented Aug 12, 2015

looks like we may be missing open-mpi/ompi@477083bc in v2.x

there's got to be a more automated way to manage these things.

@edgargabriel
Copy link
Member Author

edgargabriel commented Aug 12, 2015

I'll redo this pr. I made a mistake by not having an up to date v2.x branch when I created this pr, and that's why that commit is missing. As far as I can see, open-mpi/ompi@477083b is actually on v2.x, was just not in my clone when I created the branch.

@hppritcha
Copy link
Member

hppritcha commented Aug 12, 2015

correct, I too needed to refresh my local repo on my mac.

2015-08-12 9:40 GMT-06:00 Edgar Gabriel notifications@github.com:

I'll redo this pr. I made a mistake by not having an up to date v2.x
branch when I created this pr, and that's why that commit is missing. As
far as I can see, open-mpi/ompi@477083b
open-mpi/ompi@477083b is actually on v2.x,
was just not in my clone when I created the branch.

—
Reply to this email directly or view it on GitHub
#475 (comment)
.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

No reviews

Assignees

@hppritcha hppritcha

Labels

pushed-back

Projects

None yet

Milestone

v2.0.0

Development

Successfully merging this pull request may close these issues.

3 participants

@edgargabriel @hppritcha @jsquyres