Permalink
Switch branches/tags
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
36 lines (35 sloc) 1.75 KB
{
"alias": "video/1201/solving-the-import-problem-scalable-dynamic-load",
"category": "SciPy 2012",
"copyright_text": "CC BY-SA",
"description": "The most common programming paradigm for scientific computing, SPMD\n(Single Program Multiple Data), catastrophically interacts with the\nloading strategies of dynamically linked executables and\nnetwork-attached file systems on even moderately sized high performance\ncomputing clusters. This difficulty is further exacerbated by\n\"function-shipped\" I/O on modern supercomputer compute nodes, preventing\nthe deployment of simple solutions. In this talk, we introduce a\ntwo-component solution: collfs, a set of low-level MPI-collective file\noperations that can selectively shadow file system access in a library,\nand walla, a set of Python import hooks for seamlessly enabling parallel\ndynamic loading scalable to tens of thousands of cores.\n",
"duration": null,
"id": 1201,
"language": "eng",
"quality_notes": "",
"recorded": "2012-07-18",
"slug": "solving-the-import-problem-scalable-dynamic-load",
"speakers": [
"Aron Ahmadia",
"Jed Brown",
"William Scullin"
],
"summary": "",
"tags": [
"hpc"
],
"thumbnail_url": "http://i3.ytimg.com/vi/BpuykTOy4a0/hqdefault.jpg",
"title": "Solving the import problem: Scalable Dynamic Loading Network File Systems",
"videos": [
{
"length": null,
"type": "mp4",
"url": "http://s3.us.archive.org/nextdayvideo/enthought/scipy_2012/Solving_the_import_problem_Scalable_Dynamic_Loading_Network_File_Systems.mp4?Signature=03ccbsvqMA%2B2aqWka6hHeqFwiKI%3D&Expires=1346381205&AWSAccessKeyId=FEWGReWX3QbNk0h3"
},
{
"length": 0,
"type": "youtube",
"url": "http://youtube.com/watch?v=BpuykTOy4a0"
}
]
}