Skip to content

Build : Add more SCons 'Depends' onto the python modules #1185

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
May 10, 2022
Merged

Build : Add more SCons 'Depends' onto the python modules #1185

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 24 additions & 15 deletions SConstruct
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1764,8 +1764,8 @@ coreEnv.Alias( "installCore", pythonHeaderInstall )
corePythonModuleEnv.Append( LIBS = os.path.basename( coreEnv.subst( "$INSTALL_LIB_NAME" ) ) )
corePythonModuleEnv.Append( LIBS = os.path.basename( corePythonEnv.subst( "$INSTALL_PYTHONLIB_NAME" ) ) )
corePythonModule = corePythonModuleEnv.SharedLibrary( "python/IECore/_IECore", corePythonModuleSources )
corePythonModuleEnv.Depends( corePythonModule, coreLibrary )
corePythonModuleEnv.Depends( corePythonModule, corePythonLibrary )
corePythonModuleEnv.Depends( corePythonModule, coreLibraryInstall )
corePythonModuleEnv.Depends( corePythonModule, corePythonLibraryInstall )
Copy link
Member

@andrewkaufman andrewkaufman Nov 4, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit confused about this change. You should be able to run scons without anything actually installing, it all just compiles in-place, but this change would cause libIECorePython.so to actually install.

I think this line was correct before, and what you're needing is for corePythonModuleInstall to depend on corePythonLibraryInstall.

Copy link
Member

@andrewkaufman andrewkaufman Nov 4, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more. 

index ab0fa4b66..c482335fe 100644
--- a/SConstruct
+++ b/SConstruct
@@ -1763,10 +1763,11 @@ corePythonModuleEnv.Append( LIBS = os.path.basename( coreEnv.subst( "$INSTALL_LI
 corePythonModuleEnv.Append( LIBS = os.path.basename( corePythonEnv.subst( "$INSTALL_PYTHONLIB_NAME" ) ) )
 corePythonModule = corePythonModuleEnv.SharedLibrary( "python/IECore/_IECore", corePythonModuleSources )
 corePythonModuleEnv.Depends( corePythonModule, coreLibrary )
-corePythonModuleEnv.Depends( corePythonModule, corePythonLibraryInstall )
+corePythonModuleEnv.Depends( corePythonModule, corePythonLibrary )
 
 corePythonModuleInstall = corePythonModuleEnv.Install( "$INSTALL_PYTHON_DIR/IECore", corePythonScripts + corePythonModule )
 corePythonModuleEnv.AddPostAction( "$INSTALL_PYTHON_DIR/IECore", lambda target, source, env : makeSymLinks( corePythonEnv, corePythonEnv["INSTALL_PYTHON_DIR"] ) )
+corePythonModuleEnv.Depends( corePythonModuleInstall, corePythonLibraryInstall )

Copy link
Contributor Author

@boberfly boberfly May 4, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @andrewkaufman

I ran into this one again and the exact issue is: /opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/ld: /dependencies-5e01d53f74c525306a27c497de35e1a0510d6546-source/gafferDependenciesBuild/lib/libIECorePython.so: file not recognized: file truncated

My memory might be wrong, but your suggestion I think didn't work for me until I used the corePythonLibraryInstall as the dependency, perhaps the dependent modules try to link to the installed one? Not sure...

Copy link
Contributor Author

@boberfly boberfly May 4, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trying it now with a rebase

Copy link
Contributor Author

@boberfly boberfly May 9, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just an update, the requested change from @andrewkaufman still caused race conditions in the build for me, so I've kept the depends to corePythonLibraryInstall so this is just a rebase.


corePythonModuleInstall = corePythonModuleEnv.Install( "$INSTALL_PYTHON_DIR/IECore", corePythonScripts + corePythonModule )
corePythonModuleEnv.AddPostAction( "$INSTALL_PYTHON_DIR/IECore", lambda target, source, env : makeSymLinks( corePythonEnv, corePythonEnv["INSTALL_PYTHON_DIR"] ) )
Expand Down Expand Up @@ -1912,7 +1912,8 @@ if doConfigure :
]
)
imagePythonModule = imagePythonModuleEnv.SharedLibrary( "python/IECoreImage/_IECoreImage", imagePythonSources + imagePythonModuleSources )
imagePythonModuleEnv.Depends( imagePythonModule, imageLibrary )
imagePythonModuleEnv.Depends( imagePythonModule, imageLibraryInstall )
imagePythonModuleEnv.Depends( imagePythonModule, corePythonLibraryInstall )

# python module install
imagePythonModuleInstall = imagePythonModuleEnv.Install( "$INSTALL_PYTHON_DIR/IECoreImage", imagePythonScripts + imagePythonModule )
Expand Down Expand Up @@ -1986,9 +1987,9 @@ if doConfigure :
# python module
scenePythonModuleEnv.Append( LIBS = os.path.basename( sceneEnv.subst( "$INSTALL_LIB_NAME" ) ) )
scenePythonModule = scenePythonModuleEnv.SharedLibrary( "python/IECoreScene/_IECoreScene", scenePythonModuleSources )
scenePythonModuleEnv.Depends( scenePythonModule, coreLibrary )
scenePythonModuleEnv.Depends( scenePythonModule, corePythonLibrary )
scenePythonModuleEnv.Depends( scenePythonModule, sceneLibrary )
scenePythonModuleEnv.Depends( scenePythonModule, coreLibraryInstall )
scenePythonModuleEnv.Depends( scenePythonModule, corePythonLibraryInstall )
scenePythonModuleEnv.Depends( scenePythonModule, sceneLibraryInstall )

scenePythonModuleInstall = scenePythonModuleEnv.Install( "$INSTALL_PYTHON_DIR/IECoreScene", scenePythonScripts + scenePythonModule )
scenePythonModuleEnv.AddPostAction( "$INSTALL_PYTHON_DIR/IECoreScene", lambda target, source, env : makeSymLinks( scenePythonModuleEnv, scenePythonModuleEnv["INSTALL_PYTHON_DIR"] ) )
Expand Down Expand Up @@ -2086,7 +2087,8 @@ if doConfigure :
]
)
vdbPythonModule = vdbPythonModuleEnv.SharedLibrary( "python/IECoreVDB/_IECoreVDB", vdbPythonModuleSources )
vdbPythonModuleEnv.Depends( vdbPythonModule, vdbLibrary )
vdbPythonModuleEnv.Depends( vdbPythonModule, vdbLibraryInstall )
vdbPythonModuleEnv.Depends( vdbPythonModule, corePythonLibraryInstall )

vdbPythonModuleInstall = vdbPythonModuleEnv.Install( "$INSTALL_PYTHON_DIR/IECoreVDB", vdbPythonScripts + vdbPythonModule )
vdbPythonModuleEnv.AddPostAction( "$INSTALL_PYTHON_DIR/IECoreVDB", lambda target, source, env : makeSymLinks( vdbPythonModuleEnv, vdbPythonModuleEnv["INSTALL_PYTHON_DIR"] ) )
Expand Down Expand Up @@ -2273,7 +2275,8 @@ if env["WITH_GL"] and doConfigure :
]
)
glPythonModule = glPythonModuleEnv.SharedLibrary( "python/IECoreGL/_IECoreGL", glPythonSources )
glPythonModuleEnv.Depends( glPythonModule, glLibrary )
glPythonModuleEnv.Depends( glPythonModule, glLibraryInstall )
glPythonModuleEnv.Depends( glPythonModule, corePythonLibraryInstall )

glPythonScripts = glob.glob( "python/IECoreGL/*.py" )
glPythonModuleInstall = glPythonModuleEnv.Install( "$INSTALL_PYTHON_DIR/IECoreGL", glPythonScripts + glPythonModule )
Expand Down Expand Up @@ -2469,7 +2472,8 @@ if doConfigure :
]
)
mayaPythonModule = mayaPythonModuleEnv.SharedLibrary( "python/IECoreMaya/_IECoreMaya", mayaPythonSources )
mayaPythonModuleEnv.Depends( mayaPythonModule, mayaLibrary )
mayaPythonModuleEnv.Depends( mayaPythonModule, mayaLibraryInstall )
mayaPythonModuleEnv.Depends( mayaPythonModule, corePythonLibraryInstall )

mayaPythonModuleInstall = mayaPythonModuleEnv.Install( "$INSTALL_PYTHON_DIR/IECoreMaya", mayaPythonScripts + mayaPythonModule )
mayaPythonModuleEnv.AddPostAction( "$INSTALL_PYTHON_DIR/IECoreMaya", lambda target, source, env : makeSymLinks( mayaPythonModuleEnv, mayaPythonModuleEnv["INSTALL_PYTHON_DIR"] ) )
Expand Down Expand Up @@ -2684,7 +2688,8 @@ if doConfigure :
nukePythonModuleEnv.AddPostAction( "$INSTALL_NUKEPYTHON_DIR/IECoreNuke", lambda target, source, env : makeSymLinks( nukePythonModuleEnv, nukePythonModuleEnv["INSTALL_NUKEPYTHON_DIR"] ) )
nukePythonModuleEnv.Alias( "install", nukePythonModuleInstall )
nukePythonModuleEnv.Alias( "installNuke", nukePythonModuleInstall )
nukePythonModuleEnv.Depends( nukePythonModule, corePythonModule )
nukePythonModuleEnv.Depends( nukePythonModule, corePythonModuleInstall )
nukePythonModuleEnv.Depends( nukePythonModule, corePythonLibraryInstall )

if coreEnv["INSTALL_CORENUKE_POST_COMMAND"]!="" :
# this is the only way we could find to get a post action to run for an alias
Expand Down Expand Up @@ -2913,7 +2918,8 @@ if doConfigure :
]
)
houdiniPythonModule = houdiniPythonModuleEnv.SharedLibrary( "python/IECoreHoudini/_IECoreHoudini", houdiniPythonSources )
houdiniPythonModuleEnv.Depends( houdiniPythonModule, houdiniLib )
houdiniPythonModuleEnv.Depends( houdiniPythonModule, houdiniLibInstall )
houdiniPythonModuleEnv.Depends( houdiniPythonModule, corePythonLibraryInstall )
houdiniPythonModuleInstall = houdiniPythonModuleEnv.Install( "$INSTALL_PYTHON_DIR/IECoreHoudini", houdiniPythonScripts + houdiniPythonModule )
houdiniPythonModuleEnv.AddPostAction( "$INSTALL_PYTHON_DIR/IECoreHoudini", lambda target, source, env : makeSymLinks( houdiniPythonModuleEnv, houdiniPythonModuleEnv["INSTALL_PYTHON_DIR"] ) )
houdiniPythonModuleEnv.Alias( "install", houdiniPythonModuleInstall )
Expand Down Expand Up @@ -3114,7 +3120,8 @@ if doConfigure :
]
)
usdPythonModule = usdPythonModuleEnv.SharedLibrary( "contrib/IECoreUSD/python/IECoreUSD/_IECoreUSD", usdPythonSources )
usdPythonModuleEnv.Depends( usdPythonModule, usdLibrary )
usdPythonModuleEnv.Depends( usdPythonModule, usdLibraryInstall )
usdPythonModuleEnv.Depends( usdPythonModule, corePythonLibraryInstall )

usdPythonModuleInstall = usdPythonModuleEnv.Install( "$INSTALL_PYTHON_DIR/IECoreUSD", usdPythonScripts + usdPythonModule )
usdPythonModuleEnv.AddPostAction( "$INSTALL_PYTHON_DIR/IECoreUSD", lambda target, source, env : makeSymLinks( usdPythonModuleEnv, usdPythonModuleEnv["INSTALL_PYTHON_DIR"] ) )
Expand Down Expand Up @@ -3247,8 +3254,9 @@ if doConfigure :
]
)
alembicPythonModule = alembicPythonModuleEnv.SharedLibrary( "contrib/IECoreAlembic/python/IECoreAlembic/_IECoreAlembic", alembicPythonSources )
alembicPythonModuleEnv.Depends( alembicPythonModule, alembicLibrary )
alembicPythonModuleEnv.Depends( alembicPythonModule, scenePythonModule )
alembicPythonModuleEnv.Depends( alembicPythonModule, alembicLibraryInstall )
alembicPythonModuleEnv.Depends( alembicPythonModule, scenePythonModuleInstall )
alembicPythonModuleEnv.Depends( alembicPythonModule, corePythonLibraryInstall )

alembicPythonModuleInstall = alembicPythonModuleEnv.Install( "$INSTALL_PYTHON_DIR/IECoreAlembic", alembicPythonScripts + alembicPythonModule )
alembicPythonModuleEnv.AddPostAction( "$INSTALL_PYTHON_DIR/IECoreAlembic", lambda target, source, env : makeSymLinks( alembicPythonModuleEnv, alembicPythonModuleEnv["INSTALL_PYTHON_DIR"] ) )
Expand Down Expand Up @@ -3383,7 +3391,8 @@ if doConfigure :
]
)
appleseedPythonModule = appleseedPythonModuleEnv.SharedLibrary( "contrib/IECoreAppleseed/python/IECoreAppleseed/_IECoreAppleseed", appleseedPythonSources )
appleseedPythonModuleEnv.Depends( appleseedPythonModule, appleseedLibrary )
appleseedPythonModuleEnv.Depends( appleseedPythonModule, appleseedLibraryInstall )
appleseedPythonModuleEnv.Depends( appleseedPythonModule, corePythonLibraryInstall )

appleseedPythonModuleInstall = appleseedPythonModuleEnv.Install( "$INSTALL_PYTHON_DIR/IECoreAppleseed", appleseedPythonScripts + appleseedPythonModule )
appleseedPythonModuleEnv.AddPostAction( "$INSTALL_PYTHON_DIR/IECoreAppleseed", lambda target, source, env : makeSymLinks( appleseedPythonModuleEnv, appleseedPythonModuleEnv["INSTALL_PYTHON_DIR"] ) )
Expand Down