Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Increase UDF server startup timeout #1168

Closed
yhlam opened this issue Oct 24, 2019 · 12 comments · Fixed by #1179
Milestone

Comments

@yhlam
Copy link

@yhlam yhlam commented Oct 24, 2019

There is a hard coded 1 minute timeout of starting python server process:

// now repeatedly try to create the Python interface object, waiting up to 1 minute to do it
for(int k=0; k<600; k++)
{
// try to create the object
hr = CoCreateInstance(clsid, NULL, CLSCTX_LOCAL_SERVER, IID_IDispatch, (void**) &(this->pInterface));
if(hr != REGDB_E_CLASSNOTREG)
break;
// didn't create object - check that python process is still there!
DWORD dwExitCode;
if(0 == GetExitCodeProcess(pi.hProcess, &dwExitCode))
throw formatted_exception() << "GetExitCodeProcess failed: " << GetLastErrorMessage();
if(dwExitCode != STILL_ACTIVE)
{
formatted_exception e;
e << "Python process exited before it was possible to create the interface object.";
if(this->HasValue("RedirectOutput"))
e << " Try consulting '" << this->GetValue("RedirectOutput") << "'.";
e << "\n";
e << "Command: " << cmdLine << "\nWorking Dir: " << workingDir;
throw e;
}
Sleep(100);
}

I have a python installation on a network drive. Sometimes, it can take more than 1 minute to start the python process due to network slowness. It would be nice if the timeout can be configured like other parameters, so that the timeout can be increased for use case like mine which expects long start up time.

Thanks.

p.s. This is a great project. Thank you so much for all your hard work.

@fzumstein

This comment has been minimized.

Copy link
Member

@fzumstein fzumstein commented Oct 24, 2019

How are you solving the issue currently?

@yhlam

This comment has been minimized.

Copy link
Author

@yhlam yhlam commented Oct 24, 2019

I currently have to start a debug server and use debug mode to work around it

@fzumstein

This comment has been minimized.

Copy link
Member

@fzumstein fzumstein commented Oct 24, 2019

ok good call! How long does it approx. take to start the server?

@yhlam

This comment has been minimized.

Copy link
Author

@yhlam yhlam commented Oct 25, 2019

Just the server startup is around 1.5min. It takes another 3-4min to import all the dependencies when RunPython is called in the first time. But after that, everything are loaded into memory, so no network related delay in further execution.

@fzumstein

This comment has been minimized.

Copy link
Member

@fzumstein fzumstein commented Oct 25, 2019

It's easy enough to increase the threshold to 2 minutes for the next release, but until I'll get around to make this properly configurable will take a lot longer.

@fzumstein fzumstein added this to the 0.16.1 milestone Oct 25, 2019
@yhlam

This comment has been minimized.

Copy link
Author

@yhlam yhlam commented Oct 25, 2019

Great, thanks

@ankitagra

This comment has been minimized.

Copy link

@ankitagra ankitagra commented Nov 8, 2019

I experience the same issue.
I can load xlwings when I install Python environment (and xlwings) locally in C:.
But when I install Python on a network drive, then I get an error Could not activate python COM server, hr = -2147221164 1000 (could be related to #1036). And the error pops up exactly after a minute.
@fzumstein is there a way I can change this hard-coded time in my local (xlwings) installation? Or would I need to wait for your release? Could you please tell when you plan to do it plz?

@fzumstein

This comment has been minimized.

Copy link
Member

@fzumstein fzumstein commented Nov 8, 2019

@ankitagra can you check how long it takes on your side by launching the server manually in debug mode? (see: http://docs.xlwings.org/en/stable/debugging.html#udf-debug-server)

@ankitagra

This comment has been minimized.

Copy link

@ankitagra ankitagra commented Nov 13, 2019

@fzumstein I tested in debug mode and it takes somewhere around 2.5 mins (including the Pycharm debugger loading time), so xlwings alone may be taking around 2mins (or slightly lesser).

Btw, to give more context .... I tested loading xlwings on 2 windows PCs (exactly same build and MS office package) using the Python installation on a network drive - one of the PCs loads xlwings fast (around half a min), while the other (problematic) PC crashes (after hitting the cap of 1 min). It is the latter that takes 2+ mins in the debug mode.

I find that even the dll (xlwings32-0.15.3.dll) loading time is different on the 2 PCs (PC1 is quick, PC2 takes long). I was assuming that it should be very similar (considering both have same MSoffice packages, etc).

Please let me know if you want to test anything else.
Please let me know your plans on next xlwings release dates, or some quick fix that I can do on my side to get around with this crashing? Thank you.

@fzumstein

This comment has been minimized.

Copy link
Member

@fzumstein fzumstein commented Nov 13, 2019

@ankitagra you can download a test package from here: https://ci.appveyor.com/project/zoomeranalytics/xlwings/builds/28823558/artifacts

download the package, then install with pip: pip install xlwings-4d11f88.tar.gz.

It has the timeout doubled to 2mins. Let me know if that's good enough. Making it configurable unfortunately isn't something I have the time right now as this is a bigger thing (you can always make me change my mind via: https://www.xlwings.org/pricing)

@fzumstein

This comment has been minimized.

Copy link
Member

@fzumstein fzumstein commented Nov 19, 2019

HI @ankitagra have you had a chance to test?

@ankitagra

This comment has been minimized.

Copy link

@ankitagra ankitagra commented Nov 19, 2019

Sorry @fzumstein, dint get time. I'll try to test over this weekend and let you know.

@fzumstein fzumstein changed the title Change the hard coded UDF server startup timeout to be configurable Increase UDF server startup timeout Dec 1, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.