Internal Logging Issue with OpenStudio 3.7 when it is Installed in a Path with a Space (eg. C:\Program Files\openstudio)

[chriswmackey]

Issue overview

I am opening this issue primarily for @jmarrec after asking an initial question on another issue. The issue is that, whenever OpenStudio is installed in a file path with a space (for example, the C:\Program Files directory that IT departments make us use), we get the following message when running an OSW with a reporting measure using OpenStudio CLI:

'C:/Program' is not recognized as an internal or external command,
operable program or batch file.
RunEnergyPlus: Completed Successfully with 0 Fatal Errors, 0 Severe Errors, 34 Warnings.

Current Behavior

The message does not indicate a complete OpenStudio CLI failure since the OSW run still completes successfully, including a successful run of EnergyPlus and the creation of reports from reporting measures specified in the OSW. However, it seems that whatever part of the OpenStudio 3.7 code that is producing this exception prevents us from seeing the progress of the reporting measure, whether it's the log of steps running in the measure or any adapters specified in the OSW, like those that allow us to see the progress of the EnergyPlus simulation as the OSW is being run.

Expected Behavior

In OpenStudio 3.6.1, we did not get this message about "'C:/Program' is not recognized as an internal or external command," and we would instead see the logs of the reporting measure being run as well as the adapter we use giving us a full report of the EnergyPlus simulation as it was run. For example, at a minimum, we would see the following:

ExpandObjects Started.
No expanded file generated.
ExpandObjects Finished with Error(s). Time:     0.016
RunEnergyPlus: Completed Successfully with 0 Fatal Errors, 0 Severe Errors, 34 Warnings.

... and, if we had an adapter specified in the OSW, we would see the full EnergyPlus progress.

Steps to Reproduce

I know that @jmarrec is not a Windows user so I think he may be able to create the issue on his end just by installing OpenStudio in a path with a space in it and trying to run the OpenStudio Results measure on any model (using the /path with spaces/openstudio/bin/openstudio to call the OpenStudio CLI executable). With this, I think he should be able to see the exception message and at least figure out where it is coming from with the OpenStudio code without the need to do a whole Windows setup. Also, he should not need to set up an adapter in the OSW or do anything fancy just to see this initial problem. The issue appears to be more deeply related to the ability to log the progress of reporting measures and not specific to the adapter as I originally thought.

If anyone else wants to recreate the issue or if my suggestion above does not work, I have a minimal OSW file that will work on Windows as long as the OpenStudio executable is installed in C:\Program Files\openstudio\bin\openstudio.exe (or another path with a space in it). Here is the zip file of the sample:
https://drive.google.com/file/d/16Rb6gnid0JOxPlcLd78du7F0A5R96x3J/view?usp=sharing

And, if you extract it, this is what you should see:

You have a small shoe box OSM, a Boston EPW and a workflow.osw that is meant to run only the OpenStudio Results measure, all out of a C:\test_osm` directory. Here are the contents of that simple OSW JSON:

{
    "seed_file": "C:\\test_os\\in.osm",
    "weather_file": "C:\\test_os\\USA_MA_Boston-Logan.Intl.AP.725090_TMY3.epw", 
    "steps": [
        {
            "arguments": {
                "schedules_overview_section": true, 
                "space_type_breakdown_section": true, 
                "plant_loops_detail_section": true, 
                "measure_warning_section": true, 
                "plug_loads_section": true, 
                "zone_equipment_detail_section": true, 
                "outdoor_air_section": true, 
                "air_loops_detail_section": true, 
                "envelope_section_section": true, 
                "space_type_details_section": true, 
                "source_energy_section": true, 
                "monthly_overview_section": true, 
                "water_use_section": true, 
                "interior_lighting_section": true, 
                "hvac_load_profile": true, 
                "building_summary_section": true, 
                "annual_overview_section": true, 
                "utility_bills_rates_section": true, 
                "units": "IP", 
                "reg_monthly_details": false, 
                "exterior_light_section": true, 
                "cost_summary_section": true, 
                "zone_summary_section": true, 
                "zone_condition_section": true
            }, 
            "measure_dir_name": "openstudio_results"
        }
    ]
}

Just run the OSW using:

cd C:\test_os
"C:\Program Files\openstudio\bin\openstudio.exe" run -w workflow.osw

... and you should see the issue.

Possible Solution

I have a feeling that all that is needed to fix this issue is that the path to a file or folder needs to be put in "double quotes" somewhere in the OpenStudio code. This way, the full path will be interpreted by the command line and it won't try to interpret 'C:/Program' as a command.

Environment

I'm running on Windows 10 but, as I said, I think any OpenStudio installation with a space in the path should illustrate the issue.

Context

This is not a deal breaker for us upgrading to OpenStudio 3.7 since, as I said, the OSW files are still running and producing the correct outputs. However, several of our users are starting to complain that "OpenStudio is frozen" since they can't see any logging or progress of the reporting measure. Here is an example.

Thanks, as always.

[jmarrec]

Yeah ok, I can reproduce (on mac here). Thanks for the details and the sleuthing @chriswmackey !

My money is on the call to ExpandObjects (That stupid fortran utility)

[jmarrec]

Bingo! (or "Yatzee!" I guess as you say in the US).

$ "/Users/julien/Downloads/Official_Binaries/os 370/bin/openstudio" --verbose run --show-stdout --style-stdout -w workflow.osw

[...]

#########################Returned from State PreProcess#########################
[openstudio.workflow.OSWorkflow] <Debug> EnergyPlus - selected = true

┌────────────────────────────────────────────────────────────────────────────────┐
│                           Starting State EnergyPlus                            │
└────────────────────────────────────────────────────────────────────────────────┘
[openstudio.OSWorkflow.prepareEnergyPlusDir] <Debug> Original Directory: /Users/julien/Downloads/temp/5108/test os
[openstudio.OSWorkflow.prepareEnergyPlusDir] <Debug> Changing To run directory: /Users/julien/Downloads/temp/5108/test os/run
[ApplicationPathHelpers] <Debug> getOpenStudioModule, info.dli_fname = '/Users/julien/Downloads/Official_Binaries/os 370/lib/libopenstudiolib.dylib'
[PathHelpers] <Debug> completeAndNormalize: result = /Users/julien/Downloads/Official_Binaries/os 370/lib/libopenstudiolib.dylib
[ApplicationPathHelpers] <Debug> getOpenStudioModule, found '/Users/julien/Downloads/Official_Binaries/os 370/lib/libopenstudiolib.dylib'
[ApplicationPathHelpers] <Debug> getOpenStudioModule, info.dli_fname = '/Users/julien/Downloads/Official_Binaries/os 370/lib/libopenstudiolib.dylib'
[PathHelpers] <Debug> completeAndNormalize: result = /Users/julien/Downloads/Official_Binaries/os 370/lib/libopenstudiolib.dylib
[ApplicationPathHelpers] <Debug> getOpenStudioModule, found '/Users/julien/Downloads/Official_Binaries/os 370/lib/libopenstudiolib.dylib'
[ApplicationPathHelpers] <Debug> Searching for EnergyPlus in '/Users/julien/Downloads/Official_Binaries/os 370/lib/../EnergyPlus/energyplus'
[PathHelpers] <Debug> completeAndNormalize: result = /Users/julien/Downloads/Official_Binaries/os 370/EnergyPlus/energyplus-23.2.0
[ApplicationPathHelpers] <Debug> Found EnergyPlus at '/Users/julien/Downloads/Official_Binaries/os 370/EnergyPlus/energyplus-23.2.0'
[PathHelpers] <Debug> completeAndNormalize: result = /Users/julien/Downloads/Official_Binaries/os 370/EnergyPlus/energyplus-23.2.0
[openstudio.OSWorkflow.prepareEnergyPlusDir] <Info> EnergyPlus executable path is /Users/julien/Downloads/Official_Binaries/os 370/EnergyPlus/energyplus
[openstudio.OSWorkflow.prepareEnergyPlusDir] <Info> ExpandObjects executable path is /Users/julien/Downloads/Official_Binaries/os 370/EnergyPlus/ExpandObjects
[openstudio.workflow.OSWorkflow] <Info> Starting simulation in run directory: /Users/julien/Downloads/temp/5108/test os/run
[openstudio.workflow.OSWorkflow] <Info> Running command '/Users/julien/Downloads/Official_Binaries/os 370/EnergyPlus/ExpandObjects'
sh: /Users/julien/Downloads/Official_Binaries/os: No such file or directory
[openstudio.workflow.OSWorkflow] <Warn> ExpandObjects returned a non-zero exit code (32512).
[openstudio.workflow.OSWorkflow] <Info> Running command '"/Users/julien/Downloads/Official_Binaries/os 370/EnergyPlus/energyplus" in.idf'
EnergyPlus Starting

[jmarrec]

OpenStudio/src/workflow/RunEnergyPlus.cpp

Lines 157 to 167 in c48911e

	// TODO: workflow-gem was manually running expandObjects prior to the potential serialization to json
	// Should we rather pass -x to the E+ cmd line?
	if (!workflowJSON.runOptions()->skipExpandObjects()) {
	const std::string cmd = openstudio::toString(runDirResults.expandObjectsExe.native());
	LOG(Info, "Running command '" << cmd << "'");
	int result = 0;
	detailedTimeBlock("Running ExpandObjects", [&cmd, &result] { result = std::system(cmd.c_str()); });
	if (result != 0) {
	LOG(Warn, "ExpandObjects returned a non-zero exit code (" << result << ").");
	}

@chriswmackey So unless you have HVACTemplate:XXXX objects in your IDF, you should be fine...

[jmarrec]

MCVE, even with your openstudio installed without a space

ln -sf /Applications/EnergyPlus-23-2-0 "energy plus2320"
export ENERGYPLUS_EXE_PATH="$(pwd)/energy plus2320/energyplus"
openstudio run --style-stdout --show-stdout -w workflow.osw

[chriswmackey]

Thanks, @jmarrec . Good to know. I guess that's why we always experienced the EnergyPlus simulation succeeding since we never use the HVACTemplate objects in our workflows with OpenStudio CLI.

I'm sensing that addressing this call to Expand Objects may not fix the lack of the reporting measure progress but at least this will prevent us from getting a cryptic message. So it's progress.

[jmarrec]

@chriswmackey either use the classic CLI, or Use "--show-stdout" (and capture it) (and optionally --style-stdout) when using the C++ CLI. This is what wer're doing for the OpenStudioApplication when using the C++ CLI (while we used the socket adapter for the legacy CLI).

Example

openstudio --verbose run --style-stdout --show-stdout -w workflow.osw
openstudio --loglevel Info run --style-stdout --show-stdout -w workflow.osw

You can also add --add-timings

[jmarrec]

@chriswmackey should be fixed by #5109 . When the windows binary is available (< 1 hour), I'd appreciate if you could test it.

[chriswmackey]

Thanks, @jmarrec . This is excellent. The --show-stdout is exactly the thing that I needed to be pointed towards and I'm already getting ready to push this out to our users:
ladybug-tools/honeybee-energy#1002

I will test the new build once it is available, though I see that merging is currently blocked. I'll keep an eye out for any updates but just mention my GitHub handle here again if I'm late in seeing it.

[jmarrec]

@chriswmackey the binary will be in this post: #5109 (comment)
I think in 20 minutes or so.

[chriswmackey]

Ah, disregard my last comment. I see you just want me to grab the build from the CI. So there's no need to have the PR merged to get the build. I'll just need to wait for the Windows version to finish building here.

[chriswmackey]

Hey @jmarrec . I can see that the Windows build finished but clicking on the .exe doesn't seem to start the download. Is there something that I'm missing here?

[chriswmackey]

Nevermind. I think it was just a bothersome security feature in Chrome that was not letting my download executables from addresses that didn't start with https. I have the exe now.

[chriswmackey]

Thanks, @jmarrec .

I tested it with the lasted build from the CI and I can confirm that everything works as expected when it's installed in a path with a space:

And the --show-stdout gives me exactly what I need:

Feel free to close the issue when ready.

[jmarrec]

Thanks for testing!