Problems using oemof with Windows 10 and Python 3.5

[Konsebast]

I am starting to write my master thisis atm at the University of Bremen. The Thesis should be about the development and modelling of an energy system for a mikrogrid in northern Germany. For building the model, I would like to use oemof.

Unfortunately I am struggeling with installing and using oemof. When I try to open the oemof examples, I only get errors.
Here is what I do:
PS D:\[...]\Desktop\milp_heating_system_optimization> python -m heating_system_optimization.py

Traceback (most recent call last):

File "C:\Users\[...]\AppData\Local\Programs\Python\Python35-32\lib\runpy.py", line 151, in _run_module_as_main

mod_name, mod_spec, code = _get_module_details(mod_name)

File "C:\Users\[...]\AppData\Local\Programs\Python\Python35-32\lib\runpy.py", line 104, in _get_module_details

spec = importlib.util.find_spec(mod_name)

File "C:\Users\[...]\AppData\Local\Programs\Python\Python35-32\lib\importlib\util.py", line 88, in find_spec

parent = __import__(parent_name, fromlist=['__path__'])

File "D:\[...]\Desktop\milp_heating_system_optimization\heating_system_optimization.py", line 13, in <module>

logger.define_logging()

File "C:\Users\[...]\AppData\Local\Programs\Python\Python35-32\lib\site-packages\oemof\tools\logger.py", line 64, in define_logging

logging.config.fileConfig(os.path.join(basicpath, 'logging.ini'))

File "C:\Users\[...]\AppData\Local\Programs\Python\Python35-32\lib\logging\config.py", line 84, in fileConfig

handlers = _install_handlers(cp, formatters)

File "C:\Users\[...]\AppData\Local\Programs\Python\Python35-32\lib\logging\config.py", line 147, in _install_handlers

args = eval(args, vars(logging))

File "<string>", line 1, in <module>

File "C:\Users\[...]\AppData\Local\Programs\Python\Python35-32\lib\os.py", line 683, in __getitem__

raise KeyError(key) from None

`KeyError: 'HOME'``

I would be very happy if someone could explain me, where I am wrong.

Sorry for the bad formatting of the code...

[simnh]

Hi,

I guess the problem is caused by windows, as we all use linux. The main problem is that our logging module can't be initialized (I guess). Unfortunately this module is used everywhere in the package, this must be fixed first.

@uvchik Maybe you can help as you wrote it?

P.S.
I made some fixes today on the oemof_examples repo, so once your error is fixed you should update this example repo otherwise you ll get the next error...

[uvchik]

@simnh : There have been developers using windows and they lately fixed an issue so that everything should work on windows.

@Konsebast : I'm confident, that we will fix that, because it is just a wrong path.

Do you use version 0.0.4 or the developing version? If you use the developing version, what commit/branch do you use?

Could you post the output of the following lines:

import sys
import os
print(sys.platform)
print(os.environ['USERPROFILE'])
print(os.environ['HOME'])

[uvchik]

Do you have an folder name .oemof on your system? Is there a file named logging.ini in this folder?

[uvchik]

By the way: To copy a code block you can use three backticks:

    ```
    copy your code
    ```

You can also write the language to get code highlighting:

    ```python
    copy your python code
    ```

[Konsebast]

Thank you for your quick reaction!

I am using version 0.0.4 atm.

import sys
import os
print(sys.platform)
win32
print(os.environ['USERPROFILE'])
C:\Users\Konstantin Kubina
print(os.environ['HOME'])
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\Konstantin Kubina\AppData\Local\Programs\Python\Python35-32\lib\os.py", line 683, in __getitem__
raise KeyError(key) from None
KeyError: 'HOME'

Sorry, I just found the "C:\Users\Konstantin Kubina.oemof" Folder with the logging.ini File in it.

[c-moeller]

Same error here when running on Windows:

win32
C:\Users\christian.wernitz

Traceback (most recent call last):

  File "C:\oemof\test.py", line 8, in <module>

    print(os.environ['HOME'])

  File "C:\Program Files\Python35\lib\os.py", line 683, in __getitem__

    raise KeyError(key) from None

KeyError: 'HOME'

But we have a logging.ini at C:\Users\christian.wernitz.oemof

[Konsebast]

Sorry, I just found the "C:\Users\Konstantin Kubina.oemof" Folder with the logging.ini File in it.

[c-moeller]

We have solved the problem by the following:

Once logging folder .oemof is initialized, open logging.ini and change:
Line 30: args=(os.path.join(os.environ['USERPROFILE'], '.oemof', 'log_files', 'oemof.log'),
Line 38: args=(os.path.join(os.environ['USERPROFILE'], '.oemof', 'log_files', 'oemof.log'),)

@Konsebast if this works for you, too, I'm going to add this to the documentation.

[Konsebast]

Thank you, it worked now partly... Unfortunately now I get the following error msg:

PS D:\OneDrive\Desktop\milp_heating_system_optimization> python -m heating_system_optimization.py
15:23:33-INFO-Path for logging: C:\Users\Konstantin Kubina\.oemof\log_files
C:\Users\Konstantin Kubina\AppData\Local\Programs\Python\Python35-32\python.exe: Error while finding spec for 'heating_system_optimization.py' (<class 'ImportError'>: No module named 'networkx')

[uvchik]

That means, that you haven't installed the networkx module.
Do something like pip3 install networkx

The best thing is to test the default oemof example which is the storage_example.

Store the three files in one folder and start the storage_invest.py file. You need to have at least one solver installed (see: https://software.sandia.gov/downloads/pub/pyomo/PyomoInstallGuide.html#Solvers)

You will find the name of the solver in line 71. It is 'glpk' now, but you can change it. You can use the open and free solvers glpk or cbc, while cbc is faster on my computer. The gurobi solver is only free for students (see gurobi academic license). It is the fastest solver of the three.

[uvchik]

The following link provides an .exe-file. You have to adapt the PATH or copy the cbc.exe to a folder that is in the PATH.

http://ampl.com/dl/open/cbc/cbc-win32.zip
http://ampl.com/dl/open/cbc/cbc-win64.zip

[Konsebast]

I am comming closer I think...

PS D:\OneDrive\Desktop\oemof-dev\oemof-dev\examples\storage_optimization> python -m storage_invest.py
16:51:47-INFO-Path for logging: C:\Users\Konstantin Kubina\.oemof\log_files
16:51:47-INFO-Read data from csv file and set time index
16:51:47-INFO-Initialize the energy system
16:51:47-INFO-Create oemof objects
16:51:47-INFO-Optimise the energy system
Traceback (most recent call last):
  File "C:\Users\Konstantin Kubina\AppData\Local\Programs\Python\Python35-32\lib\runpy.py", line 151, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name)
  File "C:\Users\Konstantin Kubina\AppData\Local\Programs\Python\Python35-32\lib\runpy.py", line 104, in _get_module_details
    spec = importlib.util.find_spec(mod_name)
  File "C:\Users\Konstantin Kubina\AppData\Local\Programs\Python\Python35-32\lib\importlib\util.py", line 88, in find_spec
    parent = __import__(parent_name, fromlist=['__path__'])
  File "D:\OneDrive\Desktop\oemof-dev\oemof-dev\examples\storage_optimization\storage_invest.py", line 157, in <module>
    energysystem.optimize()
  File "C:\Users\Konstantin Kubina\AppData\Local\Programs\Python\Python35-32\lib\site-packages\oemof\core\energy_system.py", line 130, in optimize
    om = OM(energysystem=self)
  File "C:\Users\Konstantin Kubina\AppData\Local\Programs\Python\Python35-32\lib\site-packages\oemof\solph\optimization_model.py", line 116, in __init__
    assembler.registry[cls](e=None, om=self, block=block)
KeyError: <class 'oemof.core.network.entities.ExcessSlack'>

Thank you for all the help

[simnh]

I guess this is still a bug. can you get the latest version of this example (master branch). I fixed it
few days ago...

[uvchik]

@Konsebast : Are you using the storage_invest.py from here together with oemof 0.0.4? That is really strange because I tested it on a fresh installation and everything works fine. The differences are that I used Python3.4 and not 3.5 and that I used an old WinXP. But that sounds like an oemof error and not anything related to python and Windows.

Anyway you can try excess=False in line 97 but this leads to strange results. Doing this you can check if everything else works.

@simnh Do you have an idea?

[simnh]

hm...I can only think of a problem with the "register" functions in solph that is related to python 3.5?

setting option excess and shortage to false might an options to track down the error..

with python 3.4 it works just fine on master.

[Konsebast]

After setting up python 3.4 new and reinstalling all the modules and setting the PATHs, the problem changed again a little bit.

PS D:\OneDrive\Desktop\oemof-master\oemof-master\examples\storage_optimization> python -m storage_invest.py
11:07:07-INFO-Path for logging: C:\Users\Konstantin Kubina\.oemof\log_files
11:07:07-INFO-Read data from csv file and set time index
11:07:07-INFO-Initialize the energy system
11:07:07-INFO-Create oemof objects
11:07:07-INFO-Optimise the energy system
11:07:08-INFO-Building component constraints.
11:07:11-INFO-Building bus constraints.
11:07:14-INFO-Building objective function.
11:07:14-INFO-Handing problem to solver and solving.
Traceback (most recent call last):
  File "D:\Masterarbeit\python\lib\runpy.py", line 151, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name)
  File "D:\Masterarbeit\python\lib\runpy.py", line 104, in _get_module_details
    spec = importlib.util.find_spec(mod_name)
  File "D:\Masterarbeit\python\lib\importlib\util.py", line 86, in find_spec
    parent = __import__(parent_name, fromlist=['__path__'])
  File "D:\OneDrive\Desktop\oemof-master\oemof-master\examples\storage_optimization\storage_invest.py", line 157, in <mo
dule>
    energysystem.optimize()
  File "D:\Masterarbeit\python\lib\site-packages\oemof\core\energy_system.py", line 135, in optimize
    solve_kwargs=self.simulation.solve_kwargs)
  File "D:\Masterarbeit\python\lib\site-packages\oemof\solph\optimization_model.py", line 384, in solve
    results = opt.solve(self, **solve_kwargs)
  File "D:\Masterarbeit\python\lib\site-packages\pyomo\opt\base\solvers.py", line 124, in solve
    self._solver_error('solve')
  File "D:\Masterarbeit\python\lib\site-packages\pyomo\opt\base\solvers.py", line 152, in _solver_error
    + "\n\toptions: %s" % ( self.options, ) )
RuntimeError: Attempting to use an unavailable solver.

The SolverFactory was unable to create the solver "cpc"
and returned an UnknownSolver object.  This error is raised at the point
where the UnknownSolver object was used as if it were valid (by calling
method "solve").

The original solver was created with the following parameters:
        solver_io: lp
        type: cpc
        _args: ()
        options: {}

[simnh]

ok thats good: your optimization model was build successfully with python 3.4. Now we know that we have to be aware that python 3.5 does not work with our current implementation

what solver is cpc? did you install it in a correct way (do you mean 'cbc'?)
under the hood we use pyomo to create the optimizationmodel. pyomo then passes the model to a solver.
You need to be able to execute your solver from your console (make the binary system wide excessible), then it should work. I don't know if pyomo takes care of that when it is installed.

This thread might be a help:

http://stackoverflow.com/questions/20690195/how-do-you-install-glpk-solver-along-with-pyomo-in-winpython

it's for glpk solver but the same applies for other solvers.

[Konsebast]

argh.... I am sorry, that was my mistake. Now it works perfect and without errors.

I wrote cpc instead of cbc into the .py file.

[uvchik]

What Windows version do you use? And if you have time you could send us the result of the following expression: print(os.path.expanduser('~')). Inspired by your problem we think about changing the command for the default path (see #122).

If everything is fine now, you could close this issue.

[Konsebast]

Thank you for your help.

I use Windows 10 pro

>>> os.path.expanduser('~')
'C:\\Users\\Konstantin Kubina'

[uvchik]

👍 Let us know if you have further problems. If you finally use oemof in your master thesis I am personally interested in the results and how you coped with oemof (see my profile to contact me).

p.s.: I change the title just for the archive and other users, who might have the same problem.