Pywren and python ORM support (SQLAlchemy module)

[vijayaratha]

Im testing pywren to see whether we can work with python ORM (MYSQL DB runs on AMAZON RDS)

My sample program;

import pywren
import...

def saveObservations(tankNo):
   sortedList=[...] <--Hardcoded list
    listToSave = []
    values = calculate(sortedList)
    listToSave.append(mapperObservations.toDbObj(values))
    saveObservations(listToSave)


 tanks=[1, 2, 3, 4]
 wrenexec = pywren.default_executor()
 futures = wrenexec.map(saveObservations, tanks)
  print [f.result() for f in futures]

To get ORM support, we use SQLAlchemy[1] library.
I checked my python environment , it has all libraries installed. (sqlalchemy, flask-sqlalchemy,..)
But when trying to execute this test program, I get following error.My pyCharm configurations/python/libraries path are correct. I checked several times. Without Pywren library, Our normal project with ORM support (using sqlalchemy module) works.
When I modify that with Pywren I get this error. I suspect some issue at Pywren library which could not find sqlalchemy module.
Can anyone point me the what i'm doing wrong here?

Traceback (most recent call last):

  File "/Users/ratha/projects/test711/ATGWS/ParallelProcessor/ATGObservationORM.py", line 28, in <module>
    print [f.result() for f in futures]
  File "/Users/ratha/projects/test711/ATGWS/venv/lib/python2.7/site-packages/pywren/future.py", line 202, in result
    reraise(*self._traceback)
  File "/var/task/jobrunner.py", line 29, in <module>
ImportError: No module named sqlalchemy.util._collections

[1] https://www.sqlalchemy.org/

[ericmjonas]

It looks like SQLAlchemy is a module with a c extension that our runtime does not currently support. To use sqlalchemy we would have to currently build a custom runtime, whcih might be possible. You're using mysql for your db backend?

[vijayaratha]

Yes.MySql is my backend. Do you have any guide on how to build custom runtime and how to use that?. If a guide is provided which would be great help for us.So, in the future we may not need to ask for help for custom runtime.