Create a PyIodide package for Andes #109
Assignees
Labels
Comments
|
I see now that it is already possible to try Andes online here: |
|
Got it. Right now, Andes runs online in Binder. I was impressed by matpower.app's performance in WebAssembly. Will give it a shot at some later time. |
|
This is amazing!! With the latest version of ANDES, you create a customized distribution to ANDES source code to skip the code generation. The steps are as follows.
A folder named The inconvenience is that the above steps need to be executed every time after updating ANDES models. Let me know if any thoughts. |
|
Hi @rwl , I wonder if you could share the build script |
|
For some reason, I didn't use the CLAPACK package that comes with Pyodide. I have a version that applies these patches: diff -Naur a/BLAS/SRC/xerbla.c b/BLAS/SRC/xerbla.c
--- a/BLAS/SRC/xerbla.c 2010-04-27 21:23:16.000000000 +0200
+++ b/BLAS/SRC/xerbla.c 2021-03-25 19:58:44.754892352 +0100
@@ -10,6 +10,8 @@
http://www.netlib.org/f2c/libf2c.zip
*/
+#include <stdio.h>
+
#include "f2c.h"
#include "blaswrap.h"
diff -Naur a/F2CLIBS/libf2c/ef1asc_.c b/F2CLIBS/libf2c/ef1asc_.c
--- a/F2CLIBS/libf2c/ef1asc_.c 2009-08-08 00:32:18.000000000 +0200
+++ b/F2CLIBS/libf2c/ef1asc_.c 2021-03-25 20:05:18.970632776 +0100
@@ -13,7 +13,7 @@
extern VOID s_copy();
ef1asc_(a, la, b, lb) ftnint *a, *b; ftnlen *la, *lb;
#else
-extern void s_copy(char*,char*,ftnlen,ftnlen);
+extern int s_copy(char*,char*,ftnlen,ftnlen);
int ef1asc_(ftnint *a, ftnlen *la, ftnint *b, ftnlen *lb)
#endif
{
diff -Naur a/F2CLIBS/libf2c/f2ch.add b/F2CLIBS/libf2c/f2ch.add
--- a/F2CLIBS/libf2c/f2ch.add 2009-08-08 00:32:18.000000000 +0200
+++ b/F2CLIBS/libf2c/f2ch.add 2021-03-25 20:05:18.970632776 +0100
@@ -124,9 +124,9 @@
extern double r_sqrt(float *);
extern double r_tan(float *);
extern double r_tanh(float *);
-extern void s_cat(char *, char **, integer *, integer *, ftnlen);
+extern int s_cat(char *, char **, integer *, integer *, ftnlen);
extern integer s_cmp(char *, char *, ftnlen, ftnlen);
-extern void s_copy(char *, char *, ftnlen, ftnlen);
+extern int s_copy(char *, char *, ftnlen, ftnlen);
extern int s_paus(char *, ftnlen);
extern integer s_rdfe(cilist *);
extern integer s_rdue(cilist *);
diff -Naur a/F2CLIBS/libf2c/Makefile b/F2CLIBS/libf2c/Makefile
--- a/F2CLIBS/libf2c/Makefile 2010-04-27 21:09:35.000000000 +0200
+++ b/F2CLIBS/libf2c/Makefile 2021-04-08 15:33:03.414480609 +0200
@@ -20,14 +20,12 @@
# compile, then strip unnecessary symbols
.c.o:
$(CC) -c -DSkip_f2c_Undefs $(CFLAGS) $*.c
- ld -r -x -o $*.xxx $*.o
- mv $*.xxx $*.o
## Under Solaris (and other systems that do not understand ld -x),
## omit -x in the ld line above.
## If your system does not have the ld command, comment out
## or remove both the ld and mv lines above.
-MISC = f77vers.o i77vers.o main.o s_rnge.o abort_.o exit_.o getarg_.o iargc_.o\
+MISC = f77vers.o i77vers.o s_rnge.o abort_.o exit_.o getarg_.o iargc_.o\
getenv_.o signal_.o s_stop.o s_paus.o system_.o cabs.o ctype.o\
derf_.o derfc_.o erf_.o erfc_.o sig_die.o uninit.o
POW = pow_ci.o pow_dd.o pow_di.o pow_hh.o pow_ii.o pow_ri.o pow_zi.o pow_zz.o
@@ -72,8 +70,8 @@
all: f2c.h signal1.h sysdep1.h libf2c.a clapack_install
libf2c.a: $(OFILES)
- ar r libf2c.a $?
- -ranlib libf2c.a
+ $(ARCH) r libf2c.a $?
+ $(RANLIB) libf2c.a
## Shared-library variant: the following rule works on Linux
## systems. Details are system-dependent. Under Linux, -fPIC
@@ -119,7 +117,7 @@
install: libf2c.a
cp libf2c.a $(LIBDIR)
- -ranlib $(LIBDIR)/libf2c.a
+ $(RANLIB) $(LIBDIR)/libf2c.a
clapack_install: libf2c.a
mv libf2c.a ..
@@ -184,8 +182,8 @@
arith.h: arithchk.c
$(CC) $(CFLAGS) -DNO_FPINIT arithchk.c -lm ||\
$(CC) -DNO_LONG_LONG $(CFLAGS) -DNO_FPINIT arithchk.c -lm
- ./a.out >arith.h
- rm -f a.out arithchk.o
+ node a.out.js >arith.h
+ rm -f a.out.js a.out.wasm
check:
xsum Notice README abort_.c arithchk.c backspac.c c_abs.c c_cos.c \
diff -Naur a/INSTALL/tstiee.c b/INSTALL/tstiee.c
--- a/INSTALL/tstiee.c 2009-08-08 00:32:18.000000000 +0200
+++ b/INSTALL/tstiee.c 2021-03-25 19:58:44.754892352 +0100
@@ -10,6 +10,8 @@
http://www.netlib.org/f2c/libf2c.zip
*/
+#include <string.h>
+
#include "f2c.h"
#include "blaswrap.h"
diff -Naur a/Makefile b/Makefile
--- a/Makefile 2021-04-13 19:33:04.300487374 +0200
+++ b/Makefile 2021-04-13 19:33:04.320487362 +0200
@@ -15,9 +15,8 @@
clean: cleanlib cleantesting cleanblas_testing
-lapack_install:
- ( cd INSTALL; $(MAKE); ./testlsame; ./testslamch; \
- ./testdlamch; ./testsecond; ./testdsecnd; ./testversion )
+lapack_install: f2clib
+ ( cd INSTALL; $(MAKE) )
blaslib:
( cd BLAS/SRC; $(MAKE) )
diff -Naur a/SRC/Makefile b/SRC/Makefile
--- a/SRC/Makefile 2010-04-27 21:09:35.000000000 +0200
+++ b/SRC/Makefile 2021-04-08 15:43:54.616133123 +0200
@@ -50,9 +50,9 @@
#
#######################################################################
-ALLAUX = maxloc.o ilaenv.o ieeeck.o lsamen.o xerbla.o xerbla_array.o iparmq.o \
+ALLAUX = maxloc.o ilaenv.o ieeeck.o lsamen.o iparmq.o \
ilaprec.o ilatrans.o ilauplo.o iladiag.o chla_transtype.o \
- ../INSTALL/ilaver.o ../INSTALL/lsame.o
+ ../INSTALL/ilaver.o
ALLXAUX =
@@ -104,7 +104,7 @@
sggrqf.o sggsvd.o sggsvp.o sgtcon.o sgtrfs.o sgtsv.o \
sgtsvx.o sgttrf.o sgttrs.o sgtts2.o shgeqz.o \
shsein.o shseqr.o slabrd.o slacon.o slacn2.o \
- slaein.o slaexc.o slag2.o slags2.o slagtm.o slagv2.o slahqr.o \
+ slaein.o slaexc.o slag2.o slags2.o slagtm.o slagv2.o \
slahrd.o slahr2.o slaic1.o slaln2.o slals0.o slalsa.o slalsd.o \
slangb.o slange.o slangt.o slanhs.o slansb.o slansp.o \
slansy.o slantb.o slantp.o slantr.o slanv2.o \
@@ -176,7 +176,7 @@
clacgv.o clacon.o clacn2.o clacp2.o clacpy.o clacrm.o clacrt.o cladiv.o \
claed0.o claed7.o claed8.o \
claein.o claesy.o claev2.o clags2.o clagtm.o \
- clahef.o clahqr.o \
+ clahef.o \
clahrd.o clahr2.o claic1.o clals0.o clalsa.o clalsd.o clangb.o clange.o clangt.o \
clanhb.o clanhe.o \
clanhp.o clanhs.o clanht.o clansb.o clansp.o clansy.o clantb.o \
@@ -236,7 +236,7 @@
dggrqf.o dggsvd.o dggsvp.o dgtcon.o dgtrfs.o dgtsv.o \
dgtsvx.o dgttrf.o dgttrs.o dgtts2.o dhgeqz.o \
dhsein.o dhseqr.o dlabrd.o dlacon.o dlacn2.o \
- dlaein.o dlaexc.o dlag2.o dlags2.o dlagtm.o dlagv2.o dlahqr.o \
+ dlaein.o dlaexc.o dlag2.o dlags2.o dlagtm.o dlagv2.o \
dlahrd.o dlahr2.o dlaic1.o dlaln2.o dlals0.o dlalsa.o dlalsd.o \
dlangb.o dlange.o dlangt.o dlanhs.o dlansb.o dlansp.o \
dlansy.o dlantb.o dlantp.o dlantr.o dlanv2.o \
@@ -310,7 +310,7 @@
zlacgv.o zlacon.o zlacn2.o zlacp2.o zlacpy.o zlacrm.o zlacrt.o zladiv.o \
zlaed0.o zlaed7.o zlaed8.o \
zlaein.o zlaesy.o zlaev2.o zlags2.o zlagtm.o \
- zlahef.o zlahqr.o \
+ zlahef.o \
zlahrd.o zlahr2.o zlaic1.o zlals0.o zlalsa.o zlalsd.o zlangb.o zlange.o \
zlangt.o zlanhb.o \
zlanhe.o \It uses this Finally, the Makefile applies these commands: # Modify subroutines to return void instead of int
# as expected by KVXOPT and SuiteSparse.
sed -i 's|^/\* Subroutine \*/ int |/\* Subroutine \*/ void |' $(SRC)/BLAS/SRC/*.c
sed -i 's|^/\* Subroutine \*/ int |/\* Subroutine \*/ void |' $(SRC)/SRC/*.c
sed -i 's|^ extern /\* Subroutine \*/ int | extern /\* Subroutine \*/ void |' $(SRC)/BLAS/SRC/*.c
sed -i 's|^ extern /\* Subroutine \*/ int | extern /\* Subroutine \*/ void |' $(SRC)/SRC/*.c
sed -i 's|return 0;|return;|' $(SRC)/BLAS/SRC/*.c
sed -i 's|return 0;|return;|' $(SRC)/SRC/*.cThe KVXOPT package required some changes to the setup.py file: diff -Naur a/setup.py b/setup.py
--- a/setup.py 2020-11-01 20:06:27.000000000 +0100
+++ b/setup.py 2021-04-13 15:15:42.804671229 +0200
@@ -7,10 +7,10 @@
from distutils.file_util import copy_file
# Modifiy this if BLAS and LAPACK libraries are not in /usr/lib.
-BLAS_LIB_DIR = '/usr/lib'
+BLAS_LIB_DIR = '../../../F2CLAPACK/F2CLAPACK_WA'
# Default names of BLAS and LAPACK libraries
-BLAS_LIB = ['blas']
+BLAS_LIB = ['blas', 'f2c']
LAPACK_LIB = ['lapack']
BLAS_EXTRA_LINK_ARGS = []
@@ -82,7 +82,7 @@
FFTW_MACROS = []
# Directory containing SuiteSparse source
-SUITESPARSE_SRC_DIR = ''
+SUITESPARSE_SRC_DIR = '../../../SuiteSparse/SuiteSparse-WA'
# For SuiteSparse Versions before to 4.0.0 SuiteSparse_config does not exist
# We can avoid the search and link with this flagThe package:
name: kvxopt
version: 1.2.6.0
source:
sha256: b67430a7434ce2eee94da217059d6f2903ed1930a29370cc479b559699cd0ff0
url: https://files.pythonhosted.org/packages/52/61/e33d976e1954d18f8543a944ac6dc85899523739814aa01628f3a437ab06/kvxopt-1.2.6.0.tar.gz
patches:
- kvxopt.patch
requirements:
run:
- F2CLAPACK
- SuiteSparse
test:
imports:
- kvxopt
- kvxopt.amd
- kvxopt.base
- kvxopt.blas
- kvxopt.dense
- kvxopt.klu
- kvxopt.lapack
- kvxopt.sparse
- kvxopt.umfpackI found that diff -Naur a/dill/_dill.py b/dill/_dill.py
--- a/dill/_dill.py 2020-08-25 10:22:36.000000000 +0200
+++ b/dill/_dill.py 2021-04-13 14:24:02.101956995 +0200
@@ -207,13 +207,13 @@
FileType = get_file_type('rb', buffering=0)
TextWrapperType = get_file_type('r', buffering=-1)
-BufferedRandomType = get_file_type('r+b', buffering=-1)
+BufferedRandomType = get_file_type('r+b', buffering=0)
BufferedReaderType = get_file_type('rb', buffering=-1)
BufferedWriterType = get_file_type('wb', buffering=-1)
try:
from _pyio import open as _open
PyTextWrapperType = get_file_type('r', buffering=-1, open=_open)
- PyBufferedRandomType = get_file_type('r+b', buffering=-1, open=_open)
+ PyBufferedRandomType = get_file_type('r+b', buffering=0, open=_open)
PyBufferedReaderType = get_file_type('rb', buffering=-1, open=_open)
PyBufferedWriterType = get_file_type('wb', buffering=-1, open=_open)
except ImportError:The changes to Andes mostly just involved commenting out some optional imports: diff -Naur a/andes/core/model.py b/andes/core/model.py
--- a/andes/core/model.py 2021-03-21 16:00:35.000000000 +0100
+++ b/andes/core/model.py 2021-04-16 12:59:44.151276802 +0200
@@ -12,7 +12,6 @@
# Last modified: 8/16/20, 7:27 PM
import logging
-import scipy as sp
from collections import OrderedDict
from typing import Iterable, Sized, Callable, Union
diff -Naur a/andes/core/solver.py b/andes/core/solver.py
--- a/andes/core/solver.py 2021-03-21 16:00:35.000000000 +0100
+++ b/andes/core/solver.py 2021-04-15 10:19:44.034012059 +0200
@@ -4,8 +4,8 @@
import logging
-from scipy.sparse import csc_matrix
-from scipy.sparse.linalg import spsolve
+# from scipy.sparse import csc_matrix
+# from scipy.sparse.linalg import spsolve
from andes.shared import np, matrix, umfpack, klu, cupy
logger = logging.getLogger(__name__)
diff -Naur a/andes/main.py b/andes/main.py
--- a/andes/main.py 2021-03-21 16:00:35.000000000 +0100
+++ b/andes/main.py 2021-04-15 10:19:44.034012059 +0200
@@ -29,7 +29,7 @@
from andes.routines import routine_cli
from andes.utils.misc import elapsed, is_interactive
from andes.utils.paths import get_config_path, tests_root, get_log_dir
-from andes.shared import coloredlogs, unittest
+# from andes.shared import coloredlogs, unittest
from andes.shared import Pool, Process
logger = logging.getLogger(__name__)
diff -Naur a/andes/routines/eig.py b/andes/routines/eig.py
--- a/andes/routines/eig.py 2021-03-21 16:00:35.000000000 +0100
+++ b/andes/routines/eig.py 2021-04-15 10:19:44.034012059 +0200
@@ -3,7 +3,7 @@
"""
import logging
-import scipy.io
+# import scipy.io
import numpy as np
from math import ceil, pi
diff -Naur a/andes/shared.py b/andes/shared.py
--- a/andes/shared.py 2021-03-21 16:00:35.000000000 +0100
+++ b/andes/shared.py 2021-04-15 10:19:44.034012059 +0200
@@ -12,11 +12,11 @@
import math
import os
-import coloredlogs # NOQA
+# import coloredlogs # NOQA
import numpy as np # NOQA
from andes.utils.lazyimport import LazyImport
-from distutils.spawn import find_executable
+# from distutils.spawn import find_executable
# Library preference:
# KVXOPT + ipadd > CVXOPT + ipadd > KXVOPT > CVXOPT (+ KLU or not)
diff -Naur a/andes/utils/paths.py b/andes/utils/paths.py
--- a/andes/utils/paths.py 2021-03-21 16:00:35.000000000 +0100
+++ b/andes/utils/paths.py 2021-04-15 11:10:31.134064356 +0200
@@ -2,7 +2,7 @@
Utility functions for loading andes stock test cases
"""
import os
-import platform
+# import platform
import tempfile
import pathlib
import logging
@@ -219,13 +219,7 @@
str
The path to the temporary logging directory
"""
- path = ''
- if platform.system() in ('Linux', 'Darwin'):
- path = tempfile.mkdtemp(prefix='andes-')
-
- elif platform.system() == 'Windows':
- appdata = os.getenv('APPDATA')
- path = os.path.join(appdata, 'andes')
+ path = tempfile.mkdtemp(prefix='andes-')
if not os.path.exists(path):
os.makedirs(path)
diff -Naur a/setup.py b/setup.py
--- a/setup.py 2021-03-21 16:00:35.000000000 +0100
+++ b/setup.py 2021-04-15 10:19:44.034012059 +0200
@@ -56,7 +56,7 @@
# 'path/to/data_file',
]
},
- install_requires=requirements,
+ # install_requires=requirements,
license="GNU Public License v3",
classifiers=[
'Development Status :: 4 - Beta',It would be interesting to see how FORTRAN to WASM compilation is coming along. Getting ARPACK working and performing SSSA in the browser would be a first! |
|
Thank you very much for sharing the code. It still looks challenging, but let me try to build it first. |
|
The code I wrote was mixed into another repository. I separated it out and pushed it here: https://github.com/rwl/andesapp I haven't tried it, but there it a Dockerfile that should build. |
|
Hey @rwl and @cuihantao , I saw this interesting, and I'm always trying to improve KVXOPT. I will follow this if you need modifications in KVXOPT for better integration with this Pyodide interpreter. |
|
Hello @sanurielf and @rwl , Sorry that I didn't follow up until now because I didn't even make the docker image to build. A lot internals have changed in Pyodide, and the documentations was being improved. I was later dragged away by some other works... I perhaps work on it again when the semester ends. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Andes has a complex set of dependencies that potential users might not be willing to install locally. PyIoidide is an exciting new project that could allow Andes to be run within the browser using WebAssembly. Many of the packages that Andes requires have already been ported to PyIodide. If CVXOPT and SymPy were ported it might be possible to create an Andes package:
https://pyodide.readthedocs.io/en/latest/new_packages.html
This would allow users to create and share notebooks online. Andes running in WebAssembly would open the possibility to create a custom GUI such as that made with MATPOWER:
https://matpower.app/
This could, potentially, also be built into an Electron-based desktop application and made available to download by users wishing to work offline or on air-gaped machines.
The text was updated successfully, but these errors were encountered: