Missing log_prob method or endpoint

[riddell-stan]

Given a model_name and parameter values, call the log_prob function in the C++ model. Implementation can mirror that of the param_names endpoint.

When this is implemented, add a method to the PyStan 3 Model class.

(I think I have used this function one time in the last 4 years.)

[bob-carpenter]

I think this is most useful for testing and for algorithm development. I'm not sure anyone's doing algorithm development based on Stan models in Python or in R. It's just too easy to write a couple models with gradients from scratch and develop using those without bringing in the relatively heavyweight dependencies of PyStan or RStan.

[ahartikainen]

I know people using these for custom stuff.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[mjcarter95]

I'm working on a project that (due to licensing requirements) motivates the use of PyStan v3 and requires access to log_prob. Happy to help resolve this issue and expose log_prob to HTTPStan and PyStan v3.

[riddell-stan]

To make log_prob available you need to write a view which is very similar to handle_show_params

httpstan/httpstan/views.py

Line 200 in c6b5239

async def handle_show_params(request: aiohttp.web.Request) -> aiohttp.web.Response:

. This calls a function defined in C++ called param_names,

httpstan/httpstan/stan_services.cpp

Line 83 in c6b5239

std::vector<std::string> param_names(py::dict data) {

.

A lot of the work here is writing documentation and tests.

[mjcarter95]

Thank you for the guidance. I'm also keen to make log_prob_grad and write_array available - should I raise as a separate issue?

[riddell-stan]

Let's start with log_prob. Other methods should get their own issues.

[mjcarter95]

Edit looks like the issue below was something to do with my Ubuntu system, tried on my Mac and the tests run fine.

I've exposed log_prob in HTTPStan and tested by hand. I am now writing a test, but I am met with the error below for any test that relies on get_model_name, any ideas what might cause this?

platform linux -- Python 3.7.6, pytest-6.1.2, py-1.9.0, pluggy-0.13.1
rootdir: /home/sgmcart3/Code/stan/httpstan/tests
plugins: asyncio-0.14.0
collected 1 item                                                                                                                                                               

test_linear_regression.py F

=================================================================================== FAILURES ===================================================================================
____________________________________________________________________________ test_linear_regression ____________________________________________________________________________

api_url = 'http://127.0.0.1:8080/v1'

    @pytest.mark.asyncio
    async def test_linear_regression(api_url: str) -> None:
        """Test sampling from linear regression posterior with defaults."""
    
        payload = {
            "function": "stan::services::sample::hmc_nuts_diag_e_adapt",
            "data": data,
            "num_samples": 500,
            "num_warmup": 500,
            "random_seed": 1,
        }
>       beta_0 = await helpers.sample_then_extract(api_url, program_code, payload, "beta.1")

test_linear_regression.py:46: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
helpers.py:146: in sample_then_extract
    operation = await sample(api_url, program_code, fit_payload)
helpers.py:84: in sample
    model_name = await get_model_name(api_url, program_code)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

api_url = 'http://127.0.0.1:8080/v1'
program_code = '\n    data {\n        int<lower=0> N;\n        int<lower=0> p;\n        matrix[N,p] x;\n        vector[N] y;\n    }\n...      vector[p] beta;\n        real<lower=0> sigma;\n    }\n    model {\n        y ~ normal(x * beta, sigma);\n    }\n'

    async def get_model_name(api_url: str, program_code: str) -> str:
        """Compile and retrieve model name.
    
        This function is a coroutine.
        """
        models_url = f"{api_url}/models"
        payload = {"program_code": program_code}
        async with aiohttp.ClientSession() as session:
            async with session.post(models_url, json=payload) as resp:
>               assert resp.status == 201
E               AssertionError

helpers.py:20: AssertionError
------------------------------------------------------------------------------ Captured log call -------------------------------------------------------------------------------
CRITICAL httpstan:views.py:119 Exception while building model extension module: `LinkError(DistutilsExecError("command 'g++' failed with exit status 1"))`, traceback: `['  File "/home/sgmcart3/Code/stan/httpstan-development/lib/python3.7/site-packages/httpstan/views.py", line 113, in handle_create_model\n    compiler_output = await httpstan.models.build_services_extension_module(program_code)\n', '  File "/home/sgmcart3/Code/stan/httpstan-development/lib/python3.7/site-packages/httpstan/models.py", line 176, in build_services_extension_module\n    None, httpstan.build_ext.run_build_ext, extensions, build_lib\n', '  File "/home/sgmcart3/anaconda3/lib/python3.7/concurrent/futures/thread.py", line 57, in run\n    result = self.fn(*self.args, **self.kwargs)\n', '  File "/home/sgmcart3/Code/stan/httpstan-development/lib/python3.7/site-packages/httpstan/build_ext.py", line 80, in run_build_ext\n    build_extension.run()\n', '  File "/home/sgmcart3/anaconda3/lib/python3.7/distutils/command/build_ext.py", line 340, in run\n    self.build_extensions()\n', '  File "/home/sgmcart3/anaconda3/lib/python3.7/distutils/command/build_ext.py", line 449, in build_extensions\n    self._build_extensions_serial()\n', '  File "/home/sgmcart3/anaconda3/lib/python3.7/distutils/command/build_ext.py", line 474, in _build_extensions_serial\n    self.build_extension(ext)\n', '  File "/home/sgmcart3/anaconda3/lib/python3.7/distutils/command/build_ext.py", line 559, in build_extension\n    target_lang=language)\n', '  File "/home/sgmcart3/anaconda3/lib/python3.7/distutils/ccompiler.py", line 717, in link_shared_object\n    extra_preargs, extra_postargs, build_temp, target_lang)\n', '  File "/home/sgmcart3/anaconda3/lib/python3.7/distutils/unixccompiler.py", line 207, in link\n    raise LinkError(msg)\n']`
=============================================================================== warnings summary ===============================================================================
../../httpstan-development/lib/python3.7/site-packages/setuptools/depends.py:2
  /home/sgmcart3/Code/stan/httpstan-development/lib/python3.7/site-packages/setuptools/depends.py:2: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
    import imp

-- Docs: https://docs.pytest.org/en/stable/warnings.html
=========================================================================== short test summary info ============================================================================
FAILED test_linear_regression.py::test_linear_regression - AssertionError
======================================================================== 1 failed, 1 warning in 25.22s =========================================================================```


Ubuntu 18.04.4 LTS
Python 3.7.6 (default, Jan  8 2020, 19:59:22) 
gcc version 7.5.0 (Ubuntu 7.5.0-3ubuntu1~18.04)