[Bug] runner.js not bundled in fresh install on latest RC tags

[seanbyrne88]

What happened?

Working off a fresh install on the latest RC, runner.js is not bundled and errors out when running Program of Thought.

Steps to reproduce

Fresh install and run any program of thought application.

DSPy version

2.6.0rc8

[okhat]

Thanks @seanbyrne88 ! Tagging @arnavsinghvi11

[arnavsinghvi11]

Hi @seanbyrne88 , thanks for sharing!

Could you share the stack trace you're seeing and the minimal example using PoT you're running? (also, was this working for you before on any past versions?)
I just tried a fresh install with dspy 2.6.0rc8 on a new env and it worked from my end, but I'm wondering if maybe deno is not configured correctly for you so happy to investigate further.

[seanbyrne88]

Thanks will put something together tonight.

I pip installed from the release sha and it looked like PoT expected runner to be in the primitives directory. Confirmed that deno works and I dropped runner.js in there manually and it did work after that. Will put together an example though.

Installed using pip and pyenv.

[seanbyrne88]

Used example from here on 2.6.0

#Example Usage
#Define a simple signature for basic question answering
generate_answer_signature = dspy.Signature("question -> answer")
generate_answer_signature.attach(question=("Question:", ""))

# Pass signature to ProgramOfThought Module
pot = dspy.ProgramOfThought(generate_answer_signature)


question = 'Sarah has 5 apples. She buys 7 more apples from the store. How many apples does Sarah have now?'
result = pot(question=question)

print(f"Question: {question}")
print(f"Final Predicted Answer (after ProgramOfThought process): {result.answer}")

When I execute this I get:

Error in code execution

When I step through using a debugger I see in program_of_thought.py
The error variable on looping has the value:
No output from Deno subprocess. Stderr: Module not found: file:///envs/dspy-repro/lib/python3.12/site-packages/dspy/primitives/runner.js

---

To verify deno I run

deno --version
deno.2.1.7 (stable, release, aarch64-apple-darwin)
v8 13.0.245.12-rusty
typescript 5.6.2

[seanbyrne88]

When I used poetry like in the CI process it adds runner.js.

Are you able to repro with a pip install from pypi now that the latest release is there?

EDIT: Grabbed the whl and source distributions on pypi and runner.js is not there either.

[okhat]

Thanks @seanbyrne88 ! We're looking into this

[arnavsinghvi11]

Thanks @seanbyrne88 !

I still wasn't able to reproduce this error on a fresh env with the latest pypi release, but we can try a couple things here:

-we have an updated PoT tutorial that's in an open PR that you can use instead of the one currently on the page - will be merged soon! (likely won't resolve your error but just to keep your testing updated)
-could you check your deno path and ensure its aligned with the runner.js and dspy repo paths?
-we can also try manually accounting for .js files in the repo installation. Can you add this to your local setup.py and run pip install -e .

setup(
      ...
      include_package_data=True,
      package_data={
          "dspy": ["primitives/*.js"],
      ...
)

If this resolves it, I can push out a PR for this to have the pypi installation updated.

Feel free to share your OS and any additional env details that can help with debugging if none of these work.

[hung-phan]

I ran into the same issue with this. When install with poetry add dspy@2.6.2, i don't see it bundled runner.js

[arnavsinghvi11]

Hi @seanbyrne88 @hung-phan , we've migrated from using setup.py and now runner.js is bundled in the latest dspy-ai version. Feel free to try it out and let me know if it works!

[tevino]

@arnavsinghvi11 The release 2.6.6 from PyPI is still broken, there's no primitives/runner.js in the tarball.

SHA256 8fb08bb8009c51d3f972856c19620cab2cc5e2b5a4c379005441bd81267ce905