Skip to content

Commit

Permalink
Fix/clarify Python/Java versions (#1957)
Browse files Browse the repository at this point in the history
  • Loading branch information
lintool committed Aug 8, 2024
1 parent f51a2d8 commit 1bbf7a7
Show file tree
Hide file tree
Showing 3 changed files with 8 additions and 7 deletions.
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,8 @@ Correspondingly, Pyserini was upgraded to JDK 21 at commit [`b2f677`](https://gi

## 🎬 Installation

Install via PyPI (requires Python 3.10+):
Pyserini is built on Python 3.10 (other versions might work, but YMMV).
Install via PyPI:

```
pip install pyserini
Expand Down
10 changes: 5 additions & 5 deletions docs/installation.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
# Pyserini: Detailed Installation Guide

Pyserini is built on Python 3.10.
Pyserini is built on Python 3.10 (other versions might work, but YMMV).
At a high level, we try to keep our [`requirements.txt`](../requirements.txt) up to date.
Pyserini has a number of important dependencies:

+ For sparse retrieval, Pyserini depends on [Anserini](http://anserini.io/), which is built on Lucene.
[PyJNIus](https://github.com/kivy/pyjnius) is used to interact with the JVM.
[PyJNIus](https://github.com/kivy/pyjnius) is used to interact with the JVM. We depend on Java 21.
+ For dense retrieval (since it involves neural networks), we need the [🤗 Transformers library](https://github.com/huggingface/transformers), [PyTorch](https://pytorch.org/), and [Faiss](https://github.com/facebookresearch/faiss) (specifically `faiss-cpu`).
A `pip` installation will automatically pull in the first to satisfy the package requirements, but since the other two may require platform-specific custom configuration, they are _not_ explicitly listed in the package requirements.
We leave the installation of these packages to you (but provide detailed instructions below).
Expand Down Expand Up @@ -42,7 +42,7 @@ conda install -c conda-forge openjdk=21 maven -y
conda install -c conda-forge lightgbm nmslib -y

# from https://github.com/facebookresearch/faiss/blob/main/INSTALL.md
# NOTE: due to a bug in the latest 1.7.4 release, Intel MKL 2021 needs to be installed separately where applicable.
# NOTE: due to a bug in the latest 1.7.4 release (on osx-64), Intel MKL 2021 needs to be installed separately where applicable.
conda install -c pytorch faiss-cpu=1.7.4 mkl=2021 blas=1.0=mkl -y
conda install -c pytorch pytorch -y

Expand All @@ -60,7 +60,7 @@ conda install -c pytorch faiss-cpu pytorch -y
pip install pyserini
```

As of April 2024, for `faiss-cpu`, `osx-64` is still at v1.7.4, whereas `osx-arm64` is at v1.8.0; hence the differences in the instructions above.
As of August 2024, for `faiss-cpu`, `osx-64` is still at v1.7.4, whereas `osx-arm64` is at v1.8.0; hence the differences in the instructions above.

### Linux

Expand Down Expand Up @@ -161,7 +161,7 @@ Assuming all tests pass, you should be ready to go!

## Troubleshooting Tips

+ The above guide handle JVM installation via conda. If you are using your own Java environment and get an error about Java version mismatch, it's likely an issue with your `JAVA_HOME` environmental variable.
+ The above guide handles JVM installation via conda. If you are using your own Java environment and get an error about Java version mismatch, it's likely an issue with your `JAVA_HOME` environmental variable.
In `bash`, use `echo $JAVA_HOME` to find out what the environmental variable is currently set to, and use `export JAVA_HOME=/path/to/java/home` to change it to the correct path.
On a Linux system, the correct path might look something like `/usr/lib/jvm/java-21`.
Unfortunately, we are unable to offer more concrete advice since the actual path depends on your OS, which JDK you're using, and a host of other factors.
Expand Down
2 changes: 1 addition & 1 deletion project-description.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Install via PyPI:
pip install pyserini
```

Pyserini requires Python 3.10+ and Java 11 (due to its dependency on [Anserini](http://anserini.io/)).
Pyserini requires Python 3.10 and Java 21 (due to its dependency on [Anserini](http://anserini.io/)).

Since dense retrieval depends on neural networks, Pyserini requires a more complex set of dependencies to use this feature.
A `pip` installation will automatically pull in the [🤗 Transformers library](https://github.com/huggingface/transformers) to satisfy the package requirements.
Expand Down

0 comments on commit 1bbf7a7

Please sign in to comment.