Cannot build index with dense representations

[felicitywang1]

Hi,
I followed the instructions here (with some modifications to run locally on mac cpu) and got the following NoneType error. Any suggestions how I should fix it?

Thank you.

python -m pyserini.encode \
  input   --corpus tests/resources/simple_cacm_corpus.jsonl \
          --fields text \
  output  --embeddings indexes/tests/simple_cacm_corpus \
          --to-faiss \
  encoder --encoder castorini/tct_colbert-v2-hnp-msmarco \
          --fields text

Traceback (most recent call last):
File "/Users/f.wang1/miniconda3/envs/transformers/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/Users/f.wang1/miniconda3/envs/transformers/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/Users/f.wang1/miniconda3/envs/transformers/lib/python3.8/site-packages/pyserini/encode/main.py", line 118, in
encoder = init_encoder(args.encoder.encoder, args.encoder.encoder_class, device=args.encoder.device)
AttributeError: 'NoneType' object has no attribute 'encoder'
zsh: command not found: output

[lintool]

This test case might be helpful? https://github.com/castorini/pyserini/blob/master/tests/test_encoder.py#L85

[felicitywang1]

Thanks. With the command in there on cpu and some other tweaks I managed to get it running :)

[lintool]

hi @felicitywang1 can you share what the command was so the next person doesn't need to figure it out from scratch again? ;)

[felicitywang1]

It's exactly the same as in https://github.com/castorini/pyserini/blob/master/tests/test_encoder.py#L85 ...
If one's working with a cpu as I am

[lintool]

Gotcha, thanks!

Glad the test cases are doing their jobs...