Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

It seems that the current version of SPTAG only supports np.float32. #11

Open
puluwen opened this issue Apr 18, 2019 · 3 comments
Open

It seems that the current version of SPTAG only supports np.float32. #11

puluwen opened this issue Apr 18, 2019 · 3 comments
Labels
new feature
Milestone
Backlog

Comments

@puluwen
Copy link

puluwen commented Apr 18, 2019

When i use the np.float64, I turned the SPTAG.Annindex() datatype parameter from 'Float' (used in the test demo) to 'Double', 'Float64','float64', 'double'， but all failed.
@MaggieQi
Copy link
Member

We currently only support Int8, Int16 and Float (float32). Float64 is not supported. We will add support for more data types soon : )

@puluwen
Copy link
Author

puluwen commented May 13, 2019

I suggest writing this information into readme.

@puluwen
Copy link
Author

puluwen commented May 13, 2019

I also found another problem that sptag could not accept data that occupied more than 4 GB of memory.

@MaggieQi MaggieQi added this to the Backlog milestone Jul 10, 2019
PhilipBAdams added a commit that referenced this issue Jun 9, 2022
@suiguoxin @PhilipBAdams
support data compression & delta-encoding of posting lists (#297)
f0579d4 
* MVP version of Data compress (#1)

* Support delta-encoding (#2)

* support config of compress level (#3)

* integrate zstd with cmake (#4)

* Bug Fix: wrong listPageCount when listTotalBytes % pageSize==0 (#5)

* train & share dictionary (#6)

* rearrange posting list (#7)

* config minDictTraingBufferSize and dictBufferCapacity (#8)

* cmake with local installed zstd (#9)

* refine visual studio config (#10)

* parallel for get compressed size (#11)

* fix check truth bug (#12)

* change zstd branch (#13)

* remove verbose log in truth analysis; refine dockerfile

* check rvalue in Compressor.h

* resolve back compatibility by add a search option: WithDataCompressionFeatures

* remove redundant configs, change head info format only when compression enabled

* reuse buffer when decompression

* remove exit(1) from search index

* bug fix: reuse compression buffer in workspace

 # with '#' will be ignored, and an empty message aborts the commit.

* code refine

* optimize mem usage

* replace vcpkg zstd with nuget

* config issue

* fix solution build issue

Co-authored-by: Philip Adams <35666630+PhilipBAdams@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
new feature
Projects
None yet
Milestone
Backlog
Development

No branches or pull requests
2 participants
@MaggieQi @puluwen