Skip to content

Commit

Permalink
fix std type
Browse files Browse the repository at this point in the history
  • Loading branch information
kungjim committed Aug 16, 2023
1 parent f64fa88 commit be85236
Show file tree
Hide file tree
Showing 6 changed files with 19 additions and 19 deletions.
4 changes: 2 additions & 2 deletions docs/docs/modules/Embedding.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# Embedding

The Embedding module uses Rocksdb to store the values of Embedding, which is KV format. The Key of feature is u_int64_t type, the value is a list of floating point numbers and some other values.
The Embedding module uses Rocksdb to store the values of Embedding, which is KV format. The Key of feature is uint64_t type, the value is a list of floating point numbers and some other values.

## Key and Group

All features are discretization and represented by the unique u_int64_t value. We use group to represent the same type of features.Different group can have different optimizer, initializer and dimension.
All features are discretization and represented by the unique uint64_t value. We use group to represent the same type of features.Different group can have different optimizer, initializer and dimension.

## Value

Expand Down
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# Embedding

The Embedding module uses Rocksdb to store the values of Embedding, which is KV format. The Key of feature is u_int64_t type, the value is a list of floating point numbers and some other values.
The Embedding module uses Rocksdb to store the values of Embedding, which is KV format. The Key of feature is uint64_t type, the value is a list of floating point numbers and some other values.

## Key and Group

All features are discretization and represented by the unique u_int64_t value. We use group to represent the same type of features.Different group can have different optimizer, initializer and dimension.
All features are discretization and represented by the unique uint64_t value. We use group to represent the same type of features.Different group can have different optimizer, initializer and dimension.

## Value

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ First part stores the dim and count of each group, totally 256 groups. Second pa

| type | size | length | description |
| --------- | ---- | ----------------- | ----------------- |
| u_int64_t | 8bit | 1 | key value |
| uint64_t | 8bit | 1 | key value |
| int32 | 4bit | 1 | group of the key |
| float | 4bit | dim of this group | weight of the key |

Expand Down
6 changes: 3 additions & 3 deletions include/common.h
Original file line number Diff line number Diff line change
Expand Up @@ -35,12 +35,12 @@
#define Float float
const Float Epsilon = 1e-8f;

#ifndef u_int64_t
#define u_int64_t unsigned long long
#ifndef uint64_t
//#define uint64_t unsigned long long
#endif

#ifndef int64_t
#define int64_t long long
//#define int64_t long long
#endif

int64_t get_current_time();
Expand Down
10 changes: 5 additions & 5 deletions include/counting_bloom_filter.h
Original file line number Diff line number Diff line change
Expand Up @@ -43,9 +43,9 @@ using Counter = struct Counter;

const double FPR = 0.001;
const int max_count = 15;
const u_int64_t min_size = 268435456ull; // 2^28
const u_int64_t high_mask = 8589934591ull; // 2^33-1
const u_int64_t low_mask = 2147483647ull; // 2^31-1
const uint64_t min_size = 268435456ull; // 2^28
const uint64_t high_mask = 8589934591ull; // 2^33-1
const uint64_t low_mask = 2147483647ull; // 2^31-1

class CountingBloomFilter final {
private:
Expand Down Expand Up @@ -86,8 +86,8 @@ class CountingBloomFilter final {
void add(const Key &key, const int64_t &num = 1);
int get_count() const;
};
u_int64_t hash_func(const int64_t &x);
u_int64_t hash_func(const Key &x);
uint64_t hash_func(const int64_t &x);
uint64_t hash_func(const Key &x);
void create_empty_file(const std::string &filename, const size_t &size);

#endif // DAMO_EMBEDDING_COUNTING_BLOOM_FILTER_H
12 changes: 6 additions & 6 deletions src/counting_bloom_filter.cpp
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
#include "counting_bloom_filter.h"

u_int64_t hash_func(const int64_t &x) {
uint64_t hash_func(const int64_t &x) {
return ((x >> 31) & high_mask) | ((x & low_mask) << 33);
}

u_int64_t hash_func(const Key &x) {
u_int64_t hash = x.group;
uint64_t hash_func(const Key &x) {
uint64_t hash = x.group;
return hash_func(x.key) | (hash << 15);
}

Expand All @@ -23,7 +23,7 @@ CountingBloomFilter::CountingBloomFilter()

CountingBloomFilter::CountingBloomFilter(const Params &config)
: CountingBloomFilter(
config.get<u_int64_t>("capacity", min_size),
config.get<uint64_t>("capacity", min_size),
config.get<int>("count", max_count),
config.get<std::string>("path", "/tmp/COUNTING_BLOOM_FILTER_DATA"),
config.get<bool>("reload", true), config.get<double>("fpr", FPR)) {}
Expand Down Expand Up @@ -83,7 +83,7 @@ CountingBloomFilter::CountingBloomFilter(size_t capacity, int count,

bool CountingBloomFilter::check(const Key &key) {
int min_count = max_count;
u_int64_t hash = hash_func(key);
uint64_t hash = hash_func(key);

for (int i = 0; i < this->k_; i++) {
auto idx = hash % this->counter_num_;
Expand All @@ -104,7 +104,7 @@ bool CountingBloomFilter::check(const Key &key) {
}

void CountingBloomFilter::add(const Key &key, const int64_t &num) {
u_int64_t hash = hash_func(key);
uint64_t hash = hash_func(key);
for (int i = 0; i < this->k_; i++) {
auto idx = hash % this->counter_num_;
if (idx & 1) {
Expand Down

0 comments on commit be85236

Please sign in to comment.