-
-
Notifications
You must be signed in to change notification settings - Fork 614
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Removing data copy in the RAMDirectory
The fst crate recently added support for sliced `Arc<Vec<u8>>`. This called for a rewrite of the RAMDirectory for tantivy's RAMDirectory. Previously every single read was copying data. In addition: - RAMDirectory's Write object panic if someone does not flush right before the destruction of the object. - In the same spirit, the postings serializer panics if someone opens a term without closing the previous one. Closes #16
- Loading branch information
1 parent
cecc9f9
commit 0972a1c
Showing
12 changed files
with
212 additions
and
71 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,36 @@ | ||
use std::sync::Arc; | ||
|
||
#[derive(Clone)] | ||
pub struct SharedVecSlice { | ||
pub data: Arc<Vec<u8>>, | ||
pub start: usize, | ||
pub len: usize | ||
} | ||
|
||
impl SharedVecSlice { | ||
|
||
pub fn empty() -> SharedVecSlice { | ||
SharedVecSlice::new(Arc::new(Vec::new())) | ||
} | ||
|
||
pub fn new(data: Arc<Vec<u8>>) -> SharedVecSlice { | ||
let data_len = data.len(); | ||
SharedVecSlice { | ||
data: data, | ||
start: 0, | ||
len: data_len, | ||
} | ||
} | ||
|
||
pub fn as_slice(&self,) -> &[u8] { | ||
&self.data[self.start..self.start + self.len] | ||
} | ||
|
||
pub fn slice(&self, from_offset: usize, to_offset:usize) -> SharedVecSlice { | ||
SharedVecSlice { | ||
data: self.data.clone(), | ||
start: self.start + from_offset, | ||
len: to_offset - from_offset, | ||
} | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.