New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
g595.Manucharyan.task3 #279
Merged
Merged
Changes from all commits
Commits
Show all changes
28 commits
Select commit
Hold shift + click to select a range
0866a60
Merge remote-tracking branch 'refs/remotes/fediq/master'
vanderwardan 552d60a
Merge remote-tracking branch 'refs/remotes/fediq/master'
vanderwardan 3b50057
Add third task
vanderwardan da7781b
Merge remote-tracking branch 'refs/remotes/fediq/master'
vanderwardan 10c6e3d
Third task
vanderwardan 2f58fa2
Codestyle changes
vanderwardan a9c4c95
add synchronised
vanderwardan a9f1fc8
Review changes
vanderwardan f530f07
code style changes
vanderwardan 83ecff4
try do smth, that travis test my project
vanderwardan 6446586
tried again
vanderwardan 181536f
add cleaning storage not only in close()
vanderwardan af9170b
code style changes
vanderwardan a9a52c0
revert
vanderwardan f95b2d3
some bugs fixed
vanderwardan a85598c
try find what's wrong with travis
vanderwardan 0ccfc8a
code style
vanderwardan e81d659
and here we go
vanderwardan 81e49f0
optimise little bit
vanderwardan 12319c8
oops
vanderwardan b1b8e2b
Merge remote-tracking branch 'refs/remotes/fediq/master'
vanderwardan e328d85
try to pass new tests
vanderwardan 8722b58
fix problems with seek
vanderwardan 89423e5
fix problems with files
vanderwardan 6c53bd8
code style changes
vanderwardan eacb2e4
fix constructor
vanderwardan edcfe73
back to the right version of lector's file
vanderwardan 47dde1d
fucking IDEA auto fixes
vanderwardan File filter
Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
21 changes: 21 additions & 0 deletions
21
.../ru/mipt/java2016/homework/g595/manucharyan/task3/ConcreteStrategyDoubleRandomAccess.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
package ru.mipt.java2016.homework.g595.manucharyan.task3; | ||
|
||
import java.io.DataInput; | ||
import java.io.DataOutput; | ||
import java.io.IOException; | ||
|
||
/** | ||
* @author Vardan Manucharyan | ||
* @since 30.10.16 | ||
*/ | ||
public class ConcreteStrategyDoubleRandomAccess implements SerializationStrategyRandomAccess<Double> { | ||
@Override | ||
public void serializeToFile(Double value, DataOutput output) throws IOException { | ||
output.writeDouble(value); | ||
} | ||
|
||
@Override | ||
public Double deserializeFromFile(DataInput input) throws IOException { | ||
return input.readDouble(); | ||
} | ||
} |
21 changes: 21 additions & 0 deletions
21
...ru/mipt/java2016/homework/g595/manucharyan/task3/ConcreteStrategyIntegerRandomAccess.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
package ru.mipt.java2016.homework.g595.manucharyan.task3; | ||
|
||
import java.io.DataInput; | ||
import java.io.DataOutput; | ||
import java.io.IOException; | ||
|
||
/** | ||
* @author Vardan Manucharyan | ||
* @since 30.10.16 | ||
*/ | ||
public class ConcreteStrategyIntegerRandomAccess implements SerializationStrategyRandomAccess<Integer> { | ||
@Override | ||
public void serializeToFile(Integer value, DataOutput output) throws IOException { | ||
output.writeInt(value); | ||
} | ||
|
||
@Override | ||
public Integer deserializeFromFile(DataInput input) throws IOException { | ||
return input.readInt(); | ||
} | ||
} |
22 changes: 22 additions & 0 deletions
22
.../ru/mipt/java2016/homework/g595/manucharyan/task3/ConcreteStrategyStringRandomAccess.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,22 @@ | ||
package ru.mipt.java2016.homework.g595.manucharyan.task3; | ||
|
||
import java.io.DataInput; | ||
import java.io.DataOutput; | ||
import java.io.IOException; | ||
|
||
/** | ||
* @author Vardan Manucharyan | ||
* @since 30.10.16 | ||
*/ | ||
public class ConcreteStrategyStringRandomAccess implements SerializationStrategyRandomAccess<String> { | ||
@Override | ||
public void serializeToFile(String value, DataOutput output) throws IOException { | ||
output.writeUTF(value); | ||
} | ||
|
||
@Override | ||
public String deserializeFromFile(DataInput input) throws IOException { | ||
return input.readUTF(); | ||
} | ||
|
||
} |
287 changes: 287 additions & 0 deletions
287
.../main/java/ru/mipt/java2016/homework/g595/manucharyan/task3/OptimisedKeyValueStorage.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,287 @@ | ||
package ru.mipt.java2016.homework.g595.manucharyan.task3; | ||
|
||
import java.io.BufferedOutputStream; | ||
import java.io.DataOutputStream; | ||
import java.io.File; | ||
import java.io.FileOutputStream; | ||
import java.io.IOException; | ||
import java.io.RandomAccessFile; | ||
import java.util.HashMap; | ||
import java.util.Iterator; | ||
import java.util.Map; | ||
|
||
import ru.mipt.java2016.homework.base.task2.KeyValueStorage; | ||
|
||
/** | ||
* @author Vardan Manucharyan | ||
* @since 20.11.2016. | ||
*/ | ||
public class OptimisedKeyValueStorage<K, V> implements KeyValueStorage<K, V> { | ||
|
||
private static final long MAX_CACHE_SIZE = 100L; | ||
private static final double FILLING_PERCENTAGE = 0.5; //when we need clean storage from deleted values | ||
private static long deletedCount; //number of deleted elements | ||
|
||
private final SerializationStrategyRandomAccess<K> keySerializationStrategy; | ||
private final SerializationStrategyRandomAccess<V> valueSerializationStrategy; | ||
|
||
//consist keys and offsets(the pair of begin and length) | ||
private Map<K, Long> base = new HashMap<>(); | ||
private Map<K, V> cache = new HashMap<>(); | ||
|
||
private long maxOffset; | ||
private final String pathname; | ||
private final String storageName = "storage.txt"; | ||
private RandomAccessFile mapStorage; | ||
private RandomAccessFile storage; | ||
private final String mapStorageName = "mapStorage.txt"; | ||
|
||
private File mutexFile; // для многопоточности | ||
private boolean isClosed; | ||
|
||
public OptimisedKeyValueStorage(SerializationStrategyRandomAccess<K> keySerializationStrategy, | ||
SerializationStrategyRandomAccess<V> valueSerializaionStrategy, | ||
String path) throws IOException { | ||
|
||
this.keySerializationStrategy = keySerializationStrategy; | ||
this.valueSerializationStrategy = valueSerializaionStrategy; | ||
maxOffset = 0L; | ||
deletedCount = 0L; | ||
pathname = path; | ||
isClosed = false; | ||
|
||
mutexFile = new File(pathname, "Mutex"); | ||
if (!mutexFile.createNewFile()) { | ||
throw new RuntimeException("Can't synchronize!"); | ||
} | ||
|
||
File directory = new File(pathname); | ||
if (!directory.isDirectory()) { | ||
throw new RuntimeException("wrong path"); | ||
} | ||
|
||
try { | ||
File file = new File(path, mapStorageName); | ||
boolean exist = file.exists(); | ||
|
||
mapStorage = new RandomAccessFile(file, "rw"); | ||
if (exist) { | ||
downloadDataFromStorage(); | ||
} | ||
storage = new RandomAccessFile(new File(pathname, storageName), "rw"); | ||
|
||
} catch (IOException exception) { | ||
throw new RuntimeException("Can't create a storage!"); | ||
} | ||
} | ||
|
||
/** | ||
* Возвращает значение для данного ключа, если оно есть в хранилище. | ||
* Иначе возвращает null. | ||
*/ | ||
@Override | ||
public synchronized V read(K key) { | ||
isClose(); | ||
|
||
if (!exists(key)) { | ||
return null; | ||
} else { | ||
try { | ||
if (cache.get(key) != null) { | ||
return cache.get(key); | ||
} | ||
|
||
Long offset = base.get(key); | ||
storage.seek(offset); | ||
V res = valueSerializationStrategy.deserializeFromFile(storage); | ||
return res; | ||
} catch (Exception exception) { | ||
throw new RuntimeException("Can't read from storage"); | ||
} | ||
} | ||
} | ||
|
||
/** | ||
* Возвращает true, если данный ключ есть в хранилище | ||
*/ | ||
@Override | ||
public boolean exists(K key) { | ||
isClose(); | ||
|
||
if (cache.containsKey(key)) { | ||
return true; | ||
} | ||
return base.containsKey(key); | ||
} | ||
|
||
/** | ||
* Записывает в хранилище пару ключ-значение. | ||
*/ | ||
@Override | ||
public synchronized void write(K key, V value) { | ||
isClose(); | ||
|
||
cache.put(key, value); | ||
base.put(key, 0L); | ||
|
||
if (cache.size() > MAX_CACHE_SIZE) { | ||
writeCacheToStorage(); | ||
} | ||
} | ||
|
||
/** | ||
* Удаляет пару ключ-значение из хранилища. | ||
*/ | ||
@Override | ||
public synchronized void delete(K key) { | ||
isClose(); | ||
|
||
cache.remove(key); | ||
base.remove(key); | ||
deletedCount++; | ||
|
||
if (deletedCount / size() > FILLING_PERCENTAGE) { | ||
reorganiseStorage(); | ||
} | ||
} | ||
|
||
/** | ||
* Читает все ключи в хранилище. | ||
* <p> | ||
* Итератор должен бросать {@link java.util.ConcurrentModificationException}, | ||
* если данные в хранилище были изменены в процессе итерирования. | ||
*/ | ||
@Override | ||
public Iterator<K> readKeys() { | ||
isClose(); | ||
|
||
return base.keySet().iterator(); | ||
|
||
} | ||
|
||
/** | ||
* Возвращает число ключей, которые сейчас в хранилище. | ||
*/ | ||
@Override | ||
public int size() { | ||
isClose(); | ||
|
||
return base.size(); | ||
} | ||
|
||
@Override | ||
public synchronized void close() { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. При повторном закрытии ничего не нужно делать. У тебя нет этой проверки. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Да, точно |
||
if (isClosed) { | ||
return; | ||
} | ||
|
||
reorganiseStorage(); | ||
uploadDataToStorage(); | ||
|
||
try { | ||
mapStorage.close(); | ||
storage.close(); | ||
} catch (IOException excetion) { | ||
throw new RuntimeException("Can't close storage"); | ||
} finally { | ||
isClosed = true; | ||
cache.clear(); | ||
base.clear(); | ||
mutexFile.delete(); | ||
} | ||
} | ||
|
||
private void downloadDataFromStorage() { | ||
try { | ||
|
||
int count = mapStorage.readInt(); | ||
|
||
for (int i = 0; i < count; i++) { | ||
K key = keySerializationStrategy.deserializeFromFile(mapStorage); | ||
Long tmp = mapStorage.readLong(); | ||
base.put(key, tmp); | ||
maxOffset = Math.max(maxOffset, tmp); | ||
} | ||
} catch (IOException exception) { | ||
base.clear(); | ||
throw new RuntimeException("Trouble with storage.db"); | ||
} | ||
} | ||
|
||
private void uploadDataToStorage() { | ||
try { | ||
mapStorage.close(); | ||
File file = new File(pathname, mapStorageName); | ||
assert (file.delete()); | ||
mapStorage = new RandomAccessFile(new File(pathname, mapStorageName), "rw"); | ||
|
||
mapStorage.writeInt(size()); | ||
for (HashMap.Entry<K, Long> entry : base.entrySet()) { | ||
keySerializationStrategy.serializeToFile(entry.getKey(), mapStorage); | ||
mapStorage.writeLong(entry.getValue()); | ||
} | ||
} catch (IOException exception) { | ||
throw new RuntimeException("Trouble with storage.db"); | ||
} | ||
} | ||
|
||
private void writeCacheToStorage() { | ||
isClose(); | ||
|
||
if (cache.isEmpty()) { | ||
return; | ||
} | ||
|
||
try { | ||
storage.seek(maxOffset); | ||
for (HashMap.Entry<K, V> entry : cache.entrySet()) { | ||
valueSerializationStrategy.serializeToFile(entry.getValue(), storage); | ||
base.put(entry.getKey(), maxOffset); | ||
maxOffset = storage.getFilePointer(); | ||
} | ||
|
||
cache.clear(); | ||
|
||
} catch (IOException exception) { | ||
throw new RuntimeException("Can't write cache on the disk"); | ||
} | ||
|
||
} | ||
|
||
private void reorganiseStorage() { | ||
|
||
File file1 = new File(pathname, storageName); | ||
File file = new File(pathname, "newStorage.txt"); | ||
|
||
try (DataOutputStream newStorage = new DataOutputStream(new BufferedOutputStream(new FileOutputStream(file)))) { | ||
|
||
writeCacheToStorage(); | ||
|
||
for (HashMap.Entry<K, Long> entry : base.entrySet()) { | ||
storage.seek(entry.getValue()); | ||
Long tmp = (long) newStorage.size(); | ||
valueSerializationStrategy.serializeToFile(read(entry.getKey()), newStorage); | ||
base.put(entry.getKey(), tmp); | ||
} | ||
deletedCount = 0; | ||
|
||
storage.close(); | ||
assert (file1.delete()); | ||
|
||
newStorage.close(); | ||
assert (file.renameTo(file1)); | ||
storage = new RandomAccessFile(new File(pathname, storageName), "rw"); | ||
|
||
} catch (IOException exception) { | ||
throw new RuntimeException("Can't reorganise storage!"); | ||
} | ||
} | ||
|
||
private void isClose() { | ||
if (isClosed) { | ||
throw new IllegalStateException("Can't write: storage is closed"); | ||
} | ||
} | ||
|
||
} | ||
|
16 changes: 16 additions & 0 deletions
16
...a/ru/mipt/java2016/homework/g595/manucharyan/task3/SerializationStrategyRandomAccess.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
package ru.mipt.java2016.homework.g595.manucharyan.task3; | ||
|
||
import java.io.DataInput; | ||
import java.io.DataOutput; | ||
import java.io.IOException; | ||
|
||
/** | ||
* @author Vardan Manucharyan | ||
* @since 30.10.16 | ||
*/ | ||
public interface SerializationStrategyRandomAccess<Value> { | ||
|
||
void serializeToFile(Value value, DataOutput output) throws IOException; | ||
|
||
Value deserializeFromFile(DataInput input) throws IOException; | ||
} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Идея предложила тебе заменить на
return cache.containsKey(key) || base.containsKey(key)
, а ты её игноришь. Не надо так.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ну так же понятней, нет? Типа тут односложные ифы, и сразу понятно, что происходит, иначе надо думать(
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ну, не знаю, по мне так два однотипных условия в одном выглядят читабельнее. Вообще не суть.