Skip to content

Commit

Permalink
add compaction.failed event
Browse files Browse the repository at this point in the history
  • Loading branch information
Timothée Rebours committed Jan 10, 2024
1 parent d4c096b commit 013a18c
Show file tree
Hide file tree
Showing 5 changed files with 68 additions and 20 deletions.
13 changes: 12 additions & 1 deletion API.md
Original file line number Diff line number Diff line change
Expand Up @@ -241,7 +241,7 @@ Will return pointers to matched elements (shallow copies), returning full copies

**Kind**: global class
**Extends**: [<code>EventEmitter</code>](http://nodejs.org/api/events.html)
**Emits**: <code>Datastore#event:&quot;compaction.done&quot;</code>
**Emits**: <code>Datastore#event:&quot;compaction.done&quot;</code>, <code>Datastore#event:&quot;compaction.failed&quot;</code>

* [Datastore](#Datastore)[<code>EventEmitter</code>](http://nodejs.org/api/events.html)
* [new Datastore(options)](#new_Datastore_new)
Expand Down Expand Up @@ -282,6 +282,7 @@ Will return pointers to matched elements (shallow copies), returning full copies
* [.remove(query, [options], [cb])](#Datastore+remove)
* [.removeAsync(query, [options])](#Datastore+removeAsync) ⇒ <code>Promise.&lt;number&gt;</code>
* ["event:compaction.done"](#Datastore+event_compaction.done)
* ["event:compaction.failed"](#Datastore+event_compaction.failed)
* _inner_
* [~countCallback](#Datastore..countCallback) : <code>function</code>
* [~findOneCallback](#Datastore..findOneCallback) : <code>function</code>
Expand Down Expand Up @@ -743,6 +744,16 @@ if the update did not actually modify them.</p>
<p>Compaction event. Happens when the Datastore's Persistence has been compacted.
It happens when calling [compactDatafileAsync](#Datastore+compactDatafileAsync), which is called periodically if you have called
[setAutocompactionInterval](#Datastore+setAutocompactionInterval).</p>
<p>In case of failure, it emits [Datastore#event:"compaction.failed"](Datastore#event:"compaction.failed") instead.</p>

**Kind**: event emitted by [<code>Datastore</code>](#Datastore)
<a name="Datastore+event_compaction.failed"></a>

### "event:compaction.failed"
<p>Compaction event. Happens when the Datastore's Persistence compaction task has failed.
It may happen when calling [compactDatafileAsync](#Datastore+compactDatafileAsync), which is called periodically if you have called
[setAutocompactionInterval](#Datastore+setAutocompactionInterval).</p>
<p>In case of success, it emits [Datastore#event:"compaction.done"](Datastore#event:"compaction.done") instead.</p>

**Kind**: event emitted by [<code>Datastore</code>](#Datastore)
<a name="Datastore..countCallback"></a>
Expand Down
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,9 @@ to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
- Explicitly import `buffer` [#34](https://github.com/seald/nedb/pull/34).
- Fix `Cursor`'s typings [#45](https://github.com/seald/nedb/issues/45)

### Added
- Added a `compaction.failed` event [#28](https://github.com/seald/nedb/issues/28)

## [4.0.3] - 2023-12-13
### Fixed
- Fixed EPERM Exception when datastore is at the root of a disk on Windows [#48](https://github.com/seald/nedb/issues/48)
Expand Down
14 changes: 14 additions & 0 deletions lib/datastore.js
Original file line number Diff line number Diff line change
Expand Up @@ -60,10 +60,23 @@ const { isDate, pick, filterIndexNames } = require('./utils.js')
* It happens when calling {@link Datastore#compactDatafileAsync}, which is called periodically if you have called
* {@link Datastore#setAutocompactionInterval}.
*
* In case of failure, it emits {@link Datastore#event:"compaction.failed"} instead.
*
* @event Datastore#event:"compaction.done"
* @type {undefined}
*/

/**
* Compaction event. Happens when the Datastore's Persistence compaction task has failed.
* It may happen when calling {@link Datastore#compactDatafileAsync}, which is called periodically if you have called
* {@link Datastore#setAutocompactionInterval}.
*
* In case of success, it emits {@link Datastore#event:"compaction.done"} instead.
*
* @event Datastore#event:"compaction.failed"
* @type {Error}
*/

/**
* Generic document in NeDB.
* It consists of an Object with anything you want inside.
Expand Down Expand Up @@ -143,6 +156,7 @@ const { isDate, pick, filterIndexNames } = require('./utils.js')
* @classdesc The `Datastore` class is the main class of NeDB.
* @extends external:EventEmitter
* @emits Datastore#event:"compaction.done"
* @emits Datastore#event:"compaction.failed"
* @typicalname NeDB
*/
class Datastore extends EventEmitter {
Expand Down
42 changes: 23 additions & 19 deletions lib/persistence.js
Original file line number Diff line number Diff line change
Expand Up @@ -96,27 +96,31 @@ class Persistence {
* @private
*/
async persistCachedDatabaseAsync () {
const lines = []

if (this.inMemoryOnly) return
try {
const lines = []

this.db.getAllData().forEach(doc => {
lines.push(this.afterSerialization(model.serialize(doc)))
})
Object.keys(this.db.indexes).forEach(fieldName => {
if (fieldName !== '_id') { // The special _id index is managed by datastore.js, the others need to be persisted
lines.push(this.afterSerialization(model.serialize({
$$indexCreated: {
fieldName: this.db.indexes[fieldName].fieldName,
unique: this.db.indexes[fieldName].unique,
sparse: this.db.indexes[fieldName].sparse
}
})))
}
})
if (this.inMemoryOnly) return
this.db.getAllData().forEach(doc => {
lines.push(this.afterSerialization(model.serialize(doc)))
})
Object.keys(this.db.indexes).forEach(fieldName => {
if (fieldName !== '_id') { // The special _id index is managed by datastore.js, the others need to be persisted
lines.push(this.afterSerialization(model.serialize({
$$indexCreated: {
fieldName: this.db.indexes[fieldName].fieldName,
unique: this.db.indexes[fieldName].unique,
sparse: this.db.indexes[fieldName].sparse
}
})))
}
})

await storage.crashSafeWriteFileLinesAsync(this.filename, lines, this.modes)
this.db.emit('compaction.done')
await storage.crashSafeWriteFileLinesAsync(this.filename, lines, this.modes)
this.db.emit('compaction.done')
} catch (error) {
this.db.emit('compaction.failed', error)
throw error
}
}

/**
Expand Down
16 changes: 16 additions & 0 deletions test/persistence.async.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -368,13 +368,29 @@ describe('Persistence async', function () {
})

it('Can listen to compaction events', async () => {
d.insertAsync({ a: 2 })
const compacted = new Promise(resolve => {
d.once('compaction.done', function () {
resolve()
})
})
await d.compactDatafileAsync()
await compacted // should already be resolved when the function returns, but still awaiting for it

const failure = new Promise(resolve => {
d.once('compaction.failed', function (error) {
resolve(error)
})
})

const afterSerializationOriginal = d.persistence.afterSerialization
d.persistence.afterSerialization = () => { throw new Error('synthetic error') }
await d.compactDatafileAsync().then(() => { throw new Error('should have failed')}, error => {
if (!error || !(error instanceof Error) || error.message !== 'synthetic error') throw error
})
const error = await failure
if (!error || !(error instanceof Error) || error.message !== 'synthetic error') throw error
d.persistence.afterSerialization = afterSerializationOriginal
})

it('setAutocompaction fails gracefully when passed a NaN', async () => {
Expand Down

0 comments on commit 013a18c

Please sign in to comment.