Skip to content

Commit

Permalink
.Net: instead running two queries now a single query can do the same …
Browse files Browse the repository at this point in the history
…thing (#6705)

### Motivation and Context
1. Instead of executing two SQL queries, a single query can do the same
thing.
2. Performance improvement

### Description
Previously the **InternalUpsertAsync** method was executing two sql
query by calling two individual methods - First **UpdateAsync** and then
second **InsertOrIgnoreAsync**.

The new method uses the INSERT OR REPLACE statement to either update an
existing row or insert a new one based on the unique collection and key
combination

### Contribution Checklist

- [ ] The code builds clean without any errors or warnings
- [ ] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [ ] All unit tests pass, same unit test can detect the bug
- [ ] I didn't break anyone 😄

---------

Co-authored-by: Dmytro Struk <13853051+dmytrostruk@users.noreply.github.com>
  • Loading branch information
atiq-bs23 and dmytrostruk committed Jun 14, 2024
1 parent 7812017 commit bb1eb3a
Show file tree
Hide file tree
Showing 2 changed files with 5 additions and 32 deletions.
23 changes: 3 additions & 20 deletions dotnet/src/Connectors/Connectors.Memory.Sqlite/Database.cs
Original file line number Diff line number Diff line change
Expand Up @@ -56,30 +56,13 @@ public async Task CreateCollectionAsync(SqliteConnection conn, string collection
await cmd.ExecuteNonQueryAsync(cancellationToken).ConfigureAwait(false);
}

public async Task UpdateAsync(SqliteConnection conn,
public async Task UpsertAsync(SqliteConnection conn,
string collection, string key, string? metadata, string? embedding, string? timestamp, CancellationToken cancellationToken = default)
{
using SqliteCommand cmd = conn.CreateCommand();
cmd.CommandText = $@"
UPDATE {TableName}
SET metadata=@metadata, embedding=@embedding, timestamp=@timestamp
WHERE collection=@collection
AND key=@key ";
cmd.Parameters.AddWithValue("@collection", collection);
cmd.Parameters.AddWithValue("@key", key);
cmd.Parameters.AddWithValue("@metadata", metadata ?? string.Empty);
cmd.Parameters.AddWithValue("@embedding", embedding ?? string.Empty);
cmd.Parameters.AddWithValue("@timestamp", timestamp ?? string.Empty);
await cmd.ExecuteNonQueryAsync(cancellationToken).ConfigureAwait(false);
}

public async Task InsertOrIgnoreAsync(SqliteConnection conn,
string collection, string key, string? metadata, string? embedding, string? timestamp, CancellationToken cancellationToken = default)
{
using SqliteCommand cmd = conn.CreateCommand();
cmd.CommandText = $@"
INSERT OR IGNORE INTO {TableName}(collection, key, metadata, embedding, timestamp)
VALUES(@collection, @key, @metadata, @embedding, @timestamp); ";
INSERT OR REPLACE INTO {TableName}(collection, key, metadata, embedding, timestamp)
VALUES(@collection, @key, @metadata, @embedding, @timestamp);";
cmd.Parameters.AddWithValue("@collection", collection);
cmd.Parameters.AddWithValue("@key", key);
cmd.Parameters.AddWithValue("@metadata", metadata ?? string.Empty);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -246,18 +246,8 @@ private async Task<string> InternalUpsertAsync(SqliteConnection connection, stri
{
record.Key = record.Metadata.Id;

// Update
await this._dbConnector.UpdateAsync(
conn: connection,
collection: collectionName,
key: record.Key,
metadata: record.GetSerializedMetadata(),
embedding: JsonSerializer.Serialize(record.Embedding, JsonOptionsCache.Default),
timestamp: ToTimestampString(record.Timestamp),
cancellationToken: cancellationToken).ConfigureAwait(false);

// Insert if entry does not exists
await this._dbConnector.InsertOrIgnoreAsync(
// Insert or replace
await this._dbConnector.UpsertAsync(
conn: connection,
collection: collectionName,
key: record.Key,
Expand Down

0 comments on commit bb1eb3a

Please sign in to comment.