Quick Start DotNet

Quick Start: .NET

From nothing to an encrypted, indexed file you can query with LINQ, in about five minutes.

You need: .NET 10. The Jazmin package has no third-party dependencies.

dotnet add package Jazmin

If you already use Newtonsoft.Json, the first example will look familiar: see also Migrating from JSON and Newtonsoft.

1. Like Newtonsoft: objects in, bytes out

using Jazmin;
using Jazmin.Serialization;

var customers = new List<Customer>
{
    new() { Id = 1, Name = "Ann Johnson", Country = "ZA", Balance = 1520.75m, Joined = new DateTime(2024, 1, 2, 0, 0, 0, DateTimeKind.Utc) },
    new() { Id = 2, Name = "Bob Smith", Country = "NA", Balance = 99.10m, Joined = new DateTime(2024, 3, 4, 0, 0, 0, DateTimeKind.Utc) },
};

byte[] bytes = JazminConvert.SerializeObject(customers);
List<Customer> back = JazminConvert.DeserializeObject<List<Customer>>(bytes)!;
Console.WriteLine($"{back.Count} customers round-tripped");
Console.WriteLine($"{back[1].Name}: {back[1].Balance}");

public class Customer
{
    public int Id { get; set; }
    public string Name { get; set; } = "";
    public string? Country { get; set; }
    public decimal Balance { get; set; }
    public DateTime Joined { get; set; }
}

2 customers round-tripped
Bob Smith: 99.10

The columns came from the class: int became an int column, decimal an exact decimal column, DateTime a datetime column. Decimals are never rounded: 99.10m comes back as 99.10.

2. A real file: attributes, indexes, encryption and LINQ

using Jazmin;
using Jazmin.Query;
using Jazmin.Serialization;

var key = JazminKey.Generate(); // in real code: keep key.Export() in your secret manager, never in logs

var customers = new List<Customer>
{
    new() { Id = 1, Name = "Ann Johnson", Country = "ZA", Balance = 1520.75m },
    new() { Id = 2, Name = "Bob Smith", Country = "NA", Balance = 99.10m },
    new() { Id = 3, Name = "Thabo Ndlovu", Country = "ZA", Balance = 0.00m },
};
new JazminSerializer(new JazminSerializerSettings { Key = key }).Serialize("customers.jzm", customers);

// Opening reads only the header. A query reads only the index pages and chunks that can match.
using var reader = JazminReader.Open("customers.jzm", new JazminReadOptions { Key = key });
Console.WriteLine($"{reader.RowCount} rows: {string.Join(", ", reader.Columns.Select(c => $"{c.Name} ({c.Type.ToString().ToLowerInvariant()})"))}");

foreach (var c in reader.Query<Customer>(c => c.Country == "ZA" && c.Name.Contains("Ndlovu")))
    Console.WriteLine($"{c.Id} {c.Name} {c.Balance}");

var query = reader.AsQueryable<Customer>();
Console.WriteLine($"{query.Count(c => c.Country == "ZA")} customers in ZA");
Console.WriteLine(reader.Explain(JazminFilter.Eq("Id", 3)).Strategy);

public class Customer
{
    [JazminIndex] // a sorted index: fast lookups and ranges
    public int Id { get; set; }

    [JazminIndex(JazminIndexKind.Trigram)] // fast "contains" search
    public string Name { get; set; } = "";

    [JazminIndex]
    public string? Country { get; set; }

    public decimal Balance { get; set; }
}

3 rows: Id (int), Name (string), Country (string), Balance (decimal)
3 Thabo Ndlovu 0.00
2 customers in ZA
index

• The key is 32 random bytes, written as text that starts with jzk1- and carries a checksum, so a typo is reported as a typo. Lose it and the data cannot be recovered.
• Query<T> turns the parts of the condition it can into index lookups, then checks the whole condition on each candidate. A condition it can't translate is still correct, just scanned.
• AsQueryable<T>() runs Where, Skip, Take, Count, Select and more inside the reader (New in 1.2).

3. Big data: stream rows in, stream rows out

using Jazmin;
using Jazmin.Query;

var columns = new[]
{
    new JazminColumn("id", JazminType.Int) { Nullable = false },
    new JazminColumn("reading", JazminType.Float),
};
using (var writer = JazminWriter.Create("readings.jzm", columns, new JazminWriteOptions { SortedBy = ["id"] }))
{
    for (long i = 0; i < 1_000_000; i++) writer.WriteValues(i, Math.Round(Math.Sin(i / 1000.0) * 1000) / 1000);
} // Dispose finishes the file: indexes, header and trailer

using var readings = JazminReader.Open("readings.jzm");
Console.WriteLine($"{readings.RowCount} rows");
foreach (var row in readings.Find(JazminFilter.Eq("id", 765432L)))
    Console.WriteLine($"id {row["id"]}: {row["reading"]}"); // reads one chunk, not a million rows

double total = 0;
var range = JazminFilter.Gte("id", 500000L) & JazminFilter.Lt("id", 500100L);
foreach (var row in readings.Find(range, new JazminQueryOptions { Select = ["reading"] })) total += (double)row["reading"]!;
Console.WriteLine($"sum of 100 readings: {Math.Round(total, 3)}");

1000000 rows
id 765432: -0.899
sum of 100 readings: -51.072

Memory while writing is one chunk (4,096 rows by default) plus the index lists, however many rows you write. WriteValues is the fastest way in: no dictionary or object per row.

4. Typed streaming without a reader

using Jazmin;
using Jazmin.Serialization;

var serializer = new JazminSerializer();
serializer.Serialize("people.jzm", Enumerable.Range(1, 5).Select(i => new Person { Id = i, Name = $"Person {i}" }));

using var stream = File.OpenRead("people.jzm");
foreach (var p in serializer.DeserializeEnumerable<Person>(stream)) // one row at a time
    Console.Write($"{p.Name}; ");
Console.WriteLine();

public class Person
{
    public int Id { get; set; }
    public string Name { get; set; } = "";
}

Person 1; Person 2; Person 3; Person 4; Person 5;

Where next

• Core Concepts: what is inside a file, and why queries read so little
• .NET Serializer and LINQ: attributes, settings, converters, AsQueryable
• Filter Language: every operator, with examples
• Encryption and Keys and Access Control
• Recipes: complete solutions to common jobs