Data and operation extensions for faster .NET (standard2.0) development
This library groups together various data and operation extensions that I've used and I'm still using for faster .NET development. This document describes all these items along with their use case description and some code samples.
They are grouped in three main areas, each with its own set of functional areas:
- H's Necessaire
- Configs
- Models
- Extensions
- AzureExtensions
- CollectionExtensions
- DataExtensions
- ExceptionExtensions
- FileSystemExtensions
- TaskExtensions
- Operations
- Discussions
H.Necessaire runtime configs described
App.Nameoptional, defaults to"H.Necessaire"DefaultHasheroptional, defaults toSimpleSecureHasher, possible values:SimpleSecureHasher,MD5Hasher
QdActions.MaxProcessingAttemptsoptional, defaults to3QdActions.ProcessingIntervalInSecondsoptional, defaults to15QdActions.ProcessingBatchSizeoptional, defaults to10
FileSystemStorageRootFolderoptional, defaults toGetRootFolderFromStartAssembly()
IronMen.FilePathoptional, default to"ironmen.json"IronMen.PassFilePathoptional, default to"ironmen.pass.json"
SqlConnections.DefaultConnectionStringSqlConnections.DatabaseNames.Core
RavenDbConnections.DatabaseNames.CoreRavenDbConnections.ClientCertificateNamePFX cert must be an embedded resourceRavenDbConnections.DatabaseUrls- collection of stringsRavenDbConnections.ClientCertificatePassword- if client cert is pass-protected
AzureCosmosDB.URLAzureCosmosDB.Keys- collection of stringsAzureCosmosDB.DatabaseIDAzureCosmosDB.ContainerIDAzureCosmosDB.ApporAzureCosmosDB.ApplicationorAzureCosmosDB.AppNameorAzureCosmosDB.ApplicationNameoptional
Google.Firestore.ProjectIDGoogle.Firestore.HNecessaireDefaultoptional collection name, defaults toHNecessaireDefaultGoogle.Firestore.Auth.Json- googleAuthJsonPath, can also be already set inGOOGLE_APPLICATION_CREDENTIALSenv. variable and skipped from runtime config
BaseUrloptional, defaults to smart determinationBaseApiUrloptional, defaults to smart determinationSecurityContextCheckIntervalInSecondsoptional, defaults to30SyncIntervalInSecondsoptional, defaults to10Formatting.DateAndTimeoptional, defaults to"ddd, MMM dd, yyyy 'at' HH:mm 'UTC'"Formatting.Dateoptional, defaults to"ddd, MMM dd, yyyy"Formatting.Timeoptional, defaults to"HH:mm"Formatting.Monthoptional, defaults to"yyyy MMM"Formatting.DayOfWeekoptional, defaults to"dddd"Formatting.TimeStampThisYearoptional, defaults to"MMM dd 'at' HH:mm"Formatting.TimeStampOtherYearoptional, defaults to"MMM dd, yyyy 'at' HH:mm"Formatting.TimeStampIdentifieroptional, defaults to"yyyyMMdd_HHmmss_'UTC'"Security.LoginUrloptional, defaults to"/Security/Login"Security.RefreshUrloptional, defaults to"/Security/Refresh""BffApiSyncRegistryRelativeUrl"optional, defaults to"/sync/sync"
NuSpecRootFolderPathoptional, defaults to"C:\H\H.Necessaire\Src\H.Necessaire"
.NET Classes that can be generally used for various intents.
Used to model any kind of operation on which you want to know whether it was successful or not and the reason(s) why.
class OperationResult
{
bool IsSuccessful
string Reason
string[] Comments
ThrowOnFail()
FlattenReasons()
}
class OperationResult<T> : OperationResult
{
T Payload
T ThrowOnFailOrReturn()
}public OperationResult<Order> ValidateOrder(Order order)
{
if (order == null)
return OperationResult.Fail("The order is inexistent").WithoutPayload<Order>();
if (!order.Items.Any())
return OperationResult.Fail("The order has no items").WithoutPayload<Order>();
return OperationResult.Win().WithPayload(order);
}
[...]
OperationResult<Order> orderValidation = ValidateOrder(order);
if (!orderValidation.IsSuccessful)
{
//log errors
return;
}
SaveOrder(order);
[...]
Order order = ValidateOrder(order).ThrowOnFailOrReturn();Extends AggregateException and is the exception type thrown by OperationResult.ThrowOnFail() or OperationResult<T>.ThrowOnFailOrReturn().
You shouldn't manually instantiate this type of exception.
class OperationResultException : AggregateException
{
OperationResult OperationResult
}Used instead of a string-string key value pair for lighter syntax.
struct Note
{
string Id
string Value
static Note[] FromDictionary(Dictionary<string, string> keyValuePairs)
}Used to model semantic versioning.
class VersionNumber
{
int Major
int Minor
int Patch
int Build
string Suffix
string Semantic
}Used to model version number correlation with code reference.
class Version
{
VersionNumber Number
DateTime Timestamp
string Branch
string Commit
}Used to model numeric intervals.
struct NumberInterval
{
double Min
double Max
}Used for modelling various use cases when you need to iterate on stuff that are bound to specific context. Best example is iterating over a database stream, when you need to free the connection or any other type of unit of work created to access those external resources.
interface IDisposableEnumerable<T> : IEnumerable<T>, IDisposableUsed for modelling limited collections. For instance when doing pagination or virtual pagination on a huge data set.
interface ILimitedEnumerable<T> : IEnumerable<T>
{
int Offset
int Length
int TotalNumberOfItems
}The naming is super clear. It's used for working with a collection of disposable objects so that you don't need to handle each one's disposal. For instance when aggregating something from multiple streams.
class CollectionOfDisposables<T> : IDisposableEnumerable<T> where T : IDisposable
{
//Constructs
CollectionOfDisposables(params T[] disposables)
CollectionOfDisposables(IEnumerable<T> disposables)
}using(var streams = new CollectionOfDisposables(File.OpenRead(@"C:\a.txt"), File.OpenRead(@"C:\b.txt")))
{
//Process streams
}Models and object that's identified by a Guid. Usage is obvious.
interface IGuidIdentity
{
Guid ID
}Operation contract for a key-values storage resource. Use case is obvious.
interface IKeyValueStorage
{
string StoreName { get; }
Task Set(string key, string value, TimeSpan? validFor = null);
Task Set(string key, string value, DateTime? validUntil = null);
Task<string> Get(string key);
Task Zap(string key);
Task Remove(string key);//Should be just an alias for Zap. Most devs are not used to the verb "zap"
}Model used to define a sort criteria
class SortFilter
{
string By
SortDirection Direction
enum SortDirection
{
Ascending = 0,
Descending = 1,
}
}Operation Contract for sorting
interface ISortFilter
{
SortFilter[] SortFilters
OperationResult ValidateSortFilters();
}An abstract implementation of ISortFilter to make the use of it easier. When using this class you just need to override the ValidSortNames.
abstract class SortFilterBase : ISortFilter
{
protected abstract string[] ValidSortNames { get; }
}Model for pagination criteria.
class PageFilter
{
int PageIndex
int PageSize
}Operation contract for pagination
interface IPageFilter
{
PageFilter PageFilter
}Model for a pagination operation result
class Page<T>
{
T[] Content
int PageIndex
int PageSize
int? TotalNumberOfPages
static Page<T> Single(params T[] content)
static Page<T> Empty(int pageIndex = 0, int pageSize = 0, int totalNumberOfPages = 1)
}class UserFilter : SortFilterBase, IPageFilter { [...] }
Page<User> usersPage = await userResource.Browse(new UserFilter([...]));A bunch of helpful extension methods for collections, exceptions, tasks, Azure, and more. Just read down below.
A bunch of helpful Azure extension methods.
&
If you want to store a DateTime.MinValue in a storage account in Azure you'll get an exception. That's because the minimum date that azure storage supports is new DateTime(1601, 1, 1);
class MyTableEntity : ITableEntity
{
[...]
public void ReadEntity(IDictionary<string, EntityProperty> properties, OperationContext operationContext)
{
[...]
if (properties.ContainsKey(nameof(CreatedAt)))
CreatedAt = properties[nameof(CreatedAt)].DateTime?.ToNetMinDateFromAzureTableStorageMinDate() ?? DateTime.MinValue;
[...]
}
public static MyTableEntity FromMyEntity(MyEntity entity)
{
[...]
CreatedAt = entity.CreatedAt.ToAzureTableStorageSafeMinDate(),
[...]
}
[...]
}Batch, containment, array-ing, trimming, etc.
&
A simpler syntax for checking if an item exists in a collection or not.
if (workItem.Status.In(ProcessingStatus.CompletedAndWaitingForApproval, ProcessingStatus.CompletedButRejected))
return OrderStatus.Reviewing;Splits a collection in batches of the specified size.
foreach (IEnumerable<Command> batch in commands.Batch(4))
result.AddRange(await Task.WhenAll(batch.Select(ProcessCommand).ToArray()));Converts the given item into a one element array. Sugar syntax for new [] { item } or new ItemType[] { item }.
var searchResults = await orderResource.SearchOrders(new OrderFilter
{
CustomerIDs = request.Customer.ID.AsArray(),
[...]
});Safely tries to create a new array without the first numberOfElementsToJump elements. An exception-less alternative to .Skip().
var result = new [] { 1, 2, 3 }.Jump(100); //No exception, result will be an empty array.Useful when doing a wildcard search to exclude irrelevant entries and to minimize the max number of search keys and avoid performance issues or overflows.
string[] keywordsToSearchFor = filter.Customer?.Keywords?.TrimToValidKeywordsOnly();Trimmers, parsers, fluent syntaxes, etc.
Used for percent values, mainly for UI projections to make sure you don't display nasty percent value, like -13% or 983465.
string ownershipPercentage = $"{Math.Round(OwnershipPercentage.Value.TrimToPercent(), 1)}%");&
Used as sugar syntax for if(TryParse(out val)){[...]}
Guid? id = "asdasd".ParseToGuidOrFallbackTo(null);Sugar syntax for date >= from && date <= to
if(!date.IsBetweenInclusive(DateTime.Now.AddMonths(-6), DateTime.Now))
throw new InvalidOperationException("Selected date must in between today and six month ago");Used for fluent syntax.
Customer customer = new Customer()
.And(c => c.FirstName = "Hin")
.And(c => c.LastName = "Tee")
.And(c => c.ID = Guid.NewGuid())Used to check if a given type is a derivate or same type as the other. Sugar syntax for typeToCheck == typeToCompareWith || typeToCompareWith.IsSubclassOf(typeToCheck).
class Customer : User { }
typeof(Customer).IsSameOrSubclassOf(typeof(User));//trueGet the UNIX timestamp of a dateTime. Useful for compatibility with external systems, legacy code, cross-platform integration, JS integration
var unixStamp = DateTime.Now.ToUnixTimestamp()Get the DateTime out of a UNIX timestamp. Useful for compatibility with external systems, legacy code, cross-platform integration, JS integration
DateTime dateTime = 21763457162.UnixTimeStampToDateTime()Makes sure that a given time is UTC so that you don't get unexpected behavior from DateTime.Kind.Unspecified or when persisting or serializing date-times.
DateTime utcDate = DateTime.Now.EnsureUtc();Converts an IEnumerable<T> into an IDisposableEnumerable<T>. This is just an in-memory wrapper over the enumeration, useful for in-memory mocks of certain resources. For real-world use-cases this shouldn't be used. The resource should always implement IDisposableEnumerable<T> itself
int[] numbers = new [] { 1, 2, 3 };
IDisposableEnumerable<User> = numbers.AsDisposableEnumerable();Some useful extension methods for C# exceptions.
Flattens the given exception instance. In short it recursively maps the root Exception plus any InnerExceptions and AggregateExceptions to a flat array of Exceptions. Super useful for logging the actual errors when dealing with Tasks, RPCs, DB access, because these scenarios usually hold the actual exception cause inside inner exceptions tree or aggregate exceptions.
Exception[] allExceptions = aggExOrExWithInnerExOrCombinationOfBoth.Flatten();Converts the given exceptions to Note[]. Useful for persistence.
Note[] exceptions = aggregateException.Flatten().ToNotes();Some useful extension methods dealing with file names.
Converts the given string value to a new string which can safely be used as file name.
string fileName = "bubu?*:.txt".ToSafeFileName();//gets converted into bubu.txt
File.WriteAllText($"C:\Users\bubu\Downloads\{fileName}", "File Content");Sugar syntaxes for Task API.
Sugar syntax for Task.FromResult(value);
Useful mainly when implementing in-memory mocks for interfaces that use the Task API.
interface ImAResource
{
Task ShowMeTheMoney();
}
class InMemoryResource : ImAResource
{
public Task<int> ShowMeTheMoney()
{
return 199.AsTask();
}
}Transforms a synchronous Action to an asynchronous Func<Task> so you can safely use it along with the Task API.
interface ImAnEngine
{
Task RunDelayed(Func<Task> thisLogic);
}
[...]
void ReportProgress()
{
Console.WriteLine("Progress...");
}
[...]
await myEngine.RunDelayed(ReportProgress.AsAsync())Data normalizer, unsafe code execution, debounce, throttle, execution time measurement, to name a few. See the full list below.
This is used to reduce or expand numeric intervals. For instance you want to reduce a range of values between 100 and 10000 to 0 and 100, so you can present them as a percent.
class DataNormalizer
{
//Construct
DataNormalizer(NumberInterval fromInterval, NumberInterval toInterval)
//Translates the given value from fromInterval to toInterval
double Do(double value);
}var files = System.IO.Directory.EnumerateFiles(@"C:\Users\bubu\Downloads");
DataNormalizer percentNormalizer = new DataNormalizer(new NumberInterval(0, files.Count()), new NumberInterval(0, 100));
int fileIndex = 0;
foreach(var file in files)
{
await Backup(file);
fileIndex++;
await ReportProgress(percentNormalizer.Do(fileIndex));
}It is used to make sure that you invoke a specific action only once if sequential calls are made in a given time frame.
For instance I want to make sure that DoThis() is called just once even if it is repeatedly invoked 10 times in 10 seconds.
class Debouncer
{
//Construct
Debouncer(Func<Task> asyncActionToTame, TimeSpan debounceInterval)
Task Invoke();
Dispose();
}Task SearchUsers(string key){ [...] }
//If the user types faster than 1 char / second, we make sure to invoke SearchUsers just once, after he's done typing
var debouncedUserSearch = new Debouncer(SearchUsers, TimeSpan.FromSeconds(1))
async Task OnSearchInputTextChanged(TextInput textInput)
{
[...]
await debouncedUserSearch.Invoke();
[...]
}It is used to make sure that you invoke a specific action only once every given time frame while that action is continuously being invoked.
For instance I do a super frequent action (e.g.: a file iteration on a huge file tree) but I want to report progress only once a second.
This is very similar with the Debouncer except that the execution is not fully debounced to the end of the sequential invocations, but also during them.
class Throttler
{
//Construct
Throttler(Func<Task> asyncActionToTame, TimeSpan throttleInterval)
Task Invoke();
}Task ReportProgress(){ [...] }
//We only want to report progress once a second, no matter how fast an operation goes
var throttledProgressReport = new Throttler(ReportProgress, TimeSpan.FromSeconds(1))
async Task BackupFiles()
{
[...]
foreach(var file in filesToBackup)
{
[...]
await throttledProgressReport.Invoke();
[...]
}
}Extensions that help with the execution of unsafe code that we don't want to crash our app or to stop the execution flow. A good example is the interaction with external resources. Say we want to raise a web-hook but we don't want our app to crash if that endpoint is not available but rather fallback to another mechanism.
&
These are the methods used to achieve this behavior. You can see them as a sugar syntax over try{}catch{} plus a retry policy.
TryOrFailWithGrace is just an extension method wrapper over the static TryAFewTimesOrFailWithGrace.
Task CallExternalWebHook() { [...] }
async Task ProcessCustomerRequest()
{
[...]
await
CallExternalWebHook
.TryOrFailWithGrace(
numberOfTimes: 3,
onFail: async ex => await LogExceptionAndNotifyAdmins(ex)//This is safely called as well; if an exception is thrown, the execution will continue
)
}It's used to make sure a certain piece of code is executed event if 'quick return' occurs in the implementation. It does this by implementing IDisposable and thus leveraging the using syntax.
A good example here is setting a waiting state on a class.
class ScopedRunner : IDisposable
{
//Construct
ScopedRunner(Action onStart, Action onStop)
Dispose()
}using(new ScopedRunner(() => IsBusy = true, () => IsBusy = false))
{
var stuffToDo = await GetStuffToDo();
if(stuffToDo == null)
return;
await stuffToDo.Do();
}This is a practical application of the ScopedRunner used to measure the execution time of a piece of code. Internally it uses a ScopedRunner and a Stopwatch to do the measurement.
class TimeMeasurement : IDisposable
{
//Construct
TimeMeasurement(Action<TimeSpan> onDone)
Dispose()
}void LogDuration(TimeSpan duration) { [...] }
using(new TimeMeasurement(LogDuration))
{
await RunHeavyStuff();
}Questions, ideas, talks? Ping me on github.