Performance tuning and other improvements (#73)

* Add zgip comression for the posts * Avoid creating StringBuilders for each field/tag, instead use the main one. * regression fix * Add MeasurementHierarchy to IInfluxDatabase Add SeriesCount and PointsCount properties to IInfluxMeasurement Fixes #69 * Add DropDatabaseAsync Allow for deleting/dropping the Influx database by calling `DropDatabaseAsync`. Fixes #67 * Add DropMeasurement and TestDeletePoints functions Fixes #41
AdysTech · Oct 12, 2019 · 221ec19 · 221ec19
1 parent cf574f6
commit 221ec19
Show file tree

Hide file tree

Showing 16 changed files with 703 additions and 184 deletions.
diff --git a/.vscode/settings.json b/.vscode/settings.json
@@ -0,0 +1,36 @@
+{
+    "cSpell.enabledLanguageIds": [
+        "asciidoc",
+        "c",
+        "cpp",
+        "csharp",
+        "css",
+        "git-commit",
+        "go",
+        "handlebars",
+        "haskell",
+        "html",
+        "jade",
+        "java",
+        "javascript",
+        "javascriptreact",
+        "json",
+        "jsonc",
+        "latex",
+        "less",
+        "markdown",
+        "php",
+        "plaintext",
+        "pug",
+        "python",
+        "restructuredtext",
+        "rust",
+        "scala",
+        "scss",
+        "text",
+        "typescript",
+        "typescriptreact",
+        "yaml",
+        "yml"
+    ]
+}
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,3 +1,17 @@
+## v0.9.0 [10/12/2019]
+
+### Release Notes
+Add  MeasurementHierarchy to IInfluxDatabase, SeriesCount and PointsCount properties to IInfluxMeasurement
+Now calling `GetInfluxDBStructureAsync` populates the structure with retention policies, and also gives the unique series and point counts for each of the measurements
+Allow for deleting/dropping the Influx database by calling `DropDatabaseAsync`, allow delete entire measurement via	`DropMeasurementAsync` as well as specific data points with and where clause using `TestDeletePointsAsync`
+
+### Bugfixes
+
+- [#72](https://github.com/AdysTech/InfluxDB.Client.Net/issues/72): Performance improvements
+- [#69](https://github.com/AdysTech/InfluxDB.Client.Net/issues/69): easiest way to count the entries in the measurement
+- [#67](https://github.com/AdysTech/InfluxDB.Client.Net/issues/67): delete an existing database
+- [#41](https://github.com/AdysTech/InfluxDB.Client.Net/issues/41): DELETE FROM {Measurement} WHERE
+
 ## v0.8.0 [3/14/2019]
 
 ### Release Notes

diff --git a/README.md b/README.md
@@ -1,71 +1,82 @@
 #### Build Status
+
 1. AppVeyor [![Build status](https://ci.appveyor.com/api/projects/status/ryj9u8dpcasu1xur?svg=true)](https://ci.appveyor.com/project/AdysTech/influxdb-client-net)
 2. Travis CI [![Build status](https://travis-ci.org/AdysTech/InfluxDB.Client.Net.svg?branch=master)](https://travis-ci.org/AdysTech/InfluxDB.Client.Net.svg?branch=master)
 
 **Now supports .Net Core, run same .Net code in Windows and Linux**
 
-####  Nuget
+#### Nuget
+
 1. .Net Standard 2.0 > [AdysTech.InfluxDB.Client.Net.Core](https://www.nuget.org/packages/AdysTech.InfluxDB.Client.Net.Core/) _(this is the preferred version)_
 2. .Net 4.5.1 > [AdysTech.InfluxDB.Client.Net](https://www.nuget.org/packages/AdysTech.InfluxDB.Client.Net)
 
-
 # InfluxDB.Client.Net
+
 [InfluxDB](http://influxdb.com) is new awesome open source time series database. But there is no official .Net client model for it. This is a feature rich .Net client for InfluxDB. All methods are exposed as Async methods, so that the calling thread will not be blocked. 
 It currently supports
 
-1.	Connecting using credentials
-2.	Querying all existing databases
-3.	Creating new database
-4.	Querying for the whole DB structure (hierarchical structure of all measurements, and fields)
-5.	Writing single, multiple values to db
-6.  Retention policy management
-7.  Post data to specific retention policies
-8.  Query for all Continuous Queries
-9.  Create a new Continuous Query
+1. Connecting using credentials
+2. Querying all existing databases
+3. Creating new database
+4. Querying for the whole DB structure (hierarchical structure of all measurements, and fields)
+5. Writing single, multiple values to db
+6. Retention policy management
+7. Post data to specific retention policies
+8. Query for all Continuous Queries
+9. Create a new Continuous Query
 10. Drop continuous queries
 11. Chunked queries
+12. Drop database
+13. Delete entire or partial measurement data
+14. Series and point count for each of the measurements
 
 To be added are
 
-1.	Query for all tags, their unique values
-2.	Deleting data, currently drop queries can be fired to delete data
+1. Query for all tags, their unique values
+
 
 
 ### Usage
+
 #### Creating Client
+
 ```Csharp
 InfluxDBClient client = new InfluxDBClient (influxUrl, dbUName, dbpwd);
 ```
 
 #### Querying all DB names
+
 ```Csharp
 List<String> dbNames = await client.GetInfluxDBNamesAsync ();
 ```
 
 #### Querying DB structure
+
 ```Csharp
 Dictionary<string, List<String>> = await client.GetInfluxDBStructureAsync("<db name>");
 ```
+
 This returns a hierarchical structure of all measurements, and fields (but not tags)!
 
 #### Create new database
+
 ```Csharp
 bool success = await client.CreateDatabaseAsync("<db name>");
 ```
-Creates a new database in InfluxDB. Does not raise exceptions if the DB already exists. 
 
+Creates a new database in InfluxDB. Does not raise exceptions if the DB already exists.
 
 #### Type safe data points
+
 ##### `InfluxDatapoint<T>`
 
 It represents a single Point (collection of fields) in a series. In InfluxDB each point is uniquely identified by its series and timestamps.
-Currently this class (as well as InfluxDB as of 0.9) supports `Boolean`, `String`, `Integer` and `Double` (additionally `decimal` is supported in client, which gets stored as a double in InfluxDB) types for field values. 
-Multiple fields of same type are supported, as well as tags. 
-
-
+Currently this class (as well as InfluxDB as of 0.9) supports `Boolean`, `String`, `Integer` and `Double` (additionally `decimal` is supported in client, which gets stored as a double in InfluxDB) types for field values.
+Multiple fields of same type are supported, as well as tags.
 
 #### Writing data points to DB
-```Csharp    
+
+```Csharp
 var valMixed = new InfluxDatapoint<InfluxValueField>();
 valMixed.UtcTimestamp = DateTime.UtcNow;
 valMixed.Tags.Add("TestDate", time.ToShortDateString());
@@ -74,58 +85,67 @@ valMixed.Fields.Add("Doublefield", new InfluxValueField(rand.NextDouble()));
 valMixed.Fields.Add("Stringfield", new InfluxValueField(DataGen.RandomString()));
 valMixed.Fields.Add("Boolfield", new InfluxValueField(true));
 valMixed.Fields.Add("Int Field", new InfluxValueField(rand.Next()));
-    
+
 valMixed.MeasurementName = measurementName;
 valMixed.Precision = TimePrecision.Seconds;
 valMixed.Retention = new InfluxRetentionPolicy() { Name = "Test2" };
-    
+
 var r = await client.PostPointAsync(dbName, valMixed);
 ```
+
 A collection of points can be posted using `await client.PostPointsAsync (dbName, points)`, where `points` can be collection of different types of `InfluxDatapoint`
-	
+
 #### Query for data points
+
 ```Csharp
 var r = await client.QueryMultiSeriesAsync ("_internal", "select * from runtime limit 10");
 var s = await client.QueryMultiSeriesAsync("_internal", "SHOW STATS");
 var rc = await client.QueryMultiSeriesAsync ("_internal", "select * from runtime limit 100",10);
 ```
+
 `QueryMultiSeriesAsync` method returns `List<InfluxSeries>`, `InfluxSeries` is a custom object which will have a series name, set of tags (e.g. columns you used in `group by` clause. For the actual values, it will use dynamic object(`ExpandoObject` to be exact). The example #1 above will result in a single entry list, and the result can be used like `r.FirstOrDefault()?.Entries[0].time`. This also opens up a way to have an update mechanism as you can now query for data, change some values/tags etc, and write back. Since Influx uses combination of timestamp, tags as primary key, if you don't change tags, the values will be overwritten.
 
 Second example above will provide multiple series objects, and allows to get data like `r.FirstOrDefault(x=>x.SeriesName=="queryExecutor").Entries[0].QueryDurationNs`.
 
 The last example above makes InfluxDB to split the selected points (100 limited by `limit` clause) to multiple series, each having 10 points as given by `chunk` size.
 
 #### Retention Policies
+
 This library uses a cutsom .Net object to represent the Influx Retention Policy. The `Duration` concept is nicely wraped in `TimeSpan`, so it can be easily manipulated using .Net code. It also supports `ShardDuration` concept introduced in recent versions of InfluxDB.
 
 ##### Get all Retention Policies from DB
+
 ```Csharp
 var policies = await client.GetRetentionPoliciesAsync (dbName);
 ```
 
 ##### Create Retention Policy and Write points to a specific retention policy
+
 The code below will create a new retention policy with a retention period of 6 hours, and write a point to that policy.
 
 ```Csharp
 var rp = new InfluxRetentionPolicy () { Name = "Test2", DBName = dbName, Duration = TimeSpan.FromHours (6), IsDefault = false };
 if (!await client.CreateRetentionPolicyAsync (rp))
     throw new InvalidOperationException ("Unable to create Retention Policy");
-              
+
 valMixed.MeasurementName = measurementName;
 valMixed.Precision = TimePrecision.Seconds;
-valMixed.Retention = new InfluxRetentionPolicy () { Name = "Test2"}; //or you can just assign this to rp   
+valMixed.Retention = new InfluxRetentionPolicy () { Name = "Test2"}; //or you can just assign this to rp
 var r = await client.PostPointAsync (dbName, valMixed);
 ```
 
 #### Continuous Queries
+
 Similar to retention policy the library also creates a custom object to represent the `Continuous Queries`. Because the queries can be very complex, the actual query part of the CQ is exposed as just a string. But the timing part of [CQ](https://docs.influxdata.com/influxdb/v1.0/query_language/continuous_queries) specifically <intervals> given in `[RESAMPLE [EVERY <interval>] [FOR <interval>]]` are exposed as `TimeSpan` objects. Since the `GROUP BY time(<interval>)` is mandated for CQs this interval again is exposed as `TimeSpan`. This allows you to run LINQ code on collection of CQs.
 
 ##### Get all Continuous Queries present in DB
+
 ```Csharp
 var cqList = await client.GetContinuousQueriesAsync ();
 ```
 
 ##### Create a new Continuous Query
+
 ```Csharp
 var p = new InfluxContinuousQuery () { Name = "TestCQ1",
                             DBName = dbName,
@@ -134,8 +154,42 @@ var p = new InfluxContinuousQuery () { Name = "TestCQ1",
                             ResampleFrequency = TimeSpan.FromHours (0.5) };
 var r = await client.CreateContinuousQueryAsync (p);
 ```
+
 ##### Drop a Continuous Query
+
 ```Csharp
 var r = await client.DropContinuousQueryAsync (p);
 ```
-`p` has to be an existing CQ, which is already saved. 
+
+`p` has to be an existing CQ, which is already saved.
+
+
+##### Drop a Database
+
+```Csharp
+var r = await client.DropDatabaseAsync (db);
+```
+
+`db` has to be an `InfluxDatabase` instance, which can be created via `new InfluxDatabase(dbName)`
+
+
+##### Drop a Measurement
+
+```Csharp
+var r = await client.DropMeasurementAsync (m);
+```
+
+`m` has to be an `InfluxMeasurement` instance, which can be created via `new InfluxMeasurement(measurementName)`
+
+##### Delete from Measurement with optional `where` clause
+
+```Csharp
+ r = await client.DeletePointsAsync(
+                    new InfluxDatabase(dbName), 
+                    new InfluxMeasurement(measurement), 
+                    whereClause: new List<string>() { 
+                        "purge = yes", 
+                        $"time() > {DateTime.UtcNow.AddDays(-4).ToEpoch(TimePrecision.Hours)}"
+                    });
+
+Currently `where` clause is just a list of conditions, without any intelligence. SO time conversion, quoting should be handled by the consumer.
diff --git a/src/AdysTech.InfluxDB.Client.Net.Core/AdysTech.InfluxDB.Client.Net.Core.csproj b/src/AdysTech.InfluxDB.Client.Net.Core/AdysTech.InfluxDB.Client.Net.Core.csproj
@@ -6,7 +6,7 @@
     <Product>AdysTech.InfluxDB.Client.Net</Product>
     <Company>AdysTech</Company>
     <Authors>AdysTech;mvadu</Authors>
-    <Version>0.8.0.0</Version>
+    <Version>0.9.0.0</Version>
     <PackageId>AdysTech.InfluxDB.Client.Net.Core</PackageId>
     <Copyright>© AdysTech 2016-2019</Copyright>
     <PackageProjectUrl>https://github.com/AdysTech/InfluxDB.Client.Net</PackageProjectUrl>
@@ -25,8 +25,8 @@ It currently supports
 8.  Query for all Continuous Queries 
 9.  Create/Drop Continuous Queries 
 10. Chunking Support in queries</PackageReleaseNotes>
-    <AssemblyVersion>0.8.0.0</AssemblyVersion>
-    <FileVersion>0.8.0.0</FileVersion>
+    <AssemblyVersion>0.9.0.0</AssemblyVersion>
+    <FileVersion>0.9.0.0</FileVersion>
     <PackageTags>InfluxDB Influx TSDB TimeSeries InfluxData Chunking retention RetentionPolicy</PackageTags>
   </PropertyGroup>
 

diff --git a/src/AdysTech.InfluxDB.Client.Net/Properties/AssemblyInfo.cs b/src/AdysTech.InfluxDB.Client.Net/Properties/AssemblyInfo.cs
@@ -31,5 +31,5 @@
 // You can specify all the values or you can default the Build and Revision Numbers 
 // by using the '*' as shown below:
 // [assembly: AssemblyVersion("1.0.*")]
-[assembly: AssemblyVersion("0.8.0.0")]
-[assembly: AssemblyFileVersion("0.8.0.0")]
+[assembly: AssemblyVersion("0.9.0.0")]
+[assembly: AssemblyFileVersion("0.9.0.0")]