WarmupEngine. Resolves #207

README.md

@@ -20,6 +20,8 @@ StrongGrid includes a client that allows you to interact with all the "resources
 
 StrongGrid also includes a parser for webhook sent from SendGrid to your own WebAPI. This parser supports the two types of webhooks that SendGrid can post to your API: the Event Webhook and the Inbound Parse Webhook. 
 
+Since November 2017, StrongGrid also includes a "warmup engine" that allows you to warmup IP addresses using a custom schedule.
+
 If you information about how to setup the SendGrid webhooks, please consult the following resources:
 - [Webhooks Overview](https://sendgrid.com/docs/API_Reference/Webhooks/debug.html)
 - [Guide to debug webhooks](https://sendgrid.com/docs/API_Reference/Webhooks/index.html)
@@ -55,7 +57,7 @@ StrongGrid supports the `4.5.2` .NET framework as well as `.Net Core`.
 
 ## Usage
 
-#### Client
+### Client
 You declare your client variable like so:
 ```csharp
 var apiKey = "... your api key...";
@@ -92,7 +94,7 @@ var globalStats = await client.Statistics.GetGlobalStatisticsAsync(startDate, en
 var template = await client.Templates.CreateAsync("My template");
 ```
 
-#### Parser
+### Parser
 
 Here's a basic example of an API controller which parses the webhook from SendGrid into an array of Events:
 ```csharp
@@ -139,6 +141,72 @@ namespace WebApplication1.Controllers
 }
 ```
 
+### Warmup Engine
+
+SendGrid already provide a way to warm up ip addresses but you have no control over this process. StrongGrid solves this issue by providing you a warmup engine that you can tailor to your needs.
+
+#### Typical usage
+
+```csharp
+// Prepare the warmup engine
+var poolName = "warmup_pool";
+var dailyVolumePerIpAddress = new[] { 50, 100, 500, 1000 };
+var resetDays = 1; // Should be 1 if you send on a daily basis, should be 2 if you send every other day, should be 7 if you send on a weekly basis, etc.
+var warmupSettings = new WarmupSettings(poolName, dailyVolumePerIpAddress, resetDays);
+var warmupEngine = new WarmupEngine(warmupSettings, client);
+
+// This is a one-time call to create the IP pool that will be used to warmup the IP addresses
+var ipAddresses = new[] { "168.245.123.132", "168.245.123.133" };
+await warmupEngine.PrepareWithExistingIpAddressesAsync(ipAddresses, CancellationToken.None).ConfigureAwait(false);
+
+// Send emails using any othe following methods
+var result = warmupEngine.SendToSingleRecipientAsync(...);
+var result = warmupEngine.SendToMultipleRecipientsAsync(...);
+var result = warmupEngine.SendAsync(...);
+```
+
+The `Send...` methods return a `WarmupResult` object that will tell you whether the process is completed or not, and will also give you the messageId of the email sent using the IP pool (if applicable) and the messageId of the email sent using the default IP address (which is not being warmed up).
+The WarmupEngine will send emails using the IP pool until the daily volume limit is achieved and any remaining email will be sent using the default IP address.
+As you get closer and closer to your daily limit, it's possible that the Warmup engine may have to split a given "send" into two messages: one of which is sent using the ip pool and the other one sent using the default ip address.
+Let's use an example to illustrate: let's say that you have 15 emails left before you reach your daily warmup limit and you try to send an email to 20 recipients. In this scenario the first 15 emails will be sent using the warmup ip pool and the remaining 5 emails will be sent using the default ip address.
+
+#### More advanced usage
+
+**Recommended daily volume:** If you are unsure what daily limits to use, [SendGrid has provided a recommended schedule](https://sendgrid.com/docs/assets/IPWarmupSchedule.pdf) and StrongGrid provides a convenient method to use the recommended schedule tailored to number of emails you expect to send in a typical day.
+All you have to do is come up with a rough estimate of your daily volume and StrongGrid can configure the appropriate warmup settings.
+Here's an example:
+
+```csharp
+var poolName = "warmup_pool";
+var estimatedDailyVolume = 50000; // Should be your best guess: how many emails you will be sending in a typical day
+var resetDays = 1; // Should be 1 if you send on a daily basis, should be 2 if you send every other day, should be 7 if you send on a weekly basis, etc.
+var warmupSettings = WarmupSettings.FromSendGridRecomendedSettings(poolName, estimatedDailyVolume, resetDays);
+```
+
+**Progress repository:** By default StrongGrid's WarmupEngine will write progress information in a file on your computer's `temp` folder but you can override this settings. 
+First of all, you can change the folder where this file is saved. You can also decide to use a completely different repository. Out of the box, StringGrid provides `FileSystemWarmupProgressRepository` and `MemoryWarmupProgressRepository`.
+It also provide an interface called `IWarmupProgressRepository` which allows you to write your own implementation to save the progress data to a location more suitable to you.
+
+```csharp
+// You select one of the following repositories available out of the box:
+var warmupProgressRepository = new MemoryWarmupProgressRepository();
+var warmupProgressRepository = new FileSystemWarmupProgressRepository();
+var warmupProgressRepository = new FileSystemWarmupProgressRepository(@"C:\temp\myfolder\");
+var warmupEngine = new WarmupEngine(warmupSettings, client, warmupProgressRepository);
+```
+
+**Purchase new IP Addresses:** You can purchase new IP addresses using SendGrid' UI, but StrongGrid's WarmupEngine makes it even easier.
+Rather than invoking `PrepareWithExistingIpAddressesAsync` (as demonstrated previously), you can invoke `PrepareWithNewIpAddressesAsync` and StrongGrid will take care of adding new ip addresses to your account and add them to a new IP pool ready for warmup.
+As a reminder, please note that the `PrepareWithExistingIpAddressesAsync` and `PrepareWithNewIpAddressesAsync` can only be invoked once.
+Invoking either method a second time would result in an exception due to the fact that the IP pool has already been created.
+
+```csharp
+var howManyAddresses = 2; // How many ip addresses do you want to purchase?
+var subusers = new[] { "your_subuser" }; // The subusers you authorize to send emails on the new ip addresses
+await warmupEngine.PrepareWithNewIpAddressesAsync(howManyAddresses, subusers, CancellationToken.None).ConfigureAwait(false);
+```
+
+**End of warmup process:** when the process is completed, the IP pool is deleted and the warmed up IP address(es) are returned to the default pool. You can subsequently invoke the `client.Mail.SendAsync(...)` method to send your emails.
 
 ## License
 [![FOSSA Status](https://app.fossa.io/api/projects/git%2Bhttps%3A%2F%2Fgithub.com%2FJericho%2FStrongGrid.svg?type=large)](https://app.fossa.io/projects/git%2Bhttps%3A%2F%2Fgithub.com%2FJericho%2FStrongGrid?ref=badge_large)

Source/StrongGrid.IntegrationTests/Program.cs

@@ -980,57 +980,67 @@ private static async Task IpAddresses(IClient client, TextWriter log, Cancellati
 			var allIpAddresses = await client.IpAddresses.GetAllAsync(false, null, 10, 0, cancellationToken).ConfigureAwait(false);
 			await log.WriteLineAsync($"There are {allIpAddresses.Length} IP addresses on your account").ConfigureAwait(false);
 
-			/**************************************************
-				Commenting out the following tests because 
-				I do not have the necessary privileges
-			 **************************************************
+			// GET A SPECIFIC IP ADDRESS
+			if (allIpAddresses != null && allIpAddresses.Any())
+			{
+				var firstAddress = await client.IpAddresses.GetAsync(allIpAddresses.First().Address, cancellationToken).ConfigureAwait(false);
+				await log.WriteLineAsync($"IP address {firstAddress.Address} was retrieved").ConfigureAwait(false);
+			}
 
 			// GET THE WARMING UP IP ADDRESSES
 			var warmingup = await client.IpAddresses.GetWarmingUpAsync(cancellationToken).ConfigureAwait(false);
 			await log.WriteLineAsync($"There are {warmingup.Length} warming up IP addresses").ConfigureAwait(false);
 
+			// GET A SPECIFIC IP ADDRESS
+			if (warmingup != null && warmingup.Any())
+			{
+				var firstWarmingupAddress = await client.IpAddresses.GetAsync(warmingup.First().Address, cancellationToken).ConfigureAwait(false);
+				await log.WriteLineAsync($"There warmup status of {firstWarmingupAddress.Address} is {firstWarmingupAddress.Warmup}").ConfigureAwait(false);
+			}
+
 			// GET THE ASSIGNED IP ADDRESSES
 			var assigned = await client.IpAddresses.GetAssignedAsync(cancellationToken).ConfigureAwait(false);
 			await log.WriteLineAsync($"There are {assigned.Length} assigned IP addresses").ConfigureAwait(false);
 
+			// GET THE UNASSIGNED IP ADDRESSES
+			var unAssigned = await client.IpAddresses.GetUnassignedAsync(cancellationToken).ConfigureAwait(false);
+			await log.WriteLineAsync($"There are {unAssigned.Length} unassigned IP addresses").ConfigureAwait(false);
+
 			// GET THE REMAINING IP ADDRESSES
 			var remaining = await client.IpAddresses.GetRemainingCountAsync(cancellationToken).ConfigureAwait(false);
 			await log.WriteLineAsync($"You have {remaining.Remaining} remaining IP addresses for the {remaining.Period} at a cost of {remaining.PricePerIp}").ConfigureAwait(false);
-
-			**************************************************/
 		}
 
-		private static Task IpPools(IClient client, TextWriter log, CancellationToken cancellationToken)
+		private static async Task IpPools(IClient client, TextWriter log, CancellationToken cancellationToken)
 		{
-			/**************************************************
-				Commenting out the following tests because 
-				I do not have the necessary privileges
-			 **************************************************
-
 			await log.WriteLineAsync("\n***** IP POOLS *****\n").ConfigureAwait(false);
 
 			// GET ALL THE IP POOLS
 			var allIpPools = await client.IpPools.GetAllAsync(cancellationToken).ConfigureAwait(false);
 			await log.WriteLineAsync($"There are {allIpPools.Length} IP pools on your account").ConfigureAwait(false);
 
+			// CLEANUP PREVIOUS INTEGRATION TESTS THAT MIGHT HAVE BEEN INTERRUPTED BEFORE THEY HAD TIME TO CLEANUP AFTER THEMSELVES
+			foreach (var oldPool in allIpPools.Where(p => p.Name.StartsWith("StrongGrid Integration Testing:")))
+			{
+				await client.IpPools.DeleteAsync(oldPool.Name, cancellationToken).ConfigureAwait(false);
+				await log.WriteLineAsync($"Ip Pool {oldPool.Name} deleted").ConfigureAwait(false);
+			}
+
 			// CREATE A NEW POOL
-			var newPool = await client.IpPools.CreateAsync("mktg", cancellationToken).ConfigureAwait(false);
+			var newPool = await client.IpPools.CreateAsync("StrongGrid Integration Testing: new pool", cancellationToken).ConfigureAwait(false);
 			await log.WriteLineAsync($"New pool created: {newPool.Name}").ConfigureAwait(false);
 
 			// UPDATE THE IP POOL
-			await client.IpPools.UpdateAsync("mktg", "marketing", cancellationToken).ConfigureAwait(false);
+			await client.IpPools.UpdateAsync("StrongGrid Integration Testing: new pool", "StrongGrid Integration Testing: updated name", cancellationToken).ConfigureAwait(false);
 			await log.WriteLineAsync("New pool has been updated").ConfigureAwait(false);
 
 			// GET THE IP POOL
-			var marketingPool = await client.IpPools.GetAsync("marketing", cancellationToken).ConfigureAwait(false);
+			var marketingPool = await client.IpPools.GetAsync("StrongGrid Integration Testing: updated name", cancellationToken).ConfigureAwait(false);
 			await log.WriteLineAsync($"Retrieved pool '{marketingPool.Name}'").ConfigureAwait(false);
 
 			// DELETE THE IP POOL
 			await client.IpPools.DeleteAsync(marketingPool.Name, cancellationToken).ConfigureAwait(false);
 			await log.WriteLineAsync($"Deleted pool '{marketingPool.Name}'").ConfigureAwait(false);
-
-			**************************************************/
-			return Task.FromResult(0); // Success.
 		}
 
 		private static Task Subusers(IClient client, TextWriter log, CancellationToken cancellationToken)

Source/StrongGrid.UnitTests/MockSystemClock.cs

@@ -4,7 +4,7 @@
 
 namespace StrongGrid.UnitTests
 {
-	public class MockSystemClock : Mock<ISystemClock>
+	internal class MockSystemClock : Mock<ISystemClock>
 	{
 		public MockSystemClock(DateTime currentDateTime) :
 			this(currentDateTime.Year, currentDateTime.Month, currentDateTime.Day, currentDateTime.Hour, currentDateTime.Minute, currentDateTime.Second, currentDateTime.Millisecond)

Source/StrongGrid.UnitTests/Resources/IpAddressesTests.cs

@@ -111,18 +111,35 @@ public void Parse_multiple_json()
 		public async Task AddAsync()
 		{
 			// Arrange
+			var apiResponse = @"{
+				'ips': [
+					{
+						'ip': '1.2.3.4',
+						'subusers': [ 'jdesautels' ]
+					}
+				],
+				'remaining_ips':2,
+				'warmup': false
+			}";
+
 			var mockHttp = new MockHttpMessageHandler();
-			mockHttp.Expect(HttpMethod.Post, Utils.GetSendGridApiUri(ENDPOINT)).Respond(HttpStatusCode.OK);
+			mockHttp.Expect(HttpMethod.Post, Utils.GetSendGridApiUri(ENDPOINT)).Respond("application/json", apiResponse);
 
 			var client = Utils.GetFluentClient(mockHttp);
 			var ipAddresses = new IpAddresses(client);
 
 			// Act
-			await ipAddresses.AddAsync(2, new[] { "user", "subuser1" }, true, CancellationToken.None).ConfigureAwait(false);
+			var result = await ipAddresses.AddAsync(2, new[] { "user", "subuser1" }, true, CancellationToken.None).ConfigureAwait(false);
 
 			// Assert
 			mockHttp.VerifyNoOutstandingExpectation();
 			mockHttp.VerifyNoOutstandingRequest();
+			result.ShouldNotBeNull();
+			result.IpAddresses.ShouldNotBeNull();
+			result.IpAddresses.Length.ShouldBe(1);
+			result.IpAddresses[0].Address.ShouldBe("1.2.3.4");
+			result.RemainingIpAddresses.ShouldBe(2);
+			result.WarmingUp.ShouldBeFalse();
 		}
 
 		[Fact]
@@ -250,7 +267,7 @@ public async Task GetWarmUpStatusAsync()
 			var ipAddresses = new IpAddresses(client);
 
 			// Act
-			var result = await ipAddresses.GetWarmUpStatusAsync(address, CancellationToken.None).ConfigureAwait(false);
+			var result = await ipAddresses.GetWarmupStatusAsync(address, CancellationToken.None).ConfigureAwait(false);
 
 			// Assert
 			mockHttp.VerifyNoOutstandingExpectation();

Source/StrongGrid.UnitTests/Resources/MailTests.cs

@@ -31,8 +31,16 @@ public async Task SendAsync()
 			var client = Utils.GetFluentClient(mockHttp);
 			var mail = new Mail(client);
 
+			var personalizations = new[]
+			{
+				new MailPersonalization()
+				{
+					To = new[] { new MailAddress("bob@example.com", "Bob Smith") }
+				}
+			};
+
 			// Act
-			var result = await mail.SendAsync(null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, CancellationToken.None).ConfigureAwait(false);
+			var result = await mail.SendAsync(personalizations, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, CancellationToken.None).ConfigureAwait(false);
 
 			// Assert
 			mockHttp.VerifyNoOutstandingExpectation();
@@ -55,8 +63,16 @@ public async Task SendAsync_response_with_message_id()
 			var client = Utils.GetFluentClient(mockHttp);
 			var mail = new Mail(client);
 
+			var personalizations = new[]
+			{
+				new MailPersonalization()
+				{
+					To = new[] { new MailAddress("bob@example.com", "Bob Smith") }
+				}
+			};
+
 			// Act
-			var result = await mail.SendAsync(null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, CancellationToken.None).ConfigureAwait(false);
+			var result = await mail.SendAsync(personalizations, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, null, CancellationToken.None).ConfigureAwait(false);
 
 			// Assert
 			mockHttp.VerifyNoOutstandingExpectation();

Source/StrongGrid.UnitTests/StrongGrid.UnitTests.csproj

@@ -7,8 +7,8 @@
   </PropertyGroup>
 
   <ItemGroup>
-    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.3.0" />
-    <PackageReference Include="Moq" Version="4.7.142" />
+    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.5.0" />
+    <PackageReference Include="Moq" Version="4.7.145" />
     <PackageReference Include="RichardSzalay.MockHttp" Version="3.2.1" />
     <PackageReference Include="Shouldly" Version="2.8.3" />
     <PackageReference Include="xunit" Version="2.3.1" />