Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Significant performance enhancement for ASIDownloadCache's -clearCachedResponsesForStoragePolicy: #347

Open
wants to merge 5 commits into from

1 participant

Mark Allen
Mark Allen

Instead of walking each file and deleting it on the current thread, we move the cache directory to a uniquely-named directory in the temp directory and delete its contents on a background thread. For whatever reason the iOS filesystem can perform a rename very quickly but deletes - especially for larger files, e.g. images - can be very slow (orders of magnitude slower) on old devices like the iPad 1.

This approach should be safe even if the background deletion thread is unable to complete normally (e.g. the application is unexpected terminated during execution) because we can rely on the system to clean up for us eventually since we're operating on files in the temp directory.

ASIHTTPRequest is a great library and I have used it to build some high-traffic apps. Unfortunately I was really bitten by the cache-clearing performance on older devices. Users were unable to start up the app due to being killed by the iOS watchdog since the deletes took so long to execute. In our app, this call might have taken 20 seconds in the pathological case (!) on an iPad 1. Now it takes about 100ms.

Another way to address this would be to allow client code to specify ASIDownloadCache size/item limits, with some kind of LRU eviction. But this was a quicker, less invasive win for me.

revetkn added some commits
Mark Allen revetkn Significant performance enhancement for -clearCachedResponsesForStora…
…gePolicy:

Instead of walking each file and deleting it on the current thread, we move the cache directory to a uniquely-named directory in the temp directory and delete its contents on a background thread.  For whatever reason the iOS filesystem can perform a rename very quickly but deletes - especially for larger files, e.g. images - can be very slow (orders of magnitude slower) on old devices like the iPad 1.

This approach should be safe even if the background deletion thread is unable to complete normally (e.g. the application is unexpected terminated during execution) because we can rely on the system to clean up for us eventually since we're operating on files in the temp directory.

ASIHTTPRequest is a great library and I have used it to build some high-traffic apps.  Unfortunately I was really bitten by the cache-clearing performance on older devices.  Users were unable to start up the app due to being killed by the iOS watchdog since the deletes took so long to execute.  In our app, this call might have taken 20 seconds in the pathological case (!) on an iPad 1.  Now it takes about 100ms.

Another way to address this would be to allow client code to specify ASIDownloadCache size/item limits, with some kind of LRU eviction.  But this was a quicker, less invasive win for me.
9aadd7c
Mark Allen revetkn NSStreamEventErrorOccurred -> NSStreamStatusError fix 93ca247
Mark Allen revetkn Deprecated NSDate -addTimeInterval: replaced with -dateByAddingTimeIn…
…terval:
3a1eac4
Mark Allen revetkn Fix for deprecated methods 7cf10bf
Mark Allen revetkn Comment out deprecated "ignore invalid certs" code since we don't use…
… it anyway
14d10ce
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Nov 19, 2012
  1. Mark Allen

    Significant performance enhancement for -clearCachedResponsesForStora…

    revetkn authored
    …gePolicy:
    
    Instead of walking each file and deleting it on the current thread, we move the cache directory to a uniquely-named directory in the temp directory and delete its contents on a background thread.  For whatever reason the iOS filesystem can perform a rename very quickly but deletes - especially for larger files, e.g. images - can be very slow (orders of magnitude slower) on old devices like the iPad 1.
    
    This approach should be safe even if the background deletion thread is unable to complete normally (e.g. the application is unexpected terminated during execution) because we can rely on the system to clean up for us eventually since we're operating on files in the temp directory.
    
    ASIHTTPRequest is a great library and I have used it to build some high-traffic apps.  Unfortunately I was really bitten by the cache-clearing performance on older devices.  Users were unable to start up the app due to being killed by the iOS watchdog since the deletes took so long to execute.  In our app, this call might have taken 20 seconds in the pathological case (!) on an iPad 1.  Now it takes about 100ms.
    
    Another way to address this would be to allow client code to specify ASIDownloadCache size/item limits, with some kind of LRU eviction.  But this was a quicker, less invasive win for me.
Commits on Sep 30, 2013
  1. Mark Allen
  2. Mark Allen
Commits on Oct 1, 2013
  1. Mark Allen

    Fix for deprecated methods

    revetkn authored
Commits on Oct 7, 2014
  1. Mark Allen
This page is out of date. Refresh to see the latest.
10 Classes/ASIAuthenticationDialog.m
View
@@ -217,9 +217,9 @@ - (UITextField *)domainField
+ (void)dismiss
{
if ([sharedDialog respondsToSelector:@selector(presentingViewController)])
- [[sharedDialog presentingViewController] dismissModalViewControllerAnimated:YES];
+ [[sharedDialog presentingViewController] dismissViewControllerAnimated:YES completion:^{}];
else
- [[sharedDialog parentViewController] dismissModalViewControllerAnimated:YES];
+ [[sharedDialog parentViewController] dismissViewControllerAnimated:YES completion:^{}];
}
- (void)viewDidDisappear:(BOOL)animated
@@ -237,9 +237,9 @@ - (void)dismiss
[[self class] dismiss];
} else {
if ([self respondsToSelector:@selector(presentingViewController)])
- [[self presentingViewController] dismissModalViewControllerAnimated:YES];
+ [[self presentingViewController] dismissViewControllerAnimated:YES completion:^{}];
else
- [[self parentViewController] dismissModalViewControllerAnimated:YES];
+ [[self parentViewController] dismissViewControllerAnimated:YES completion:^{}];
}
}
@@ -315,7 +315,7 @@ - (void)show
}
#endif
- [[self presentingController] presentModalViewController:self animated:YES];
+ [[self presentingController] presentViewController:self animated:YES completion:^{}];
}
#pragma mark button callbacks
4 Classes/ASIDataCompressor.m
View
@@ -161,7 +161,7 @@ + (BOOL)compressDataFromFile:(NSString *)sourcePath toFile:(NSString *)destinati
readLength = [inputStream read:inputData maxLength:DATA_CHUNK_SIZE];
// Make sure nothing went wrong
- if ([inputStream streamStatus] == NSStreamEventErrorOccurred) {
+ if ([inputStream streamStatus] == NSStreamStatusError) {
if (err) {
*err = [NSError errorWithDomain:NetworkRequestErrorDomain code:ASICompressionError userInfo:[NSDictionary dictionaryWithObjectsAndKeys:[NSString stringWithFormat:@"Compression of %@ failed because we were unable to read from the source data file",sourcePath],NSLocalizedDescriptionKey,[inputStream streamError],NSUnderlyingErrorKey,nil]];
}
@@ -187,7 +187,7 @@ + (BOOL)compressDataFromFile:(NSString *)sourcePath toFile:(NSString *)destinati
[outputStream write:(const uint8_t *)[outputData bytes] maxLength:[outputData length]];
// Make sure nothing went wrong
- if ([inputStream streamStatus] == NSStreamEventErrorOccurred) {
+ if ([inputStream streamStatus] == NSStreamStatusError) {
if (err) {
*err = [NSError errorWithDomain:NetworkRequestErrorDomain code:ASICompressionError userInfo:[NSDictionary dictionaryWithObjectsAndKeys:[NSString stringWithFormat:@"Compression of %@ failed because we were unable to write to the destination data file at %@",sourcePath,destinationPath],NSLocalizedDescriptionKey,[outputStream streamError],NSUnderlyingErrorKey,nil]];
}
4 Classes/ASIDataDecompressor.m
View
@@ -158,7 +158,7 @@ + (BOOL)uncompressDataFromFile:(NSString *)sourcePath toFile:(NSString *)destina
readLength = [inputStream read:inputData maxLength:DATA_CHUNK_SIZE];
// Make sure nothing went wrong
- if ([inputStream streamStatus] == NSStreamEventErrorOccurred) {
+ if ([inputStream streamStatus] == NSStreamStatusError) {
if (err) {
*err = [NSError errorWithDomain:NetworkRequestErrorDomain code:ASICompressionError userInfo:[NSDictionary dictionaryWithObjectsAndKeys:[NSString stringWithFormat:@"Decompression of %@ failed because we were unable to read from the source data file",sourcePath],NSLocalizedDescriptionKey,[inputStream streamError],NSUnderlyingErrorKey,nil]];
}
@@ -184,7 +184,7 @@ + (BOOL)uncompressDataFromFile:(NSString *)sourcePath toFile:(NSString *)destina
[outputStream write:(Bytef*)[outputData bytes] maxLength:[outputData length]];
// Make sure nothing went wrong
- if ([inputStream streamStatus] == NSStreamEventErrorOccurred) {
+ if ([inputStream streamStatus] == NSStreamStatusError) {
if (err) {
*err = [NSError errorWithDomain:NetworkRequestErrorDomain code:ASICompressionError userInfo:[NSDictionary dictionaryWithObjectsAndKeys:[NSString stringWithFormat:@"Decompression of %@ failed because we were unable to write to the destination data file at %@",sourcePath,destinationPath],NSLocalizedDescriptionKey,[outputStream streamError],NSUnderlyingErrorKey,nil]];
}
91 Classes/ASIDownloadCache.m
View
@@ -19,6 +19,7 @@
@interface ASIDownloadCache ()
+ (NSString *)keyForURL:(NSURL *)url;
- (NSString *)pathToFile:(NSString *)file;
+- (NSString *)generateUniqueIdentifier;
@end
@implementation ASIDownloadCache
@@ -393,35 +394,69 @@ - (void)setDefaultCachePolicy:(ASICachePolicy)cachePolicy
- (void)clearCachedResponsesForStoragePolicy:(ASICacheStoragePolicy)storagePolicy
{
- [[self accessLock] lock];
- if (![self storagePath]) {
- [[self accessLock] unlock];
- return;
- }
- NSString *path = [[self storagePath] stringByAppendingPathComponent:(storagePolicy == ASICacheForSessionDurationCacheStoragePolicy ? sessionCacheFolder : permanentCacheFolder)];
-
- NSFileManager *fileManager = [[[NSFileManager alloc] init] autorelease];
+ [[self accessLock] lock];
+ if (![self storagePath]) {
+ [[self accessLock] unlock];
+ return;
+ }
+ NSString *path = [[self storagePath] stringByAppendingPathComponent:(storagePolicy == ASICacheForSessionDurationCacheStoragePolicy ? sessionCacheFolder : permanentCacheFolder)];
+
+ NSFileManager *fileManager = [[[NSFileManager alloc] init] autorelease];
+
+ BOOL isDirectory = NO;
+ BOOL exists = [fileManager fileExistsAtPath:path isDirectory:&isDirectory];
+ if (!exists || !isDirectory) {
+ [[self accessLock] unlock];
+ return;
+ }
+
+ // It is significantly faster to perform a move than a delete on the iOS filesystem, especially on older devices.
+ // We move the existing cache directory so it lives in the temp directory and has a unique name.
+ // We clear the contents of the moved directory in a background thread.
+ NSString *renamedPath = [NSTemporaryDirectory() stringByAppendingPathComponent:[self generateUniqueIdentifier]];
+
+ NSError *error = nil;
+ BOOL renamed = [fileManager moveItemAtPath:path toPath:renamedPath error:&error];
+ if (!renamed) {
+ [[self accessLock] unlock];
+ [NSException raise:@"FailedToRenameCacheDirectory" format:@"Renaming cache directory failed at path '%@'",path];
+ }
+
+ BOOL recreated = [fileManager createDirectoryAtPath:path withIntermediateDirectories:YES attributes:nil error:&error];
+ if (!recreated) {
+ [[self accessLock] unlock];
+ [NSException raise:@"FailedToRecreateCacheDirectory" format:@"Recreating cache directory failed at path '%@'",path];
+ }
+
+ dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^(void) {
+ NSError *backgroundError = nil;
+ NSArray *cacheFiles = [fileManager contentsOfDirectoryAtPath:renamedPath error:&backgroundError];
+ if (backgroundError) {
+ [NSException raise:@"FailedToTraverseCacheDirectory" format:@"Listing cache directory failed at path '%@'",renamedPath];
+ }
+ for (NSString *file in cacheFiles) {
+ [fileManager removeItemAtPath:[renamedPath stringByAppendingPathComponent:file] error:&backgroundError];
+ if (backgroundError) {
+ [NSException raise:@"FailedToRemoveCacheFile" format:@"Failed to remove cached data at path '%@'",renamedPath];
+ }
+ }
+
+ // Remove the now-empty temporary directory
+ [fileManager removeItemAtPath:renamedPath error:&backgroundError];
+ if (backgroundError) {
+ [NSException raise:@"FailedToRemoveCacheDirectory" format:@"Failed to remove cached directory at path '%@'",renamedPath];
+ }
+ });
+
+ [[self accessLock] unlock];
+}
- BOOL isDirectory = NO;
- BOOL exists = [fileManager fileExistsAtPath:path isDirectory:&isDirectory];
- if (!exists || !isDirectory) {
- [[self accessLock] unlock];
- return;
- }
- NSError *error = nil;
- NSArray *cacheFiles = [fileManager contentsOfDirectoryAtPath:path error:&error];
- if (error) {
- [[self accessLock] unlock];
- [NSException raise:@"FailedToTraverseCacheDirectory" format:@"Listing cache directory failed at path '%@'",path];
- }
- for (NSString *file in cacheFiles) {
- [fileManager removeItemAtPath:[path stringByAppendingPathComponent:file] error:&error];
- if (error) {
- [[self accessLock] unlock];
- [NSException raise:@"FailedToRemoveCacheFile" format:@"Failed to remove cached data at path '%@'",path];
- }
- }
- [[self accessLock] unlock];
+- (NSString *)generateUniqueIdentifier
+{
+ CFUUIDRef uniqueIdentifier = CFUUIDCreate(NULL);
+ CFStringRef uniqueIdentifierString = CFUUIDCreateString(NULL, uniqueIdentifier);
+ CFRelease(uniqueIdentifier);
+ return [(NSString *)uniqueIdentifierString autorelease];
}
+ (BOOL)serverAllowsResponseCachingForRequest:(ASIHTTPRequest *)request
24 Classes/ASIHTTPRequest.m
View
@@ -1211,17 +1211,19 @@ - (void)startRequest
if (![self validatesSecureCertificate]) {
// see: http://iphonedevelopment.blogspot.com/2010/05/nsstream-tcp-and-ssl.html
- NSDictionary *sslProperties = [[NSDictionary alloc] initWithObjectsAndKeys:
- [NSNumber numberWithBool:YES], kCFStreamSSLAllowsExpiredCertificates,
- [NSNumber numberWithBool:YES], kCFStreamSSLAllowsAnyRoot,
- [NSNumber numberWithBool:NO], kCFStreamSSLValidatesCertificateChain,
- kCFNull,kCFStreamSSLPeerName,
- nil];
+ // This is deprecated and we don't need it for now
+
+ // NSDictionary *sslProperties = [[NSDictionary alloc] initWithObjectsAndKeys:
+ // [NSNumber numberWithBool:YES], kCFStreamSSLAllowsExpiredCertificates,
+ // [NSNumber numberWithBool:YES], kCFStreamSSLAllowsAnyRoot,
+ // [NSNumber numberWithBool:NO], kCFStreamSSLValidatesCertificateChain,
+ // kCFNull,kCFStreamSSLPeerName,
+ // nil];
- CFReadStreamSetProperty((CFReadStreamRef)[self readStream],
- kCFStreamPropertySSLSettings,
- (CFTypeRef)sslProperties);
- [sslProperties release];
+ // CFReadStreamSetProperty((CFReadStreamRef)[self readStream],
+ // kCFStreamPropertySSLSettings,
+ // (CFTypeRef)sslProperties);
+ // [sslProperties release];
}
// Tell CFNetwork to use a client certificate
@@ -4866,7 +4868,7 @@ + (NSDate *)expiryDateForRequest:(ASIHTTPRequest *)request maxAge:(NSTimeInterva
// RFC 2612 says max-age must override any Expires header
if (maxAge) {
- return [[NSDate date] addTimeInterval:maxAge];
+ return [[NSDate date] dateByAddingTimeInterval:maxAge];
} else {
NSString *expires = [responseHeaders objectForKey:@"Expires"];
if (expires) {
Something went wrong with that request. Please try again.