atom.xml

<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
  <title><![CDATA[Mark S. Rasmussen]]></title>
  
  <link href="/atom.xml" rel="self"/>
  <link href="http://improve.dk/"/>
  <updated>2014-08-25T07:22:59.326Z</updated>
  <id>http://improve.dk/</id>
  
  <author>
    <name><![CDATA[Mark S. Rasmussen]]></name>
    <email><![CDATA[mark@improve.dk]]></email>
  </author>
  
  <generator uri="http://zespia.tw/hexo/">Hexo</generator>
  
  <entry>
    <title><![CDATA[New Horizons]]></title>
    <link href="http://improve.dk/new-horizons/"/>
    <id>http://improve.dk/new-horizons/</id>
    <published>2014-08-25T00:00:00.000Z</published>
    <updated>2014-08-25T07:22:22.000Z</updated>
    <content type="html"><![CDATA[<p>Over the last years I’ve been doing less and less coding while doing more and more management. As of today I’m taking it a step further as I’m assuming the role of CEO at <a href="(http://www.ipaper.dk">iPaper A/S</a>.</p>
<a id="more"></a>

<p><a href="http://www.ipaper-cms.com/pages/ipaper-as-appoints-new-ceo/" target="_blank">Official press release</a></p>
<p>While this does mean less coding at work, I won’t be going full manager. I’ll still be reverse engineering databases in my spare time, I’ll still be debugging interesting problems, just as I’ll still continue contributing to open source.</p>
<h2 id="Me,_Myself_&amp;_SQL_Server">Me, Myself &amp; SQL Server</h2>
<p>Though I won’t be giving up coding, this will have an impact on the amount of time I can spend on extracurricular activies, such as presenting. Unfortunately this also means I won’t be able to participate in neither the MVP nor the SQL PASS summits this year. I had really looked forward to joining the MVP summit for the first time, but unfortunately I will have to prioritize time differently for now.</p>
<p>This is <strong>not</strong> a goodbye to the family, simply an explanation for why I won’t be seeing you this November. </p>
]]></content>
    <summary type="html"><![CDATA[<p>Over the last years I’ve been doing less and less coding while doing more and more management. As of today I’m taking it a step further as I’m assuming the role of CEO at <a href="(http://www.ipaper.dk">iPaper A/S</a>.</p>
]]></summary>
    
      <category term="Misc" scheme="http://improve.dk/category/Misc/"/>
    
  </entry>
  
  <entry>
    <title><![CDATA[Presenting at Microsoft DevCon 2014]]></title>
    <link href="http://improve.dk/presenting-at-microsoft-devcon-2014/"/>
    <id>http://improve.dk/presenting-at-microsoft-devcon-2014/</id>
    <published>2014-05-18T13:27:00.000Z</published>
    <updated>2014-05-19T07:07:16.000Z</updated>
    <content type="html"><![CDATA[<p>I’m happy to announce that I’ll be presenting at <a href="http://www.msdevcon.ru/en/" target="_blank">Microsoft DevCon 2014</a> in Russia!</p>
<a id="more"></a>

<div class="imgwrapper" style=""><div><a href="/presenting-at-microsoft-devcon-2014/devcon.png" class="fancy"><img src="/presenting-at-microsoft-devcon-2014/devcon.png" style="max-height: 250px"/></a></div></div>

<p>While visiting Russia will be a first-time experience for me, the topic I’m presenting on is not. In just 30 minutes I will try to give an overview of not only how SQL Server stores data internally, but also how it keeps track of that data.</p>
<h2 id="Full_Abstract">Full Abstract</h2>
<blockquote>
<p><strong>Understanding SQL Server Data Files at the Byte Level</strong></p>
<p><em>Think SQL Server is magical? You’re right! However, there’s some sense to the magic, and that’s what I’ll show you in this level 500 deep dive session. I will walk you through the internal storage format of MDF files, how we might go about parsing a complete database ourselves, using nothing but a hex editor. I will cover how SQL Server stores its own internal metadata about objects, how it knows where to find your data on disk, and once it finds it, how to read it.</em></p>
</blockquote>
]]></content>
    <summary type="html"><![CDATA[<p>I’m happy to announce that I’ll be presenting at <a href="http://www.msdevcon.ru/en/" target="_blank">Microsoft DevCon 2014</a> in Russia!</p>
]]></summary>
    
      <category term="Conferences and Presenting" scheme="http://improve.dk/category/Conferences%20and%20Presenting/"/>
    
  </entry>
  
  <entry>
    <title><![CDATA[Redirecting Old Permalinks on Statically Generated Blogs]]></title>
    <link href="http://improve.dk/redirecting-old-permalinks-on-statically-generated-blogs/"/>
    <id>http://improve.dk/redirecting-old-permalinks-on-statically-generated-blogs/</id>
    <published>2014-04-20T23:56:12.000Z</published>
    <updated>2014-05-04T16:12:23.000Z</updated>
    <content type="html"><![CDATA[<p>Having <a href="/migrating-from-wordpress-to-hexo/">just migrated from Wordpress to Hexo</a>, I quickly realized I forgot something. I forgot to redirect my old permalinks to the new ones…</p>
<h2 id="Permalinks_Aren’t_Necessarily_Permanent">Permalinks Aren’t Necessarily Permanent</h2>
<p>A permalink ought to live for the duration of your content, and most importantly, never change. However, having been through a number of different blog engines, not all of them support the same permalink structures, and might not even support redirecting old ones. As such, throgh the years my posts have ended up with multiple permalinks:</p>
<ul>
<li><a href="http://improve.dk/archive/2008/03/23/sql-server-mirroring-a-practical-approach.aspx">http://improve.dk/archive/2008/03/23/sql-server-mirroring-a-practical-approach.aspx</a></li>
<li><a href="http://improve.dk/blog/2008/03/23/sql-server-mirroring-a-practical-approach/">http://improve.dk/blog/2008/03/23/sql-server-mirroring-a-practical-approach/</a></li>
<li><a href="http://improve.dk/sql-server-mirroring-a-practical-approach/">http://improve.dk/sql-server-mirroring-a-practical-approach/</a></li>
</ul>
<p>As you can see, I’ve dropped both the /archive/ and the /blog/ prefixes, as well as the dates. Redirecting old incoming links to the new ones was easy enough when I ran Wordpress on Apache. All it requires were a couple of lines in the .htaccess file:</p>
<figure class="highlight"><pre><span class="comment"># Redirect old permalink structure</span>
<span class="tag">&lt;IfModule mod_rewrite.c&gt;</span>
	<span class="keyword"><span class="common">RewriteEngine</span></span> <span class="literal">On</span>
	<span class="keyword"><span class="common">RewriteRule</span></span> ^archive/([0-9]{4})/([0-9]{2})/([0-9]{2})/([^\.]+)\.aspx$ http://improve.dk/<span class="number">$4</span>/<span class="sqbracket"> [NC,R=301,L]</span>
	<span class="keyword"><span class="common">RewriteRule</span></span> ^blog/([0-9]{4})/([0-9]{2})/([0-9]{2})/([^\.]+)$ http://improve.dk/<span class="number">$4</span>/<span class="sqbracket"> [NC,R=301,L]</span>
<span class="tag">&lt;/IfModule&gt;</span>
</pre></figure>

<h2 id="Static_Woes">Static Woes</h2>
<p>Since I’ve migrated to <a href="http://hexo.io" target="_blank">Hexo</a> it’s not as simple, unfortunately. I no longer host my site on Apache, but on <a href="https://pages.github.com/" target="_blank">GitHub Pages</a>. GitHub Pages only allow static files to be served, so I’m no longer able to utilize the .htaccess rewriting rules. There’s also no server-side functionality available, so I can’t even manually send out a 302-redirect, needed to preserve my incoming links SEO value.</p>
<p>What I ended up doing was to write a small script that would parse my Wordpress backup file and then recreate the /blog/ and /archive/ directories, as if the posts were actually stored there:</p>
<figure class="highlight cs"><pre><span class="keyword">string</span> template = <span class="string">@"layout: false
---
&lt;!DOCTYPE html&gt;
&lt;html&gt;
	&lt;head&gt;
		&lt;title&gt;Redirecting to [Title]&lt;/title&gt;
  		&lt;link rel=""canonical"" href=""[Permalink]""/&gt;
		&lt;meta http-equiv=""content-type"" content=""text/html; charset=utf-8"" /&gt;
		&lt;meta http-equiv=""refresh"" content=""0;url=[Permalink]"" /&gt;
	&lt;/head&gt;
	&lt;body&gt;
		Redirecting to &lt;a href=""[Permalink]""&gt;[Title]&lt;/a&gt;...
	&lt;/body&gt;
&lt;/html&gt;"</span>;

<span class="keyword">void</span> Main()
{
	<span class="keyword">var</span> outputPath = <span class="string">@"D:\Projects\improve.dk (GIT)\source\"</span>;
	<span class="keyword">var</span> xmlPath = <span class="string">@"D:\Projects\improve.dk (GIT)\marksrasmussen-blog.wordpress.2014-03-08.xml"</span>;
	<span class="keyword">var</span> xml = File.ReadAllText(xmlPath);
	<span class="keyword">var</span> xd = <span class="keyword">new</span> XmlDocument();
	xd.LoadXml(xml);
	
	<span class="keyword">var</span> nsmgr = <span class="keyword">new</span> XmlNamespaceManager(xd.NameTable);
	nsmgr.AddNamespace(<span class="string">"content"</span>, <span class="string">"http://purl.org/rss/1.0/modules/content/"</span>);
	nsmgr.AddNamespace(<span class="string">"wp"</span>, <span class="string">"http://wordpress.org/export/1.2/"</span>);
	
	<span class="keyword">foreach</span> (XmlNode item <span class="keyword">in</span> xd.SelectNodes(<span class="string">"//item"</span>))
	{
		<span class="keyword">var</span> title = item.SelectSingleNode(<span class="string">"title"</span>).InnerText;
		<span class="keyword">var</span> date = Convert.ToDateTime(item.SelectSingleNode(<span class="string">"pubDate"</span>).InnerText);
		<span class="keyword">var</span> slug = item.SelectSingleNode(<span class="string">"wp:post_name"</span>, nsmgr).InnerText;
		
		<span class="keyword">var</span> indexHtml = template
			.Replace(<span class="string">"[Permalink]"</span>, <span class="string">"http://improve.dk/"</span> + slug + <span class="string">"/"</span>)
			.Replace(<span class="string">"[Title]"</span>, HttpUtility.HtmlEncode(title));
		
		<span class="comment">// First create the /archive/ entry</span>
		<span class="keyword">var</span> outputFolder = Path.Combine(outputPath, <span class="string">"archive"</span>, date.Year.ToString(), date.Month.ToString().PadLeft(<span class="number">2</span>, <span class="string">'0'</span>), date.Day.ToString().PadLeft(<span class="number">2</span>, <span class="string">'0'</span>), slug + <span class="string">".aspx"</span>);
		<span class="keyword">var</span> indexPath = Path.Combine(outputFolder, <span class="string">"index.html"</span>);
		Directory.CreateDirectory(outputFolder);
		File.WriteAllText(indexPath, indexHtml);
		
		<span class="comment">// Then the /blog/ entry</span>
		outputFolder = Path.Combine(outputPath, <span class="string">"blog"</span>, date.Year.ToString(), date.Month.ToString().PadLeft(<span class="number">2</span>, <span class="string">'0'</span>), date.Day.ToString().PadLeft(<span class="number">2</span>, <span class="string">'0'</span>), slug);
		indexPath = Path.Combine(outputFolder, <span class="string">"index.html"</span>);
		Directory.CreateDirectory(outputFolder);
		File.WriteAllText(indexPath, indexHtml);
	}
}
</pre></figure>

<p>Now the post stored directly in the root, while placeholders have been put in place in the old /blog/ and /archive/ directories. The placeholder code is very simple:</p>
<figure class="highlight"><pre>layout: false
---
<span class="doctype">&lt;!DOCTYPE html&gt;</span>
<span class="tag">&lt;<span class="title">html</span>&gt;</span>
	<span class="tag">&lt;<span class="title">head</span>&gt;</span>
		<span class="tag">&lt;<span class="title">title</span>&gt;</span>Redirecting to TxF presentation materials<span class="tag">&lt;/<span class="title">title</span>&gt;</span>
  		<span class="tag">&lt;<span class="title">link</span> <span class="attribute">rel</span>=<span class="value">"canonical"</span> <span class="attribute">href</span>=<span class="value">"http://improve.dk/txf-presentation-materials/"</span>/&gt;</span>
		<span class="tag">&lt;<span class="title">meta</span> <span class="attribute">http-equiv</span>=<span class="value">"content-type"</span> <span class="attribute">content</span>=<span class="value">"text/html; charset=utf-8"</span> /&gt;</span>
		<span class="tag">&lt;<span class="title">meta</span> <span class="attribute">http-equiv</span>=<span class="value">"refresh"</span> <span class="attribute">content</span>=<span class="value">"0;url=http://improve.dk/txf-presentation-materials/"</span> /&gt;</span>
	<span class="tag">&lt;/<span class="title">head</span>&gt;</span>
	<span class="tag">&lt;<span class="title">body</span>&gt;</span>
		Redirecting to <span class="tag">&lt;<span class="title">a</span> <span class="attribute">href</span>=<span class="value">"http://improve.dk/txf-presentation-materials/"</span>&gt;</span>TxF presentation materials<span class="tag">&lt;/<span class="title">a</span>&gt;</span>...
	<span class="tag">&lt;/<span class="title">body</span>&gt;</span>
<span class="tag">&lt;/<span class="title">html</span>&gt;</span>
</pre></figure>

<p>It’s simply a small script that contains a meta refresh tag that sends the user on to the new URL. By utilizing the ´rel=”canonical”` meta tag, I ensure that this retains the SEO value as if I had performed a 302 redirect.</p>
<h2 id="Going_Forward">Going Forward</h2>
<p>Creating the placeholder files is a one-off task, seeing as I’ll only ever need to redirect posts that precede the time when I changed my URL structure to contain neither the /blog/ and /archive/ prefixes, nor the dates. All posts from the beginning of 2013 were published using the current URL scheme, which I intend to keep for the foreseeable future.</p>
]]></content>
    
    
      <category term="Miscellaneous" scheme="http://improve.dk/category/Miscellaneous/"/>
    
  </entry>
  
  <entry>
    <title><![CDATA[Migrating from Wordpress to Hexo]]></title>
    <link href="http://improve.dk/migrating-from-wordpress-to-hexo/"/>
    <id>http://improve.dk/migrating-from-wordpress-to-hexo/</id>
    <published>2014-04-19T00:00:00.000Z</published>
    <updated>2014-04-22T10:39:11.000Z</updated>
    <content type="html"><![CDATA[<p>It’s this time of the year again - the time to migrate from one blog engine to another.</p>
<a id="more"></a>

<p>About a year ago, I migrated from <a href="http://subtextproject.com/" target="_blank">Subtext</a> to <a href="http://wordpress.org/" target="_blank">Wordpress</a>. While I was initially happy, I still wasn’t completely satisfied with the workflow. My primary peeves were:</p>
<ul>
<li>Complexity - I had to pay a host to run a stack consisting of PHP and MySQL and keep it updated.</li>
<li>Security - I needed to constantly keep watch over Wordpress and keep it updated, seeing as it’s a popular target for mass defacements, etc.</li>
<li>Backups - While I did run an automated backup plugin, it was cumbersome as I needed an offsite location (i used FTP).</li>
<li>Writing - While the WYSIWYG editor works for some, it didn’t for me. As such I ended up writing all my posts in pure HTML.</li>
<li>Openness - I’m a big proponent of open source and while I did publish the source code for my <a href="https://github.com/improvedk/improve.dk_Wordpress" target="_blank">custom Wordpress theme</a>, I wanted to also open up my blog posts themselves.</li>
<li>Speed - I’ve spent more time than I’d like to, just keeping Wordpress running smoothly. A lot of things were outside of my control though, seeing as performance optimization was typically relegated to third party plugins.</li>
</ul>
<p>While considering the above list, I ended up settling on <a href="http://hexo.io" target="_blank">Hexo</a> - a static site generator powered by <a href="http://nodejs.org" target="_blank">Node.js</a>.</p>
<h2 id="Migration">Migration</h2>
<p>The migration process was simple enough, though it required some manual labor. All my Wordpress posts are written in HTML and since Hexo posts are based on Markdown, they needed to be converted. After dumping my old Wordpress site into a backup XML file, I was able to <a href="https://github.com/improvedk/improve.dk/blob/master/WP%20Conversion.linq" target="_blank">write a script</a> that parsed the backup XML file and converted each post into the Hexo Markdown format. There were some misses that required manual intervention, seeing as I had invalid HTML, special cases, etc. But overall, 95% of the posts were converted automatically.</p>
<p>Since Hexo is a static site generator, I needed to host my comments offsite. Thankfully <a href="http://disqus.com/" target="_blank">Disqus</a> has native support for the Wordpress comment backup format so importing the comments was a breeze.</p>
<p>Hexo does not support storing assets and posts in folders but prefers to store posts and assets seperately. As I like to keep them together (seeing as I’ve got close to 300 posts), I had to write a small script that copied the assets into the right output locations:</p>
<figure class="highlight js"><pre><span class="keyword">var</span> fs = <span class="built_in">require</span>(<span class="string">'fs'</span>);
<span class="keyword">var</span> path = <span class="built_in">require</span>(<span class="string">'path'</span>);
<span class="keyword">var</span> publicDir = hexo.public_dir;
<span class="keyword">var</span> sourceDir = hexo.source_dir;
<span class="keyword">var</span> postsDir = path.join(sourceDir, <span class="string">'_posts'</span>);
<span class="keyword">var</span> htmlTag = hexo.util.html_tag;
<span class="keyword">var</span> route = hexo.route;

<span class="comment">// Stores assets that'll need to be copied to the post output folders</span>
<span class="keyword">var</span> filesToCopy = [];

<span class="comment">// After Hexo's done generating, we'll copy post assets to their public folderse</span>
hexo.on(<span class="string">'generateAfter'</span>, <span class="function"><span class="keyword">function</span><span class="params">()</span> {</span>
	filesToCopy.forEach(<span class="function"><span class="keyword">function</span><span class="params">(obj)</span> {</span>
		fs.writeFileSync(obj.destination, fs.readFileSync(obj.source));
	});
});

<span class="comment">// Each time a post is rendered, note that we need to copy its assets</span>
hexo.extend.filter.register(<span class="string">'post'</span>, <span class="function"><span class="keyword">function</span><span class="params">(data, cb)</span> {</span>
	<span class="keyword">if</span> (data.slug) {
		<span class="keyword">var</span> postDir = path.join(postsDir, data.slug);
		<span class="keyword">var</span> files = fs.readdirSync(postDir);

		files.forEach(<span class="function"><span class="keyword">function</span><span class="params">(file)</span> {</span>
			<span class="comment">// Skip the markdown files themselves</span>
			<span class="keyword">if</span> (path.extname(file) == <span class="string">'.md'</span>)
				<span class="keyword">return</span>;

			<span class="keyword">var</span> outputDir = path.join(publicDir, data.slug);
			<span class="keyword">var</span> outputPath = path.join(publicDir, data.slug, file);
			<span class="keyword">var</span> inputPath = path.join(postDir, file);

			<span class="keyword">if</span> (!fs.existsSync(outputDir))
				fs.mkdirSync(path.join(outputDir));
			
			filesToCopy.push({ source: inputPath, destination: outputPath });
		});
	}

	cb();
});
</pre></figure>

<p>Though Hexo has a number of helpers to easily insert image links, I prefer to be able to just write an image name on a line by itself and then have the asset link inserted. Enabling that was easy enough too:</p>
<figure class="highlight js"><pre><span class="comment">// Replaces lines with image names with the actual image markup</span>
hexo.extend.filter.register(<span class="string">'pre'</span>, <span class="function"><span class="keyword">function</span><span class="params">(data, cb)</span> {</span>
	<span class="comment">// Find all matching image tags</span>
	<span class="keyword">var</span> regex = <span class="keyword">new</span> <span class="built_in">RegExp</span>(<span class="regexp">/^([a-z_0-9\-\.]+(?:.jpg|png|gif))(?: ([a-z]+)( \d+)?)?$/gim</span>);
	
	data.content = data.content.replace(regex, <span class="function"><span class="keyword">function</span><span class="params">(match, file, type, maxHeight)</span> {</span>
		<span class="comment">// Create image link</span>
		<span class="keyword">var</span> imgLink;
		<span class="keyword">if</span> (data.slug) <span class="comment">// Posts need to reference image absolutely</span>
			imgLink = <span class="string">'/'</span> + data.slug + <span class="string">'/'</span> + file;
		<span class="keyword">else</span>
			imgLink = file;

		<span class="comment">// Max height of image</span>
		<span class="keyword">var</span> imgMaxHeight = <span class="string">'250px'</span>;
		<span class="keyword">if</span> (maxHeight)
			imgMaxHeight = maxHeight + <span class="string">'px'</span>;

		<span class="comment">// Set style depending on type</span>
		<span class="keyword">var</span> style = <span class="string">''</span>;
		<span class="keyword">if</span> (type) {
			<span class="keyword">switch</span> (type) {
				<span class="keyword">case</span> <span class="string">'right'</span>:
					style = <span class="string">'float: right; margin: 20px'</span>;
					<span class="keyword">break</span>;

				<span class="keyword">case</span> <span class="string">'left'</span>:
					style = <span class="string">'float: left'</span>;
					<span class="keyword">break</span>;
			}
		}

		<span class="keyword">return</span> <span class="string">'&lt;div class="imgwrapper" style="'</span> + style + <span class="string">'"&gt;&lt;div&gt;&lt;a href="'</span> + imgLink + <span class="string">'" class="fancy"&gt;&lt;img src="'</span> + imgLink + <span class="string">'" style="max-height: '</span> + imgMaxHeight + <span class="string">'"/&gt;&lt;/a&gt;&lt;/div&gt;&lt;/div&gt;'</span>;
	});
	
	<span class="comment">// Let hexo continue</span>
	cb();
});
</pre></figure>

<h2 id="Hosting,_Security,_Backup_&amp;_Speed">Hosting, Security, Backup &amp; Speed</h2>
<p>Due to its static nature, there are no logins to protect, per se - seeing as there’s no backend. The blog itself is hosted on Github, both the <a href="https://github.com/improvedk/improve.dk" target="_blank">source</a> as well as the statically generated <a href="https://github.com/improvedk/improvedk.github.io" target="_blank">output files</a>. This means I’ve got full backup in the form of distributed git repositories, as well as very easy rollback in case of mistakes.</p>
<p>As for speed, it doesn’t get much faster than serving static files. Comments are lazily loaded after the post itself is loaded. While I can’t utilize the Github CDN (seeing as I’m hosting the blog at an apex domain, making it impossible for me to setup a CNAME - which is required to use the Github CDN), the speed is way faster than it used to be on Wordpress. I could move my DNS to a registrar that supports apex aliasing, but I’m happy with the speed for now.</p>
<h2 id="Openness">Openness</h2>
<p>Finally, since the source for the blog itself is hosted on Github, including the posts themselves, each post is actually editable directly on Github. You’ll notice that I’ve added an Octocat link at the bottom of each post, next to the social sharing icons. Clicking the Octocat will lead you directly to the source of the post you’re looking at. If you find an error or have a suggestion for an edit, feel free to fork the post and submit a pull request.</p>
]]></content>
    <summary type="html"><![CDATA[<p>It’s this time of the year again - the time to migrate from one blog engine to another.</p>
]]></summary>
    
      <category term="Miscellaneous" scheme="http://improve.dk/category/Miscellaneous/"/>
    
  </entry>
  
  <entry>
    <title><![CDATA[Presenting at SQL Saturday 275]]></title>
    <link href="http://improve.dk/presenting-sql-saturday-275/"/>
    <id>http://improve.dk/presenting-sql-saturday-275/</id>
    <published>2014-02-04T00:00:00.000Z</published>
    <updated>2014-04-22T10:39:11.000Z</updated>
    <content type="html"><![CDATA[<p>I’m happy to announce that I’ll be presenting at <a href="http://sqlsaturday.com/275/" target="_blank">SQLSaturday #275</a> in Copenhagen on March 29th!</p>
<a id="more"></a>

<p>I’ll be presenting my <strong>Recovering Data from Fatally Corrupt Databases</strong> session:</p>
<blockquote>
<p>Imagine the worst case scenario: Your database won’t come online. Lots of checksum errors logged. DBCC CheckDB won’t even run on the database. And worst of all - you have no backups! Now what do you do with this 20GB binary blob of an MDF file? In this demo-rich session I will briefly introduce the internals of MDF files while primarly concentrating on how to manually extract data from corrupt databases. I will be using the OrcaMDF RawDatabase framework to do most of the parsing, which will also be explained during the session.</p>
</blockquote>
<p>If you want to be able to <a href="/sql-server-corruption-recovery-when-all-else-fails/">save the day</a> when all other options are exhausted, you shouldn’t miss this session.</p>
]]></content>
    <summary type="html"><![CDATA[<p>I’m happy to announce that I’ll be presenting at <a href="http://sqlsaturday.com/275/" target="_blank">SQLSaturday #275</a> in Copenhagen on March 29th!</p>
]]></summary>
    
      <category term="SQL Server - Community" scheme="http://improve.dk/category/SQL%20Server%20-%20Community/"/>
    
      <category term="Conferences and Presenting" scheme="http://improve.dk/category/Conferences%20and%20Presenting/"/>
    
      <category term="SQL Server - OrcaMDF" scheme="http://improve.dk/category/SQL%20Server%20-%20OrcaMDF/"/>
    
  </entry>
  
  <entry>
    <title><![CDATA[SQL Server Corruption Recovery - When All Else Fails]]></title>
    <link href="http://improve.dk/sql-server-corruption-recovery-when-all-else-fails/"/>
    <id>http://improve.dk/sql-server-corruption-recovery-when-all-else-fails/</id>
    <published>2013-11-06T00:00:00.000Z</published>
    <updated>2014-05-04T16:12:23.000Z</updated>
    <content type="html"><![CDATA[<p>In this post I want to walk through a number of SQL Server corruption recovery techniques for when you’re out of luck, have no backups, and the usual methods don’t work. I’ll be using the <a href="http://msftdbprodsamples.codeplex.com/releases/view/93587" target="_blank">AdventureWorksLT2008R2 sample database</a> as my victim.</p>
<a id="more"></a>

<h2 id="A_Clean_Start">A Clean Start</h2>
<p>To start out, I’ve attached the downloaded database and it’s available on my SQL Server 2008 R2 instance, under the name of <strong>AWLT2008R2</strong>.</p>
<div class="imgwrapper" style=""><div><a href="/sql-server-corruption-recovery-when-all-else-fails/A9.png" class="fancy"><img src="/sql-server-corruption-recovery-when-all-else-fails/A9.png" style="max-height: 250px"/></a></div></div>

<p>To ensure we’ve got a clean start, I’ll run DBCC CHECKDB with the DATA_PURITY flag set, just to make sure the database is OK.</p>
<figure class="highlight sql"><pre>DBCC CHECKDB (AWLT2008R2) WITH ALL_ERRORMSGS, DATA_PURITY
</pre></figure>


<figure class="highlight"><pre>DBCC results <span class="keyword">for</span> <span class="string">'AWLT2008R2'</span>.
Service Broker Msg <span class="number">9675</span>, State <span class="number">1</span>: Message Types analyzed: <span class="number">14.</span>
Service Broker Msg <span class="number">9676</span>, State <span class="number">1</span>: Service Contracts analyzed: <span class="number">6.</span>
Service Broker Msg <span class="number">9667</span>, State <span class="number">1</span>: Services analyzed: <span class="number">3.</span>
Service Broker Msg <span class="number">9668</span>, State <span class="number">1</span>: Service Queues analyzed: <span class="number">3.</span>
Service Broker Msg <span class="number">9669</span>, State <span class="number">1</span>: Conversation Endpoints analyzed: <span class="number">0.</span>
Service Broker Msg <span class="number">9674</span>, State <span class="number">1</span>: Conversation Groups analyzed: <span class="number">0.</span>
Service Broker Msg <span class="number">9670</span>, State <span class="number">1</span>: Remote Service Bindings analyzed: <span class="number">0.</span>
Service Broker Msg <span class="number">9605</span>, State <span class="number">1</span>: Conversation Priorities analyzed: <span class="number">0.</span>
DBCC results <span class="keyword">for</span> <span class="string">'sys.sysrscols'</span>.
There are <span class="number">805</span> rows <span class="keyword">in</span> <span class="number">9</span> pages <span class="keyword">for</span> object <span class="string">"sys.sysrscols"</span>.
DBCC results <span class="keyword">for</span> <span class="string">'sys.sysrowsets'</span>.
There are <span class="number">125</span> rows <span class="keyword">in</span> <span class="number">1</span> pages <span class="keyword">for</span> object <span class="string">"sys.sysrowsets"</span>.
DBCC results <span class="keyword">for</span> <span class="string">'SalesLT.ProductDescription'</span>.
There are <span class="number">762</span> rows <span class="keyword">in</span> <span class="number">18</span> pages <span class="keyword">for</span> object <span class="string">"SalesLT.ProductDescription"</span>.
<span class="keyword">...</span>
CHECKDB found <span class="number">0</span> allocation errors and <span class="number">0</span> consistency errors <span class="keyword">in</span> database <span class="string">'AWLT2008R2'</span>.
DBCC execution completed. If DBCC printed error messages, contact your system administrator.
</pre></figure>

<h2 id="Enter_Corruption">Enter Corruption</h2>
<p>As I don’t want to kill my disk drives just to introduce corruption, I’ll be using <a href="/corrupting-databases-purpose-using-orcamdf-corruptor/">OrcaMDF’s Corruptor class</a> instead. First up we need to shut down SQL Server:</p>
<figure class="highlight sql"><pre>SHUTDOWN WITH NOWAIT
</pre></figure>


<figure class="highlight"><pre><span class="built_in">Server</span> shut down by NOWAIT <span class="built_in">request</span> from login MSR\Mark S. Rasmussen.
SQL <span class="built_in">Server</span> <span class="keyword">is</span> terminating this process.
</pre></figure>

<p>Once the instance has been shut down, I’ve located my MDF file, stored at <strong>D:\MSSQL Databases\AdventureWorksLT2008R2.mdf</strong>. Knowing the path to the MDF file, I’ll now intentially corrupt 5% of the pages in the database (at a database size of 5,312KB this will end up corrupting 33 random pages, out of a total of 664 pages).</p>
<figure class="highlight cs"><pre>Corruptor.CorruptFile(<span class="string">@"D:\MSSQL Databases\AdventureWorksLT2008R2.mdf"</span>, <span class="number">0.05</span>);
</pre></figure>

<p>At this point I have no idea about which pages were actually corrupted, I just know that 33 random pages just got overwritten by all zeros.</p>
<h2 id="Uh_Oh">Uh Oh</h2>
<p>After restarting the SQL Server instance and looking at the tree of databases, it’s obvious we’re in trouble…</p>
<div class="imgwrapper" style=""><div><a href="/sql-server-corruption-recovery-when-all-else-fails/A11.png" class="fancy"><img src="/sql-server-corruption-recovery-when-all-else-fails/A11.png" style="max-height: 250px"/></a></div></div>

<p>Running DBCC CHECKDB doesn’t help much:</p>
<figure class="highlight sql"><pre>DBCC CHECKDB (AWLT2008R2) WITH ALL_ERRORMSGS, DATA_PURITY
</pre></figure>


<figure class="highlight"><pre>Msg <span class="number">926</span>, Level <span class="number">14</span>, State <span class="number">1</span>, Line <span class="number">1</span>
Database <span class="string">'AWLT2008R2'</span> cannot be opened. It has been marked SUSPECT <span class="keyword">by</span> recovery.
See <span class="operator">the</span> SQL Server errorlog <span class="keyword">for</span> more information.
</pre></figure>

<p>What does the errorlog say?</p>
<ul>
<li>Starting up database ‘AWLT2008R2’.</li>
<li>1 transactions rolled forward in database ‘AWLT2008R2’ (13). This is an informational message only. No user action is required.</li>
<li>Error: 824, Severity: 24, State: 2.</li>
<li><strong>SQL Server detected a logical consistency-based I/O error</strong>: incorrect pageid (expected 1:2; actual 0:0). It occurred during a read of page (1:2) in database ID 13 at offset 0x00000000004000 in file ‘D:\MSSQL Databases\AdventureWorksLT2008R2.mdf’.  Additional messages in the SQL Server error log or system event log may provide more detail. <strong>This is a severe error condition that threatens database integrity and must be corrected immediately. Complete a full database consistency check (DBCC CHECKDB).</strong> This error can be caused by many factors; for more information, see SQL Server Books Online.</li>
<li>Error: 3414, Severity: 21, State: 1.</li>
<li><strong>An error occurred during recovery, preventing the database ‘AWLT2008R2’ (database ID 13) from restarting. Diagnose the recovery errors and fix them, or restore from a known good backup. If errors are not corrected or expected, contact Technical Support.</strong></li>
<li>CHECKDB for database ‘AWLT2008R2’ finished without errors on 2013-11-05 20:02:07.810 (local time). This is an informational message only; no user action is required.</li>
<li>Recovery is complete. This is an informational message only. No user action is required.</li>
</ul>
<p>This is officially not good. Our database failed to recover and can’t be put online at the moment, due to I/O consistency errors. We’ve also got our first hint:</p>
<figure class="highlight"><pre><span class="tag">incorrect</span> <span class="tag">pageid</span> (<span class="tag">expected</span> 1<span class="pseudo">:2</span>; <span class="tag">actual</span> 0<span class="pseudo">:0)</span>
</pre></figure>

<p>What this tells us is that the header of page 2 has been overwritten by zeros since SQL Server expected to find the value 1:2, but found 0:0 instead. Page 2 is the first GAM page in the database and is an essential part of the metadata.</p>
<p>SQL Server also wisely told us to either fix the errors or <strong>restore from a known good backup</strong>. And this is why you should always have a recovery strategy. If you ever end up in a situation like this, without a backup, you’ll have to continue reading.</p>
<h2 id="DBCC_CHECKDB">DBCC CHECKDB</h2>
<p>SQL Server recommended that we run a <strong>full database consistency check</strong> using DBCC CHECKDB. Unfortunately, given the state of our database, DBCC CHECKDB is unable to run:</p>
<figure class="highlight sql"><pre>DBCC CHECKDB (AWLT2008R2) WITH ALL_ERRORMSGS, DATA_PURITY
</pre></figure>


<figure class="highlight"><pre>Msg <span class="number">926</span>, Level <span class="number">14</span>, State <span class="number">1</span>, Line <span class="number">1</span>
Database <span class="string">'AWLT2008R2'</span> cannot be opened. It has been marked SUSPECT <span class="keyword">by</span> recovery.
See <span class="operator">the</span> SQL Server errorlog <span class="keyword">for</span> more information.
</pre></figure>

<p>In some cases you may be able to force the database online, by putting it into <strong>EMERGENCY</strong> mode. If we could get the database into EMERGENCY mode, we might just be able to run DBCC CHECKDB.</p>
<figure class="highlight sql"><pre><span class="operator"><span class="keyword">ALTER</span> <span class="keyword">DATABASE</span> AWLT2008R2 <span class="keyword">SET</span> EMERGENCY</span>
</pre></figure>


<figure class="highlight"><pre>Msg <span class="number">824</span>, Level <span class="number">24</span>, State <span class="number">2</span>, Line <span class="number">1</span>
SQL Server detected <span class="operator">a</span> logical consistency-based I/O error: incorrect pageid
(expected <span class="number">1</span>:<span class="number">16</span>; actual <span class="number">0</span>:<span class="number">0</span>). It occurred during <span class="operator">a</span> <span class="built_in">read</span> <span class="operator">of</span> page (<span class="number">1</span>:<span class="number">16</span>) <span class="operator">in</span> database
ID <span class="number">13</span> <span class="keyword">at</span> <span class="built_in">offset</span> <span class="number">0x00000000020000</span> <span class="operator">in</span> <span class="built_in">file</span> <span class="string">'D:\MSSQL Databases\AdventureWorksLT2008R2.mdf'</span>.
Additional messages <span class="operator">in</span> <span class="operator">the</span> SQL Server error <span class="built_in">log</span> <span class="operator">or</span> <span class="keyword">system</span> event <span class="built_in">log</span> may provide more
detail. This is <span class="operator">a</span> severe error condition that threatens database integrity <span class="operator">and</span> must
be corrected immediately. Complete <span class="operator">a</span> full database consistency check (DBCC CHECKDB).
This error can be caused <span class="keyword">by</span> many factors; <span class="keyword">for</span> more information, see SQL Server
Books Online.
</pre></figure>

<p>Even worse, it seems that page 16 has also been hit by corruption. Page 16 is the root page of the sysallocunits base table, holding all of the allocation unit storage metadata. Without page 16 there is no way for SQL Server to access any of its metadata. In short, there’s no way we’re getting this database online!</p>
<h2 id="Enter_OrcaMDF">Enter OrcaMDF</h2>
<p>The OrcaMDF Database class won’t be able to open the database, seeing as it does not handle corruption very well. Even so, I want to try anyway, you never know. First off you’ll have to shut down SQL Server to release the locks on the corrupt MDF file.</p>
<figure class="highlight sql"><pre>SHUTDOWN WITH NOWAIT
</pre></figure>

<p>If you then try opening the database using the OrcaMDF Database class, you’ll get a result like this:</p>
<figure class="highlight cs"><pre><span class="keyword">var</span> db = <span class="keyword">new</span> Database(<span class="string">@"D:\MSSQL Databases\AdventureWorksLT2008R2.mdf"</span>);
</pre></figure>

<div class="imgwrapper" style=""><div><a href="/sql-server-corruption-recovery-when-all-else-fails/Capture.png" class="fancy"><img src="/sql-server-corruption-recovery-when-all-else-fails/Capture.png" style="max-height: 250px"/></a></div></div>

<p>Interestingly the Database class didn’t puke on the boot page (ID 9) itself, so we know that that one’s OK, at least. But as soon as it hit page 16, things started to fall apart - and we already knew page 16 was corrupt.</p>
<h3 id="RawDatabase">RawDatabase</h3>
<p>While the OrcaMDF <strong>Database</strong> class can’t read the database file either, <strong>RawDatabase</strong> can. RawDatabase doesn’t care about metadata, it doesn’t read anything but what you tell it to, and as a result of that, it’s much more resilient to corruption.</p>
<p>Given that we know the corruption has resulted in pages being zeroed out, we could easily gather a list of corrupted pages by just searching for pages whose logical page ID doesn’t match the one in the header:</p>
<figure class="highlight cs"><pre><span class="keyword">var</span> db = <span class="keyword">new</span> RawDatabase(<span class="string">@"D:\MSSQL Databases\AdventureWorksLT2008R2.mdf"</span>)
db.Pages
  .Where(x =&gt; x.Header.PageID != x.PageID)
  .Select(x =&gt; x.PageID)
  .ToList()
  .ForEach(Console.WriteLine);
</pre></figure>


<figure class="highlight"><pre><span class="number">2</span>
<span class="number">4</span>
<span class="number">5</span>
<span class="number">16</span>
<span class="number">55</span>
<span class="keyword">...</span>
<span class="number">639</span>
<span class="number">649</span>
<span class="number">651</span>
<span class="number">662</span>
<span class="number">663</span>
</pre></figure>

<p>This is only possible since we know the corruption caused pages to be zeroed out, so you’ll rarely be this lucky. However, sometimes you may be able to detect the exact result of the corruption, thus enabling you to pinpoint the corrupted pages, just like we did here. However, this doesn’t really help us much - all we have now is a list of some page ID’s that are useless to us.</p>
<h3 id="Getting_a_List_of_Objects">Getting a List of Objects</h3>
<p>For this next part we’ll need a working database, any database, on an instance running the same version that our corrupted database this. This could be the master database - literally any working database. First you’ll want to connect to the database using the <a href="http://technet.microsoft.com/en-us/library/ms178068(v=sql.105" target="_blank">Dedicated Administrator Connection</a>.aspx). Connecting through the DAC allows us to query the base tables of the database.</p>
<p>The base table beneath sys.tables is called <strong>sys.sysschobjs</strong>, and if we can get to that, we can get a list of all the objects in the database, which might be a good start. Having connected to the working database, we can get the sys.sysschobjs details like so:</p>
<figure class="highlight sql"><pre><span class="operator"><span class="keyword">SELECT</span> * <span class="keyword">FROM</span> sys.sysschobjs <span class="keyword">WHERE</span> name = <span class="string">'sysschobjs'</span></span>
</pre></figure>

<div class="imgwrapper" style=""><div><a href="/sql-server-corruption-recovery-when-all-else-fails/Capture1.png" class="fancy"><img src="/sql-server-corruption-recovery-when-all-else-fails/Capture1.png" style="max-height: 250px"/></a></div></div>

<p>The only thing I’m looking for here is the object id, provided by the <strong>id</strong> column. In contrast to all user tables, the system tables have their actual object id stored in the page header, which allows us to easily query for pages by their id. Knowing sys.sysschobjs has ID <strong>34</strong>, let’s see if we can get a list of all the pages belonging to it (note that the .Dump() method is native to <a href="http://www.linqpad.net/" target="_blank">LinqPad</a> - all it does is to output the resulting objects as a table):</p>
<figure class="highlight cs"><pre><span class="keyword">var</span> db = <span class="keyword">new</span> RawDatabase(<span class="string">@"D:\MSSQL Databases\AdventureWorksLT2008R2.mdf"</span>);
db.Pages
  .Where(x =&gt; x.Header.ObjectID == <span class="number">34</span>)
  .Dump();
</pre></figure>

<div class="imgwrapper" style=""><div><a href="/sql-server-corruption-recovery-when-all-else-fails/Capture2.png" class="fancy"><img src="/sql-server-corruption-recovery-when-all-else-fails/Capture2.png" style="max-height: 250px"/></a></div></div>

<p>Now that we have a list of pages belonging to the sys.sysschobjs table, we need to retrieve the actual rows from there. Using <strong>sp_help</strong> on the working database, we can see the underlying schema of sys.sysschobjs:</p>
<figure class="highlight sql"><pre>sp_help 'sys.sysschobjs'
</pre></figure>

<div class="imgwrapper" style=""><div><a href="/sql-server-corruption-recovery-when-all-else-fails/Capture3.png" class="fancy"><img src="/sql-server-corruption-recovery-when-all-else-fails/Capture3.png" style="max-height: 250px"/></a></div></div>

<p>Once we have the schema of sys.sysschobjs, we can make RawDatabase parse the actual rows for us, after which we can filter it down to just the user tables, seeing as we don’t care about procedures, views, indexes and so forth:</p>
<figure class="highlight cs"><pre><span class="keyword">var</span> db = <span class="keyword">new</span> RawDatabase(<span class="string">@"D:\MSSQL Databases\AdventureWorksLT2008R2.mdf"</span>);
<span class="keyword">var</span> pages = db.Pages.Where(x =&gt; x.Header.ObjectID == <span class="number">34</span> && x.Header.Type == PageType.Data);
<span class="keyword">var</span> records = pages.SelectMany(x =&gt; x.Records).Select(x =&gt; (RawPrimaryRecord)x);
<span class="keyword">var</span> rows = RawColumnParser.Parse(records, <span class="keyword">new</span> IRawType[] {
	RawType.Int(<span class="string">"id"</span>),
	RawType.NVarchar(<span class="string">"name"</span>),
	RawType.Int(<span class="string">"nsid"</span>),
	RawType.TinyInt(<span class="string">"nsclass"</span>),
	RawType.Int(<span class="string">"status"</span>),
	RawType.Char(<span class="string">"type"</span>, <span class="number">2</span>),
	RawType.Int(<span class="string">"pid"</span>),
	RawType.TinyInt(<span class="string">"pclass"</span>),
	RawType.Int(<span class="string">"intprop"</span>),
	RawType.DateTime(<span class="string">"created"</span>),
	RawType.DateTime(<span class="string">"modified"</span>)
});

rows.Where(x =&gt; x[<span class="string">"type"</span>].ToString().Trim() == <span class="string">"U"</span>)
	.Select(x =&gt; <span class="keyword">new</span> {
		ObjectID = (<span class="keyword">int</span>)x[<span class="string">"id"</span>],
		Name = x[<span class="string">"name"</span>]
	}).Dump();
</pre></figure>

<div class="imgwrapper" style=""><div><a href="/sql-server-corruption-recovery-when-all-else-fails/Capture4.png" class="fancy"><img src="/sql-server-corruption-recovery-when-all-else-fails/Capture4.png" style="max-height: 250px"/></a></div></div>

<p>We just went from a completely useless suspect database, with no knowledge of the schema, to now having a list of each user table name &amp; object id. Sure, if one of the pages belonging to sys.syschobjs was corrupt, we’d be missing some of the tables without knowing it. Even so, this is a good start, and there are ways of detecting the missing pages (we could look for broken page header references, for example).</p>
<h3 id="Getting_Schemas">Getting Schemas</h3>
<p>As we saw for sys.sysschobjs, if we are to parse any of the user table data, we need to know the schema of the tables. The schema happens to be stored in the <strong>sys.syscolpars</strong> base table, and if we lookup in sys.sysschobjs for ‘sys.syscolpars’, we’ll get an object ID of <strong>41</strong>. As we did before, we can get a list of all pages belonging to sys.syscolpars:</p>
<figure class="highlight cs"><pre><span class="keyword">var</span> db = <span class="keyword">new</span> RawDatabase(<span class="string">@"D:\MSSQL Databases\AdventureWorksLT2008R2.mdf"</span>);
db.Pages
  .Where(x =&gt; x.Header.ObjectID == <span class="number">41</span>)
  .Dump();
</pre></figure>

<div class="imgwrapper" style=""><div><a href="/sql-server-corruption-recovery-when-all-else-fails/Capture5.png" class="fancy"><img src="/sql-server-corruption-recovery-when-all-else-fails/Capture5.png" style="max-height: 250px"/></a></div></div>

<p>By looking up the schema of sys.syscolpars using sp_help, in the working database, we can parse the actual rows much the same way:</p>
<figure class="highlight cs"><pre><span class="comment">// Parse sys.syscolpars</span>
<span class="keyword">var</span> db = <span class="keyword">new</span> RawDatabase(<span class="string">@"D:\MSSQL Databases\AdventureWorksLT2008R2.mdf"</span>);
<span class="keyword">var</span> pages = db.Pages.Where(x =&gt; x.Header.ObjectID == <span class="number">41</span> && x.Header.Type == PageType.Data);
<span class="keyword">var</span> records = pages.SelectMany(x =&gt; x.Records).Select(x =&gt; (RawPrimaryRecord)x);
<span class="keyword">var</span> rows = RawColumnParser.Parse(records, <span class="keyword">new</span> IRawType[] {
	RawType.Int(<span class="string">"id"</span>),
	RawType.SmallInt(<span class="string">"number"</span>),
	RawType.Int(<span class="string">"colid"</span>),
	RawType.NVarchar(<span class="string">"name"</span>),
	RawType.TinyInt(<span class="string">"xtype"</span>),
	RawType.Int(<span class="string">"utype"</span>),
	RawType.SmallInt(<span class="string">"length"</span>),
	RawType.TinyInt(<span class="string">"prec"</span>),
	RawType.TinyInt(<span class="string">"scale"</span>),
	RawType.Int(<span class="string">"collationid"</span>),
	RawType.Int(<span class="string">"status"</span>),
	RawType.SmallInt(<span class="string">"maxinrow"</span>),
	RawType.Int(<span class="string">"xmlns"</span>),
	RawType.Int(<span class="string">"dflt"</span>),
	RawType.Int(<span class="string">"chk"</span>),
	RawType.VarBinary(<span class="string">"idtval"</span>)
});

rows.Select(x =&gt; <span class="keyword">new</span> {
	ObjectID = (<span class="keyword">int</span>)x[<span class="string">"id"</span>],
	ColumnID = (<span class="keyword">int</span>)x[<span class="string">"colid"</span>],
	Number = (<span class="keyword">short</span>)x[<span class="string">"number"</span>],
	TypeID = (<span class="keyword">byte</span>)x[<span class="string">"xtype"</span>],
	Length = (<span class="keyword">short</span>)x[<span class="string">"length"</span>],
	Name = x[<span class="string">"name"</span>]
}).Dump();
</pre></figure>

<div class="imgwrapper" style=""><div><a href="/sql-server-corruption-recovery-when-all-else-fails/Capture6.png" class="fancy"><img src="/sql-server-corruption-recovery-when-all-else-fails/Capture6.png" style="max-height: 250px"/></a></div></div>

<h3 id="Recovering_the_Customer_Table_Schema">Recovering the Customer Table Schema</h3>
<p>While there are 12 tables, none are probably more important than the <strong>Customer</strong> table. Based on parsing the sys.sysschobjs base table, we know that the customer table has an object ID of <strong>117575457</strong>. Let’s try and filter down to just that object ID, using the code above:</p>
<figure class="highlight cs"><pre>rows.Where(x =&gt; (<span class="keyword">int</span>)x[<span class="string">"id"</span>] == <span class="number">117575457</span>).Select(x =&gt; <span class="keyword">new</span> {
	ObjectID = (<span class="keyword">int</span>)x[<span class="string">"id"</span>],
	ColumnID = (<span class="keyword">int</span>)x[<span class="string">"colid"</span>],
	Number = (<span class="keyword">short</span>)x[<span class="string">"number"</span>],
	TypeID = (<span class="keyword">byte</span>)x[<span class="string">"xtype"</span>],
	Length = (<span class="keyword">short</span>)x[<span class="string">"length"</span>],
	Name = x[<span class="string">"name"</span>]
}).OrderBy(x =&gt; x.Number).Dump();
</pre></figure>

<div class="imgwrapper" style=""><div><a href="/sql-server-corruption-recovery-when-all-else-fails/Capture7.png" class="fancy"><img src="/sql-server-corruption-recovery-when-all-else-fails/Capture7.png" style="max-height: 250px"/></a></div></div>

<p>Running the following query in any working database, we can correlate the TypeID values with the SQL Server type names:</p>
<figure class="highlight sql"><pre><span class="operator"><span class="keyword">SELECT</span>
	*
<span class="keyword">FROM</span>
	sys.types
<span class="keyword">WHERE</span>
	system_type_id <span class="keyword">IN</span> (<span class="number">56</span>, <span class="number">104</span>, <span class="number">231</span>, <span class="number">167</span>, <span class="number">36</span>, <span class="number">61</span>) <span class="keyword">AND</span>
	system_type_id = user_type_id</span>
</pre></figure>

<div class="imgwrapper" style=""><div><a href="/sql-server-corruption-recovery-when-all-else-fails/Capture8.png" class="fancy"><img src="/sql-server-corruption-recovery-when-all-else-fails/Capture8.png" style="max-height: 250px"/></a></div></div>

<p>Using the output from syscolpars and the type names, we can now deduce the schema of the Customer table (note that the syscolpars lengths are physical, meaning a length of 16 for an nvarchar column means a logical length of 8):</p>
<figure class="highlight sql"><pre><span class="operator"><span class="keyword">CREATE</span> <span class="keyword">TABLE</span> Customer (
	CustomerID <span class="keyword">int</span>,
	NameStyle <span class="keyword">bit</span>,
	Title nvarchar(<span class="number">8</span>),
	FirstName nvarchar(<span class="number">50</span>),
	MiddleName nvarchar(<span class="number">50</span>),
	LastName nvarchar(<span class="number">50</span>),
	Suffix nvarchar(<span class="number">10</span>),
	CompanyName nvarchar(<span class="number">128</span>),
	SalesPerson nvarchar(<span class="number">256</span>),
	EmailAddress nvarchar(<span class="number">50</span>),
	Phone nvarchar(<span class="number">25</span>),
	PasswordHash <span class="keyword">varchar</span>(<span class="number">128</span>),
	PasswordSalt <span class="keyword">varchar</span>(<span class="number">10</span>),
	rowguid uniqueidentifier,
	ModifiedDate datetime
)</span>
</pre></figure>

<p>All we need now is to find the pages belonging to the Customer table. That’s slightly easier said than done however. While each object has an object ID, as can be verified using sys.sysschobjs, that object ID is not what’s stored in the page headers, except for system objects. Thus we can’t just query for all pages whose Header.ObjectID == 117575457, as the value 117575457 won’t be stored in the header.</p>
<h3 id="Recovering_the_Customer_Allocation_Unit">Recovering the Customer Allocation Unit</h3>
<p>To find the pages belonging to the Customer table, we’ll first need to find the allocation unit to which it belongs. Unfortunately we already know that page 16 is corrupt - the first page of the <strong>sys.sysallocunits</strong> table, containing all of the metadata. However, we might just be lucky enough for that first page to contain the allocation units for all of the internal tables, which we do not care about. Let’s see if there are any other pages belonging to sys.sysallocunits:</p>
<figure class="highlight cs"><pre><span class="keyword">var</span> db = <span class="keyword">new</span> RawDatabase(<span class="string">@"D:\MSSQL Databases\AdventureWorksLT2008R2.mdf"</span>);
db.Pages
  .Where(x =&gt; x.Header.ObjectID == <span class="number">7</span>)
  .Dump();
</pre></figure>

<div class="imgwrapper" style=""><div><a href="/sql-server-corruption-recovery-when-all-else-fails/Capture9.png" class="fancy"><img src="/sql-server-corruption-recovery-when-all-else-fails/Capture9.png" style="max-height: 250px"/></a></div></div>

<p>There are 5 other pages available. Let’s try and parse them out so we have as much of the allocation unit data available, as possible. Once again we’ll get the schema from the working database, using sp_help, after which we can parse the remaining rows using RawDatabase. By looking up ‘sysallocunits’ in sysschobjs, we know it has an object ID of 7:</p>
<figure class="highlight cs"><pre><span class="keyword">var</span> db = <span class="keyword">new</span> RawDatabase(<span class="string">@"D:\MSSQL Databases\AdventureWorksLT2008R2.mdf"</span>);
<span class="keyword">var</span> pages = db.Pages.Where(x =&gt; x.Header.ObjectID == <span class="number">7</span> && x.Header.Type == PageType.Data);
<span class="keyword">var</span> records = pages.SelectMany(x =&gt; x.Records).Select(x =&gt; (RawPrimaryRecord)x);
<span class="keyword">var</span> rows = RawColumnParser.Parse(records, <span class="keyword">new</span> IRawType[] {
	RawType.BigInt(<span class="string">"auid"</span>),
	RawType.TinyInt(<span class="string">"type"</span>),
	RawType.BigInt(<span class="string">"ownerid"</span>),
	RawType.Int(<span class="string">"status"</span>),
	RawType.SmallInt(<span class="string">"fgid"</span>),
	RawType.Binary(<span class="string">"pgfirst"</span>, <span class="number">6</span>),
	RawType.Binary(<span class="string">"pgroot"</span>, <span class="number">6</span>),
	RawType.Binary(<span class="string">"pgfirstiam"</span>, <span class="number">6</span>),
	RawType.BigInt(<span class="string">"pcused"</span>),
	RawType.BigInt(<span class="string">"pcdata"</span>),
	RawType.BigInt(<span class="string">"pcreserved"</span>),
	RawType.Int(<span class="string">"dbfragid"</span>)
});

rows.Select(x =&gt; <span class="keyword">new</span> {
	AllocationUnitID = (<span class="keyword">long</span>)x[<span class="string">"auid"</span>],
	Type = (<span class="keyword">byte</span>)x[<span class="string">"type"</span>],
	ContainerID = (<span class="keyword">long</span>)x[<span class="string">"ownerid"</span>]
}).Dump();
</pre></figure>

<div class="imgwrapper" style=""><div><a href="/sql-server-corruption-recovery-when-all-else-fails/Capture10.png" class="fancy"><img src="/sql-server-corruption-recovery-when-all-else-fails/Capture10.png" style="max-height: 250px"/></a></div></div>

<p>By itself, we can’t use this data, but we’ll need it in just a moment. First we need to get a hold of the Customer table partitions as well. We do so by looking up the schema of <strong>sys.sysrowsets</strong> using sp_help, after which we can parse it. Looking up ‘sysrowsets’ in sysschobjs, we know that sys.sysrowsets has an object ID of 5:</p>
<figure class="highlight cs"><pre><span class="keyword">var</span> db = <span class="keyword">new</span> RawDatabase(<span class="string">@"D:\MSSQL Databases\AdventureWorksLT2008R2.mdf"</span>);
<span class="keyword">var</span> pages = db.Pages.Where(x =&gt; x.Header.ObjectID == <span class="number">5</span> && x.Header.Type == PageType.Data);
<span class="keyword">var</span> records = pages.SelectMany(x =&gt; x.Records).Select(x =&gt; (RawPrimaryRecord)x);
<span class="keyword">var</span> rows = RawColumnParser.Parse(records, <span class="keyword">new</span> IRawType[] {
	RawType.BigInt(<span class="string">"rowsetid"</span>),
	RawType.TinyInt(<span class="string">"ownertype"</span>),
	RawType.Int(<span class="string">"idmajor"</span>),
	RawType.Int(<span class="string">"idminor"</span>),
	RawType.Int(<span class="string">"numpart"</span>),
	RawType.Int(<span class="string">"status"</span>),
	RawType.SmallInt(<span class="string">"fgidfs"</span>),
	RawType.BigInt(<span class="string">"rcrows"</span>),
	RawType.TinyInt(<span class="string">"cmprlevel"</span>),
	RawType.TinyInt(<span class="string">"fillfact"</span>),
	RawType.SmallInt(<span class="string">"maxnullbit"</span>),
	RawType.Int(<span class="string">"maxleaf"</span>),
	RawType.SmallInt(<span class="string">"maxint"</span>),
	RawType.SmallInt(<span class="string">"minleaf"</span>),
	RawType.SmallInt(<span class="string">"minint"</span>),
	RawType.VarBinary(<span class="string">"rsguid"</span>),
	RawType.VarBinary(<span class="string">"lockres"</span>),
	RawType.Int(<span class="string">"dbfragid"</span>)
});

rows.Where(x =&gt; (<span class="keyword">int</span>)x[<span class="string">"idmajor"</span>] == <span class="number">117575457</span>).Select(x =&gt; <span class="keyword">new</span> {
	RowsetID = (<span class="keyword">long</span>)x[<span class="string">"rowsetid"</span>],
	ObjectID = (<span class="keyword">int</span>)x[<span class="string">"idmajor"</span>],
	IndexID = (<span class="keyword">int</span>)x[<span class="string">"idminor"</span>]
}).Dump();
</pre></figure>

<div class="imgwrapper" style=""><div><a href="/sql-server-corruption-recovery-when-all-else-fails/Capture11.png" class="fancy"><img src="/sql-server-corruption-recovery-when-all-else-fails/Capture11.png" style="max-height: 250px"/></a></div></div>

<p>By filtering down to just the Customer table’s object ID, we’ve now got the three partitions that belongs to the table - one for each allocation unit type - ROW_OVERFLOW_DATA (3), LOB_DATA (2) and IN_ROW_DATA (1). We don’t care about LOB and SLOB for now, all we need is the IN_ROW_DATA partition - giving us a RowsetID value of <strong>72057594039697408</strong>.</p>
<p>Now that we have the RowsetID, let’s lookup the allocation unit using the data we got from sys.sysallocunits earlier on:</p>
<figure class="highlight cs"><pre><span class="keyword">var</span> db = <span class="keyword">new</span> RawDatabase(<span class="string">@"D:\MSSQL Databases\AdventureWorksLT2008R2.mdf"</span>);
<span class="keyword">var</span> pages = db.Pages.Where(x =&gt; x.Header.ObjectID == <span class="number">7</span> && x.Header.Type == PageType.Data);
<span class="keyword">var</span> records = pages.SelectMany(x =&gt; x.Records).Select(x =&gt; (RawPrimaryRecord)x);
<span class="keyword">var</span> rows = RawColumnParser.Parse(records, <span class="keyword">new</span> IRawType[] {
	RawType.BigInt(<span class="string">"auid"</span>),
	RawType.TinyInt(<span class="string">"type"</span>),
	RawType.BigInt(<span class="string">"ownerid"</span>),
	RawType.Int(<span class="string">"status"</span>),
	RawType.SmallInt(<span class="string">"fgid"</span>),
	RawType.Binary(<span class="string">"pgfirst"</span>, <span class="number">6</span>),
	RawType.Binary(<span class="string">"pgroot"</span>, <span class="number">6</span>),
	RawType.Binary(<span class="string">"pgfirstiam"</span>, <span class="number">6</span>),
	RawType.BigInt(<span class="string">"pcused"</span>),
	RawType.BigInt(<span class="string">"pcdata"</span>),
	RawType.BigInt(<span class="string">"pcreserved"</span>),
	RawType.Int(<span class="string">"dbfragid"</span>)
});

rows.Where(x =&gt; (<span class="keyword">long</span>)x[<span class="string">"ownerid"</span>] == <span class="number">72057594039697408</span>).Select(x =&gt; <span class="keyword">new</span> {
	AllocationUnitID = (<span class="keyword">long</span>)x[<span class="string">"auid"</span>],
	Type = (<span class="keyword">byte</span>)x[<span class="string">"type"</span>],
	ContainerID = (<span class="keyword">long</span>)x[<span class="string">"ownerid"</span>]
}).Dump();
</pre></figure>

<div class="imgwrapper" style=""><div><a href="/sql-server-corruption-recovery-when-all-else-fails/Capture12.png" class="fancy"><img src="/sql-server-corruption-recovery-when-all-else-fails/Capture12.png" style="max-height: 250px"/></a></div></div>

<h3 id="Recovering_the_Customers">Recovering the Customers</h3>
<p>Now that we have the allocation unit ID, we can convert that into the object ID value, as stored in the page headers (big thanks goes out to <a href="http://www.sqlskills.com/blogs/paul/" target="_blank">Paul Randal</a> who was kind enough to blog about the <a href="http://www.sqlskills.com/blogs/paul/inside-the-storage-engine-how-are-allocation-unit-ids-calculated/" target="_blank">relationship between the allocation unit ID and the page header m_objId and m_indexId fields</a>):</p>
<figure class="highlight cs"><pre><span class="keyword">var</span> allocationUnitID = <span class="number">72057594041270272</span>;
<span class="keyword">var</span> indexID = allocationUnitID &gt;&gt; <span class="number">48</span>;
<span class="keyword">var</span> objectID = (allocationUnitID - (indexID &lt;&lt; <span class="number">48</span>)) &gt;&gt; <span class="number">16</span>;

Console.WriteLine(<span class="string">"IndexID: "</span> + indexID);
Console.WriteLine(<span class="string">"ObjectID: "</span> + objectID);
</pre></figure>


<figure class="highlight"><pre><span class="attribute">IndexID</span>: <span class="string">256</span>
<span class="attribute">ObjectID</span>: <span class="string">51</span>
</pre></figure>

<p>Now that we have not only the object ID, but also the index ID, we can easily get a list of all the pages belonging to the Customer table:</p>
<figure class="highlight cs"><pre><span class="keyword">var</span> db = <span class="keyword">new</span> RawDatabase(<span class="string">@"D:\MSSQL Databases\AdventureWorksLT2008R2.mdf"</span>);
db.Pages
  .Where(x =&gt; x.Header.ObjectID == <span class="number">51</span> && x.Header.IndexID == <span class="number">256</span>)
  .Dump();
</pre></figure>

<div class="imgwrapper" style=""><div><a href="/sql-server-corruption-recovery-when-all-else-fails/Capture13.png" class="fancy"><img src="/sql-server-corruption-recovery-when-all-else-fails/Capture13.png" style="max-height: 250px"/></a></div></div>

<p>And since we already know the schema for the Customer table, it’s a simple matter of making RawDatabase parse the actual rows:</p>
<figure class="highlight cs"><pre><span class="keyword">var</span> db = <span class="keyword">new</span> RawDatabase(<span class="string">@"D:\MSSQL Databases\AdventureWorksLT2008R2.mdf"</span>);
<span class="keyword">var</span> pages = db.Pages.Where(x =&gt; x.Header.ObjectID == <span class="number">51</span> && x.Header.IndexID == <span class="number">256</span> && x.Header.Type == PageType.Data);
<span class="keyword">var</span> records = pages.SelectMany(x =&gt; x.Records).Select(x =&gt; (RawPrimaryRecord)x);
<span class="keyword">var</span> rows = RawColumnParser.Parse(records, <span class="keyword">new</span> IRawType[] {
	RawType.Int(<span class="string">"CustomerID"</span>),
	RawType.Bit(<span class="string">"NameStyle"</span>),
	RawType.NVarchar(<span class="string">"Title"</span>),
	RawType.NVarchar(<span class="string">"FirstName"</span>),
	RawType.NVarchar(<span class="string">"MiddleName"</span>),
	RawType.NVarchar(<span class="string">"LastName"</span>),
	RawType.NVarchar(<span class="string">"Suffix"</span>),
	RawType.NVarchar(<span class="string">"CompanyName"</span>),
	RawType.NVarchar(<span class="string">"SalesPerson"</span>),
	RawType.NVarchar(<span class="string">"EmailAddress"</span>),
	RawType.NVarchar(<span class="string">"Phone"</span>),
	RawType.Varchar(<span class="string">"PasswordHash"</span>),
	RawType.Varchar(<span class="string">"PasswordSalt"</span>),
	RawType.UniqueIdentifier(<span class="string">"rowguid"</span>),
	RawType.DateTime(<span class="string">"ModifiedDate"</span>)
});

rows.Select(x =&gt; <span class="keyword">new</span> {
	CustomerID = (<span class="keyword">int</span>)x[<span class="string">"CustomerID"</span>],
	FirstName = (<span class="keyword">string</span>)x[<span class="string">"FirstName"</span>],
	MiddleName = (<span class="keyword">string</span>)x[<span class="string">"MiddleName"</span>],
	LastName = (<span class="keyword">string</span>)x[<span class="string">"LastName"</span>],
	CompanyName = (<span class="keyword">string</span>)x[<span class="string">"CompanyName"</span>],
	EmailAddress = (<span class="keyword">string</span>)x[<span class="string">"EmailAddress"</span>]
}).Dump();
</pre></figure>

<div class="imgwrapper" style=""><div><a href="/sql-server-corruption-recovery-when-all-else-fails/Capture15.png" class="fancy"><img src="/sql-server-corruption-recovery-when-all-else-fails/Capture15.png" style="max-height: 250px"/></a></div></div>

<p>And there we have it. 795 customers were just recovered from an otherwise unrecoverable state. Now it’s just a matter of repeating this process for the other tables as well.</p>
<h2 id="Summary">Summary</h2>
<p>As I’ve just shown, even though all hope seems lost, there are still options. If you know what you’re doing, a tool like OrcaMDF, or another homebrewn solution, might come in as an invaluable out, during a disaster. This is not, and should never be, a replacement for a good recovery strategy. That being said, not a week goes by without someone posting on a forum somewhere about a corrupt database without any backups.</p>
<p>In this case we went from fatal corruption to recovering 795 customers from the Customer table. Looking at the database, before it was corrupted, there was originally 847 customers in the table. Thus 52 customers were lost due to the corruption. If the pages really are hit by corruption, nothing will get that data back, unless you have a backup. However, if you’re unlucky and end up with metadata corruption, and/or a database that won’t come online, this may be a viable solution.</p>
<p>Should you come across a situation where OrcaMDF might come in handy, I’d love to hear about it - nothing better to hear than success stories! If you don’t feel like going through this process yourself, feel free to contact me; I may be able to help.</p>
]]></content>
    <summary type="html"><![CDATA[<p>In this post I want to walk through a number of SQL Server corruption recovery techniques for when you’re out of luck, have no backups, and the usual methods don’t work. I’ll be using the <a href="http://msftdbprodsamples.codeplex.com/releases/view/93587" target="_blank">AdventureWorksLT2008R2 sample database</a> as my victim.</p>
]]></summary>
    
      <category term=".NET" scheme="http://improve.dk/category/.NET/"/>
    
      <category term="SQL Server - Internals" scheme="http://improve.dk/category/SQL%20Server%20-%20Internals/"/>
    
      <category term="SQL Server - OrcaMDF" scheme="http://improve.dk/category/SQL%20Server%20-%20OrcaMDF/"/>
    
      <category term="SQL Server" scheme="http://improve.dk/category/SQL%20Server/"/>
    
  </entry>
  
  <entry>
    <title><![CDATA[Corrupting Databases on Purpose Using the OrcaMDF Corruptor]]></title>
    <link href="http://improve.dk/corrupting-databases-purpose-using-orcamdf-corruptor/"/>
    <id>http://improve.dk/corrupting-databases-purpose-using-orcamdf-corruptor/</id>
    <published>2013-11-05T00:00:00.000Z</published>
    <updated>2014-05-04T16:12:23.000Z</updated>
    <content type="html"><![CDATA[<p>Sometimes you must first do evil, to do good. Such is the case when you want to hone your skills in corruption recovery of SQL Server databases.</p>
<a id="more"></a>

<p>To give me more material to test the new <a href="/orcamdf-rawdatabase-a-swiss-army-knife-for-mdf-files/">RawDatabase</a> functionality, I’ve now added a <a href="https://github.com/improvedk/OrcaMDF/blob/master/src/OrcaMDF.Framework/Corruptor.cs" target="_blank">Corruptor class</a> to OrcaMDF. Corruptor does more or less what the name says - it corrupts database files on purpose.</p>
<p>The corruption itself is quite simple. Corruptor will choose a number of random pages and simply overwrite the page completely with all zeros. Depending on what pages are hit, this can be quite fatal.</p>
<p>I shouldn’t have to say this, but just in case… Please do not use this on anything valuable. <strong>It will fatally corrupt your data.</strong></p>
<h2 id="Examples">Examples</h2>
<p>There are two overloads for the Corruptor.CorruptFile method, both of them return an IEnumerable of integers - a list of the page IDs that have been overwritten by zeros.</p>
<p>The following code will corrupt 5% of the pages in the AdventureWorks2008R2LT.mdf file, after which it will output each page ID that has been corrupted. You can specify the percentage of pages to corrupt by changing the second parameter.</p>
<figure class="highlight cs"><pre><span class="keyword">var</span> corruptedPageIDs = Corruptor.CorruptFile(<span class="string">@"C:\AdventureWorks2008R2LT.mdf"</span>, <span class="number">0.05</span>);
Console.WriteLine(<span class="keyword">string</span>.Join(<span class="string">", "</span>, corruptedPageIDs));
</pre></figure>


<figure class="highlight"><pre>606, 516, 603, 521, 613, 621, 118, 47, 173, 579,
323, 217, 358, 515, 615, 271, 176, 596, 417, 379,
269, 409, 558, 103, 8, 636, 200, 361, 60, 486,
366, 99, 87
</pre></figure>

<p>To make the corruption hit even harder, you can also use the second overload of the CorruptFile method, allowing you to specify the exact number of pages to corrupt, within a certain range of page IDs. The following code will corrupt exactly 10 pages within the first 50 pages (zero-based), thus hitting mostly metadata.</p>
<figure class="highlight cs"><pre><span class="keyword">var</span> corruptedPageIDs = Corruptor.CorruptFile(<span class="string">@"C:\AdventureWorks2008R2LT.mdf"</span>, <span class="number">10</span>, <span class="number">0</span>, <span class="number">49</span>);
Console.WriteLine(<span class="keyword">string</span>.Join(<span class="string">", "</span>, corruptedPageIDs));
</pre></figure>


<figure class="highlight"><pre>16, 4, 0, 32, 15, 14, 30, 2, 49, 9
</pre></figure>

<p>In the above case I was extraordinarily unlucky seeing as page 0 is the file header page, page 2 is the first GAM page, page 9 is the boot page and finally page 16 is the page that contains the allocation unit metadata. With corruption like this, you can be certain that DBCC CHECKDB will be giving up, leaving you with no other alternative than to restore from a backup.</p>
<p>Or… You could try to recover as much data as possible using <a href="/orcamdf-rawdatabase-a-swiss-army-knife-for-mdf-files/">OrcaMDF RawDatabase</a>, but I’ll get back to that later :)</p>
]]></content>
    <summary type="html"><![CDATA[<p>Sometimes you must first do evil, to do good. Such is the case when you want to hone your skills in corruption recovery of SQL Server databases.</p>
]]></summary>
    
      <category term=".NET" scheme="http://improve.dk/category/.NET/"/>
    
      <category term="SQL Server - Internals" scheme="http://improve.dk/category/SQL%20Server%20-%20Internals/"/>
    
      <category term="SQL Server - OrcaMDF" scheme="http://improve.dk/category/SQL%20Server%20-%20OrcaMDF/"/>
    
      <category term="SQL Server" scheme="http://improve.dk/category/SQL%20Server/"/>
    
  </entry>
  
  <entry>
    <title><![CDATA[OrcaMDF RawDatabase - A Swiss Army Knife for MDF Files]]></title>
    <link href="http://improve.dk/orcamdf-rawdatabase-a-swiss-army-knife-for-mdf-files/"/>
    <id>http://improve.dk/orcamdf-rawdatabase-a-swiss-army-knife-for-mdf-files/</id>
    <published>2013-11-04T00:00:00.000Z</published>
    <updated>2014-05-04T16:12:23.000Z</updated>
    <content type="html"><![CDATA[<p>When I initially started working on <a href="/introducing-orcamdf/">OrcaMDF</a> I had just one goal, to gain a deeper knowledge of MDF file internals than I could through most books available.</p>
<a id="more"></a>

<p>As time progressed, so did OrcaMDF. While I had no initial plans of doing so, OrcaMDF has ended up being capable of parsing base tables, metadata and even <a href="/orcamdf-now-exposes-metadata-through-system-dmvs/">dynamically recreating common DMVs</a>. On top of this, I made a <a href="/orcamdf-studio-release-feature-recap/">simple GUI</a>, just to make OrcaMDF easier to use.</p>
<p>While that’s great, it comes at the price of extreme complexity. To be able to automatically parse table metadata like schemas, partitions, allocation units and more, not to mention abstracting away details like heaps and indexes, it takes a lot of code and it requires intimate knowledge of the database itself. Seeing as metadata changes between versions, OrcaMDF currently only supports SQL Server 2008 R2. While the data structures themselves are rather stable, there are minor differences in the way metadata is stored, the data exposed by DMVs and so forth. And on top of this, requiring all of the metadata to be perfect, for OrcaMDF to work, results in OrcaMDF being just as vulnerable to corruption as SQL Server is itself. Got a corrupt boot page? Neither SQL Server nor OrcaMDF will be able to parse the database.</p>
<h2 id="Say_Hello_to_RawDatabase">Say Hello to RawDatabase</h2>
<p>I tried to imagine the future of OrcaMDF and how to make it the most useful. I could march on make it support more and more of the same features that SQL Server does, eventually being able to parse 100% of an MDF file. But what would the value be? Sure, it would be a great learning opportunity, but the thing is, if you’ve got a working database, SQL Server does a pretty good job too. So what’s the alternative?</p>
<p><em>RawDatabase</em>, in contrast to the <em>Database</em> class, doesn’t try to parse anything besides what you tell it to. There’s no automatic parsing of schemas. It doesn’t know about base tables. It doesn’t know about DMVs. It does however know about the SQL Server data structures and it gives you an interface for working with the MDF file directly. Letting RawDatabase parse nothing but the data structures means it’s significantly less vulnerable to corruption or bad data.</p>
<h2 id="Examples">Examples</h2>
<p>It’s still early in the development, but let me show some examples of what can be done using RawDatabase. While I’m running the code in <a href="http://www.linqpad.net/" target="_blank">LINQPad</a>, as that makes it easy to show the results, the result are just standard .NET objects. All examples are run against the AdventureWorks 2008R2 LT (Light Weight) database.</p>
<h3 id="Getting_a_Single_Page">Getting a Single Page</h3>
<p>In the most basic example, we’ll parse just a single page.</p>
<figure class="highlight cs"><pre><span class="comment">// Get page 197 in file 1</span>
<span class="keyword">var</span> db = <span class="keyword">new</span> RawDatabase(<span class="string">@"C:\AWLT2008R2.mdf"</span>);
db.GetPage(<span class="number">1</span>, <span class="number">197</span>).Dump();
</pre></figure>

<div class="imgwrapper" style=""><div><a href="/orcamdf-rawdatabase-a-swiss-army-knife-for-mdf-files/A.png" class="fancy"><img src="/orcamdf-rawdatabase-a-swiss-army-knife-for-mdf-files/A.png" style="max-height: 250px"/></a></div></div>

<h3 id="Parsing_the_Page_Header">Parsing the Page Header</h3>
<p>Now that we’ve got a page, how about we dump the header values?</p>
<figure class="highlight cs"><pre><span class="comment">// Get the header of page 197 in file 1</span>
<span class="keyword">var</span> db = <span class="keyword">new</span> RawDatabase(<span class="string">@"C:\AWLT2008R2.mdf"</span>);
db.GetPage(<span class="number">1</span>, <span class="number">197</span>).Header.Dump();
</pre></figure>

<div class="imgwrapper" style=""><div><a href="/orcamdf-rawdatabase-a-swiss-army-knife-for-mdf-files/A1.png" class="fancy"><img src="/orcamdf-rawdatabase-a-swiss-army-knife-for-mdf-files/A1.png" style="max-height: 250px"/></a></div></div>

<h3 id="Parsing_the_Slot_Array">Parsing the Slot Array</h3>
<p>Just as the header is available, you can also get the raw slot array entries.</p>
<figure class="highlight cs"><pre><span class="comment">// Get the slot array entries of page 197 in file 1</span>
<span class="keyword">var</span> db = <span class="keyword">new</span> RawDatabase(<span class="string">@"C:\AWLT2008R2.mdf"</span>);
db.GetPage(<span class="number">1</span>, <span class="number">197</span>).SlotArray.Dump();
</pre></figure>

<div class="imgwrapper" style=""><div><a href="/orcamdf-rawdatabase-a-swiss-army-knife-for-mdf-files/A2.png" class="fancy"><img src="/orcamdf-rawdatabase-a-swiss-army-knife-for-mdf-files/A2.png" style="max-height: 250px"/></a></div></div>

<h3 id="Parsing_Records">Parsing Records</h3>
<p>While getting the raw slot array entries can be useful, you’ll usually want to look at the records themselves. Fortunately, that’s easy to do too.</p>
<figure class="highlight cs"><pre><span class="comment">// Get all records on page 197 in file 1</span>
<span class="keyword">var</span> db = <span class="keyword">new</span> RawDatabase(<span class="string">@"C:\AWLT2008R2.mdf"</span>);
db.GetPage(<span class="number">1</span>, <span class="number">197</span>).Records.Dump();
</pre></figure>

<div class="imgwrapper" style=""><div><a href="/orcamdf-rawdatabase-a-swiss-army-knife-for-mdf-files/A3.png" class="fancy"><img src="/orcamdf-rawdatabase-a-swiss-army-knife-for-mdf-files/A3.png" style="max-height: 250px"/></a></div></div>

<h3 id="Retrieving_Data_from_Records">Retrieving Data from Records</h3>
<p>Once you’ve got the records, you could now access the FixedLengthData or the VariableLengthOffsetValues properties to get the raw fixed length and variable length column values. However, what you’ll typically want is to get the actually parsed values. To spare you the work, OrcaMDF can parse it for you, if you just provide it the schema.</p>
<figure class="highlight cs"><pre><span class="comment">// Read the record contents of the first record on page 197 of file 1</span>
<span class="keyword">var</span> db = <span class="keyword">new</span> RawDatabase(<span class="string">@"C:\AWLT2008R2.mdf"</span>);
RawPrimaryRecord firstRecord = (RawPrimaryRecord)db.GetPage(<span class="number">1</span>, <span class="number">197</span>).Records.First();

<span class="keyword">var</span> values = RawColumnParser.Parse(firstRecord, <span class="keyword">new</span> IRawType[] {
	RawType.Int(<span class="string">"AddressID"</span>),
	RawType.NVarchar(<span class="string">"AddressLine1"</span>),
	RawType.NVarchar(<span class="string">"AddressLine2"</span>),
	RawType.NVarchar(<span class="string">"City"</span>),
	RawType.NVarchar(<span class="string">"StateProvince"</span>),
	RawType.NVarchar(<span class="string">"CountryRegion"</span>),
	RawType.NVarchar(<span class="string">"PostalCode"</span>),
	RawType.UniqueIdentifier(<span class="string">"rowguid"</span>),
	RawType.DateTime(<span class="string">"ModifiedDate"</span>)
});
	
values.Dump();
</pre></figure>

<div class="imgwrapper" style=""><div><a href="/orcamdf-rawdatabase-a-swiss-army-knife-for-mdf-files/A4.png" class="fancy"><img src="/orcamdf-rawdatabase-a-swiss-army-knife-for-mdf-files/A4.png" style="max-height: 250px"/></a></div></div>

<p>RawColumnParser.Parse will, given a schema, automatically convert the raw bytes into a Dictionary&lt;string, object&gt;, the key being the column name from the schema and the value being the actual type of the column, e.g. int, short, Guid, string, etc. By letting you, the user, specify the schema, OrcaMDF can get rid of a slew of dependencies on metadata, thus ignoring any possible corruption in metadata. Given the availability of the Next &amp; PreviousPageID properties of the header, it would be simple to iterate through all linked pages, parsing all records of each page - basically performing a scan on a given allocation unit.</p>
<h3 id="Filtering_Pages">Filtering Pages</h3>
<p>Besides retrieving a specific page, RawDatabase also has a Pages property that enumerates over all pages in a database. Using this you could, for example, get a list of all IAM pages in the database.</p>
<figure class="highlight cs"><pre><span class="comment">// Get a list of all IAM pages in the database</span>
<span class="keyword">var</span> db = <span class="keyword">new</span> RawDatabase(<span class="string">@"C:\AWLT2008R2.mdf"</span>);
db.Pages
	.Where(x =&gt; x.Header.Type == PageType.IAM)
	.Dump();
</pre></figure>

<div class="imgwrapper" style=""><div><a href="/orcamdf-rawdatabase-a-swiss-army-knife-for-mdf-files/A5.png" class="fancy"><img src="/orcamdf-rawdatabase-a-swiss-army-knife-for-mdf-files/A5.png" style="max-height: 250px"/></a></div></div>

<p>And since this is powered by LINQ, it’s easy to project just the properties you want. For example, you could get all index pages and their slot counts like this:</p>
<figure class="highlight cs"><pre><span class="comment">// Get all index pages and their slot counts</span>
<span class="keyword">var</span> db = <span class="keyword">new</span> RawDatabase(<span class="string">@"C:\AWLT2008R2.mdf"</span>);
db.Pages
	.Where(x =&gt; x.Header.Type == PageType.Index)
	.Select(x =&gt; <span class="keyword">new</span> {
		x.PageID,
		x.Header.SlotCnt
	}).Dump();
</pre></figure>

<div class="imgwrapper" style=""><div><a href="/orcamdf-rawdatabase-a-swiss-army-knife-for-mdf-files/A6.png" class="fancy"><img src="/orcamdf-rawdatabase-a-swiss-army-knife-for-mdf-files/A6.png" style="max-height: 250px"/></a></div></div>

<p>Or let’s say you wanted to get all data pages with at least one record and more than 7000 bytes of free space - with the page id, free count, record count and average record size as the output:</p>
<figure class="highlight cs"><pre><span class="keyword">var</span> db = <span class="keyword">new</span> RawDatabase(<span class="string">@"C:\AWLT2008R2.mdf"</span>);
db.Pages
	.Where(x =&gt; x.Header.FreeCnt &gt; <span class="number">7000</span>)
	.Where(x =&gt; x.Header.SlotCnt &gt;= <span class="number">1</span>)
	.Where(x =&gt; x.Header.Type == PageType.Data)
	.Select(x =&gt; <span class="keyword">new</span> {
	    x.PageID,
		x.Header.FreeCnt,
		RecordCount = x.Records.Count(),
		RecordSize = (<span class="number">8096</span> - x.Header.FreeCnt) / x.Records.Count()
	}).Dump();
</pre></figure>

<div class="imgwrapper" style=""><div><a href="/orcamdf-rawdatabase-a-swiss-army-knife-for-mdf-files/A7.png" class="fancy"><img src="/orcamdf-rawdatabase-a-swiss-army-knife-for-mdf-files/A7.png" style="max-height: 250px"/></a></div></div>

<p>And as a final example, imagine you’ve got just an MDF file but you seem to have forgotten what objects are stored inside of it. Fret not, we’ll just get the data from the sysschobjs base table! Sysschobjs is the base table that stores all object data, and fortunately it has a static object ID of <em>34</em>. Using this, we can filter down to all of the data pages for object 34, get all the records and then parse just the two first columns of the schema (you may specify a partial schema, as long as you only omit columns at the end), ending up in us dumping just the names (we could of course have gotten the full schema, if we wanted to).</p>
<figure class="highlight cs"><pre><span class="keyword">var</span> db = <span class="keyword">new</span> RawDatabase(<span class="string">@"C:\AWLT2008R2.mdf"</span>);

<span class="keyword">var</span> records = db.Pages
	.Where(x =&gt; x.Header.ObjectID == <span class="number">34</span> && x.Header.Type == PageType.Data)
	.SelectMany(x =&gt; x.Records);
	
<span class="keyword">var</span> rows = records.Select(x =&gt; RawColumnParser.Parse((RawPrimaryRecord)x, <span class="keyword">new</span> IRawType[] {
	RawType.Int(<span class="string">"id"</span>),
	RawType.NVarchar(<span class="string">"name"</span>)
}));

rows.Select(x =&gt; x[<span class="string">"name"</span>]).Dump();
</pre></figure>

<div class="imgwrapper" style=""><div><a href="/orcamdf-rawdatabase-a-swiss-army-knife-for-mdf-files/A8.png" class="fancy"><img src="/orcamdf-rawdatabase-a-swiss-army-knife-for-mdf-files/A8.png" style="max-height: 250px"/></a></div></div>

<h2 id="Compatibility">Compatibility</h2>
<p>Seeing as RawDatabase doesn’t rely on metadata, it’s much easier to support multiple SQL Server versions. Thus, I’m happy to say that RawDatabase fully supports SQL Server 2005, 2008, 2008R2 and 2012. It probably supports 2014 too, I just haven’t tested that. Speaking of testing, all unit tests are automatically run against AdventureWorksLT for both 2005, 2008, 2008R2 and 2012 during testing. Right now there are tests demonstrating that OrcaMDF RawDatabase is able to parse the first record of each and every table in the AdventureWorks LT databases.</p>
<h2 id="Corruption">Corruption</h2>
<p>One of the really interesting use cases for RawDatabase is in the case of corrupted databases. You could filter pages on the object id you’re searching for and then brute-force parse each of them, retrieving whatever data is readable. If metadata is corrupted, you could ignore it, provide the schema manually and the just follow the linked lists of pages, or parse the IAM pages to read heaps. During the next couple of weeks I’ll be blogging more on OrcaMDF RawDatabase to show various use case examples, including ones on corruption.</p>
<h2 id="Source_&amp;_Feedback">Source &amp; Feedback</h2>
<p>I’m really excited about the new RawDatabase addition to OrcaMDF and I hope I’m not the only one who can see the potential. If you try it out, have any ideas, suggestions or other kinds of feedback, I’d love to hear it.</p>
<p>If you want to try it out, head on over to the <a href="https://github.com/improvedk/OrcaMDF" target="_blank">OrcaMDF project on GitHub</a>. Once it’s just a bit more polished, I’ll make it available on NuGet as well. Just like the rest of OrcaMDF, the code is licensed under GPL v3.</p>
]]></content>
    <summary type="html"><![CDATA[<p>When I initially started working on <a href="/introducing-orcamdf/">OrcaMDF</a> I had just one goal, to gain a deeper knowledge of MDF file internals than I could through most books available.</p>
]]></summary>
    
      <category term=".NET" scheme="http://improve.dk/category/.NET/"/>
    
      <category term="SQL Server - Internals" scheme="http://improve.dk/category/SQL%20Server%20-%20Internals/"/>
    
      <category term="SQL Server - OrcaMDF" scheme="http://improve.dk/category/SQL%20Server%20-%20OrcaMDF/"/>
    
      <category term="SQL Server" scheme="http://improve.dk/category/SQL%20Server/"/>
    
      <category term="Tools of the Trade" scheme="http://improve.dk/category/Tools%20of%20the%20Trade/"/>
    
  </entry>
  
  <entry>
    <title><![CDATA[PowerPad - Powerpoint Presenters View for Tablets & Phones]]></title>
    <link href="http://improve.dk/powerpad-powerpoint-presenters-view-for-tablets-phones/"/>
    <id>http://improve.dk/powerpad-powerpoint-presenters-view-for-tablets-phones/</id>
    <published>2013-10-28T00:00:00.000Z</published>
    <updated>2014-04-22T10:39:11.000Z</updated>
    <content type="html"><![CDATA[<p>I love presenting, especially so when it’s possible for me to do so alongside Powerpoints presenters view. Unfortunately I’m an even bigger fan of <a href="http://technet.microsoft.com/en-us/sysinternals/bb897434.aspx" target="_blank">ZoomIt</a> and I use it extensively when presenting. Why is that an issue? To use ZoomIt effectively, not just in demos but when showing slides as well, I need to duplicate my screen rather than extending it. Duplicating the screen means presenters view is not an option :(</p>
<a id="more"></a>

<h2 id="Introducing_PowerPad">Introducing PowerPad</h2>
<p>Seeing <a href="/keeping-track-of-time-while-presenting/">as I’ve already got my iPad next to me when presenting</a> it seems obvious to use that for the presenters view. However, even though I’ve scoured the app store for solutions, I have yet to find something that doesn’t require me to install invasive clients on my computer or suffice with a fixed &amp; lagging UI on the iPad. Even worse, most require me to pay up front, meaning I can’t perform a meaningful trial.</p>
<p>And so I decided to <a href="https://github.com/improvedk/PowerPad" target="_blank">do something about it</a>. PowerPad is a simple console application that runs on your computer, detects when you run a presentation and automatically provides a “presenters view” served over HTTP. The overall goal for PowerPad is to provide a Powerpoint presenters view for tablets &amp; phones.</p>
<div class="imgwrapper" style=""><div><a href="/powerpad-powerpoint-presenters-view-for-tablets-phones/presentation_started.png" class="fancy"><img src="/powerpad-powerpoint-presenters-view-for-tablets-phones/presentation_started.png" style="max-height: 250px"/></a></div></div>

<p>As soon as you’re running PowerPad, and a presentation, you’ll now be able to access the host IP through any device with a browser. I personally use my iPad:</p>
<div class="imgwrapper" style=""><div><a href="/powerpad-powerpoint-presenters-view-for-tablets-phones/screen_notes.png" class="fancy"><img src="/powerpad-powerpoint-presenters-view-for-tablets-phones/screen_notes.png" style="max-height: 250px"/></a></div></div>

<p>And in a pinch I might even use my phone:</p>
<div class="imgwrapper" style=""><div><a href="/powerpad-powerpoint-presenters-view-for-tablets-phones/screen_mobile.png" class="fancy"><img src="/powerpad-powerpoint-presenters-view-for-tablets-phones/screen_mobile.png" style="max-height: 250px"/></a></div></div>

<h2 id="Getting_Started">Getting Started</h2>
<p>PowerPad is open source and completely free to use, licensed under the MIT license. It currently supports Powerpoint 2013 and only requires you to have the .NET 2.0 Framework installed. As long as your devices are on the same network, you can hook up any number of secondary monitors to your presentation - even your attendees, should you want to.</p>
<p>For more screenshots as well as the code &amp; downloads, please check out the <a href="https://github.com/improvedk/PowerPad" target="_blank">PowerPad page on Github</a>.</p>
]]></content>
    <summary type="html"><![CDATA[<p>I love presenting, especially so when it’s possible for me to do so alongside Powerpoints presenters view. Unfortunately I’m an even bigger fan of <a href="http://technet.microsoft.com/en-us/sysinternals/bb897434.aspx" target="_blank">ZoomIt</a> and I use it extensively when presenting. Why is that an issue? To use ZoomIt effectively, not just in demos but when showing slides as well, I need to duplicate my screen rather than extending it. Duplicating the screen means presenters view is not an option :(</p>
]]></summary>
    
      <category term=".NET" scheme="http://improve.dk/category/.NET/"/>
    
      <category term="Conferences and Presenting" scheme="http://improve.dk/category/Conferences%20and%20Presenting/"/>
    
      <category term="Tools of the Trade" scheme="http://improve.dk/category/Tools%20of%20the%20Trade/"/>
    
  </entry>
  
  <entry>
    <title><![CDATA[Announcing Tribal SQL]]></title>
    <link href="http://improve.dk/announcing-tribalsql/"/>
    <id>http://improve.dk/announcing-tribalsql/</id>
    <published>2013-10-23T00:00:00.000Z</published>
    <updated>2014-04-22T10:39:11.000Z</updated>
    <content type="html"><![CDATA[<p>I’m slightly late to announce this, but better late than never!</p>
<a id="more"></a>

<p>Just a few weeks ago, the book <a href="http://www.amazon.com/Tribal-SQL-Tony-Davis/dp/1906434808/" target="_blank">Tribal SQL</a> went for sale! I authored a chapter on “Storage Internals 101” and alongside 14 other first-time authors, this is our first book to have published!</p>
<div class="imgwrapper" style=""><div><a href="/announcing-tribalsql/tribalSQLcover.png" class="fancy"><img src="/announcing-tribalsql/tribalSQLcover.png" style="max-height: 250px"/></a></div></div>

<blockquote>Tribal SQL: New voices in SQL Server<br><br>15 first-time authors answer the question: What makes you passionate about working with SQL Server?<br><br><a href="http://midnightdba.itbookworm.com/" target="_blank">MidnightDBA</a> and <a href="http://www.red-gate.com/" target="_blank">Red Gate</a> partnered to produce a book filled with community, Tribal, knowledge on SQL Server. The resulting book is a series of chapters on lessons learned, perhaps the hard way, which you won’t find in traditional training or technical guidance material.<br><br>As a truly community-driven book, the authors are all generously donating 100% of their royalties to the charity <a href="http://www.computers4africa.org.uk/" target="_blank">Computers 4 Africa</a>.<br><br>A DBA’s core responsibilities are constant. A DBA must have the hard skills necessary to maintain and enforce security mechanisms on the data, prepare effectively for disaster recovery, ensure the performance and availability of all the databases in their care.<br><br>Side by side with these, our authors have also recognized the importance of communication skills to the business and their careers. We have chapters on the importance to a DBA of communicating clearly with their co-workers and business leaders, presenting data as useful information that the business can use to make decisions, and sound project management skills.<br><br>The resulting book, <a href="http://www.amazon.com/Tribal-SQL-Tony-Davis/dp/1906434808" target="_blank">Tribal SQL</a>, is a reflection of how a DBA’s core and long-standing responsibilities and what it means to be a DBA in today’s businesses.</blockquote>

<p>If you want to get a <a href="https://www.simple-talk.com/sql/database-administration/sql-server-storage-internals-101/" target="_blank">sneak peek of my chapter</a>, it has been posted on <a href="https://www.simple-talk.com/sql/database-administration/sql-server-storage-internals-101/" target="_blank">Simple-Talk</a> as an extract of the complete book.</p>
]]></content>
    <summary type="html"><![CDATA[<p>I’m slightly late to announce this, but better late than never!</p>
]]></summary>
    
      <category term="SQL Server - Community" scheme="http://improve.dk/category/SQL%20Server%20-%20Community/"/>
    
      <category term="SQL Server - Internals" scheme="http://improve.dk/category/SQL%20Server%20-%20Internals/"/>
    
      <category term="SQL Server" scheme="http://improve.dk/category/SQL%20Server/"/>
    
  </entry>
  
  <entry>
    <title><![CDATA[Optimizing Performance by Programmaticaly Setting ReadOnlySessionState]]></title>
    <link href="http://improve.dk/optimizing-performance-programmatically-setting-readonlysessionstate/"/>
    <id>http://improve.dk/optimizing-performance-programmatically-setting-readonlysessionstate/</id>
    <published>2013-10-08T00:00:00.000Z</published>
    <updated>2014-05-04T16:12:23.000Z</updated>
    <content type="html"><![CDATA[<p>One of the main culprits when it comes to ASP.NET concurrency is caused by the fact that default sesion state has been implemented using a pessimistic locking pattern. Basically, any standard handler, whether that be an ASPX page, a generic handler or an ASMX web service, goes through the following steps:</p>
<a id="more"></a>

<ul>
<li>Retrieve &amp; exclusively lock session</li>
<li>Execute request handler</li>
<li>Save &amp; unlock updated session (whether updates have been made or not)</li>
</ul>
<p>What this means is that, for a given session, <em>only one request can execute concurrently</em>. Any other requests, from that same session, will block, waiting for the session to be released. For the remainder of this post I’ll concentrate on generic HttpHandlers, but this problem &amp; solution is common to for ASPX and ASMX pages as well.</p>
<h2 id="Disabling_Session_State">Disabling Session State</h2>
<p>If your handler doesn’t require session state, all you have to do is to <em>not</em> implement the IRequiresSessionState interface, given that HttpHandlers by default do not have access to session state:</p>
<figure class="highlight cs"><pre><span class="keyword">public</span> <span class="keyword">class</span> MyHandler : IHttpHandler
{
	<span class="keyword">public</span> <span class="keyword">void</span> <span class="title">ProcessRequest</span>(HttpContext context)
	{
		<span class="comment">// Perform some task</span>
	}
	
	<span class="keyword">public</span> <span class="keyword">bool</span> IsReusable { <span class="keyword">get</span> { <span class="keyword">return</span> <span class="keyword">false</span>; } }
}
</pre></figure>

<p>By not enabling session state, no session will be locked and you can execute as many concurrent requsts as your server can handle.</p>
<h2 id="Enabling_Session_State">Enabling Session State</h2>
<p>If you <em>do</em> need session state, simply implement the IRequiresSessionState interface, like so:</p>
<figure class="highlight cs"><pre><span class="keyword">public</span> <span class="keyword">class</span> MyHandler : IHttpHandler, IRequiresSessionState
{
	<span class="keyword">public</span> <span class="keyword">void</span> <span class="title">ProcessRequest</span>(HttpContext context)
	{
		<span class="comment">// Perform some task</span>
	}
	
	<span class="keyword">public</span> <span class="keyword">bool</span> IsReusable { <span class="keyword">get</span> { <span class="keyword">return</span> <span class="keyword">false</span>; } }
}
</pre></figure>

<p>The IRequiresSessionState interface carries no functionality at all, it’s simply a marker interface that tells the ASP.NET request pipeline to acquire session state for the given request. By implementing this interface you now have read+write access to the current session.</p>
<h2 id="Read-Only_Session_State">Read-Only Session State</h2>
<p>If all you need is to read session state, while not having to be able to write it, you should implement the IReadOnlySessionState interface instead, like so:</p>
<figure class="highlight cs"><pre><span class="keyword">public</span> <span class="keyword">class</span> MyHandler : IHttpHandler, IReadOnlySessionState
{
	<span class="keyword">public</span> <span class="keyword">void</span> <span class="title">ProcessRequest</span>(HttpContext context)
	{
		<span class="comment">// Perform some task</span>
	}
	
	<span class="keyword">public</span> <span class="keyword">bool</span> IsReusable { <span class="keyword">get</span> { <span class="keyword">return</span> <span class="keyword">false</span>; } }
}
</pre></figure>

<p>Implementing this interface changes the steps performed by the page slightly:</p>
<ul>
<li>Retrieve session, without locking</li>
<li>Execute request handler</li>
<li><del>Save &amp; unlock updated session (whether updates have been made or not)</del></li>
</ul>
<p>While session is still read as usual, it’s just not persisted back after the request is done. This means you can actually update the session, without causing any exceptions. However, as the session is never persisted, your changes won’t be saved after the request is done. For read-only use this also saves the superfluous save operation which can be costly if you’re using out-of-process session state like State or SQL Server.</p>
<h2 id="Switching_Between_Read+Write_and_Read-Only_Session_State_Programmatically">Switching Between Read+Write and Read-Only Session State Programmatically</h2>
<p>While this is great, we sometimes need something in between. Consider the following scenario:<7p></p>
<ul>
<li>You’ve got a single handler that’s heavily requested.</li>
<li>On the first request you need to perform some expensive lookup to load some data that will be used in all further requests, but is session specific, and will thus be stored in session state.</li>
<li>If you implement IRequiresSessionState, you can easily detect the first request (Session[“MyData”] == null), load the data, store it in session and then reuse it in all subsequent requests. However, this ensures only one request may execute at a time, due to the session being exclusively locked while the handler executes.</li>
<li>If you instead implement IReadOnlySessionState, you can execute as many handlers concurrently as you please, but you’ll have to do that expensive data loading on each request, seeing as you can’t store it in session.</li>
</ul>
<p>Imagine if you could dynamically decide whether to implement the full read+write enabled IRequiresSessionState or just the read enabled IReadOnlySession state. That way you could implement IRequiresSessionState for the first request and just implement IReadOnlySessionState for all of the subsequent requests, once a session has been established.</p>
<p>And guess what, from .NET 4.0 onwards, that’s possible!</p>
<h2 id="Enter_HttpContext-SetSessionStateBehavior">Enter HttpContext.SetSessionStateBehavior</h2>
<p>Looking at the <a href="http://msdn.microsoft.com/En-US/library/bb470252.aspx" target="_blank">ASP.NET request pipeline</a>, session state is loaded in the “Acquire state” event. At any point, before this event, we can set the session behavior programmatically by calling HttpContext.SetSessionStateBehavior. Setting the session programmatically through HttpContext.SetSessionStateBehavior will override any interfaces implemented by the handler itself.</p>
<p>Here’s a full example of an HttpModule that runs on each request. In the PostMapRequestHandler event (which fires just before the AcquireState event), we inspect the HttpHandler assigned to the request. If it implements the IPreferReadOnlySessionState interface (a custom marker interface), the SessionStateBehavior is set to ReadOnly, provided there already is an active session (which the presence of an ASP.NET_SessionId cookie indicates). If there is no session cookie present, or if the handler doesn’t implement IPreferReadOnlySessionState, then it’s left up to the handler default - that is, the implemented interface, to decide.</p>
<figure class="highlight cs"><pre><span class="keyword">public</span> <span class="keyword">class</span> RequestHandler : IHttpModule
{
	<span class="keyword">public</span> <span class="keyword">void</span> <span class="title">Init</span>(HttpApplication context)
	{
		context.PostMapRequestHandler += context_PostMapRequestHandler;
	}
	
	<span class="keyword">void</span> context_PostMapRequestHandler(<span class="keyword">object</span> sender, EventArgs e)
	{
		<span class="keyword">var</span> context = HttpContext.Current;
		
		<span class="keyword">if</span> (context.Handler <span class="keyword">is</span> IPreferReadOnlySessionState)
		{
			<span class="keyword">if</span> (context.Request.Headers[<span class="string">"Cookie"</span>] != <span class="keyword">null</span> && context.Request.Headers[<span class="string">"Cookie"</span>].Contains(<span class="string">"ASP.NET_SessionId="</span>))
				context.SetSessionStateBehavior(SessionStateBehavior.ReadOnly);
		}
	}
}
</pre></figure>

<p>Now all we need to do is to also implement the IPreferReadOnlySessionState interface in the handlers that can do with read-only sesion state, provided a session is already present:</p>
<figure class="highlight cs"><pre><span class="keyword">public</span> <span class="keyword">interface</span> IPreferReadOnlySessionState
{ }
</pre></figure>


<figure class="highlight cs"><pre><span class="keyword">public</span> <span class="keyword">class</span> MyHandler : IHttpHandler, IRequiresSessionState, IPreferReadOnlySessionState
{
	<span class="keyword">public</span> <span class="keyword">void</span> <span class="title">ProcessRequest</span>(HttpContext context)
	{
		<span class="comment">// Perform some task</span>
	}
	
	<span class="keyword">public</span> <span class="keyword">bool</span> IsReusable { <span class="keyword">get</span> { <span class="keyword">return</span> <span class="keyword">false</span>; } }
}
</pre></figure>

<p>And just like that, the first request has read+write access to the session state, while all subsequent requests only have read access, greatly increasing the concurrency of the handler.</p>
]]></content>
    <summary type="html"><![CDATA[<p>One of the main culprits when it comes to ASP.NET concurrency is caused by the fact that default sesion state has been implemented using a pessimistic locking pattern. Basically, any standard handler, whether that be an ASPX page, a generic handler or an ASMX web service, goes through the following steps:</p>
]]></summary>
    
      <category term=".NET" scheme="http://improve.dk/category/.NET/"/>
    
      <category term="IIS" scheme="http://improve.dk/category/IIS/"/>
    
      <category term="Performance" scheme="http://improve.dk/category/Performance/"/>
    
      <category term="Web" scheme="http://improve.dk/category/Web/"/>
    
  </entry>
  
  <entry>
    <title><![CDATA[Verifying Mailgun Webhooks]]></title>
    <link href="http://improve.dk/verifying-mailgun-webhooks/"/>
    <id>http://improve.dk/verifying-mailgun-webhooks/</id>
    <published>2013-09-23T00:00:00.000Z</published>
    <updated>2014-05-04T16:12:23.000Z</updated>
    <content type="html"><![CDATA[<p><a href="http://www.mailgun.com/" target="_blank">Mailgun</a> has a very neat feature that enables you to basically convert incoming emails to a POST request to a URL of your choice, also known as a webhook. Using this, you can easily have your application respond to email events. However, as this URL/service needs to be publically available, verifying Mailgun webhooks is very important, ensuring requests actually come from Mailgun, and not someone impersonating Mailgun.</p>
<a id="more"></a>

<p>The code required for verifying Mailgun forwards is very simple and doesn’t require much explanation:</p>
<figure class="highlight cs"><pre><span class="comment"><span class="xmlDocTag">///</span> <span class="xmlDocTag">&lt;summary&gt;</span></span>
<span class="comment"><span class="xmlDocTag">///</span> Verifies that the signature matches the timestamp & token.</span>
<span class="comment"><span class="xmlDocTag">///</span> <span class="xmlDocTag">&lt;/summary&gt;</span></span>
<span class="comment"><span class="xmlDocTag">///</span> <span class="xmlDocTag">&lt;returns&gt;</span>True if the signature is valid, otherwise false.<span class="xmlDocTag">&lt;/returns&gt;</span></span>
<span class="keyword">public</span> <span class="keyword">static</span> <span class="keyword">bool</span> <span class="title">VerifySignature</span>(<span class="keyword">string</span> key, <span class="keyword">int</span> timestamp, <span class="keyword">string</span> token, <span class="keyword">string</span> signature)
{
	<span class="keyword">var</span> encoding = Encoding.ASCII;
	<span class="keyword">var</span> hmacSha256 = <span class="keyword">new</span> HMACSHA256(encoding.GetBytes(key));
	<span class="keyword">var</span> cleartext = encoding.GetBytes(timestamp + token);
	<span class="keyword">var</span> hash = hmacSha256.ComputeHash(cleartext);
	<span class="keyword">var</span> computedSignature = BitConverter.ToString(hash).Replace(<span class="string">"-"</span>, <span class="string">""</span>).ToLower();

	<span class="keyword">return</span> computedSignature == signature;
}
</pre></figure>

<p>Use sample:</p>
<figure class="highlight cs"><pre><span class="comment">// All these values are provided by the Mailgun request</span>
<span class="keyword">var</span> key = <span class="string">"key-x3ifab7xngqxep7923iuab251q5vhox0"</span>;
<span class="keyword">var</span> timestamp = <span class="number">1568491354</span>;
<span class="keyword">var</span> token = <span class="string">"asdoij2893dm98m2x0a9sdkf09k423cdm"</span>;
<span class="keyword">var</span> signature = <span class="string">"AF038C73E912A830FFC830239ABFF"</span>;

<span class="comment">// Verify if request is valid</span>
<span class="keyword">bool</span> isValid = VerifySignature(key, timestamp, token, signature);
</pre></figure>

<p>As the <a href="http://documentation.mailgun.com/user_manual.html#securing-webhooks" target="_blank">manual says</a> you simply need to calculate a SHA256 HMAC of the concatenated timestamp and token values, after which you can verify that it matches the Mailgun provided signature. The key is the private API key, retrievable from the Mailgun control panel.</p>
]]></content>
    <summary type="html"><![CDATA[<p><a href="http://www.mailgun.com/" target="_blank">Mailgun</a> has a very neat feature that enables you to basically convert incoming emails to a POST request to a URL of your choice, also known as a webhook. Using this, you can easily have your application respond to email events. However, as this URL/service needs to be publically available, verifying Mailgun webhooks is very important, ensuring requests actually come from Mailgun, and not someone impersonating Mailgun.</p>
]]></summary>
    
      <category term=".NET" scheme="http://improve.dk/category/.NET/"/>
    
  </entry>
  
  <entry>
    <title><![CDATA[Analyzing BSOD Minidump Files Using Windbg]]></title>
    <link href="http://improve.dk/analyzing-bsod-minidump-files-using-windbg/"/>
    <id>http://improve.dk/analyzing-bsod-minidump-files-using-windbg/</id>
    <published>2013-06-04T00:00:00.000Z</published>
    <updated>2014-04-22T10:39:11.000Z</updated>
    <content type="html"><![CDATA[<p>Unfortunately, once in a while, computers fail. If you’re running Windows you’ve probably witnessed the dreaded <a href="http://en.wikipedia.org/wiki/Blue_Screen_of_Death" target="_blank">Blue Screen of Death</a>, commonly referred to as a BSOD. Once the BSOD occurs, some machines will immediately restart, before you’ve got a chance to actually see what happened. Other times users will just report that the BSOD happened, without noting anything down about what the message actually said. In this post I’ll show you how analyzing BSOD minidump files using <a href="http://en.wikipedia.org/wiki/WinDbg" target="_blank">Windbg</a> will enable you to find the cause of the BSOD after the fact.</p>
<a id="more"></a>


<h2 id="Enabling_Dump_Files">Enabling Dump Files</h2>
<p>By default, never Windows installs will automatically create minidump files once a BSOD occurs. Once restarted, you should be able to see a .dmp file here:</p>
<figure class="highlight"><pre>C:<span class="command">\Windows</span><span class="command">\Minidump</span>
</pre></figure>

<p>If you don’t see any .dmp files there, or if the directory doesn’t exist, you may have to tell Windows to create minidump files when the BSOD occurs. To do so, press the <strong>Win+Break</strong> keys to open up the System control panel. Now click <strong>Advanced system settings</strong> in the left menu. Once there, go to the <strong>Advanced</strong> tab and click the <strong>Settings…</strong> button under the <strong>Startup and Recovery</strong> section. Now make sure the <strong>Write debugging information</strong> setting is set to <strong>anything but “none”</strong>:</p>
<div class="imgwrapper" style=""><div><a href="/analyzing-bsod-minidump-files-using-windbg/Capture.png" class="fancy"><img src="/analyzing-bsod-minidump-files-using-windbg/Capture.png" style="max-height: 250px"/></a></div></div>


<h2 id="Analyzing_BSOD_Minidump_Files_Using_Windbg">Analyzing BSOD Minidump Files Using Windbg</h2>
<p>Once a dump file has been created, you can analyze it using Windbg. Start by opening Windbg and pressing the <strong>Ctrl+D</strong> keys. Now select the .dmp file you want to analyze and click <strong>Open</strong>. This should yield something like this:</p>
<figure class="highlight"><pre>Microsoft (R) Windows Debugger Version 6.12.0002.633 AMD64
Copyright (c) Microsoft Corporation. All rights reserved.


Loading Dump File [C:\Windows\Minidump\040813-15974-01.dmp]
Mini Kernel Dump File: Only registers and stack trace are available

Symbol search path is: symsrv<span class="emphasis">*symsrv.dll*</span>c:\symbols*http://msdl.microsoft.com/download/symbols
Executable search path is: 
Windows 7 Kernel Version 7601 (Service Pack 1) MP (12 procs) Free x64
Product: WinNt, suite: TerminalServer SingleUserTS
Built by: 7601.18044.amd64fre.win7sp1_gdr.130104-1431
Machine Name:
Kernel base = 0xfffff800<span class="code">`0300c000 PsLoadedModuleList = 0xfffff800`</span>03250670
Debug session time: Mon Apr  8 22:17:47.016 2013 (UTC + 2:00)
System Uptime: 0 days 1:36:19.860
Loading Kernel Symbols
...............................................................
................................................................
........................
Loading User Symbols
Loading unloaded module list
...............
<span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="emphasis">***</span>*
<span class="bullet">*                                                                             </span>*
<span class="bullet">*                        </span>Bugcheck Analysis                                    *
<span class="bullet">*                                                                             </span>*
<span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="emphasis">***</span>*

Use !analyze -v to get detailed debugging information.

BugCheck FE, {4, fffffa803c3c89e0, fffffa803102e230, fffffa803e765010}

Probably caused by : FiioE17.sys ( FiioE17+1d21 )

Followup: MachineOwner
</pre></figure>

<p>Already this tells us a couple of things - your OS details, when exactly the problem occurred as well as what module probably caused the issue (FiioE17.sys in this case). Also, it tells you how to proceed:</p>
<blockquote>
<p>Use !analyze -v to get detailed debugging information.</p>
</blockquote>
<p>As suggested, let’s try and run the !analyze -v command:</p>
<figure class="highlight"><pre>11: kd&gt; !analyze -v
<span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="emphasis">***</span>*
<span class="bullet">*                                                                             </span>*
<span class="bullet">*                        </span>Bugcheck Analysis                                    *
<span class="bullet">*                                                                             </span>*
<span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="strong">*****</span><span class="emphasis">***</span>*

BUGCODE<span class="emphasis">_USB_</span>DRIVER (fe)
USB Driver bugcheck, first parameter is USB bugcheck code.
Arguments:
Arg1: 0000000000000004, IRP<span class="emphasis">_URB_</span>DOUBLE_SUBMIT The caller has submitted an irp
<span class="code">	that is already pending in the USB bus driver.</span>
Arg2: fffffa803c3c89e0, Address of IRP
Arg3: fffffa803102e230, Address of URB
Arg4: fffffa803e765010

<span class="header">Debugging Details:
------------------</span>

CUSTOMER<span class="emphasis">_CRASH_</span>COUNT:  1

DEFAULT<span class="emphasis">_BUCKET_</span>ID:  VISTA<span class="emphasis">_DRIVER_</span>FAULT

BUGCHECK_STR:  0xFE

PROCESS_NAME:  audiodg.exe

CURRENT_IRQL:  2

LAST<span class="emphasis">_CONTROL_</span>TRANSFER:  from fffff88008326f4b to fffff80003081c40

STACK_TEXT:  
fffff880<span class="code">`0e482fd8 fffff880`</span>08326f4b : 00000000<span class="code">`000000fe 00000000`</span>00000004 fffffa80<span class="code">`3c3c89e0 fffffa80`</span>3102e230 : nt!KeBugCheckEx
fffff880<span class="code">`0e482fe0 fffff880`</span>0833244a : fffffa80<span class="code">`3ae97002 fffffa80`</span>3b8caad0 00000000<span class="code">`00000000 fffffa80`</span>3ae97050 : USBPORT!USBPORT<span class="emphasis">_Core_</span>DetectActiveUrb+0x127
fffff880<span class="code">`0e483030 fffff880`</span>0833ae74 : fffffa80<span class="code">`3c3c89e0 fffffa80`</span>3af7000a fffffa80<span class="code">`3c3c89e0 fffffa80`</span>3102e230 : USBPORT!USBPORT_ProcessURB+0xad6
fffff880<span class="code">`0e4830e0 fffff880`</span>08314af4 : 00000000<span class="code">`00000000 fffffa80`</span>3af7b050 fffffa80<span class="code">`3e5d1720 fffffa80`</span>3c3c89e0 : USBPORT!USBPORT_PdoInternalDeviceControlIrp+0x138
fffff880<span class="code">`0e483120 fffff880`</span>00fa97a7 : fffffa80<span class="code">`3c3c89e0 fffffa80`</span>31192040 fffffa80<span class="code">`3c3c89e0 fffffa80`</span>3c3c89e0 : USBPORT!USBPORT_Dispatch+0x1dc
fffff880<span class="code">`0e483160 fffff880`</span>00fb1789 : fffff880<span class="code">`00fcfb50 fffffa80`</span>3d944ed1 fffffa80<span class="code">`3c3c8d38 fffffa80`</span>3c3c8d38 : ACPI!ACPIDispatchForwardIrp+0x37
fffff880<span class="code">`0e483190 fffff880`</span>00fa9a3f : fffff880<span class="code">`00fcfb50 fffffa80`</span>316a7a90 fffffa80<span class="code">`3c3c89e0 fffffa80`</span>3ab6c050 : ACPI!ACPIIrpDispatchDeviceControl+0x75
fffff880<span class="code">`0e4831c0 fffff880`</span>088ca566 : 00000000<span class="code">`00000000 00000000`</span>00000004 fffffa80<span class="code">`3ab6c050 fffffa80`</span>3c2bd440 : ACPI!ACPIDispatchIrp+0x12b
fffff880<span class="code">`0e483240 fffff880`</span>088fad8f : 00000000<span class="code">`00000000 00000000`</span>00000000 fffffa80<span class="code">`3c2bd440 00000000`</span>00000000 : usbhub!UsbhFdoUrbPdoFilter+0xde
fffff880<span class="code">`0e483270 fffff880`</span>088c8fb7 : fffffa80<span class="code">`3c3c89e0 fffffa80`</span>3a976ce0 fffffa80<span class="code">`3c3c89e0 fffffa80`</span>3c3c89e0 : usbhub!UsbhPdoInternalDeviceControl+0x373
fffff880<span class="code">`0e4832c0 fffff880`</span>00fa97a7 : fffffa80<span class="code">`3c3c89e0 fffff800`</span>031b630d fffffa80<span class="code">`3b7be100 00000000`</span>00000801 : usbhub!UsbhGenDispatch+0x57
fffff880<span class="code">`0e4832f0 fffff880`</span>00fb1789 : fffff880<span class="code">`00fcfb50 00000000`</span>00000001 fffffa80<span class="code">`3c393b58 fffffa80`</span>3c3c8d38 : ACPI!ACPIDispatchForwardIrp+0x37
fffff880<span class="code">`0e483320 fffff880`</span>00fa9a3f : fffff880<span class="code">`00fcfb50 fffffa80`</span>316a8a90 fffffa80<span class="code">`3c3c89e0 fffffa80`</span>3c393b58 : ACPI!ACPIIrpDispatchDeviceControl+0x75
fffff880<span class="code">`0e483350 fffff880`</span>08c9bec4 : 00000000<span class="code">`00000000 fffffa80`</span>3c326938 fffffa80<span class="code">`3c393b58 00000000`</span>00000000 : ACPI!ACPIDispatchIrp+0x12b
fffff880<span class="code">`0e4833d0 fffff880`</span>08c98812 : fffffa80<span class="code">`3c393b58 fffffa80`</span>3c3c89e0 fffffa80<span class="code">`00000324 fffffa80`</span>3c3c89e0 : usbccgp!UsbcForwardIrp+0x30
fffff880<span class="code">`0e483400 fffff880`</span>08c98aba : fffffa80<span class="code">`3c326838 00000000`</span>00220003 fffffa80<span class="code">`3c3c89e0 fffffa80`</span>3c393b58 : usbccgp!DispatchPdoUrb+0xfa
fffff880<span class="code">`0e483440 fffff880`</span>08c9672e : 00000000<span class="code">`0000000f fffffa80`</span>3c393b50 fffffa80<span class="code">`3c393b58 fffffa80`</span>3c3c89e0 : usbccgp!DispatchPdoInternalDeviceControl+0x17a
fffff880<span class="code">`0e483470 fffff880`</span>08cb3d21 : fffffa80<span class="code">`3c393a00 fffffa80`</span>3c3c8901 fffffa80<span class="code">`3c3c8900 00000000`</span>00000000 : usbccgp!USBC_Dispatch+0x2de
fffff880<span class="code">`0e4834f0 fffffa80`</span>3c393a00 : fffffa80<span class="code">`3c3c8901 fffffa80`</span>3c3c8900 00000000<span class="code">`00000000 fffffa80`</span>3c373010 : FiioE17+0x1d21
fffff880<span class="code">`0e4834f8 fffffa80`</span>3c3c8901 : fffffa80<span class="code">`3c3c8900 00000000`</span>00000000 fffffa80<span class="code">`3c373010 00000000`</span>00000000 : 0xfffffa80`3c393a00
fffff880<span class="code">`0e483500 fffffa80`</span>3c3c8900 : 00000000<span class="code">`00000000 fffffa80`</span>3c373010 00000000<span class="code">`00000000 fffffa80`</span>3c3b7f30 : 0xfffffa80`3c3c8901
fffff880<span class="code">`0e483508 00000000`</span>00000000 : fffffa80<span class="code">`3c373010 00000000`</span>00000000 fffffa80<span class="code">`3c3b7f30 fffff880`</span>08cb47fd : 0xfffffa80`3c3c8900


STACK_COMMAND:  kb

FOLLOWUP_IP: 
FiioE17+1d21
fffff880`08cb3d21 ??              ???

SYMBOL<span class="emphasis">_STACK_</span>INDEX:  12

SYMBOL_NAME:  FiioE17+1d21

FOLLOWUP_NAME:  MachineOwner

MODULE_NAME: FiioE17

IMAGE_NAME:  FiioE17.sys

DEBUG<span class="emphasis">_FLR_</span>IMAGE_TIMESTAMP:  50b30686

FAILURE<span class="emphasis">_BUCKET_</span>ID:  X64<span class="emphasis">_0xFE_</span>FiioE17+1d21

BUCKET<span class="emphasis">_ID:  X64_</span>0xFE_FiioE17+1d21

Followup: MachineOwner
</pre></figure>

<p>This tells us a number of interesting things:</p>
<ul>
<li>The BSOD error was: <strong>BUGCODE_USB_DRIVER</strong></li>
<li>This is the error caused by the driver: IRP_URB_DOUBLE_SUBMIT <strong>The caller has submitted an irp that is already pending in the USB bus driver</strong>.</li>
<li>The process that invoked the error: <strong>audiodg.exe</strong></li>
<li>The stack trace of the active thread on which the error occurred. Note that Windbg can’t find the right symbols as this is a proprietary driver with no public symbols. Even so, to the developer of said driver, the above details will help immensely.</li>
<li>The driver name: <strong>FiioE17.sys</strong></li>
</ul>
<p>With the above options, you’ve got a lot of details that can be sent to the developer, hopefully enabling him/her/them to fix the issue. For now, I’ll have to unplug my <a href="http://www.amazon.com/FiiO-Headphone-Amplifier-Docking-Interface/dp/B0070UFMOW" target="_blank">Fiio E17 USB DAC</a> :(</p>
]]></content>
    <summary type="html"><![CDATA[<p>Unfortunately, once in a while, computers fail. If you’re running Windows you’ve probably witnessed the dreaded <a href="http://en.wikipedia.org/wiki/Blue_Screen_of_Death" target="_blank">Blue Screen of Death</a>, commonly referred to as a BSOD. Once the BSOD occurs, some machines will immediately restart, before you’ve got a chance to actually see what happened. Other times users will just report that the BSOD happened, without noting anything down about what the message actually said. In this post I’ll show you how analyzing BSOD minidump files using <a href="http://en.wikipedia.org/wiki/WinDbg" target="_blank">Windbg</a> will enable you to find the cause of the BSOD after the fact.</p>
]]></summary>
    
      <category term="Windbg" scheme="http://improve.dk/category/Windbg/"/>
    
  </entry>
  
  <entry>
    <title><![CDATA[Speaking at SQL PASS Summit 2013]]></title>
    <link href="http://improve.dk/speaking-at-sql-pass-summit-2013/"/>
    <id>http://improve.dk/speaking-at-sql-pass-summit-2013/</id>
    <published>2013-05-28T00:00:00.000Z</published>
    <updated>2014-04-22T10:39:11.000Z</updated>
    <content type="html"><![CDATA[<p>I’m delighted to announce that <a href="http://www.sqlpass.org/summit/2013/Sessions/SpeakerDetails.aspx?spid=245" target="_blank">I’ll be speaking</a> at this years <a href="http://www.sqlpass.org/summit/2013/" target="_blank">SQL PASS Summit in Charlotte, North Carolina</a>. Having submitted several times before, unsuccessfully, I’m really happy to have made the cut this year. Looking at the <a href="http://www.sqlpass.org/summit/2013/Sessions.aspx" target="_blank">lineup of speakers</a>, I take great pride in being given the opportunity.</p>
<a id="more"></a>

<h2 id="My_Sessions">My Sessions</h2>
<p>That’s right, not just one session, but two! And as if that wasn’t enough, the two selected sessions are my absolute favorite ones to perform! I’ve presented both several times before and thanks to great feedback from the audiences I’ve slowly fine tuned the format and content.</p>
<h2 id="Top_Tricks_and_Best_Practices_for_-NET_SQL_Server_Developers">Top Tricks and Best Practices for .NET SQL Server Developers</h2>
<p>This is a session chock-full of easy-to-use tips, tricks and gotchas that can be implemented immediately. If you’re either a .NET developer yourself, or if you have .NET developers on your team, using SQL Server, this session is sure to be an eye opener with valuable lessons.</p>
<blockquote>
<p>Being the acting DBA while doing development and managing a team of .NET developers, I’ve learned a trick or two through the years. For this session, I’ve gathered my list of top tricks any .NET developer should know and use when dealing with SQL Server. We’ll cover how to use TransactionScopes without locking up the database, avoiding MSDTC escalation, using internal batching functions in the BCL through reflection, avoiding unnecessary round trips, and much more. These are tips, tricks, and best practices that I ensure all my developers are taught before they have a chance of committing code to our production systems.</p>
</blockquote>
<h2 id="Understanding_Data_Files_at_the_Byte_Level">Understanding Data Files at the Byte Level</h2>
<p>The best part about this session, for me, is watching heads explode only 15 minutes in when I make a live demonstration of how to reverse engineer SQL Server, to persuade it into describing its own data file format. In just 75 minutes I will give you not only a thorough tour of the MDF file format, but also a plethora of techniques on how to analyze your own databases internal storage as well. Using these techniques you’ll be well armed when it comes to schema discussions, column type choice and for those rare events where you need to dive just a bit below the surface to discover what’s really happening.</p>
<blockquote>
<p>This session won’t explain when to use a heap instead of an index, but you will learn how they work – and differ – behind the scenes. Demonstrations will show how data files are organized on the disk and how that organization allows SQL Server to effectively query the data. Knowing how data files are organized will in turn help immensely when it comes to optimizing databases for both performance and storage efficiency.</p>
</blockquote>
]]></content>
    <summary type="html"><![CDATA[<p>I’m delighted to announce that <a href="http://www.sqlpass.org/summit/2013/Sessions/SpeakerDetails.aspx?spid=245" target="_blank">I’ll be speaking</a> at this years <a href="http://www.sqlpass.org/summit/2013/" target="_blank">SQL PASS Summit in Charlotte, North Carolina</a>. Having submitted several times before, unsuccessfully, I’m really happy to have made the cut this year. Looking at the <a href="http://www.sqlpass.org/summit/2013/Sessions.aspx" target="_blank">lineup of speakers</a>, I take great pride in being given the opportunity.</p>
]]></summary>
    
      <category term="SQL Server - Community" scheme="http://improve.dk/category/SQL%20Server%20-%20Community/"/>
    
      <category term="Conferences and Presenting" scheme="http://improve.dk/category/Conferences%20and%20Presenting/"/>
    
  </entry>
  
  <entry>
    <title><![CDATA[Could Not Load Type 'NewRelic.Agent.Core.AgentApi']]></title>
    <link href="http://improve.dk/could-not-load-type-newrelic-agent-core-agentapi/"/>
    <id>http://improve.dk/could-not-load-type-newrelic-agent-core-agentapi/</id>
    <published>2013-05-23T00:00:00.000Z</published>
    <updated>2014-04-22T10:39:11.000Z</updated>
    <content type="html"><![CDATA[<p>Recently I’ve begun using <a href="http://newrelic.com/" target="_blank">New Relic</a>, and so far it’s been an excellent experience. About two weeks ago I started using their <a href="https://newrelic.com/docs/dotnet/the-net-agent-api" target="_blank">.NET Agent API</a> to customize some of the data reported by our application to their servers. This makes the data way more valuable to us as we can now selectively ignore certain parts of our application while getting better reporting from other, more critical, parts of the application.</p>
<a id="more"></a>

<h2 id="Random_Outages">Random Outages</h2>
<p>Unfortunately, in the last couple of weeks, ever since introducing the .NET Agent API, we’ve had a number of outages (thankfully invisible to the customers due to a self-healing load-balancer setup shielding the individual application servers) where one of our applications servers would randomly start throwing the same exception on all requests:</p>
<figure class="highlight"><pre>System.TypeLoadException: Could not <span class="operator"><span class="keyword">load</span> type <span class="string">'NewRelic.Agent.Core.AgentApi'</span> <span class="keyword">from</span> assembly <span class="string">'NewRelic.Api.Agent, Version=2.5.112.0, Culture=neutral, PublicKeyToken=06552fced0b33d87'</span>. 
<span class="keyword">at</span> NewRelic.Api.Agent.NewRelic.SetTransactionName(String category, String name) 
<span class="keyword">at</span> System.Web.HttpApplication.SyncEventExecutionStep.System.Web.HttpApplication.IExecutionStep.<span class="keyword">Execute</span>() 
<span class="keyword">at</span> System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)</span>
</pre></figure>

<p>The error seemed to crop up randomly on all of our servers, though not at the same time and in with no predictable patterns - except it was always just after an application pool recycle. Once the error occurred it would continue happening until we either recycled the pool manually or it was recycled automatically according to its schedule.</p>
<h2 id="The_Support_Experience">The Support Experience</h2>
<p>To make a long story short, I opened a support case with New Relic as I couldn’t find anything in neither their docs, nor on Google, related to the specific exception. After about a week of going back and forth between their engineers and me they managed to track down the root cause:</p>
<blockquote>
<p>It appears that some of the caching we do is not being correctly invalidated. I have removed the caching code and you should see this fix in our next release.</p>
</blockquote>
<p>In the meantime I’ve had to stop using the .NET Agent API to avoid the issue from happening again. This doesn’t mean we won’t get any data; it’s just not as well polished as before. I’m eagerly looking forward to the next agent release so we can get back to using the .NET Agent API again.</p>
<p>In conclusion I must say I’m impressed by the overall support experience. The responses have been quick and professional. Naturally I’d prefer not to have had any issues, but we all know they can happen, and in those cases it’s a matter of having a solid triage process - and in this case I’m just happy to be able to assist in identifying the cause.</p>
]]></content>
    <summary type="html"><![CDATA[<p>Recently I’ve begun using <a href="http://newrelic.com/" target="_blank">New Relic</a>, and so far it’s been an excellent experience. About two weeks ago I started using their <a href="https://newrelic.com/docs/dotnet/the-net-agent-api" target="_blank">.NET Agent API</a> to customize some of the data reported by our application to their servers. This makes the data way more valuable to us as we can now selectively ignore certain parts of our application while getting better reporting from other, more critical, parts of the application.</p>
]]></summary>
    
      <category term=".NET" scheme="http://improve.dk/category/.NET/"/>
    
  </entry>
  
  <entry>
    <title><![CDATA[Partial DNS Forwarding Using Individual Windows DNS Zones]]></title>
    <link href="http://improve.dk/partial-dns-forwarding-using-individual-windows-dns-zones/"/>
    <id>http://improve.dk/partial-dns-forwarding-using-individual-windows-dns-zones/</id>
    <published>2013-05-21T00:00:00.000Z</published>
    <updated>2014-04-22T10:39:11.000Z</updated>
    <content type="html"><![CDATA[<p>At our office, all machines are using a local Windows DNS server for their outgoing DNS queries. This allows us to make internal zones like .ipaperlan that points to all of our internal systems, while setting up the DNS server to forward all unknown queries to Google DNS. One feature I’m missing in the standard Windows DNS server is the option to partially forward individual zones. However, there is a workaround that will allow you to setup partial DNS forwarding using individual Windows DNS zones.</p>
<a id="more"></a>

<h2 id="The_Scenario">The Scenario</h2>
<p>Imagine you have a domain <em>improve.dk</em> that already has a number of public DNS records like the following.</p>
<div class="imgwrapper" style=""><div><a href="/partial-dns-forwarding-using-individual-windows-dns-zones/Capture.png" class="fancy"><img src="/partial-dns-forwarding-using-individual-windows-dns-zones/Capture.png" style="max-height: 250px"/></a></div></div>

<p>In this case all I want to do is to add a record on our internal network, <em>jira.improve.dk</em>. As this record should only be made available internally, we can’t just add it to the public DNS records for the domain.</p>
<p>I could make a new DNS zone for the improve.dk domain in our local DNS server, but that would result in all DNS queries for improve.dk being answered by our local DNS server, rather than being forwarded. As long as I recreate all public DNS records in our local DNS server, this would work fine, but it’s not a viable solution as I’d now have to keep the two DNS setups in sync manually.</p>
<h2 id="The_Solution">The Solution</h2>
<p>Instead of creating a zone for the whole improve.dk domain, you can make a zone specifically for just the record you need to add. First right click “Forward Lookup Zones” and select “New Zone…” and then follow these steps (pretty much all defaults):</p>
<div class="imgwrapper" style=""><div><a href="/partial-dns-forwarding-using-individual-windows-dns-zones/1.png" class="fancy"><img src="/partial-dns-forwarding-using-individual-windows-dns-zones/1.png" style="max-height: 250px"/></a></div></div>

<div class="imgwrapper" style=""><div><a href="/partial-dns-forwarding-using-individual-windows-dns-zones/2.png" class="fancy"><img src="/partial-dns-forwarding-using-individual-windows-dns-zones/2.png" style="max-height: 250px"/></a></div></div>

<div class="imgwrapper" style=""><div><a href="/partial-dns-forwarding-using-individual-windows-dns-zones/3.png" class="fancy"><img src="/partial-dns-forwarding-using-individual-windows-dns-zones/3.png" style="max-height: 250px"/></a></div></div>

<div class="imgwrapper" style=""><div><a href="/partial-dns-forwarding-using-individual-windows-dns-zones/4.png" class="fancy"><img src="/partial-dns-forwarding-using-individual-windows-dns-zones/4.png" style="max-height: 250px"/></a></div></div>

<div class="imgwrapper" style=""><div><a href="/partial-dns-forwarding-using-individual-windows-dns-zones/5.png" class="fancy"><img src="/partial-dns-forwarding-using-individual-windows-dns-zones/5.png" style="max-height: 250px"/></a></div></div>

<div class="imgwrapper" style=""><div><a href="/partial-dns-forwarding-using-individual-windows-dns-zones/6.png" class="fancy"><img src="/partial-dns-forwarding-using-individual-windows-dns-zones/6.png" style="max-height: 250px"/></a></div></div>

<p>Now that the zone has been created, simply right click it and choose “New Host (A or AAAA)…”. In the dialog, leave the Name blank as that’ll affect the record itself, while entering the desired IP like so:</p>
<div class="imgwrapper" style=""><div><a href="/partial-dns-forwarding-using-individual-windows-dns-zones/6_b.png" class="fancy"><img src="/partial-dns-forwarding-using-individual-windows-dns-zones/6_b.png" style="max-height: 250px"/></a></div></div>

<div class="imgwrapper" style=""><div><a href="/partial-dns-forwarding-using-individual-windows-dns-zones/7.png" class="fancy"><img src="/partial-dns-forwarding-using-individual-windows-dns-zones/7.png" style="max-height: 250px"/></a></div></div>

<p>And just like that, DNS lookups for jira.improve.dk will now be answered locally while all other requests will be forwarded to whatever DNS server is set up as the forwarding server.</p>
<p><strong>One word of warning</strong> - You might not want to do this on Active Directory domain servers as they’re somewhat more finicky about their DNS setup. I’m honestly not aware of what complications might arise, so I’d advice you to be careful or perhaps find another solution.</p>
]]></content>
    <summary type="html"><![CDATA[<p>At our office, all machines are using a local Windows DNS server for their outgoing DNS queries. This allows us to make internal zones like .ipaperlan that points to all of our internal systems, while setting up the DNS server to forward all unknown queries to Google DNS. One feature I’m missing in the standard Windows DNS server is the option to partially forward individual zones. However, there is a workaround that will allow you to setup partial DNS forwarding using individual Windows DNS zones.</p>
]]></summary>
    
      <category term="Windows" scheme="http://improve.dk/category/Windows/"/>
    
  </entry>
  
  <entry>
    <title><![CDATA[OrcaMDF Is Now Available on NuGet]]></title>
    <link href="http://improve.dk/orcamdf-is-now-available-on-nuget/"/>
    <id>http://improve.dk/orcamdf-is-now-available-on-nuget/</id>
    <published>2013-05-13T00:00:00.000Z</published>
    <updated>2014-05-04T16:12:23.000Z</updated>
    <content type="html"><![CDATA[<p>Thanks to Justin Dearing (<a href="http://www.justaprogrammer.net/" target="_blank">b</a>|<a href="https://twitter.com/zippy1981" target="_blank">t</a>), OrcaMDF is now available on <a href="https://www.nuget.org/packages/OrcaMDF.Core" target="_blank">NuGet</a>!</p>
<a id="more"></a>

<p>OrcaMDF being on NuGet means the bar just got lowered even more if you want to try it out. Let me show you how easy it is to read the Adventureworks 2008 R2 Database using OrcaMDF:</p>
<p>To begin, let’s create a vanilla .NET <em>Console Application</em>:</p>
<div class="imgwrapper" style=""><div><a href="/orcamdf-is-now-available-on-nuget/1.png" class="fancy"><img src="/orcamdf-is-now-available-on-nuget/1.png" style="max-height: 250px"/></a></div></div>

<p>Once the solution has been made, right click <em>References</em> and go to <em>Manage NuGet Packages</em>:</p>
<div class="imgwrapper" style=""><div><a href="/orcamdf-is-now-available-on-nuget/2.png" class="fancy"><img src="/orcamdf-is-now-available-on-nuget/2.png" style="max-height: 250px"/></a></div></div>

<p>Once the dialog opens, simply search for <em>OrcaMDF</em> and click the <em>Install</em> button for the OrcaMDF.Core package:</p>
<div class="imgwrapper" style=""><div><a href="/orcamdf-is-now-available-on-nuget/3.png" class="fancy"><img src="/orcamdf-is-now-available-on-nuget/3.png" style="max-height: 250px"/></a></div></div>

<p>When done, you should now see a small green checkmark next to the OrcaMDF.Core package:</p>
<div class="imgwrapper" style=""><div><a href="/orcamdf-is-now-available-on-nuget/4.png" class="fancy"><img src="/orcamdf-is-now-available-on-nuget/4.png" style="max-height: 250px"/></a></div></div>

<p>At this point the OrcaMDF.Core assembly will be available and all you have to do is start using it. For example you could print out all of the products along with their prices by modifying the Program.cs file like so (you’ll have to alter the path to AdventureWorks2008R2_Data.mdf file so it points to a local copy (which must not be in use by SQL Server) on your machine):</p>
<figure class="highlight cs"><pre><span class="keyword">using</span> System;
<span class="keyword">using</span> OrcaMDF.Core.Engine;

namespace ConsoleApplication1
{
	class Program
	{
		<span class="keyword">static</span> <span class="keyword">void</span> Main()
		{
			<span class="keyword">using</span> (<span class="keyword">var</span> db = <span class="keyword">new</span> Database(<span class="string">@"C:\AdventureWorks2008R2_Data.mdf"</span>))
			{
				<span class="keyword">var</span> scanner = <span class="keyword">new</span> DataScanner(db);

				<span class="keyword">foreach</span> (<span class="keyword">var</span> row <span class="keyword">in</span> scanner.ScanTable(<span class="string">"Product"</span>))
				{
					Console.WriteLine(row.Field&lt;<span class="keyword">string</span>&gt;(<span class="string">"Name"</span>));
					Console.WriteLine(<span class="string">"Price: "</span> + row.Field&lt;<span class="keyword">double</span>&gt;(<span class="string">"ListPrice"</span>));
					Console.WriteLine();
				}
			}
		}
	}
}
</pre></figure>

<p>And then just running the solution:</p>
<div class="imgwrapper" style=""><div><a href="/orcamdf-is-now-available-on-nuget/5.png" class="fancy"><img src="/orcamdf-is-now-available-on-nuget/5.png" style="max-height: 250px"/></a></div></div>

<p>And there you have it, in just a few quick short steps you’ve now fetched OrcaMDF and read the Products table, from the standard AdventureWorks 2008 R2 database, without even touching SQL Server.</p>
<p>With OrcaMDF now being available on NuGet as well as with <a href="/orcamdf-studio-release-feature-recap/">a simple GUI</a>, it really doesn’t get any simpler to take it for a spin :)</p>
]]></content>
    <summary type="html"><![CDATA[<p>Thanks to Justin Dearing (<a href="http://www.justaprogrammer.net/" target="_blank">b</a>|<a href="https://twitter.com/zippy1981" target="_blank">t</a>), OrcaMDF is now available on <a href="https://www.nuget.org/packages/OrcaMDF.Core" target="_blank">NuGet</a>!</p>
]]></summary>
    
      <category term=".NET" scheme="http://improve.dk/category/.NET/"/>
    
      <category term="SQL Server - OrcaMDF" scheme="http://improve.dk/category/SQL%20Server%20-%20OrcaMDF/"/>
    
  </entry>
  
  <entry>
    <title><![CDATA[How to Identify Which Request Caused a Runaway Thread, Using Windbg]]></title>
    <link href="http://improve.dk/how-to-identify-which-request-caused-a-runaway-thread-using-windbg/"/>
    <id>http://improve.dk/how-to-identify-which-request-caused-a-runaway-thread-using-windbg/</id>
    <published>2013-05-07T00:00:00.000Z</published>
    <updated>2014-04-22T10:39:11.000Z</updated>
    <content type="html"><![CDATA[<p>When your w3wp process is stuck at 100% like, <a href="/debugging-in-production-part-1-analyzing-100-cpu-usage-using-windbg/">like when I used a non-thread-safe Dictionary concurrently</a>, you may want to identify what request the runaway thread is actually serving. Let me show you how to identify which request caused a runaway thread, using windbg.</p>
<a id="more"></a>

<p>First you’ll want to identify the process ID (PID) of the w3wp process. In my case, that’s <strong>102600</strong>:</p>
<div class="imgwrapper" style=""><div><a href="/how-to-identify-which-request-caused-a-runaway-thread-using-windbg/Taskmgr.png" class="fancy"><img src="/how-to-identify-which-request-caused-a-runaway-thread-using-windbg/Taskmgr.png" style="max-height: 250px"/></a></div></div>

<p>Next you’ll want to start up Windbg (make sure to use the correct bitness (x86 vs x64) that corresponds to the bitness of your process). Once started, press <strong>F6</strong> to open up the <em>Attach to Process</em> dialog. Once open, enter your process ID and click OK.</p>
<div class="imgwrapper" style=""><div><a href="/how-to-identify-which-request-caused-a-runaway-thread-using-windbg/Attach-to-Process.png" class="fancy"><img src="/how-to-identify-which-request-caused-a-runaway-thread-using-windbg/Attach-to-Process.png" style="max-height: 250px"/></a></div></div>

<p>Doing so should bring up the Command window, ready for your command:</p>
<div class="imgwrapper" style=""><div><a href="/how-to-identify-which-request-caused-a-runaway-thread-using-windbg/Windbg11.png" class="fancy"><img src="/how-to-identify-which-request-caused-a-runaway-thread-using-windbg/Windbg11.png" style="max-height: 250px"/></a></div></div>

<p>As the first thing, start out by loading the <a href="http://msdn.microsoft.com/en-us/library/bb190764.aspx" target="_blank">Son of Strike</a> extension, allowing us to debug managed code.</p>
<figure class="highlight"><pre><span class="number">0</span>:<span class="number">039</span>&gt; <span class="preprocessor">.loadby</span> sos <span class="keyword">clr</span>
</pre></figure>

<p>Then continue by running the !runaway command to get a list of runaway (basically threads using lots of CPU) threads:</p>
<figure class="highlight"><pre>0<span class="pseudo">:039</span>&gt; !<span class="tag">runaway</span>

 <span class="tag">User</span> <span class="tag">Mode</span> <span class="tag">Time</span>
  <span class="tag">Thread</span>       <span class="tag">Time</span>
  20<span class="pseudo">:14930</span>      0 <span class="tag">days</span> 0<span class="pseudo">:21</span><span class="pseudo">:44</span><span class="class">.261</span>
  21<span class="pseudo">:15204</span>      0 <span class="tag">days</span> 0<span class="pseudo">:21</span><span class="pseudo">:00</span><span class="class">.878</span>
  27<span class="pseudo">:19d48</span>      0 <span class="tag">days</span> 0<span class="pseudo">:04</span><span class="pseudo">:23</span><span class="class">.860</span>
  32<span class="pseudo">:18748</span>      0 <span class="tag">days</span> 0<span class="pseudo">:02</span><span class="pseudo">:59</span><span class="class">.260</span>
  31<span class="pseudo">:18bcc</span>      0 <span class="tag">days</span> 0<span class="pseudo">:02</span><span class="pseudo">:19</span><span class="class">.277</span>
  30<span class="pseudo">:19d80</span>      0 <span class="tag">days</span> 0<span class="pseudo">:01</span><span class="pseudo">:44</span><span class="class">.083</span>
  25<span class="pseudo">:19ec0</span>      0 <span class="tag">days</span> 0<span class="pseudo">:01</span><span class="pseudo">:32</span><span class="class">.446</span>
  24<span class="pseudo">:16534</span>      0 <span class="tag">days</span> 0<span class="pseudo">:01</span><span class="pseudo">:31</span><span class="class">.135</span>
  29<span class="pseudo">:19a80</span>      0 <span class="tag">days</span> 0<span class="pseudo">:01</span><span class="pseudo">:08</span><span class="class">.297</span>
  23<span class="pseudo">:19110</span>      0 <span class="tag">days</span> 0<span class="pseudo">:00</span><span class="pseudo">:30</span><span class="class">.591</span>
   6<span class="pseudo">:19b40</span>      0 <span class="tag">days</span> 0<span class="pseudo">:00</span><span class="pseudo">:00</span><span class="class">.109</span>
  26<span class="pseudo">:18a14</span>      0 <span class="tag">days</span> 0<span class="pseudo">:00</span><span class="pseudo">:00</span><span class="class">.015</span>
   0<span class="pseudo">:19dcc</span>      0 <span class="tag">days</span> 0<span class="pseudo">:00</span><span class="pseudo">:00</span><span class="class">.015</span>
  39<span class="pseudo">:16fa8</span>      0 <span class="tag">days</span> 0<span class="pseudo">:00</span><span class="pseudo">:00</span><span class="class">.000</span>
  ...
</pre></figure>

<p>Threads 20 &amp; 21 seem to be the interesting ones. Let’s start out by selecting thread #20 as the active thread:</p>
<figure class="highlight"><pre>0<span class="pseudo">:039</span>&gt; ~20<span class="tag">s</span>

000007<span class="tag">fe</span>`913<span class="tag">a15d9</span> 3<span class="tag">bc5</span>            <span class="tag">cmp</span>     <span class="tag">eax</span>,<span class="tag">ebp</span>
</pre></figure>

<p>Once selected, we can analyze the stack and its parameters by running the !CLRStack command with the -p parameter:</p>
<figure class="highlight"><pre><span class="number">0</span>:<span class="number">020</span>&gt; !CLRStack -p

OS Thread Id: <span class="number">0x14930</span> (<span class="number">20</span>)
        Child SP               IP <span class="keyword">Call</span> Site
<span class="number">000000000</span>dccdb00 <span class="number">000007</span>fe913a15d9 System<span class="preprocessor">.Collections</span><span class="preprocessor">.Generic</span><span class="preprocessor">.Dictionary</span>`<span class="number">2</span>[[System<span class="preprocessor">.Int</span>16, mscorlib],[System.__Canon, mscorlib]]<span class="preprocessor">.FindEntry</span>(Int16)
    PARAMETERS:
        this = &lt;no data&gt;
        key = &lt;no data&gt;

<span class="number">000000000</span>dccdb50 <span class="number">000007</span>fe913a14c0 System<span class="preprocessor">.Collections</span><span class="preprocessor">.Generic</span><span class="preprocessor">.Dictionary</span>`<span class="number">2</span>[[System<span class="preprocessor">.Int</span>16, mscorlib],[System.__Canon, mscorlib]]<span class="preprocessor">.get</span>_Item(Int16)
    PARAMETERS:
        this = &lt;no data&gt;
        key = &lt;no data&gt;

<span class="number">000000000</span>dccdb80 <span class="number">000007</span>fe91421cbb iPaper<span class="preprocessor">.BL</span><span class="preprocessor">.Backend</span><span class="preprocessor">.Modules</span><span class="preprocessor">.Languages</span><span class="preprocessor">.LanguageCache</span><span class="preprocessor">.GetLanguageByID</span>(Int32, iPaper<span class="preprocessor">.BL</span><span class="preprocessor">.Backend</span><span class="preprocessor">.Infrastructure</span><span class="preprocessor">.PartnerConfiguration</span><span class="preprocessor">.IPartnerConfig</span>) [e:\iPaperCMS\BL\Backend\Modules\Languages\LanguageCache<span class="preprocessor">.cs</span> @ <span class="number">44</span>]
    PARAMETERS:
        languageID (<span class="number">0x000000000dccdc20</span>) = <span class="number">0x0000000000000001</span>
        partnerConfig (<span class="number">0x000000000dccdc28</span>) = <span class="number">0x00000000fffc3e50</span>

<span class="number">000000000</span>dccdc20 <span class="number">000007</span>fe91421dfa iPaper<span class="preprocessor">.BL</span><span class="preprocessor">.Backend</span><span class="preprocessor">.Modules</span><span class="preprocessor">.Languages</span><span class="preprocessor">.Language</span><span class="preprocessor">.GetFontFileForLanguage</span>(Int32, iPaper<span class="preprocessor">.BL</span><span class="preprocessor">.Backend</span><span class="preprocessor">.Infrastructure</span><span class="preprocessor">.PartnerConfiguration</span><span class="preprocessor">.IPartnerConfig</span>) [e:\iPaperCMS\BL\Backend\Modules\Languages\Language<span class="preprocessor">.cs</span> @ <span class="number">37</span>]
    PARAMETERS:
        languageID (<span class="number">0x000000000dccdc70</span>) = <span class="number">0x0000000000000001</span>
        partnerConfig (<span class="number">0x000000000dccdc78</span>) = <span class="number">0x00000000fffc3e50</span>

<span class="number">000000000</span>dccdc70 <span class="number">000007</span>fe91417400 iPaper<span class="preprocessor">.Web</span><span class="preprocessor">.FlexFrontend</span><span class="preprocessor">.BL</span><span class="preprocessor">.Common</span><span class="preprocessor">.CachedUrlInformation</span><span class="preprocessor">.GetFromUrlDirectoryPath</span>(System<span class="preprocessor">.String</span>, System<span class="preprocessor">.String</span>, iPaper<span class="preprocessor">.BL</span><span class="preprocessor">.Backend</span><span class="preprocessor">.Infrastructure</span><span class="preprocessor">.PartnerConfiguration</span><span class="preprocessor">.IPartnerConfig</span>) [e:\iPaperCMS\Frontend\BL\Common\CachedUrlInformation<span class="preprocessor">.cs</span> @ <span class="number">89</span>]
    PARAMETERS:
        url (<span class="number">0x000000000dccde80</span>) = <span class="number">0x00000003fff27e30</span>
        host (<span class="number">0x000000000dccde88</span>) = <span class="number">0x00000003fff29618</span>
        partnerConfig (<span class="number">0x000000000dccde90</span>) = <span class="number">0x00000000fffc3e50</span>

<span class="number">000000000</span>dccde80 <span class="number">000007</span>fe91417576 iPaper<span class="preprocessor">.Web</span><span class="preprocessor">.FlexFrontend</span><span class="preprocessor">.BL</span><span class="preprocessor">.Common</span><span class="preprocessor">.CachedUrlInformation</span><span class="preprocessor">.GetFromHttpContext</span>(System<span class="preprocessor">.String</span>, System<span class="preprocessor">.Web</span><span class="preprocessor">.HttpContext</span>, iPaper<span class="preprocessor">.BL</span><span class="preprocessor">.Backend</span><span class="preprocessor">.Infrastructure</span><span class="preprocessor">.PartnerConfiguration</span><span class="preprocessor">.IPartnerConfig</span>) [e:\iPaperCMS\Frontend\BL\Common\CachedUrlInformation<span class="preprocessor">.cs</span> @ <span class="number">122</span>]
    PARAMETERS:
        paperPath (<span class="number">0x000000000dcce010</span>) = <span class="number">0x00000003fff27e30</span>
        context (<span class="number">0x000000000dcce018</span>) = <span class="number">0x00000000fffa6040</span>
        partnerConfig (<span class="number">0x000000000dcce020</span>) = <span class="number">0x00000000fffc3e50</span>

<span class="number">000000000</span>dcce010 <span class="number">000007</span>fe91415529 iPaper<span class="preprocessor">.Web</span><span class="preprocessor">.FlexFrontend</span><span class="preprocessor">.BL</span><span class="preprocessor">.RequestHandler</span><span class="preprocessor">.RequestHandler</span><span class="preprocessor">.loadFrontendContext</span>(System<span class="preprocessor">.String</span>) [e:\iPaperCMS\Frontend\BL\RequestHandler\RequestHandler<span class="preprocessor">.cs</span> @ <span class="number">469</span>]
    PARAMETERS:
        this (<span class="number">0x000000000dcce260</span>) = <span class="number">0x00000000fffa9590</span>
        paperPath (<span class="number">0x000000000dcce268</span>) = <span class="number">0x00000003fff27e30</span>

<span class="number">000000000</span>dcce260 <span class="number">000007</span>fe91414b73 iPaper<span class="preprocessor">.Web</span><span class="preprocessor">.FlexFrontend</span><span class="preprocessor">.BL</span><span class="preprocessor">.RequestHandler</span><span class="preprocessor">.RequestHandler</span><span class="preprocessor">.context</span>_PostAcquireRequestState(System<span class="preprocessor">.Object</span>, System<span class="preprocessor">.EventArgs</span>) [e:\iPaperCMS\Frontend\BL\RequestHandler\RequestHandler<span class="preprocessor">.cs</span> @ <span class="number">95</span>]
    PARAMETERS:
        this (<span class="number">0x000000000dcce5f0</span>) = <span class="number">0x00000000fffa9590</span>
        sender (<span class="number">0x000000000dcce5f8</span>) = <span class="number">0x00000000fffa8a50</span>
        e (<span class="number">0x000000000dcce600</span>) = <span class="number">0x00000000fffaebb0</span>

<span class="number">000000000</span>dcce5f0 <span class="number">000007</span>fedb72c520 System<span class="preprocessor">.Web</span><span class="preprocessor">.HttpApplication</span>+SyncEventExecutionStep<span class="preprocessor">.System</span><span class="preprocessor">.Web</span><span class="preprocessor">.HttpApplication</span><span class="preprocessor">.IExecutionStep</span><span class="preprocessor">.Execute</span>()
    PARAMETERS:
        this = &lt;no data&gt;

<span class="number">000000000</span>dcce650 <span class="number">000007</span>fedb70b745 System<span class="preprocessor">.Web</span><span class="preprocessor">.HttpApplication</span><span class="preprocessor">.ExecuteStep</span>(IExecutionStep, Boolean ByRef)
    PARAMETERS:
        this (<span class="number">0x000000000dcce6f0</span>) = <span class="number">0x00000000fffa8a50</span>
        step (<span class="number">0x000000000dcce6f8</span>) = <span class="number">0x00000000fffabc28</span>
        completedSynchronously (<span class="number">0x000000000dcce700</span>) = <span class="number">0x000000000dcce77a</span>

<span class="number">000000000</span>dcce6f0 <span class="number">000007</span>fedb72a4e1 System<span class="preprocessor">.Web</span><span class="preprocessor">.HttpApplication</span>+PipelineStepManager<span class="preprocessor">.ResumeSteps</span>(System<span class="preprocessor">.Exception</span>)
    PARAMETERS:
        this (<span class="number">0x000000000dcce7d0</span>) = <span class="number">0x00000000fffac718</span>
        error = &lt;no data&gt;

<span class="number">000000000</span>dcce7d0 <span class="number">000007</span>fedb70b960 System<span class="preprocessor">.Web</span><span class="preprocessor">.HttpApplication</span><span class="preprocessor">.BeginProcessRequestNotification</span>(System<span class="preprocessor">.Web</span><span class="preprocessor">.HttpContext</span>, System<span class="preprocessor">.AsyncCallback</span>)
    PARAMETERS:
        this = &lt;no data&gt;
        context = &lt;no data&gt;
        cb = &lt;no data&gt;

<span class="number">000000000</span>dcce820 <span class="number">000007</span>fedb704c8e System<span class="preprocessor">.Web</span><span class="preprocessor">.HttpRuntime</span><span class="preprocessor">.ProcessRequestNotificationPrivate</span>(System<span class="preprocessor">.Web</span><span class="preprocessor">.Hosting</span><span class="preprocessor">.IIS</span>7WorkerRequest, System<span class="preprocessor">.Web</span><span class="preprocessor">.HttpContext</span>)
    PARAMETERS:
        this (<span class="number">0x000000000dcce8c0</span>) = <span class="number">0x00000000fff3fb20</span>
        wr (<span class="number">0x000000000dcce8c8</span>) = <span class="number">0x00000000fffa5eb0</span>
        context (<span class="number">0x000000000dcce8d0</span>) = <span class="number">0x00000000fffa6040</span>

<span class="number">000000000</span>dcce8c0 <span class="number">000007</span>fedb70e771 System<span class="preprocessor">.Web</span><span class="preprocessor">.Hosting</span><span class="preprocessor">.PipelineRuntime</span><span class="preprocessor">.ProcessRequestNotificationHelper</span>(IntPtr, IntPtr, IntPtr, Int32)
    PARAMETERS:
        rootedObjectsPointer = &lt;no data&gt;
        nativeRequestContext (<span class="number">0x000000000dccea58</span>) = <span class="number">0x0000000000ccccc0</span>
        moduleData = &lt;no data&gt;
        flags = &lt;no data&gt;

<span class="number">000000000</span>dccea50 <span class="number">000007</span>fedb70e2c2 System<span class="preprocessor">.Web</span><span class="preprocessor">.Hosting</span><span class="preprocessor">.PipelineRuntime</span><span class="preprocessor">.ProcessRequestNotification</span>(IntPtr, IntPtr, IntPtr, Int32)
    PARAMETERS:
        rootedObjectsPointer = &lt;no data&gt;
        nativeRequestContext = &lt;no data&gt;
        moduleData = &lt;no data&gt;
        flags = &lt;no data&gt;

<span class="number">000000000</span>dcceaa0 <span class="number">000007</span>fedbe6b461 DomainNeutralILStubClass<span class="preprocessor">.IL</span>_STUB_ReversePInvoke(Int64, Int64, Int64, Int32)
    PARAMETERS:
        &lt;no data&gt;
        &lt;no data&gt;
        &lt;no data&gt;
        &lt;no data&gt;

<span class="number">000000000</span>dccf298 <span class="number">000007</span>fef0a9334e [InlinedCallFrame: <span class="number">000000000</span>dccf298] System<span class="preprocessor">.Web</span><span class="preprocessor">.Hosting</span><span class="preprocessor">.UnsafeIISMethods</span><span class="preprocessor">.MgdIndicateCompletion</span>(IntPtr, System<span class="preprocessor">.Web</span><span class="preprocessor">.RequestNotificationStatus</span> ByRef)
<span class="number">000000000</span>dccf298 <span class="number">000007</span>fedb7b9c4b [InlinedCallFrame: <span class="number">000000000</span>dccf298] System<span class="preprocessor">.Web</span><span class="preprocessor">.Hosting</span><span class="preprocessor">.UnsafeIISMethods</span><span class="preprocessor">.MgdIndicateCompletion</span>(IntPtr, System<span class="preprocessor">.Web</span><span class="preprocessor">.RequestNotificationStatus</span> ByRef)
<span class="number">000000000</span>dccf270 <span class="number">000007</span>fedb7b9c4b DomainNeutralILStubClass<span class="preprocessor">.IL</span>_STUB_PInvoke(IntPtr, System<span class="preprocessor">.Web</span><span class="preprocessor">.RequestNotificationStatus</span> ByRef)
    PARAMETERS:
        &lt;no data&gt;
        &lt;no data&gt;

<span class="number">000000000</span>dccf340 <span class="number">000007</span>fedb70e923 System<span class="preprocessor">.Web</span><span class="preprocessor">.Hosting</span><span class="preprocessor">.PipelineRuntime</span><span class="preprocessor">.ProcessRequestNotificationHelper</span>(IntPtr, IntPtr, IntPtr, Int32)
    PARAMETERS:
        rootedObjectsPointer = &lt;no data&gt;
        nativeRequestContext = &lt;no data&gt;
        moduleData = &lt;no data&gt;
        flags = &lt;no data&gt;

<span class="number">000000000</span>dccf4d0 <span class="number">000007</span>fedb70e2c2 System<span class="preprocessor">.Web</span><span class="preprocessor">.Hosting</span><span class="preprocessor">.PipelineRuntime</span><span class="preprocessor">.ProcessRequestNotification</span>(IntPtr, IntPtr, IntPtr, Int32)
    PARAMETERS:
        rootedObjectsPointer = &lt;no data&gt;
        nativeRequestContext = &lt;no data&gt;
        moduleData = &lt;no data&gt;
        flags = &lt;no data&gt;

<span class="number">000000000</span>dccf520 <span class="number">000007</span>fedbe6b461 DomainNeutralILStubClass<span class="preprocessor">.IL</span>_STUB_ReversePInvoke(Int64, Int64, Int64, Int32)
    PARAMETERS:
        &lt;no data&gt;
        &lt;no data&gt;
        &lt;no data&gt;
        &lt;no data&gt;

<span class="number">000000000</span>dccf768 <span class="number">000007</span>fef0a935a3 [ContextTransitionFrame: <span class="number">000000000</span>dccf768]
</pre></figure>

<p>This returns the full stack with a lot of frames that we’re not really interested in. What we’re looking for is the first instance of an HttpContext. If we start from the bottom and work our way up, this seems to be the first time an HttpContext is present:</p>
<figure class="highlight"><pre><span class="number">000000000</span>dcce820 <span class="number">000007</span>fedb704c8e System<span class="preprocessor">.Web</span><span class="preprocessor">.HttpRuntime</span><span class="preprocessor">.ProcessRequestNotificationPrivate</span>(System<span class="preprocessor">.Web</span><span class="preprocessor">.Hosting</span><span class="preprocessor">.IIS</span>7WorkerRequest, System<span class="preprocessor">.Web</span><span class="preprocessor">.HttpContext</span>)
    PARAMETERS:
        this (<span class="number">0x000000000dcce8c0</span>) = <span class="number">0x00000000fff3fb20</span>
        wr (<span class="number">0x000000000dcce8c8</span>) = <span class="number">0x00000000fffa5eb0</span>
        context (<span class="number">0x000000000dcce8d0</span>) = <span class="number">0x00000000fffa6040</span>
</pre></figure>

<p>Knowing that the HttpContext contains a reference to an HttpRequest, and that HttpRequest contains the RawUrl string value, we’ll start digging in. Start out by dumping the HttpContext object using the !do command:</p>
<figure class="highlight"><pre><span class="number">0</span>:<span class="number">020</span>&gt; !do <span class="number">0x00000000fffa6040</span>

<span class="label">Name:</span>        System<span class="preprocessor">.Web</span><span class="preprocessor">.HttpContext</span>
<span class="label">MethodTable:</span> <span class="number">000007</span>fedb896398
<span class="label">EEClass:</span>     <span class="number">000007</span>fedb4882e0
<span class="label">Size:</span>        <span class="number">416</span>(<span class="number">0x1a0</span>) bytes
<span class="label">File:</span>        C:\Windows\Microsoft<span class="preprocessor">.Net</span>\assembly\GAC_64\System<span class="preprocessor">.Web</span>\v4<span class="number">.0</span>_4<span class="number">.0</span><span class="number">.0</span><span class="number">.0</span>__b03f5f7f11d50a3a\System<span class="preprocessor">.Web</span><span class="preprocessor">.dll</span>
<span class="label">Fields:</span>
              MT    Field   Offset                 Type VT     Attr            Value Name
<span class="number">000007</span>fedb897c80  <span class="number">40010</span>a3        <span class="number">8</span> ..<span class="preprocessor">.IHttpAsyncHandler</span>  <span class="number">0</span> instance <span class="number">0000000000000000</span> _asyncAppHandler
<span class="number">000007</span>fedb88e618  <span class="number">40010</span>a4      <span class="number">158</span>         System<span class="preprocessor">.Int</span>32  <span class="number">1</span> instance                <span class="number">0</span> _asyncPreloadModeFlags
<span class="number">000007</span>feef9fdc30  <span class="number">40010</span>a5      <span class="number">168</span>       System<span class="preprocessor">.Boolean</span>  <span class="number">1</span> instance                <span class="number">0</span> _asyncPreloadModeFlagsSet
<span class="number">000007</span>fedb895610  <span class="number">40010</span>a6       <span class="number">10</span> ..<span class="preprocessor">.b</span><span class="preprocessor">.HttpApplication</span>  <span class="number">0</span> instance <span class="number">00000000</span>fffa8a50 _appInstance
<span class="number">000007</span>fedb897ce8  <span class="number">40010</span>a7       <span class="number">18</span> ...<span class="preprocessor">.Web</span><span class="preprocessor">.IHttpHandler</span>  <span class="number">0</span> instance <span class="number">00000003</span>fff28c20 _handler
<span class="number">000007</span>fedb898170  <span class="number">40010</span>a8       <span class="number">20</span> ..<span class="preprocessor">.m</span><span class="preprocessor">.Web</span><span class="preprocessor">.HttpRequest</span>  <span class="number">0</span> instance <span class="number">00000000</span>fffa61f8 _request
<span class="number">000007</span>fedb898550  <span class="number">40010</span>a9       <span class="number">28</span> ...<span class="preprocessor">.Web</span><span class="preprocessor">.HttpResponse</span>  <span class="number">0</span> instance <span class="number">00000000</span>fffa6378 _response
<span class="number">000007</span>fedb893cb0  <span class="number">40010</span>aa       <span class="number">30</span> ..<span class="preprocessor">.HttpServerUtility</span>  <span class="number">0</span> instance <span class="number">00000003</span>fff27ed8 _server
<span class="number">000007</span>feefa05ac0  <span class="number">40010</span>ab       <span class="number">38</span> ..<span class="preprocessor">.Collections</span><span class="preprocessor">.Stack</span>  <span class="number">0</span> instance <span class="number">0000000000000000</span> _traceContextStack
<span class="number">000007</span>fedb8a41d8  <span class="number">40010</span>ac       <span class="number">40</span> ...<span class="preprocessor">.Web</span><span class="preprocessor">.TraceContext</span>  <span class="number">0</span> instance <span class="number">0000000000000000</span> _topTraceContext
<span class="number">000007</span>feefa00548  <span class="number">40010</span>ad       <span class="number">48</span> ..<span class="preprocessor">.ections</span><span class="preprocessor">.Hashtable</span>  <span class="number">0</span> instance <span class="number">00000000</span>fffab198 _items
<span class="number">000007</span>feef9f85e0  <span class="number">40010</span>ae       <span class="number">50</span> ..<span class="preprocessor">.ections</span><span class="preprocessor">.ArrayList</span>  <span class="number">0</span> instance <span class="number">0000000000000000</span> _errors
<span class="number">000007</span>feef9fc588  <span class="number">40010</span>af       <span class="number">58</span>     System<span class="preprocessor">.Exception</span>  <span class="number">0</span> instance <span class="number">0000000000000000</span> _tempError
...
</pre></figure>

<p>This contains a lot of fields (some of which I’ve snipped out). The interesting part however, is this line:</p>
<figure class="highlight"><pre><span class="number">000007</span>fedb898170  <span class="number">40010</span>a8       <span class="number">20</span> ..<span class="preprocessor">.m</span><span class="preprocessor">.Web</span><span class="preprocessor">.HttpRequest</span>  <span class="number">0</span> instance <span class="number">00000000</span>fffa61f8 _request
</pre></figure>

<p>This contains a pointer to the HttpRequest instance. Let’s try dumping that one:</p>
<figure class="highlight"><pre><span class="number">0</span>:<span class="number">020</span>&gt; !do <span class="number">00000000</span>fffa61f8 

<span class="label">Name:</span>        System<span class="preprocessor">.Web</span><span class="preprocessor">.HttpRequest</span>
<span class="label">MethodTable:</span> <span class="number">000007</span>fedb898170
<span class="label">EEClass:</span>     <span class="number">000007</span>fedb488c00
<span class="label">Size:</span>        <span class="number">384</span>(<span class="number">0x180</span>) bytes
<span class="label">File:</span>        C:\Windows\Microsoft<span class="preprocessor">.Net</span>\assembly\GAC_64\System<span class="preprocessor">.Web</span>\v4<span class="number">.0</span>_4<span class="number">.0</span><span class="number">.0</span><span class="number">.0</span>__b03f5f7f11d50a3a\System<span class="preprocessor">.Web</span><span class="preprocessor">.dll</span>
<span class="label">Fields:</span>
              MT    Field   Offset                 Type VT     Attr            Value Name
<span class="number">000007</span>fedb89aa30  <span class="number">4001150</span>        <span class="number">8</span> ..<span class="preprocessor">.HttpWorkerRequest</span>  <span class="number">0</span> instance <span class="number">00000000</span>fffa5eb0 _wr
<span class="number">000007</span>fedb896398  <span class="number">4001151</span>       <span class="number">10</span> ..<span class="preprocessor">.m</span><span class="preprocessor">.Web</span><span class="preprocessor">.HttpContext</span>  <span class="number">0</span> instance <span class="number">00000000</span>fffa6040 _context
...
<span class="number">000007</span>fee6e1dc48  <span class="number">4001165</span>       <span class="number">90</span>           System<span class="preprocessor">.Uri</span>  <span class="number">0</span> instance <span class="number">00000003</span>fff29588 _url
<span class="number">000007</span>fee6e1dc48  <span class="number">4001166</span>       <span class="number">98</span>           System<span class="preprocessor">.Uri</span>  <span class="number">0</span> instance <span class="number">0000000000000000</span> _referrer
<span class="number">000007</span>fedb900718  <span class="number">4001167</span>       a0 ..<span class="preprocessor">.b</span><span class="preprocessor">.HttpInputStream</span>  <span class="number">0</span> instance <span class="number">0000000000000000</span> _inputStream
<span class="number">000007</span>fedb8c43d0  <span class="number">4001168</span>       a8 ..<span class="preprocessor">.ClientCertificate</span>  <span class="number">0</span> instance <span class="number">0000000000000000</span> _clientCertificate
<span class="number">000007</span>feefa07e90  <span class="number">4001169</span>       b0 ..<span class="preprocessor">.l</span><span class="preprocessor">.WindowsIdentity</span>  <span class="number">0</span> instance <span class="number">0000000000000000</span> _logonUserIdentity
<span class="number">000007</span>fedb8d7fd0  <span class="number">400116</span>a       b8 ..<span class="preprocessor">.ng</span><span class="preprocessor">.RequestContext</span>  <span class="number">0</span> instance <span class="number">0000000000000000</span> _requestContext
<span class="number">000007</span>feef9fc358  <span class="number">400116</span>b       c0        System<span class="preprocessor">.String</span>  <span class="number">0</span> instance <span class="number">00000000</span>fffa64f0 _rawUrl
<span class="number">000007</span>feefa008b8  <span class="number">400116</span>c       c8     System<span class="preprocessor">.IO</span><span class="preprocessor">.Stream</span>  <span class="number">0</span> instance <span class="number">0000000000000000</span> _readEntityBodyStream
<span class="number">000007</span>fedb8d5ac8  <span class="number">400116</span>d      <span class="number">160</span>         System<span class="preprocessor">.Int</span>32  <span class="number">1</span> instance                <span class="number">0</span> _readEntityBodyMode
<span class="number">000007</span>fedb8bbcb0  <span class="number">400116</span>e       d0 ..<span class="preprocessor">.atedRequestValues</span>  <span class="number">0</span> instance <span class="number">00000003</span>fff27fe8 _unvalidatedRequestValues
...
</pre></figure>

<p>Once again there are a lot of fields that we don’t care about. The interesting one is this one:</p>
<figure class="highlight"><pre><span class="number">000007f</span>eef9fc358  <span class="number">400116</span>b       c0        System.String  <span class="number">0</span> instance <span class="number">00000000f</span>ffa64f0 _rawUrl
</pre></figure>

<p>Dumping the RawUrl property reveals the actual URL that made the request which eventually ended up causing a runaway thread:</p>
<figure class="highlight"><pre><span class="number">0</span>:<span class="number">020</span>&gt; !do <span class="number">00000000</span>fffa64f0 

<span class="label">Name:</span>        System<span class="preprocessor">.String</span>
<span class="label">MethodTable:</span> <span class="number">000007</span>feef9fc358
<span class="label">EEClass:</span>     <span class="number">000007</span>feef363720
<span class="label">Size:</span>        <span class="number">150</span>(<span class="number">0x96</span>) bytes
<span class="label">File:</span>        C:\Windows\Microsoft<span class="preprocessor">.Net</span>\assembly\GAC_64\mscorlib\v4<span class="number">.0</span>_4<span class="number">.0</span><span class="number">.0</span><span class="number">.0</span>__b77a5c561934e089\mscorlib<span class="preprocessor">.dll</span>
<span class="label">String:</span>      /Catalogs/SomeClient/Uge45/Image<span class="preprocessor">.ashx</span>?PageNumber=<span class="number">1</span>&ImageType=Thumb
<span class="label">Fields:</span>
              MT    Field   Offset                 Type VT     Attr            Value Name
<span class="number">000007</span>feef9ff108  <span class="number">40000</span>aa        <span class="number">8</span>         System<span class="preprocessor">.Int</span>32  <span class="number">1</span> instance               <span class="number">62</span> m_stringLength
<span class="number">000007</span>feef9fd640  <span class="number">40000</span>ab        c          System<span class="preprocessor">.Char</span>  <span class="number">1</span> instance               <span class="number">2</span>f m_firstChar
<span class="number">000007</span>feef9fc358  <span class="number">40000</span>ac       <span class="number">18</span>        System<span class="preprocessor">.String</span>  <span class="number">0</span>   shared           static Empty
                                 &gt;&gt; Domain:Value  <span class="number">0000000001</span>ec80e0:NotInit  <span class="number">0000000001</span>f8e840:NotInit
</pre></figure>

<p>And there we go! The offending URL seems to be:</p>
<figure class="highlight"><pre>/Catalogs/SomeClient/Uge45/<span class="keyword">Image</span>.ashx?PageNumber=<span class="number">1</span>&<span class="keyword">ImageType</span>=Thumb
</pre></figure>

<p>If you want the complete URL, including hostname, you could dig your way into the _url field on the HttpRequest object and work your way from there. In just the same way you can dig into pretty much any object, whether it’s in your code or in the IIS codebase.</p>
]]></content>
    <summary type="html"><![CDATA[<p>When your w3wp process is stuck at 100% like, <a href="/debugging-in-production-part-1-analyzing-100-cpu-usage-using-windbg/">like when I used a non-thread-safe Dictionary concurrently</a>, you may want to identify what request the runaway thread is actually serving. Let me show you how to identify which request caused a runaway thread, using windbg.</p>
]]></summary>
    
      <category term=".NET" scheme="http://improve.dk/category/.NET/"/>
    
      <category term="IIS" scheme="http://improve.dk/category/IIS/"/>
    
      <category term="Windbg" scheme="http://improve.dk/category/Windbg/"/>
    
  </entry>
  
  <entry>
    <title><![CDATA[Debugging in Production Part 3 - Thread-Safe Dictionaries]]></title>
    <link href="http://improve.dk/debugging-in-production-part-3-thread-safe-dictionaries/"/>
    <id>http://improve.dk/debugging-in-production-part-3-thread-safe-dictionaries/</id>
    <published>2013-04-30T00:00:00.000Z</published>
    <updated>2014-05-04T16:12:23.000Z</updated>
    <content type="html"><![CDATA[<p><a href="/debugging-in-production-part-2-latent-race-condition-bugs">In part 2 we found out that the concurrent access to a generic dictionary triggered a race condition bug</a> that caused threads to get stuck at 100% CPU usage. In this part, I’ll show how easy it is to rewrite the code, using the new thread-safe dictionaries in .NET 4.0, so it’s protected from race condition bugs like the one I encountered.</p>
<a id="more"></a>


<h2 id="Enter_ConcurrentDictionary">Enter ConcurrentDictionary</h2>
<p>The problem can be solved by changing just two lines of code. Instead of using a generic Dictionary, we’ll change it to a generic ConcurrentDictionary like so:</p>
<figure class="highlight cs"><pre><span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">readonly</span> ConcurrentDictionary&lt;<span class="keyword">short</span>, ConcurrentDictionary&lt;SettingDescription, SettingDescriptionContainer&gt;&gt; cache =
	<span class="keyword">new</span> ConcurrentDictionary&lt;<span class="keyword">short</span>, ConcurrentDictionary&lt;SettingDescription, SettingDescriptionContainer&gt;&gt;();
</pre></figure>

<p>As described by this <a href="http://msdn.microsoft.com/en-us/library/dd997369.aspx" target="_blank">MSDN article on adding and removing items from a ConcurrentDictionary</a>, it’s fully thread-safe:</p>
<blockquote>
<p>ConcurrentDictionary&lt;TKey, TValue&gt; is designed for multithreaded scenarios. You do not have to use locks in your code to add or remove items from the collection.</p>
</blockquote>
<p>Performance wise ConcurrentDictionary is about 50% slower (anecdotally) than the regular Dictionary type but even if this code is run very often, that is absolutely negligible compared to making just a single database access call.</p>
<p>Besides switching the Dictionary out with a ConcurrentDictionary, we also need to modify the init function since the ConcurrentDictionary way of adding items is slightly different:</p>
<figure class="highlight cs"><pre><span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">object</span> syncRoot = <span class="keyword">new</span> <span class="keyword">object</span>();

<span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">void</span> <span class="title">init</span>(IPartnerConfig partnerConfig)
{
	<span class="comment">// We only want one inside the init method at a time</span>
	<span class="keyword">lock</span> (syncRoot)
	{
		<span class="keyword">if</span> (cache.ContainsKey(partnerConfig.PartnerID))
			<span class="keyword">return</span>;

		<span class="keyword">var</span> dict = <span class="keyword">new</span> ConcurrentDictionary&lt;SettingDescription, SettingDescriptionContainer&gt;();

		... <span class="comment">// Populate the dict variable with data from the database</span>

		cache.AddOrUpdate(partnerConfig.PartnerID, dict, (k, ov) =&gt; dict);
	}
}
</pre></figure>

<p>The syncRoot lock ensures that only one initialization is going on at the same time. While not necessary in regards of avoiding the race condition, this will avoid hitting the database multiple times if the init method is being called concurrently. This could be optimized in that there could be a syncRoot object per PartnerID to allow concurrently initializing the cache for each PartneriD. But, alas, I opt to keep it simple as the init method is only called once in the lifetime of the application.</p>
<p>Instead of just adding an item to the cache, we have to use the AddOrUpdate() signature that takes in the key, value and a lambda that returns a new value, in case the key already exists in the dictionary. In this case, no matter if the key exists or not, we want to set it to the new value, so the lambda just returns the same value as passed in the second parameter.</p>
]]></content>
    <summary type="html"><![CDATA[<p><a href="/debugging-in-production-part-2-latent-race-condition-bugs">In part 2 we found out that the concurrent access to a generic dictionary triggered a race condition bug</a> that caused threads to get stuck at 100% CPU usage. In this part, I’ll show how easy it is to rewrite the code, using the new thread-safe dictionaries in .NET 4.0, so it’s protected from race condition bugs like the one I encountered.</p>
]]></summary>
    
      <category term=".NET" scheme="http://improve.dk/category/.NET/"/>
    
      <category term="Windbg" scheme="http://improve.dk/category/Windbg/"/>
    
  </entry>
  
  <entry>
    <title><![CDATA[Debugging in Production Part 2 - Latent Race Condition Bugs]]></title>
    <link href="http://improve.dk/debugging-in-production-part-2-latent-race-condition-bugs/"/>
    <id>http://improve.dk/debugging-in-production-part-2-latent-race-condition-bugs/</id>
    <published>2013-04-15T00:00:00.000Z</published>
    <updated>2014-05-04T16:12:23.000Z</updated>
    <content type="html"><![CDATA[<p>Having <a href="/debugging-in-production-part-1-analyzing-100-cpu-usage-using-windbg">analyzed the process dump in part 1</a>, let’s take a look at the code we suspect of causing the issue, in particular how race condition bugs can be avoided.</p>
<a id="more"></a>


<h2 id="Looking_at_the_User_Code">Looking at the User Code</h2>
<p>There were three methods in action, all of them in the SettingDescriptionCache class: GetAllDescriptions, init and GetAllDescriptionsAsDictionary. GetAllDescriptions and GetAllDescriptionsAsDictionary are for all intents and purposes identical and both implement a pattern like this:</p>
<figure class="highlight cs"><pre><span class="keyword">public</span> <span class="keyword">static</span> IEnumerable&lt;SettingDescriptionContainer&gt; <span class="title">GetAllDescriptions</span>(IPartnerConfig partnerConfig)
{
	<span class="comment">// Optimistic return. If it fails we'll populate the cache and return it.</span>
	<span class="keyword">try</span>
	{
		<span class="keyword">return</span> cache[partnerConfig.PartnerID].Values;
	}
	<span class="keyword">catch</span> (KeyNotFoundException)
	{
		init(partnerConfig);
	}

	<span class="keyword">return</span> cache[partnerConfig.PartnerID].Values;
}
</pre></figure>

<p>Both methods access a static variable defined in the class like so:</p>
<figure class="highlight cs"><pre><span class="keyword">private</span> <span class="keyword">static</span> <span class="keyword">readonly</span> Dictionary&lt;<span class="keyword">short</span>, Dictionary&lt;SettingDescription, SettingDescriptionContainer&gt;&gt; cache =
	<span class="keyword">new</span> Dictionary&lt;<span class="keyword">short</span>, Dictionary&lt;SettingDescription, SettingDescriptionContainer&gt;&gt;();
</pre></figure>

<p>As this code is being called quite a lot, it’s written using an optimistic pattern that assumes the cache is populated. This is faster than checking if the cache is populated beforehand, or performing a TryGet(). I’ve previously blogged about <a href="/defending-against-the-improbable/">why you shouldn’t defend against the improbable</a>.</p>
<h2 id="Dictionaries_are_Not_Thread_Safe">Dictionaries are Not Thread Safe</h2>
<p>Looking up the <a href="http://msdn.microsoft.com/en-us/library/dd997305.aspx" target="_blank">MSDN article on thread-safe collections</a>, you’ll notice the following paragraph describes how the standard Dictionary collections are not thread-safe:</p>
<blockquote>
<p>The collection classes introduced in the .NET Framework 2.0 are found in the System.Collections.Generic namespace. These include List&lt;T&gt;, Dictionary&lt;TKey, TValue&gt;, and so on. These classes provide improved type safety and performance compared to the .NET Framework 1.0 classes. However, the .NET Framework 2.0 collection classes do not provide any thread synchronization; user code must provide all synchronization when items are added or removed on multiple threads concurrently.</p>
</blockquote>
<p>But is this the issue we’re running into? As there are two dictionaries in action, either one of them could potentially be the culprit. If the partnerConfig.PartnerID value was the same there would be a somewhat higher chance of this really being the issue - but how can find out what PartnerID values were being passed in to the methods?</p>
<h2 id="Analyzing_Method_Parameters_Using_Windbg">Analyzing Method Parameters Using Windbg</h2>
<p>Back in Windbg, for each of the threads we can run the !CLRStack command once again, but with the -p parameter. This doesn’t just list the stack trace, but also all of the parameters for each frame.</p>
<figure class="highlight"><pre>~232s
<span class="change">!CLRStack -p</span>
</pre></figure>

<div class="imgwrapper" style=""><div><a href="/debugging-in-production-part-2-latent-race-condition-bugs/Windbg5.png" class="fancy"><img src="/debugging-in-production-part-2-latent-race-condition-bugs/Windbg5.png" style="max-height: 250px"/></a></div></div>

<p>In the fifth frame, there’s a value for the IPartnerConfig parameter:</p>
<figure class="highlight"><pre>iPaper<span class="preprocessor">.BL</span><span class="preprocessor">.Backend</span><span class="preprocessor">.Modules</span><span class="preprocessor">.Paper</span><span class="preprocessor">.Settings</span><span class="preprocessor">.SettingDescriptionCache</span><span class="preprocessor">.GetAllDescriptions</span>(iPaper<span class="preprocessor">.BL</span><span class="preprocessor">.Backend</span><span class="preprocessor">.Infrastructure</span><span class="preprocessor">.PartnerConfiguration</span><span class="preprocessor">.IPartnerConfig</span>)
	PARAMETERS:
		partnerConfig (<span class="number">0x00000000543ac650</span>) = <span class="number">0x0000000260a7bd98</span>
</pre></figure>

<p>The left side value is the local memory address of the pointer itself whilst the right side is the memory location where the actual PartnerConfig instance is stored. By issuing the do (dump object) command, we can inspect the value itself:</p>
<figure class="highlight"><pre>!<span class="keyword">do</span> <span class="number">0</span>x0000000260a7bd98
</pre></figure>

<div class="imgwrapper" style=""><div><a href="/debugging-in-production-part-2-latent-race-condition-bugs/Windbg6.png" class="fancy"><img src="/debugging-in-production-part-2-latent-race-condition-bugs/Windbg6.png" style="max-height: 250px"/></a></div></div>

<p>If you look under the Name column then you’ll be able to pinpoint the individual fields in the PartnerConfiguration instance. In the Value column you can see that the PartnerID field has a value of 230. Doing this for the other four threads yields the same result - all of them are trying to access the cache value belonging to the PartnerID value of 230!</p>
<p>At this point I can quite confidently say that I’m sure this is a threading issue related to the non thread-safe Dictionary usage. I would’ve expected hard failures like like KeyNotFoundException, NullReferenceException and so on. But apparently, under the exact right race conditions, the dictionaries may get stuck at 100% CPU usage.</p>
<p>Stay tuned for part 3 where I’ll show how to use the Dictionaries in a safe way that avoids issues like these!</p>
]]></content>
    <summary type="html"><![CDATA[<p>Having <a href="/debugging-in-production-part-1-analyzing-100-cpu-usage-using-windbg">analyzed the process dump in part 1</a>, let’s take a look at the code we suspect of causing the issue, in particular how race condition bugs can be avoided.</p>
]]></summary>
    
      <category term=".NET" scheme="http://improve.dk/category/.NET/"/>
    
      <category term="IIS" scheme="http://improve.dk/category/IIS/"/>
    
      <category term="Tools of the Trade" scheme="http://improve.dk/category/Tools%20of%20the%20Trade/"/>
    
      <category term="Windbg" scheme="http://improve.dk/category/Windbg/"/>
    
  </entry>
  
</feed>