Dajbych.net


Post-mortem debugging with CLR V4 and Visual Studio 2010

, 8 minutes to read

vs2010 logo

The new Common Language Runtime in .NET 4 allows you to save the state of an application in a special file called a dump in the event of a crash. This can be used by the developer to accurately analyze the cause of the application crash. It sees the contents of variables and data structures in the same way as when it debugs the application. Describes how to configure the environment so that application crashes generate a dump file that is then sent to the corporate server.

If the developer is debugging the application, it runs under Visual Studio, so it has a very good overview of what is happening. If the application is tested by a tester, it is a little worse. It describes the process of how the error occurred and, in the best case, even points out a possible cause that will not later turn out to be completely misleading. If the customer uses the application to a wilder extent than the developer expected, a screenshot will be given back to the developer at best. It rarely has any informative value. A log extract is useful, but it is again only a guide to reconstructing the error. Visual Studio 2010 and .NET 4 give us a very elegant solution that brings so much information to the developer that the only difference between debugging in the development environment and the production environment is that we can't continue to run the program.

JIT, which is a compiler from MSIL to machine code, and Windows can be set to guard a certain application and if an unhandled exception occurs in it, it attaches a debugger to it, which writes a dump of its memory to disk. This is called a minidump. It does not contain a listing of the entire memory of the process, but only those parts that are expected to be of interest to us. To set up automatic minidump file generation, just write a few keys to the Windows registry and install the debugger included in Debugging Tools for Windows. Memory dumps are stored only in the folder that we specify. This folder can be monitored by another program that sends the file to the corporate server. The developer just opens it in Visual Studio 2010 and sees the state of the program as well as if the error occurred with him. However, the application must be in .NET 4 and compiled in Debug mode. A program debug database (pdb) file is not needed to generate a minidump file.

The following contents of the reg file set up debug and generate a minidump file for the application named HelloWorld.exe:

Windows Registry Editor Version 5.00 

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\.NETFramework]
"DbgManagedDebugger"="\"C:\\Program Files\\Debugging Tools for Windows (x64)\\cdb.exe\" -pv -p %ld -c \".dump /u /ma c:\\crash_dumps\\crash.dmp;.kill;qd\""
"DbgJITDebugLaunchSetting"=dword:00000000

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\AeDebug]
"Debugger"="\"C:\\Program Files\\Debugging Tools for Windows (x64)\\cdb.exe\" -pv -p %ld -c \".dump /u /ma c:\\crash_dumps\\crash.dmp;.kill;qd\""
"Auto"="0"

[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\Windows NT\CurrentVersion\AeDebug]
"Debugger"="\"C:\\Program Files (x86)\\Debugging Tools for Windows (x86)\\cdb.exe\" -pv -p %ld -c \".dump /u /ma c:\\crash_dumps\\crash.dmp;.kill;qd\""
"Auto"="0"

[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\.NETFramework]
"DbgManagedDebugger"="\" C:\\Program Files (x86)\\Debugging Tools for Windows (x86)\\cdb.exe\" -pv -p %ld -c \".dump /u /ma c:\\crash_dumps\\crash.dmp;.kill;qd\""
"DbgJITDebugLaunchSetting"=dword:00000000 

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\Windows Error Reporting\DebugApplications]
"HelloWorld.exe"=dword:00000001

This is the most complex example that works on x64 and stands for both x64 and x86 applications. A corresponding debugger must be attached to the corresponding application type. In node Wow6432Node there is a setting for x86 applications on an x64 machine. The other settings are for x64. On an x86 machine, Wow6432Node entries are not valid, and for other machines, only the paths to the x86 debugger are adjusted. On the other hand, if we have all applications on x64 cost purely x64, the records in the Wow6432Node node are not needed. We will also install Debugging Toos for Windows in the necessary bit version. To generate the minidump itself, you don't need the entire content of the installer, just the cdb.exe application. The last branch is used to set up applications to be debugged after a crash. Several applications can also be set up in this way.

Let’s see what it looks like in practice. A simple console application will serve as an example. The Main method contains a string and a linked list that illustrates some deeper structure. First, we initialize the variables and then call the MyCode method, which causes the application to crash.

When the application is launched, a minidump file is generated. The following image shows Visual Studio 2010 after it is opened. In addition to basic information, we have a list of loaded dynamic libraries. To debug in managed mode, click on Debug with Mixed.

If we have a project from which the original application was compiled, its source code is displayed and the line on which the exception occurred is highlighted. We can see a list of variables and their contents in the current stack. We can pin to the parent stack at will, where we have access to all variables again.

The only problem I encountered that prevented this comfortable debugging was that Visual Studio couldn't handle a minidump catching an exception that occurred in the window’s WPF constructor. Difficulties can also be caused by a slow customer line, because minidump files are tens of megabytes.

The rest of the article shows uploading a minidump file to the server. The following code monitors the C:\crash_dumps directory and if it detects a new dmp file, it sends it to the server:

FileSystemWatcher fsw = new FileSystemWatcher(@"C:\crash_dumps", "*.dmp");
fsw.Created += new FileSystemEventHandler(fsw_Created);

void fsw_Created(object sender, FileSystemEventArgs e) {
    string filename = e.FullPath;  

    ServiceClient client = new ServiceClient();
    FileStream fs = File.Open(filename, FileMode.Open);
    string serverHash = client.UploadDump(fs);

    fs.Seek(0, SeekOrigin.Begin);


    MD5CryptoServiceProvider md5 = new MD5CryptoServiceProvider();
    byte[] hash = md5.ComputeHash(fs);
    string localHash = BitConverter.ToString(hash);

    fs.Close();

    if (localHash == serverHash) File.Delete(filename);

}

The service receives the stream and stores it in a file in 4kB blocks. At the end, it computes the MD5 hash and returns it to the client:

[ServiceContract]
public interface IService {

    [OperationContract]
    string UploadDump(Stream content);

}

public class Service : IService {

    string IService.UploadDump(Stream content) {

        string file = null;

        try {

            file = Path.GetTempFileName();

            FileStream fs = File.Open(file, FileMode.Create, FileAccess.Write);

            const int bufferLen = 4096;
            byte[] buffer = new byte[bufferLen];
            int count = 0;
            while ((count = content.Read(buffer, 0, bufferLen)) > 0) fs.Write(buffer, 0, count);
            fs.Close();
            content.Close();

            fs = File.Open(file, FileMode.Open, FileAccess.Read);
            MD5CryptoServiceProvider md5 = new MD5CryptoServiceProvider();
            byte[] hash = md5.ComputeHash(fs);
            fs.Close();

            File.Move(file, Path.Combine(@"C:\Client Crash Dumps\HelloWorld", Path.GetFileNameWithoutExtension(file) + ".dmp"));

            return BitConverter.ToString(hash);

        } catch {
            if (file != null && File.Exists(file)) File.Delete(file);
            return null;
        }
    }

}

The corresponding web.config should be applied only to this service. We will not publish its endpoint anywhere, because it opens the door to possible DoS attacks.

<configuration>
  <system.web>
    <httpRuntime maxRequestLength="1048576" executionTimeout="3600"/><!-- 1 GB   1 hod -->
  </system.web>
  <system.serviceModel>
    <behaviors>
      <serviceBehaviors>
        <behavior name="ServiceUpload.ServiceBehavior">
          <serviceMetadata httpGetEnabled="true"/>
          <serviceDebug includeExceptionDetailInFaults="false"/>
        </behavior>
      </serviceBehaviors>
    </behaviors>
    <services>
      <service behaviorConfiguration="ServiceUpload.ServiceBehavior" name="ServiceUpload.Service">
        <endpoint address="" binding="basicHttpBinding" bindingConfiguration="HttpStreaming" contract="ServiceUpload.IService"/>
        <endpoint address="mex" binding="mexHttpBinding" contract="IMetadataExchange"/>
      </service>
    </services>
    <bindings>
      <basicHttpBinding>
        <binding name="HttpStreaming" maxReceivedMessageSize="1073741824" transferMode="Streamed"/><!-- 1 GB -->
      </basicHttpBinding>
    </bindings>
  </system.serviceModel>
</configuration>

The service is of the basic http binding type, so communication does not take place over a secure channel unless we encrypt the stream ourselves. The advantage is that the file is sent as a stream and the HTTP binding is not in the way of firewalls. A method implementing stream processing cannot have any additional parameter, which should not matter in this case.