Many times something seems simple on the surface, but once you start you find it is a pain in the neck. Then, once in awhile, you find it is beyond a pain in the neck. For example, spawning a process and capturing stdout\/stderr in .NET. Sure, it doesn't seem hard, create an instance of the Process class, fill in some members, kick it off and wait until it finishes. Works great! On to the next problem...<\/p>\n
Until the day it locks up on you unexpectedly. Hmm.<\/p>\n
The next step is then to ask the modern Oracle - Google, to see if others have this problem. Sure enough, you find out that if the child process' buffer becomes full, it blocks, and that is causing your deadlock. Even better, there are tons of helpful people on the Internet willing to share the solution to this problem, along with source code snippets. Score! Asynchronous reads for the win. Hook it up, and on to the next problem...<\/p>\n
Until the day it locks up on you unexpectedly. Hmm.<\/p>\n
I got to this point, and Google began to fail me. It seems either no one ran into this problem, or, the only solution people talk about is the method I'm already using. We write all our crunching tools using C\/C++ as \u00a0command line tools, and then a GUI in C# that spawns these tools.\u00a0Most of the time, things ran fine for me - but, under heavy stress conditions, such as the child writing a lot of output, the whole thing would lock up. I even had a reproducible case, where it would lock up 100% of the time for a given command line tool with a certain set of arguments. We debugged our hearts out, trying every which way we could to keep things moving on, but we kept coming up short.<\/p>\n
I can't say with 100% confidence, only 99%, that the issue was not mine, but Microsoft's. If you try to spawn processes that [may] write to both stdout and stderr (we'll ignore reading from stdin for now), you may find yourself in a world of hurt if one of those buffers fills up. This was a few months ago, and I'm having a hard time remembering the exact details, but it may have been related to the asynchronous call backs for the redirected stdout\/stderr being called on the main thread, and, in my case, I was using threadpools for tasks that spawned the processes. My main thread of my application would wait until it got some sort of event from a worker thread that it was done, or there was an error. But, with Microsoft trying to invoke the callback on the main thread for the IO redirection, it never would call to process the buffer, which left the child process waiting for the buffer to clear, and my application's main thread waiting on the worker thread to finish...deadlock.<\/p>\n
All I wanted to do was spawn a process and capture all of its stdout and all of its stderr. I did not have to interact with it via stdin at all. You would think this is a simple request. If I only wanted one or the other, I could have just used\u00a0p.StandardOutput.ReadToEnd(); <\/span>and just executed the process synchronously. You do not have that luxury if you want both stdout and stderr - you must use\u00a0asynchronous\u00a0reads, with the callbacks coming only on the main thread. Throw my need for threadpools into the mix and you find things don't work out so well.<\/p>\n All is not lost, I found a solution. The solution is to not use System.Diagnostics.Process and use raw Win32 API to get the same results. I could of \u00a0just wrote a C\/C++ DLL, or a C++\/CLI assembly, but I hate having a mess of DLLs laying around. I wanted a solution that stayed in C#, and so P\/Invoke to the rescue. First I got it all working using straight C\/C++ and then moved it all to C# using P\/Invoke calls to the Win32 API. I'll save you the work, you can find it here:<\/p>\n Many times something seems simple on the surface, but once you start you find it is a pain in the neck. Then, once in awhile, you find it is beyond a pain in the neck. For example, spawning a process and capturing stdout\/stderr in .NET. Sure, it doesn’t seem hard, create an instance of the […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[1],"tags":[],"_links":{"self":[{"href":"https:\/\/gigglenuts.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/10"}],"collection":[{"href":"https:\/\/gigglenuts.com\/blog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/gigglenuts.com\/blog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/gigglenuts.com\/blog\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/gigglenuts.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=10"}],"version-history":[{"count":8,"href":"https:\/\/gigglenuts.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/10\/revisions"}],"predecessor-version":[{"id":18,"href":"https:\/\/gigglenuts.com\/blog\/index.php?rest_route=\/wp\/v2\/posts\/10\/revisions\/18"}],"wp:attachment":[{"href":"https:\/\/gigglenuts.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=10"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/gigglenuts.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=10"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/gigglenuts.com\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=10"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}\r\nusing System;\r\nusing System.Collections.Generic;\r\nusing System.Text;\r\nusing System.Runtime.InteropServices;\r\n\r\nnamespace ProcessAPI\r\n{\r\n #region WIN32 API\r\n class Win32\r\n {\r\n [StructLayout(LayoutKind.Sequential, CharSet = CharSet.Auto)]\r\n public struct SECURITY_ATTRIBUTES\r\n {\r\n public Int32 nLength;\r\n public IntPtr lpSecurityDescriptor;\r\n public bool bInheritHandle;\r\n }\r\n\r\n [StructLayout(LayoutKind.Sequential, CharSet = CharSet.Auto)]\r\n public struct STARTUPINFO\r\n {\r\n public Int32 cb;\r\n public string lpReserved;\r\n public string lpDesktop;\r\n public string lpTitle;\r\n public Int32 dwX;\r\n public Int32 dwY;\r\n public Int32 dwXSize;\r\n public Int32 dwYSize;\r\n public Int32 dwXCountChars;\r\n public Int32 dwYCountChars;\r\n public Int32 dwFillAttribute;\r\n public Int32 dwFlags;\r\n public Int16 wShowWindow;\r\n public Int16 cbReserved2;\r\n public IntPtr lpReserved2;\r\n public IntPtr hStdInput;\r\n public IntPtr hStdOutput;\r\n public IntPtr hStdError;\r\n }\r\n\r\n [StructLayout(LayoutKind.Sequential, CharSet = CharSet.Auto)]\r\n public struct PROCESS_INFORMATION\r\n {\r\n public IntPtr hProcess;\r\n public IntPtr hThread;\r\n public Int32 dwProcessId;\r\n public Int32 dwThreadId;\r\n }\r\n\r\n [Flags]\r\n public enum DuplicateOptions : uint\r\n {\r\n DUPLICATE_CLOSE_SOURCE = (0x00000001),\/\/ Closes the source handle. This occurs regardless of any error status returned.\r\n DUPLICATE_SAME_ACCESS = (0x00000002), \/\/Ignores the dwDesiredAccess parameter. The duplicate handle has the same access as the source handle.\r\n }\r\n\r\n [DllImport(\"kernel32.dll\")]\r\n public static extern Boolean CreatePipe(\r\n out IntPtr hReadPipe,\r\n out IntPtr hWritePipe,\r\n ref SECURITY_ATTRIBUTES lpPipeAttributes,\r\n uint nSize);\r\n\r\n [DllImport(\"kernel32.dll\")]\r\n public static extern IntPtr GetCurrentProcess();\r\n\r\n [DllImport(\"kernel32.dll\", SetLastError = true)]\r\n [return: MarshalAs(UnmanagedType.Bool)]\r\n public static extern Boolean DuplicateHandle(\r\n IntPtr hSourceProcessHandle,\r\n IntPtr hSourceHandle,\r\n IntPtr hTargetProcessHandle,\r\n out IntPtr lpTargetHandle,\r\n UInt32 dwDesiredAccess,\r\n [MarshalAs(UnmanagedType.Bool)] bool bInheritHandle,\r\n UInt32 dwOptions);\r\n\r\n [DllImport(\"kernel32.dll\", SetLastError = true, CallingConvention = CallingConvention.Winapi, CharSet = CharSet.Auto)]\r\n public static extern IntPtr CreateEvent(IntPtr lpEventAttributes,\r\n [In, MarshalAs(UnmanagedType.Bool)] Boolean bManualReset,\r\n [In, MarshalAs(UnmanagedType.Bool)] Boolean bIntialState,\r\n [In, MarshalAs(UnmanagedType.BStr)] string lpName);\r\n\r\n [DllImport(\"kernel32.dll\")]\r\n public static extern void SetLastError(Int32 dwErrCode);\r\n\r\n [DllImport(\"kernel32.dll\")]\r\n public static extern bool SetEvent(IntPtr hEvent);\r\n\r\n [DllImport(\"kernel32.dll\")]\r\n public static extern Boolean WriteFile(\r\n IntPtr hFile,\r\n Byte[] lpBuffer,\r\n UInt32 nNumberOfBytesToWrite,\r\n out UInt32 lpNumberOfBytesWritten,\r\n IntPtr lpOverlapped);\r\n\r\n [DllImport(\"kernel32.dll\", SetLastError = true)]\r\n public static extern bool ReadFile(\r\n IntPtr hFile,\r\n [Out] Byte[] lpBuffer,\r\n UInt32 nNumberOfBytesToRead,\r\n out UInt32 lpNumberOfBytesRead,\r\n IntPtr lpOverlapped);\r\n\r\n [DllImport(\"kernel32.dll\")]\r\n public static extern bool CreateProcess(\r\n string lpApplicationName,\r\n string lpCommandLine,\r\n IntPtr lpProcessAttributes,\r\n IntPtr lpThreadAttributes,\r\n Boolean bInheritHandles,\r\n UInt32 dwCreationFlags,\r\n IntPtr lpEnvironment,\r\n string lpCurrentDirectory,\r\n [In] ref STARTUPINFO lpStartupInfo,\r\n out PROCESS_INFORMATION lpProcessInformation);\r\n\r\n [DllImport(\"kernel32.dll\", SetLastError = true)]\r\n [return: MarshalAs(UnmanagedType.Bool)]\r\n public static extern bool TerminateProcess(IntPtr hProcess, UInt32 uExitCode);\r\n\r\n [DllImport(\"kernel32.dll\", SetLastError = true, CallingConvention = CallingConvention.Winapi, CharSet = CharSet.Auto)]\r\n [return: MarshalAs(UnmanagedType.Bool)]\r\n public static extern bool CloseHandle(IntPtr hObject);\r\n\r\n [DllImport(\"kernel32.dll\", EntryPoint = \"PeekNamedPipe\", SetLastError = true)]\r\n public static extern bool PeekNamedPipe(\r\n IntPtr handle,\r\n Byte[] buffer,\r\n UInt32 nBufferSize,\r\n ref UInt32 bytesRead,\r\n ref UInt32 bytesAvail,\r\n ref UInt32 BytesLeftThisMessage);\r\n\r\n [DllImport(\"kernel32.dll\", EntryPoint = \"WaitForMultipleObjects\", SetLastError = true)]\r\n public static extern int WaitForMultipleObjects(\r\n UInt32 nCount,\r\n IntPtr[] lpHandles,\r\n Boolean fWaitAll,\r\n UInt32 dwMilliseconds);\r\n\r\n [DllImport(\"kernel32.dll\", SetLastError = true)]\r\n [return: MarshalAs(UnmanagedType.Bool)]\r\n public static extern Boolean GetExitCodeProcess(IntPtr hProcess, out Int32 lpExitCode);\r\n }\r\n #endregion\r\n\r\n public class ProcessRedirector : IDisposable\r\n {\r\n #region Internal Members\r\n private System.Threading.Thread m_hThread; \/\/ thread to receive the output of the child process\r\n private IntPtr m_hEvtStop; \/\/ event to notify the redir thread to exit\r\n private Int32 m_dwThreadId; \/\/ id of the redir thread\r\n private Int32 m_resultCode; \/\/ returned result code of the process\r\n private UInt32 m_dwWaitTime; \/\/ wait time to check the status of the child process\r\n\r\n protected IntPtr m_hStdinWrite; \/\/ write end of child's stdin pipe\r\n protected IntPtr m_hStdoutRead; \/\/ read end of child's stdout pipe\r\n protected IntPtr m_hStderrRead; \/\/ read end of child's stderr pipe\r\n protected IntPtr m_hChildProcess;\r\n #endregion\r\n\r\n #region Constructor\r\n public ProcessRedirector()\r\n {\r\n m_hStdinWrite = IntPtr.Zero;\r\n m_hStdoutRead = IntPtr.Zero;\r\n m_hStderrRead = IntPtr.Zero;\r\n m_hChildProcess = IntPtr.Zero;\r\n m_hThread = null;\r\n m_hEvtStop = IntPtr.Zero;\r\n m_dwThreadId = 0;\r\n m_dwWaitTime = 100;\r\n m_resultCode = -1;\r\n }\r\n #endregion\r\n\r\n #region Finalizer\r\n \/\/ NOTE: Leave out the finalizer altogether if this class doesn't \r\n \/\/ own unmanaged resources itself, but leave the other methods\r\n \/\/ exactly as they are. \r\n ~ProcessRedirector()\r\n {\r\n \/\/ Finalizer calls Dispose(false)\r\n Dispose(false);\r\n }\r\n #endregion\r\n\r\n #region Dispose\r\n public void Dispose()\r\n {\r\n Dispose(true);\r\n GC.SuppressFinalize(this);\r\n }\r\n\r\n \/\/ The bulk of the clean-up code is implemented in Dispose(bool)\r\n protected virtual void Dispose(bool disposing)\r\n {\r\n if (disposing)\r\n {\r\n \/\/ free managed resources\r\n }\r\n\r\n \/\/ free native resources if there are any.\r\n Close(true);\r\n }\r\n #endregion\r\n\r\n #region Property: ResultCode\r\n \/\/\/