0

Closed

Does not run with VS host enabled

description

Once you build MSE, it does not run under VS2005 because of VSHost issues.
 
The problem is that under the debugger, the Main() function will try to get the current module and re-execute it in some GUI mode. But with VSHost, that name is wrong.
 
It gets the application string via:
Process p = Process.GetCurrentProcess();
string Application = System.IO.Path.GetFullPath(p.MainModule.FileName);
 
However, with VSHost, the application is
"C:\dev\MSE\StackExplorer\MSE\bin\Debug\MSEPart.vshost.exe"
 
 
So Main() will call CreateProcess() directly on a vshost process without actually having VS start debugging it, which obviously will not work. (Note managed-debugging, unlike native, does not support debugging a child-process, and so the debugger won't automatically attach to the newly created process to start cooperating with VSHost).
 
 
If you Ctrl+F5 (launch outside of debugger), then it runs fine. In that case, Application will be:
"C:\dev\MSE\StackExplorer\MSE\bin\Debug\MSEPart.exe"
and so it launches the correct process.
 
I notice there's a 'debugMode' bool in Main() that you can manually change to override this behavior, but you shouldn't have to physically change Main() in order to launch it under the debugger.
 
This makes a bad first impression of using the tool because you build it under VS, press F5, and expect it to just work without having to modify Main().
 
 
 
 
 
Here's the main function from the version I'm looking at:
 
public static int Main(string$$ args)
{
//if true then new process isnt created which allows easier debugging
bool debugMode = false;
 
IntPtr INVALID_HANDLE_VALUE = (IntPtr)(-1);
if (args == null || args.Length == 0)
{
const int STD_OUTPUT_HANDLE = -11;
IntPtr stdOut = GetStdHandle(STD_OUTPUT_HANDLE);
 
//if the stdOuit handle is -1 or we are in debug mode then run the gui
if ((stdOut == INVALID_HANDLE_VALUE) || debugMode)
{
Application.EnableVisualStyles();
Application.Run(new MainGui());
}
else
{
//create a new process to run the gui in
 
const int DETACHED_PROCESS = 0x00000008;
const int STARTF_USESHOWWINDOW = 0x00000001;
const int STARTF_USESTDHANDLES = 0x00000100;
const int SW_SHOW = 5;
 
Process p = Process.GetCurrentProcess();
 
//path to the exe
string Application = System.IO.Path.GetFullPath(p.MainModule.FileName);
string CommandLine = "";
PROCESS_INFORMATION pInfo = new PROCESS_INFORMATION();
STARTUPINFO sInfo = new STARTUPINFO();
sInfo.cb = System.Runtime.InteropServices.Marshal.SizeOf(sInfo);
sInfo.dwFlags = STARTF_USESHOWWINDOW | STARTF_USESTDHANDLES;
sInfo.wShowWindow = SW_SHOW;
 
//send stdOutput as -1 to tell new process to open as gui
sInfo.hStdOutput = (IntPtr)(-1);
 
SECURITY_ATTRIBUTES pSec = new SECURITY_ATTRIBUTES();
SECURITY_ATTRIBUTES tSec = new SECURITY_ATTRIBUTES();
pSec.nLength = Marshal.SizeOf(pSec);
tSec.nLength = Marshal.SizeOf(tSec);
 
//create the new process
CreateProcess(Application, CommandLine,ref pSec, ref tSec, true, DETACHED_PROCESS,
IntPtr.Zero, null, ref sInfo, out pInfo);
}
}
else
{
//run in command line mode if arguments are present
int error = 1;
using (ArgumentExecuter argExecute = new ArgumentExecuter(args))
{
try
{
if ((error = argExecute.ExecuteArguments()) != 0)
{
Console.WriteLine(argExecute.ErrorMessage);
}
}
catch (System.IO.IOException ex)
{
error = 1;
Console.WriteLine(ex.Message);
}
}
return error;
}
 
return 0;
}
Closed Feb 23, 2007 at 2:29 PM by feelite

comments

wrote May 24, 2006 at 8:45 PM

Thanks Mike for the bug and the suggested fix. We'll investigate taking this fix in for the 1.1 release scheduled in the month.

wrote Jul 16, 2006 at 3:42 PM

How about detecting if the application is running in a debugger and set the debugMode value accordingly, provided we make it non-const first.

if (Debugger.IsAttached)
debugMode = true;

wrote Feb 23, 2007 at 2:29 PM

Resolved with changeset 18992.

wrote Feb 23, 2007 at 2:32 PM

Fix checked-in by making the debugMode a variable instead of constant and check for Debugger.IsAttached property

wrote Feb 14, 2013 at 1:48 AM

wrote May 16, 2013 at 6:31 AM