Debugging VBScript Custom Actions: Fact or Fiction?

I’ve long enjoyed/hated VBScript as a quick and dirty tool for getting a job done. As well, it’s much favored by Enterprise Application Packagers because of its maintainability; everybody knows VBScript.

There is a problem though, in that in its Quick and Dirty-ness it can often lead to the process of running an install over and over and over again after tweaking lines of code, hoping that you’ve figured out the issue. The purpose of this post is to delve into the well-documented and also the ill-documented options for getting your VBScript custom actions to work properly.

On the Well-Documented side of things is the capability for writing to the MSI log file. Regardless of how good your coding skills are, this is a good idea for the consumer of the install to debug why your code might not be performing as expected in a certain scenario. The MSI VBScript engine exposes many of the APIs in MSI.dll via the Session object. Below is a simple function I lifted off of a fellow Flexera consultant for this purpose; it relies on the VBScript version of MsiProcessMessage():

‘################################

Function MSILog(Message)

 

‘ Create a Windows Installer Error Log entry and pass it to the session.

Dim LogRecord

 

Set LogRecord = Installer.CreateRecord(1)

LogRecord.StringData(1) = “HKU CA: ” & Message

Session.Message &H04000000, LogRecord

Set LogRecord = Nothing

 

End Function

 

‘################################

 

This is all well and good, but what about for scripters that are more used to stepping through the execution line-by-line in an IDE? This question was posed to me in a training class recently, and I’d always assumed there was some easy way out there should users be curious. I began investigating.

As it turns out, it doesn’t appear that Microsoft provided documentation on how to accomplish this. Fortunately, an enterprising user worked out how to do this with Microsoft Visual InterDev some years ago, and published it to the InstallShield forums:

http://community.flexerasoftware.com/showthread.php?t=149399

Heartened, I consulted the MSDN Subscriber Downloads—only to find its cold, emotionless façade blocked me at every turn:

Cary: Show me the Visual InterDev Downloads.

MSDN Subscriber Downloads: I can’t do that, Dave.

Cary: Hm. Show me the Visual Studio 6.0 downloads, as I read InterDev is a component included with it.

MSDN Subscriber Downloads: I can’t do that either, Dave.

Cary: Why do you keep calling me Dave??

Needless to say, you can’t download this component from MSDN anymore. Neither can you download Office 2000 (which it was included with), nor Visual J++, all of which contain some version of Visual InterDev.

After a little more digging, it would appear that the Microsoft Script Debugger was the key, which primarily was used for debugging Client-side scripts in IE, but evidently had the ability to hook into any application hosting JavaScript/VBScript code.

Once you’ve acquired this and installed it, usage is relatively straightforward. First, drop in a MessageBox (or other similar blocking function) in the code of the custom action. This is important because if the script is not active, it won’t be available in the coming steps with the script debugger:

 

Run the installation, stopping at your blocking code:

 

Next, enable the JIT Debugging policy per: http://support.microsoft.com/kb/308364

And lastly, load up the MS Script Debugger (c:Program Files (x86)Microsoft Script Debuggermsscrdbg.exe), and attach to Msiexec.exe via the Tools > Active Documents menu items:

 

In retrospect, it seems obvious!

Hopefully this will be useful to you users out there that are looking for true line-by-line debugging of VBScript, rather than retroactive cryptic log file reading.

To learn about the new features in InstallShield 2012 Spring, the latest version of InstallShield, see the Webinar What’s New in InstallShield 2012 Spring.

Leave a Reply

Your email address will not be published. Required fields are marked *