Automated Install Testing - MSI Validation

An introduction to this article can be found in this previous post. -- Automated Install Testing

MSI Validation is the first series of tests that come to mind. MSI Validation runs tests on the MSI package, these test are called Internal Consistency Evaluators (ICE). They can find bad conditions, duplicate file targets, invalid records, per/user issues and so on...

MSI Validation is nice because, the ICEs are written by Microsoft you just need to run them. You can also write your own custom ones however, for now we will just use Microsoft’s.

To run validation manually, just edit you MSI package with Orca, choose Tools->Validate.

Validating the MSI package manually is fine however, we want to automate it. Then the validation can be fit into the nightly build process. As the MSI package evolves over the course of the development cycle, the ICEs will continually run.

The Platform SDK contains a command line tool that will run the tests for us. MsiVal2.exe -- to install it find MsiVal2.msi, which is located in the Platform SDK directory.

MsiVal2.msi contains MsiVa2.exe and some .cub files. The CUB files contain the actual ICEs to execute. Darice.cub contains all of the MSI ICE's and equates to Full MSI Validation Suite, the other ones contain a subset of them. So, for our example we will stick with Darice.cub



Running MSI Validation from the command line:
MsiVal2.exe <Msi Package> <.cub file> /l"Output file" /f

Example:
MsiVal2.exe "C:\My Path\My.msi" "darice.cub” /l “C:\My Path\Validation.log” /f

The output:
MSI Validation typically contains 3 levels of output (Info, Warnings, and Errors). For the most part, you will want to fix most (if not all errors). However, we don’t need or really concern ourselves so much with the info messages and warnings. Now that we know the command line to run MSI Validation, all we have to do is fit it into our build process.  Regardless, of the build tools (VBScript, Pearl, Visual Build, Batch Files ...), MSI Validation can be run on the installation package.

The default MS ICEs can help you resolve numerous problems with the installation package. However, they won’t find them all. Next we will look at how to write your own custom ICEs so that you can extend the MSI Package validation.  Coming soon ...

MSI Package Validation:
http://msdn2.microsoft.com/en-us/library/aa370569(VS.85).aspx

Internal Consistency Evaluators Reference:
http://msdn2.microsoft.com/en-us/library/aa369206.aspx

MsiVal2.exe
http://msdn2.microsoft.com/en-us/library/aa370504(VS.85).aspx

Validation Automation:
MsiVal2.exe works great, but it is limited. If more functionality is required automated validation can also be performed via COM (using the Evalcom2 interface). More information can be found here;
http://msdn2.microsoft.com/en-us/library/aa372415(VS.85).aspx

Automated Install Testing

Audience: Install Developers, Quality Assurance, Beginner to Advanced, Windows Installer (MSI)

I think everyone will agree, once your installation package is built it is a good idea to test it.

Testing installs of any size can be turn out to be tedious and time consuming. There can be many reasons for this, to list a few;

• The install responds differently on various operating systems (like Vista and XP)
• The install needs to support administrative images
• The application supports run from source installations
• There are different code paths taken depending on what is already installed on the system
• The installer upgrades one or several previous versions
• ... The list can go on and on depending on your specific requirements ...

One way is black box testing. Typically this involves nothing more than making a grid of the various scenarios, run them and see what happens. The more requirements the installation must fulfill, the larger the grid will be. This works however, it can be difficult catch every defect and if you are performing each install manually it can eat up a lot of time, which isn’t very cost effective. Most companies leave this strictly up to their quality assurance team, which in turn eats into application testing time.

If you want to save some money, impress your boss and QA team, think about white box testing. The nice thing about white box testing, is scripts can be written that fit into nightly build process which can help uncover known issues. You don’t really have to do anything more than look at some log files and fix the issues... besides write the tests.

If done properly you can uncover defects without even running the installation. These defects can then be fixed before the install even makes it to the QA team, in turn they spend more time testing the application, instead of the install. It’s win, win in my book.

The next few articles will share some handy scripts that I have found useful.

MSI Validation

Windows Installer Dll - Part 3

Audience: Install Developers, Beginner, Windows Installer (MSI) Knowledge

This is the final part of a three part series that demonstrates how to use C++ in a Windows Installer Custom Action.

• Introduction
• Part 1 - Creating a Windows Installer DLL
• Part 2 - Calling the Windows Installer DLL from a setup
• Part 3 - Debugging a Windows Installer DLL

Once your custom action is ready to go, it is usually a good idea to debug it. The two best methods I’ve seen are:

1. Stepping thought the code
2. Writing to the log file

1. Step through the code

Visual Studio makes it easy, there is only a few steps to the process.

A) - Add a message box to the debug build of your DLL

Nothing new here...

UINT __stdcall MyFunction(MSIHANDLE hMsi)
{
#if _DEBUG
     AfxMessageBox(_T("MyFunction() -> Debug"));
#endif

     int x = 0;

     //complete custom tasks ...

     return ERROR_SUCCESS;
}

B) - Build your debug DLL and insert it into your .msi file

Use a program like orca to insert your debug DLL into the MSI file’s appropriate Binary table record.


Orca is a utility that will allow you to modify MSI packages. It comes with Microsoft’s Windows Installer SDK. If you have the SDK installed, Orac’s install can be found in the SDK’s bin folder. (usually – C:\Program Files\Microsoft SDKs\Windows\v6.0\Bin)

C) - Run the install and attach to process

When the install executes your custom action, the debug message box pops up. With the project open in Visual Studio, use the Debug Menu and select Attach to Process. Find the process who’s title matches the message box’s title, and Attach to it.

Debug->Attach to Process

D) - Set a breakpoint, dismiss the message box

Set a breakpoint in your code just after the message box. Once the message box is dismissed VS will break into the debugger. From here you will have full access to the usual debug windows, and tools.

 

2 - Writing to the log file

MsiProcessMessage() can be used to write data to the log file. It is always a good idea to write important information about your action to the log file. If the install fails, you only need to collect the log file to tell what is going on. The following code writes data to the log file.

UINT __stdcall MyFunction(MSIHANDLE hMsi)
{
#if _DEBUG
     AfxMessageBox(_T("MyFunction() -> Debug"));
#endif

     WriteToLog(hMsi, L"MyFunction - doing something...");

     return ERROR_SUCCESS;
}

void WriteToLog(MSIHANDLE hMsi, LPCTSTR szMsg)
{
     PMSIHANDLE hRec = MsiCreateRecord(1);
     MsiRecordSetString(hRec, 1, szMsg);
     MsiProcessMessage(hMSI, INSTALLMESSAGE_INFO, hRec);
}


 

There it is... This concludes the series on how to use C++ in a Windows Installer custom action. Of course, the code here hasn’t really done anything. The intent is to demonstrate how to hook up a DLL. Most, if not all, the samples given at this blog will be in C++ so this seemed like a logical place to start.

 

Windows Installer Dll - Part 2

Audience: Install Developers, Beginner, Windows Installer (MSI) Knowledge

This is Part 2 of a three part series that demonstrates how to use C++ in a Windows Installer Custom Action.

• Introduction
• Part 1 - Creating a Windows Installer DLL
• Part 2 - Calling the Windows Installer DLL from a setup
• Part 3 - Debugging a Windows Installer DLL

Once your function is written and the DLL is created, you need to call it through a custom action. There are 2 ways to call a Windows Installer DLL.

1. Custom Action Type 1 (store the DLL in the MSI file’s Binary table)
2. Custom Action Type 17 (install the DLL with the product)

Hooking up the DLL in your setup project is easy. No matter what install authoring software you are using (InstallShield, Vise, Wise, Advanced Installer) – there will be some sort of wizard that guides you through the process.

I don’t think there is any value in me putting up screen shots, or demonstrating how these wizards work. Instead, lets look at some tips that will help you produce easy to maintain and 'readable' setup projects.

1 - Use Comments

This one gets missed a lot. Especially, when there is only one install developer that works on the project.

Some authoring software (InstallShield) has a comment field for the custom actions. I have found it is nice to see the comment used to tell why the custom action is needed. The function definition in you DLL’s source code should also have a quick description that describes the purpose of the function. Nothing to fancy, just enough that people can tell what’s going on.

// Example:
// checks to see if [INSTALLDIR] is too long
UINT __stdcall CA_EvaluateInstallPath(MSIHANDLE hMsi)
{
     /* ... code does something ... */

     return ERROR_SUCCESS;
}


2 - Use a Naming Convention for your Custom Actions

A) Use appropriate names for your custom actions and functions. -- You and other developers may need to maintain the code at a later date, your names should typically reflect something about what the custom action is responsible for.

Matching your Action name (custom action name) with the Target (exported function in the DLL) can be a big help if there are many functions in the same DLL.

B) Explicitly name important the Binary table entries. -- Most install authoring software will just pick a name like ‘NewBinary0' – if you take the time to rename it to something that reflects the actual source then your project will be easier to maintain thought it’s life cycle. There is no need to rename them all, just the ones that you many want to keep track off. The point here is it is better to see something like My.dll in the binary table, instead of NewBinary88.

3 - Custom Action Type 1 is easier than Custom Action Type 17

If you can get away with it, always choose Type 1 (DLL stored in the binary table) over, Type 17. It is just easier. You can use Type 1, and call the custom action anytime during the setup.

However, Type 17 can only be executed after InstallFiles and you need to use a condition on your action so that it will only run if the component has been installed.

4 - Any Action that Modifies the System Needs to be Deferred in System Context

If your action needs to modify the system, then it should be executed in the system context. This will ensure that the action will have the required permissions on the computer.

Secondly, if your action modifies the system, consider writing a rollback action as well. If the install fails after the system modification has been preformed, then the change should be undone when the install rolls back.

That’s it. The above tips will help ensure your setup project readable, easy to maintain and hopefully prevent a bug or two.

Next will move on to some debugging techniques for C++ custom actions.

Part 3 >

---

Windows Installer Dll - Part 1

Audience: Install Developers, Beginner, Windows Installer (MSI) Knowledge

This is Part 1 of a three part series that demonstrates how to use C++ in a Windows Installer Custom Action.

• Introduction
• Part 1 - Creating a Windows Installer .dll
• Part 2 - Calling the Windows Installer .dll from a setup
• Part 3 - Debugging a Windows Installer .dll

Creating a dll to be called in a custom action is easy. I use Visual Studio 2005 (why? – because that's what the company I work at uses, and I like it). Obviously, there are other compilers out there that will get you the same results but, for our purpose we will demonstrate the VC++ way. You’ll be up and running in minutes.

1 - Start a new project.

• Project Type: MFC DLL
• DLL Type: Regular DLL with MFC statically linked
• Name: CustomActions

I find this the quickest and easiest method, the wizard creates everything you need. You could choose other types (like Win32 project) however, MFC DLL is my preference.

I always recommend static linking for Windows Installer Dlls – Typically the Dll will be streamed from the .msi to the %temp% folder, and called from there. Statically linking helps ensure everything you need is contained in one dll.

2 - Update Project Settigns, and Link to Msi.lib

Open the project properties for CustomActions, and link to Msi.lib

Open stdafx.h, add the following includes.

#include <MsiQuery.h>       //Windows Installer API
#include <Msi.h>

3 - Write your Custom Action(s), Export the Function

Open CustomActions.cpp, and add your code. Each Custom Action requires the following prototype.

/*     MyFunction     **************************************

Args: [in] MSIHANDLE hMsi
       - handle to running install (passed by the Windows Installer)

Desc:     Example
************************************************************/
UINT __stdcall MyFunction(MSIHANDLE hMsi)
{

    return ERROR_SUCCESS;
}


Open CustomActions.def and export the function name.

; CustomActions.def : Declares the module parameters for the DLL.

LIBRARY     "CustomActions"

EXPORTS
    ; Explicit exports can go here
     MyFunction


4 - Build the dll

That’s it the dll is ready to go...

The next step is adding the .dll to your setup project, which is covered in Part 2.

Part 2 >

---

Windows Installer Dll - Introduction

Audience: Install Developers, Beginner, Windows Installer (MSI) Knowledge

This is the introduction to a three part series that demonstrates how to use C++ in a Windows Installer Custom Action.

• Introduction
• Part 1 - Creating a Windows Installer .dll
• Part 2 - Calling the Windows Installer .dll from a setup
• Part 3 - Debugging a Windows Installer .dll

Windows Installer Actions:

Every task the Windows Installer performs is called an action. Actions are equivalent to what common programming languages refer to as functions, methods or procedures.

There are over 70 Standard Windows Installer Actions. http://msdn2.microsoft.com/en-us/library/aa372022(VS.85).aspx

To extend the standard functionality, use a Custom Action.

Custom Actions:

Setup authors can write their own actions (Custom Actions), to extend the Windows Installer’s default functionality.

There are several types of Custom Actions. http://msdn2.microsoft.com/en-us/library/aa368066(VS.85).aspx

This article will deal specifically with using a .dll though a custom action. (Type 1 or Type 17).

Windows Installer Dll

Even if you are using InstallScript in a basic MSI project, behind the scenes a .dll is being called. I prefer to skip InstallScript all together, and go for C++.

Not that there is anything wrong with InstallScript. It has just been my experience that making your own .dll is much faster, more portable, and you have more options as to what can be done.

This article will show you how to get up and running. Part 1 >

---

Starting up!

This is the start of my Release Engineering and Install Developing blog.

Release Engineering commonly refers to the process of building, deploying and maintaining software through a product’s life cycle. This is going to be more of a ‘how to’ blog. If you don’t know what Release Engineering and Install Developing is, then blogs is likely not for you.

I’ll be posting code snippets, solutions to common challenges, tips, tricks and general articles here. The intent is to create a blog where ideas and solutions can be freely exchanged.

I have been developing installation packages on Mac and PC for about 8 years now. My main area of expertise are;

• Windows Installer
• InstallShield, Installer Wise
• Scripting (InstallScript, JScript, VBScript, Pearl, Batch files, WSH)
• Automation for Quality Assurance
• C++ (MFC)
• Html/CSS

I’ll be posting my first ‘real’ article soon.

Thanks for visiting.

Note: If you want to know more about what Release Engineering is, here’s a great explanation. One More Pointless Blog: Software Release Engineering