Spinner Software Home About
KB Contact

Packaging and Repackaging 101

Published: 23 March 2015 - Last update: 12 Dec 2015 (c) Nicolai Kjaer / Spinner Software B.V.

This document is meant as an introduction to application packaging, with additional tips and tricks along the way.

Packaging is the art of creating an automated installation package that performs a clean installation and later removal of a piece of software. It's typically done by making minor modifications to vendor supplied software setups, but can also involve the creation of a Windows Installer package from scratch (repackaging).

This document is not meant for installation builders at software vendors. This document covers tailoring (or customizing) silent and unattended installations. While the basics are the same, there will be no focus on installation wizards.

Chapters:

1. Intake

2. Preparing - MSI extraction, snapshotting, deployments, patches and updates

3. Customization - Creating the transform

4. Testing - Procedures, profiles, scripts

5. Documentation and reporting - Validations

6. Delivery

7. Virtualization

8. Tools

9. Finished

Layout:

This is an example section. Throughout the document, these sections will describe the steps that were just explained, as relevant to making a package for Oracle Java (JRE 8.0.40 64bit) with updates disabled.

Tips are shown in these sections.

This section contains scripts for automating the task.

If you have the Windows Installer SDK installed, there are some very handy scripts in the x86 folder (e.g. "C:\Program Files (x86)\Windows Kits\8.1\bin\x86\"). Either copy them, include the folder in your PATH, or use a fully qualified path when referencing them.

1. Intake

First, you must ensure that you have all the items needed to create the package. This is, at a minimum:

1. Intake / order document describing what should be installed, and any customizations needed.
2. Software or download link.
3. Software license as needed.

Using the above, the idea is to create an unattended (silent) installation that will install the software in the described way (with the desired features, and any additional files etc) onto a system. It should preferably be able to repair itself, and it must be able to uninstall itself cleanly. All of this are standard features of Windows Installer when used properly.

Java 8 Example:

  • 1. We will make a simple package with no changes other than disabling updates.
  • 2. The URL is https://java.com/nl/download/
  • 3. No license needed.

2. Preparing

The preferred way of customizing installations is to obtain the vendors original Windows Installer package (.msi file), and make any changes to the installation in a separate transform (.mst file). You then deploy the msi and mst file together for a properly customized installation. These installations are typically run silently (unattended) as a System Administrator.

The msi file is essentially a database containing a large and complex set of predefined tables and keywords. A Transform (mst file) contains the differences between the original installation, and the customized one.

Most vendors have an .msi file available for download, although they are occasionally hidden. If you do not see an .msi package for download, check for alternative downloads (such as those for "other operating systems" etc - proper deployment packages can sometimes be found there).

It happens quite often that vendors distribute a setup.exe containing the msi package. See 2.1 for more info.

Some vendors (e.g. Adobe) requires you to sign a distribution agreement before giving access to the msi package. You need to have the appropriate internally signed approvals before using such packages. Since vendors like Adobe are used on most PCs (Flash, Reader), and the agreements are nothing special, getting the approval internally should not be an issue - and you then have access to vendor developed MSI deployment packages.

Once you have a vendor MSI and any associated files, you are ready to start customizing the installation package (3).

Java 8 Example (continued):

  • 1. We note that Java has a page for "other" downloads, where we can pick up a better installer (the Windows Offline (64bit) version).
  • 2. The URL is https://java.com/nl/download/manual.jsp
  • 3. We download and save jre-8u40-windows-x64.exe.

2.1 MSI extraction

There is not always an easy way to determine if a Setup.exe contains an msi package ; you typically find out by essentially pausing the installation.

To extract the msi from the Setup.exe file, run setup.exe and wait for the first installation window to appear. This is typically a Welcome window with a Next button to begin the installation process. Do not proceed.

Explore the %temp% folder. To do so, run (Windows Key + "R", or type it in the Start menu search field) the following command:

%temp%

Look for the latests directories made, and check if any of them look like an installation package. It's usually fairly easy to see - the folder probably has the vendor and/or product name in it, and contains productname.msi along with a few files. Copy the files out of the folder (or, the entire folder if you're uncertain) to your scripting folder. Finally, move back up to the main %temp% folder so the files can be removed when you cancel.

You can now cancel the (open) installation window. If you switch back to Explorer, the folder you just copied out should be removed automatically by setup.exe when it cancelled.

If you doubleclick the MSI in your newly copied scripting folder, you should get the same initial Welcome window. Always good to doublecheck before proceeding.

In cases where the vendor has multiple products available with a common installer (e.g. Sun/Oracle, Adobe), the msi package may be found in one of their appdata locations, or their ProgramFiles or ProgramData folders.

Tips:

If you have 7zip installed, you can right-click a lot of setup files and extract the files directly.

Locations of common installers:
Sun Java : C:\Users\<user name>\AppData\LocalLow\Sun\Java\<JRE version>\
Adobe msp : ProgramData\Adobe\ARM\<Reader version>\

For Microsoft Self-Extractor software, you can also run their exe file with the commandline parameters "/extract:drive:\path\", or just "/extract" to get prompted for the target folder.

Java 8 Example (continued):

We will extract the vendor msi from the Java setup exe:
  • Run jre-8u40-windows-x64.exe and wait for the initial Welcome window.
  • Switch away from the installer and open an explorer (run %temp%) and browse around looking for the msi installation.
  • We find the original msi at C:\Users\username\AppData\LocalLow\Sun\Java\jre1.8.0_40_x64\.
  • The folder contains a single file, jre1.8.0_40.msi.
  • Copy the msi file to the scripting folder.
  • Move back out of the folder containing the msi.
  • Switch back to the Welcome window and cancel the installation.
  • Note that Java does not clean up properly, and actually left behind the extracted msi anyway.
  • We now have a single .msi file in our scripting folder. Do not doubleclick it, as it will autoinstall silently.
  • We can continue to perform our customization (3).

2.2 Snapshotting

Note: This should only be used as a last resort for legacy setup.exe installations. Most installations made in the past 5 years have a Windows Installer equivalent readily available.

To use snapshotting, it is vital that you have a fairly clean test machine. Virtual machines are perfect for this. While VMWare is commonly used in bigger environments, VirtualBox is a free alternative and works fine for simple use. As with all Oracle installations, take extreme care and precaution to avoid unwanted apps.

Tip: You need to be able to restore the virtual machine to it's previous point, or your test machine will eventually be filled with unnecessary items and your snapshots will no longer capture everything needed by an installation.

The act of "Snapshotting" an installation is this:

  • 1. Take a snapshot of the virtual machine.
  • 2. Take a snapshot of the current system, using a special tool for it. This captures the current registry, and all files and folders on the system.
  • 3. Install the product as desired, and make any modifications as needed.
  • 4. Take another snapshot using the tool. The tool should then compare the differences in registry, files and folders and build a new MSI file for you.
  • 5. Revert the virtual machine, and remove the virtual snapshot.
  • 6. Open the newly build MSI in an editor, and clean it up.

Your new MSI will include a lot of entries that probably shouldn't be there. You need to clean it up as much as possible - that means selectively going through all components, files, folders, registry entries etc and make sure only the relevant entries remain. Everything else is removed from the package.

It's expected that the tool you use to build the snapshot MSI will also ignore or remove most of the trash entries. The tool uses internal exclusion lists to make sure you never see all the stuff that actually changed. For example, Microsoft Windows will store a huge amount of information about the current session in the registry and temporary files. This is not relevant to the software installation you did, and the tool should automatically remove or exclude it.

You should try and ensure that the MSI only contains:

1. The program files, to their desired folder (e.g. C:\Program Files\Vendor\Program\)
2. Any registry keys relevant to the vendor or program (e.g. HKLM\Software\Vendor, HKCU\Software\Vendor\)
3. Any shared components not already expected to be on the target system (e.g. C:\Windows\System32\Vendor.dll).
4. Only the desired shortcuts (e.g. in Start Menu, not on Desktop).

The above should allow you to capture what most smaller programs need in order to function. Almost everything else can be thrown out of the package. As your experience grows, you will know what to delete and what to keep. Should the package no longer work, a bit of troubleshooting should tell you what component you should reinstate.

Tips:

  • Check the programs registry keys for ways to disable updates, ads, etc.
  • Some older applications use .ini files for configurations ; some tools convert these to IniFile entries.
  • Some newer applications use XML files for configurations, especially .NET applications.
  • In a managed environment, entries for auto-updates and auto-startup should be removed from packages.

To avoid automatic startup entries, check your package for:

Remove HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\Windows\CurrentVersion\Run
Remove HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Run
Remove HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Run
Remove RunOnce (just below the Run keys) entries, which can be used to launch the program once.
Shortcuts in the StartUp folder

2.3 Deployments

Some of the bigger vendors (e.g. Autodesk and their AutoCAD product series) have made specific tools to help ease the deployment of their software. When available, always use vendor tools for deployments.

Tips:

  • Adobe Customization Wizard is great for their products (e.g. Adobe Reader).
  • AutoCAD installations have a built-in deployment builder.
  • Microsoft deployments use configuration (xml) files. They sometimes require you to cancel the installation at the last step, in order for it to save the configuration file.
  • Some software have a special interface if you run it with "/ADMIN"
  • SAP GUI is a very complex deployment, but their network setup allows you to customize everything. You can even run scripts.

2.4 Patches and updates

Last-minute patches should be available as .msp files from the vendor. If not, perhaps you need to extract the msp from the hotfix/patch in some vendor specific way.

Tips:

To extract the msp file from an AutoCAD hotfix, use hotfix.exe /e my.msp
For Microsoft self-extractor exe files, use /extract:path

There are a few main ways to distribute a patch:

  • If the software is already in use, just distribute the patch (msiexec /p patch.msp)
  • To create a patched AdminImage, use msiexec /a base.msi /p patch.msp
  • To deploy patches along with the installation, use msiexec /i base.msi /p patch.msp

Normally, vendors provide a setup that will install the very latest version, along with patches for those that have already distributed the software. You rarely need both.

Adobe

A separate mention of patching Adobe products is probably good here. For home users, this is handled automatically by Adobes own updater. For companies deploying software internally, Adobe currently rely on distributing a full point version, followed by separate quarterly updates and security hotfixes. This can be a mess, since security hotfixes block any further non-security hotfixes from being applied. So you need to apply all updates manually, in sequence, to an AdminImage before distributing a full version.

Customization of Adobe products are easy though: Updates are disabled either via properties, and/or via the Adobe Customization Wizard which allows you to change all relevant settings.

Example: Let's say that your organization decides to distribute a Dutch version of Adobe Reader to everyone. The latest version is (e.g.) 10.2.4. Adobe supplies version 10.0.0, and a list of patches. In order to make an initial installation package with a version of 10.2.4, you need to:

  • Create an admin image of 10.0 (msiexec /A Reader.msi) to a new folder
  • Patch the new admin image to 10.1 (msiexec /a adobe.msi /p 10.1.msp)
  • Patch the new 10.1 admin image to 10.2
  • Patch the new 10.2 admin image to 10.2.1
  • Rinse and repeat for 10.2.2, 10.2.3, 10.2.4
  • You now have a full english installation of 10.2.4
  • Use Adobe Customization Wizard to get the proper settings
  • Use the dutch transform file from Adobe with your english installation.

Adobe does not allow updating from a security hotfix, only from normal updates. So you have to create a patch list: all quarterly updates until last one, then further updates *except* security updates. This is why it's ok to waste some HD space while building the adminimage - should you accidentially include a hotfix, you can redo from the previous step. When done, remove the temporary images.

I hope the above has been fixed by Adobe, but it's been a while.

3. Customization

At this point you should have a scripting folder with the basic installation package. It should preferably only contain a single msi file, and no other irrelevant files (e.g. setup.ini, setup.exe).

If the msi is from a vendor, you should make custom changes in what is known as a Transform (.mst file). If the msi was built by you (e.g. via snapshotting), you can make the changes directly in the msi using an editor.

Java 8 Example (continued):

  • Since we found the vendor msi file for Java, we will create a Transform.

3.1 Creating the transform (MST)

A Transform file is a delta / difference file. It is always used in combination with a base msi, and describes any and all changes made to the original installation package in order to get to the desired state.

You can create a transform in various ways:

  • Use a tool to generate the transform for you. InstallShield (commercial) and others allow you to either generate a simple transform, or go through the UI Wizards and generate what is sometimes known as a Response Transform.
  • Use the free Orca editor from Microsoft, part of the Windows SDK. It allows you to generate a new transform, make any changes you want, and save the changes in the transform file.
  • Some tools (like mine) autogenerate a transform based on pre-described standard changes. These are beneficial if you always have to make the same changes regardless of what the product was (e.g. remove shortcuts, change Add/Remove Programs information). You can then open the created mst file with an editor such as Orca for any additional customizations.

If you are using a commercial tool, you can make most customizations via a User Interface. For advanced use, you should probably have a good idea of the various database tables and their use.

Java 8 Example (continued):

We will use Orca to create our transform. If you do not have it installed, you can find it on Microsofts site.
  • First, mark the original vendor msi in your scripting folder as ReadOnly. This is to prevent mistakes.
  • Open the msi with Orca.
  • Use the menu for Transform | New Transform. Further changes are now made in the transform, and we will save it in the end.

We can use the Windows SDKs WI*.vbs scripts to automatically generate a transform for us.

We will take a copy of the original msi, perform all our actions on the copy, then generate a transform (difference file) and document what was changed.

REM Take a copy to work on
Copy Test.msi Test2.msi
REM Change a property value
CScript //nologo WIRunSql.vbs test2.msi "UPDATE Property SET Value='0' WHERE Property='ARPNOMODIFY'"
REM Insert a property
CScript //nologo WIRunSql.vbs test2.msi "INSERT INTO Property (Property, Value) VALUES ('REBOOT', 'ReallySuppress')"
REM Delete a property
CScript //nologo WIRunSql.vbs test2.msi "DELETE FROM Property WHERE Property='ARPURLINFOABOUT'"
REM Generate a transform
CScript //nologo WIGenXfm.vbs test.msi test2.msi test.mst
REM Update transform summary info
CScript //nologo WiSumInf.vbs test.mst 4=TestAuthor 3=TestSubject
REM Document the transform changes
CScript //nologo WiLstXfm.vbs test.msi test.mst
REM Remove work copy
Del Test2.msi

Note: The changes to the Summary Information Stream is not listed by WiLstXfm, but if you hold your mouse over the mst in Windows Explorer, the popup tooltip shows the new values TestAuthor and TestSubject.

There are 3 special tables that are handled differently depending on your tool: Properties, InstallExecuteSequence and the Summary Information stream.

Properties

Properties are simple key-value pairs. They are stored in the Property table, and all other tables can use their values by enclosing the key in brackets (e.g. [ProductCode]). Most common customizations are made via simple changes to Property values.

Properties are not case sensitive, but they must be in upper case in order to be "public". A public property can be changed via the commandline. Try and always use upper case for your own properties.

The majority of vendor msis have properties with logical names controlling the most common aspects of the installation. Most vendors also document their custom properties and what their value do - if possible, try and have this information available. Look for "Admin" or "Deployment" guides on their website, or google the msi filename along with the property name.

Msi.chm, installed as part of either the Windows Installer SDK or Orca, contains details about the standard properties supported by Windows Installer. The Add/Remove Programs information about the installed product is handled by the ARP* standard properties.

Examples:

To hide the "Modify" button for an application in the Add/Remove Programs control panel, 
add a property named ARPNOMODIFY with a value of 1.

To change the phone number shown in the Add/Remove Programs control panel, change the value
of the ARPHELPTELEPHONE property.

You can use WIRunSql.vbs from the Windows SDK to execute sql statements directly against an msi.

Change a property value:

CScript //nologo WIRunSql.vbs test2.msi "UPDATE Property SET Value='0' WHERE Property='ARPNOMODIFY'"

Insert the REBOOT property with a value of ReallySuppress:

CScript //nologo WIRunSql.vbs my.msi "INSERT INTO Property (Property, Value) VALUES ('REBOOT', 'ReallySuppress')"

Delete the REBOOT property:

CScript //nologo WIRunSql.vbs my.msi "DELETE FROM Property WHERE Property='REBOOT'"

Summary Information

The Summary Information stream is a special table typically shown in a different user interface. In Orca, you can see it via View | Summary Information. The information here is shown when you hold your mouse over the msi file, and also control access to the installation (e.g. target platforms, minimum Windows Installer level, etc).

Elevation of privileges (UAC compliance) is also controlled by flags in the Summary Information.

The free Windows Installer SDK includes the WiSumInfo.vbs script that can update the Summary Information Stream for both MSI packages and MST transforms. For example, the following command sets the Author and Subject of the transform summary information stream:

CScript //nologo "WiSumInf.vbs" My.mst 4=TestAuthor 3=TestSubject

InstallExecuteSequence

This table is very special: it contains the logic and step-by-step guide of the setup. Each entry has 3 fields: Action, Condition, and Sequence.

During the actual installation or removal of a package, this table is first sorted by the Sequence number column. Each record is then checked in order: If the condition is empty or evaluates to true (or 1), the Action function is executed.

The Action values are special keywords, each one causing Windows Installer to perform a single function. The InstallFiles action, for example, is the actual function that copies files onto the target system during installation. Each function uses data from their relevant table.

In some tools, you can't modify this sequence directly without editing the tables. Instead, you can just select a position in the sequence for your custom actions. The tools will then automatically re-number the sequence table.

You may occasionally need to disable a problematic Custom Action (CA) by a vendor. If the CA has no impact on the installation itself, you can put a condition of "0" on the CA in the InstallExecuteSequence table. This will cause the condition to be false, and the custom action will not run.

Installation types - Minimum, Typical, All.

If you make no changes to a basic transform, the installation will be "Typical". If you need to install specific features, or have a full installation, you can either:

  • If your tool support it, change the individual feature states via wizards.
  • Change the INSTALLLEVEL to match the desired installation state. Each entry in the Feature table has a Level field that indicates the minimum INSTALLLEVEL needed in order to install this feature.
  • Use command line parameters to specify the features during installation (ADDLOCAL).
  • Put conditions on unwanted features, or raise their Level beyond INSTALLLEVEL.

Disabling Updates

In almost all cases of managed deployments, automatic software updates must be disabled. This is typically done by finding out which property or registry value should be changed in order for the update feature to be disabled.

How you disable updates depends on the application and vendor installation package. These are fairly common scenarios, in order of preference:

  • Updates are impossible unless you install an optional feature.
  • Updates are disabled by setting a property in the installation package
  • Updates are controlled via a HKLM registry key.
  • Updates are controlled by a setting in a system file.
  • Updates are disabled by removing a local .dll.
  • Updates are disabled per user, via registry/file configuration.
While the method may vary, most applications have a way to disable auto-updates.

Java 8 Example (continued):

You should have Orca open with the jre1.8.0_40.msi from Oracle and a newly created transform.
  • Go to the Property table, and look for relevant names. Tip: Googling the suspected property names along with the product name is likely to provide the relevant documentation.
  • Change the AUTOUPDATECHECK value from 1 to 0.
  • Change the JAVAUPDATE value from 1 to 0.
  • This should do it, but we'll check some more (from memory, the property used to be JU).
  • Open the Registry table.
  • Note that they still store the value of any JU property in the registry - probably for backwards compatibility.
  • We decide to add the JU property, just to be sure (a quick search reveals this was the indeed the old property).
  • Go back to the Property table.
  • Press Insert to add a new property (since JU didn't exist before).
  • The name is JU, and the value is 0. (0 is the value for 'false').
  • 3 entries should be green in Orca, meaning they have changed from the original installation: the AUTOUPDATECHECK value, the JAVAUPDATE value, and our new JU property and value.
  • Use the Transform menu to Generate the transform. When prompted for the name, place it along with your jre1.8.0_40.msi and call it Java.mst.
  • Note the title of the Orca window changes to jre1.8.0_40.msi (transformed by Java.mst).
  • You can now close Orca, and prepare to test the msi and transform.

Occasionally there is no documentation from the vendors side, but you can disable updates from within the application. You would then attempt to locate the specific setting that controls the update mechanism, before implementing or updating the setting in the installation package.

Example:

  • Install the software on a test virtual machine.
  • Start the software and find the setting to disable updates.
  • Disable updates and exit the application.
  • Check the applications registry keys and other configuration settings for changes.
  • In this example, changing a single value under the vendors registry key is enough.
  • Open the installation package and find the corresponding registry entry.
  • If the Value of the registry entry contains brackets (e.g. [UPDATESENABLED]), it references a property and you can find it in the Property table.
  • In your transform file, change the value of the property or registry key to the value that disables updates.
  • Test that your deployment ends up with the proper value in the registry.

Add/Remove Programs

The information listed for applications in the Add/Remove Programs control panel is extensive, and custom fields can be added for more detail. It is common to clean up the information, for example by making sure there are no links to websites, or that the IT Helpdesk phone number is shown instead of the vendor hotline.

All this ARP information is gathered from msi properties during installation. For detailed information about the available properties, see ->here.

User Content

If you need to have something put into a users profile before the application is launched, you have a few options:

  • Use the self-heal capabilities of the primary shortcut.
  • Deploy the user settings via other means (e.g. Active Directory or RES).
  • Create an Active Setup entry (the "Personalizing.." windows during login).

Self-Heal

In order to enable self-heal (auto-repair) for a component, it must have a KeyPath. Windows will check for the specified file, folder or registry key before launching the application. If the file, folder or registry key was not found, a default repair of the installation if started. Upon succesfully repairing the application, it is launched.

If you include a component with file content for a user profile, include a user (HKCU) registry key and have the components KeyPath set to point to that registry key. The Attribute for the component must include the RegistryKeyPath flag (4). The component must belong to the same, or parent, feature of the component containing the executable launched by the primary shortcut.

Example:

When the primary shortcut is clicked, Windows knows which Product, Feature and Component it points to. The component contains the main executable program, so Windows knows what to launch.

Windows then checks that each component belonging to that feature are available (marked as installed). For components that have a KeyPath, it then verifies that it was actually installed - by checking for the registry key or file.

In this example, Windows checks the HKCU user registry and finds our registry key missing. A repair is then started, and by default this also repairs user settings. The user registry key is then installed, along with the file content in the user profile. Lastly, the application is launched.

User settings (GPO/AppSense/RES):

You may have to deploy a few choice user settings via Group Policy Objects (Active Directory), or your deployment platform (AppSense, RES). If so, you could export them from the package, and provide the settings in your deployment file and documentation.

Many applications do not install all user settings initially, but instead rely on the application to create them. It's therefore better to use a test image where you install and use the application, and note any settings that should be maintained by RES/AS. Export the relevant part of the user registry (via RegEdit), and copy the application files from the %appdata% folder to another location. Document what needs to be imported by the AD/AS/RES server or deployment team.

Active Setup:

Active Setup is an older Microsoft technology, primarily used for Personalization of Microsoft software. It's reponsible for those short repair popups during your very first login, when setting up Windows Media Player and Internet Explorer for the user.

Two similar registry keys are used: one in HKLM, and one in HKCU. Durin login, Windows checks that all entries found under

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Active Setup\Installed Components
are also in the
HKEY_CURRENT_USER\Software\Microsoft\Active Setup\Installed Components
If not found (or, if the Version in the HKLM entry is higher than the one in HKCU entry), the command specified in the HKLM entrys StubPath entry is run. This would typically be an
MsiExec /fomus [ProductCode] /Qb
command, which would repair the installation. Repairs are run under the current user with elevated privileges, and you can therefore install files to the current users profile and write to the HKCU registry hive.

The big disadvantage of this system is that it requires you to log off after installation, so Windows can check on next login.

Windows Installer Table Overview

The main table structure is this:

	Feature -> 				One entry per feature.
		FeatureComponents -> 		List of components associated with each feature
			Component(s) ->		Information about the folder, conditions and self-heal
				Registry	Each entry here is associated with a component
				Files		...
				Shortcuts	...
				...		...
While there are many more tables, most are self-explanatory and serve a single purpose or function.

Features are used to group various parts of the application into smaller bits of content, optional or required. Components serve as logical groupings of files and registry entries.

For example, a simple package could have very few components: 1 component to install 10 files into a folder, and 1 component to install 10 HKLM registry keys.

Best practices would quickly dictate further sub-grouping: An executable file should have its own component. If each of the 10 files were an executable, and had 1 associated registry key with it that should only be installed if the executable is installed : 10 components, each with 1 executable and 1 registry key.

4. Testing

Testing is typically done by dedicated Testers and their checklists, to ensure consistency in the delivered output. But the initial testing is done by the packager, who must verify that the installation is sound. An example is found below.

4.1 Test procedure

The typical initial testing procedure for a packager is this:

  • Take a snapshot of the virtual test machine.
  • Install your package (silently, using script)
  • Go through installed checklist and note any items to fix.
  • Remove your package (silently, using script).
  • Go through removal checklist and note any items to fix.
  • Revert virtual machine to snapshot.
  • Remove virtual snapshot.
  • Fix and retest, or deliver.

Having built the installation, you should already be certain that no unnecessary items are installed. So you can mostly just verify that:

  • The installation worked without errors.
  • Shortcuts are in the right place.
  • The Add/Remove Programs information is correct.
  • The installation directory is correct.
  • The application starts without issues.
  • The application seems usable (try using it!)
  • Help and various other menu functions work.
  • Printing works as expected.
  • Load/Save works as expected.
  • Data is written to the correct location.
  • Configuration settings are correct.
  • Updates and auto-startups are correctly disabled.
  • The application closes correctly, and acts correctly on restarts.

More advanced tests would include:

  • Repair functionality and self-heals.
  • Logging in as another user and using the application.
  • Testing access/security issues.

The packager also needs to test that after removal, not much remains of the product. It's fairly common for applications to automatically generate data in the users appdata folder or user registry, and these would be left over after removal as they were not installed along with the application. This is acceptable. Preferably, there should be no other trace of the application left behind at this point.

The following is typically checked after removing an application during initial testing. Any exceptions should be noted or fixed (if possible).

  • The HKLM\Software\Vendor\Application\ keys are gone.
  • The installation directory under C:\Program Files\ is gone.
  • The applications shortcuts are gone.

When an msi package is authored, the default for uninstall is to remove every item that was installed. Certain files and registry may be marked as "shared", and will only be removed if no other software had installed the same shared file. It's known as a reference counter: each installation will increase the counter by one, while removing the software will decrease the counter. A shared file will not be removed until the associated reference counter is 0.

It's also common to see empty folders left behind. This is usually due to a missing RemoveFolder table entry for folders created via the CreateFolder table.

User Profiles

When in managed environments, the application is never installed as the actual user. It's installed under a System Administrator account, and becomes available to all users on the PC. The installation is said to be made per machine (ALLUSERS=1).

This also means that the MSI will *not* have the ability to install or remove content in the current user profile, as it's technically another user. So you cannot easily remove items from users profiles during automatic removal of the software.

When you run the scripts to test installation or removal, you are running them as the current user (although with elevated privileges). Be aware that while installing/removing user content work at that test, it will likely not work when deployed via other means.

Scripts

To speed up the process and automate the install and removal of the software, a script is often used. This is a small command (.cmd) file that simply installs or removes the software silently. Copy the desired MsiExec command text to a new file named install.cmd or uninstall.cmd, change the names as needed, and save it the scripting folder along with the msi/mst. From then on you can just doubleclick on the .cmd file to quickly install or remove the application. Some tools generate these automatically for convenience.

An additional benefit is that you can right-click on the script, and chose Run as Administrator. This is different from being prompted to run with elevated privileges, and result in an installation closely resembling the actual deployment that will happen later.

Example basic install script (install.cmd):
MsiExec /I "My.msi" /Qb

Example with transform and message at end:
MsiExec /I "My.msi" TRANSFORMS="My.mst" /Qb+

Example with properties, logging and a Cancel button:
MsiExec /I "My.msi" ALLUSERS=1 REBOOT="ReallySuppress" /L*v "%TEMP%\My.log" /Qb-!

Example basic removal script (uninstall.cmd):
MsiExec /X "My.msi" /Qb-!

Example basic removal script via ProductCode, with a message at end.
MsiExec /X{ProductCode} /Qb+

Note:   When using a ProductCode, you must include the curly brackets. E.g.:
	MsiExec.exe /X{11111111-1111-1111-1111-111111111111} /Qb+

For uninstalls, it's best to use the /X{ProductCode} as it uses the cached msi and information. The ProductCode is an msi property ; use an MSI editor to look up the value, or copy it from the registry on an installed machine.

We can create a script that will generate an install and uninstall.cmd for a specified msi file. This example uses WiRunSql.vbs (from the Windows SDK) to obtain the ProductCode from the msi before creating the batch files.

CreateInstallScripts.cmd:

REM Provide the name of the msi on the commandline, or change the next line
SET MYMSI=%1
CScript //nologo WIRunSql.vbs %MYMSI%.msi "SELECT Value FROM Property WHERE Property='ProductCode'">%MYMSI%.pc
SET /P MYUNINSTALL=<%MYMSI%.pc
ECHO MsiExec /X%MYUNINSTALL% /Qb-!>%MYMSI%-uninstall.cmd
ECHO MsiExec /I %MYMSI%.msi /Qb-!>%MYMSI%-install.cmd

The ProductCode is stored along with other product information under your applications subkey in the HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\ registry after installation.

MsiExec.exe is the Microsoft Windows Installer executable. It is used to install, remove or configure products both automatically and interactively. In the above examples, we instruct MsiExec to install (/I) or remove (/X) an msi, with a UI interface set to basic (no wizards shown).

Run MsiExec /? for more detailed information.

Java 8 Example (continued):

To test the java installation with our transform, we create 2 scripts (command files) :
  • Create a new text file named install.cmd, with the line below:
  • MsiExec /I "D:\Test\Java\jre1.8.0_40.msi" TRANSFORMS="Java.mst" /Qb+
  • Doubleclick the install.cmd file. This should result in an error as the installation initially fails.
  • Right-click install.cmd, and choose Run As Adminstrator. You should now get a message that the installation was succesful.
  • Verify the Java version was installed (usually to C:\Program Files\Java\).
  • Create a new text file named uninstall.cmd:
  • We can also find the uninstall command line by checking the registry after an installation. Run RegEdit, and browse to HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall.
  • Search for Java, and find the correct entry (you may have multiple).
  • For our Java package, it was found under the subkey {26A24AE4-039D-4CA4-87B4-2F86418040F0}. The subkey is the ProductCode.
  • Copy the contents of the UninstallString entry: MsiExec.exe /X{26A24AE4-039D-4CA4-87B4-2F86418040F0}
  • Paste this into the uninstall.cmd script. You can add /Qb+ at the end to get a message after removal.
  • Test the uninstall.cmd - don't forget to Run as Administrator.
  • Verify that the Java version was removed.

Java 8 Example (continued):

The purpose of the transform was to disable updates, so verify that this was done. Open the Configure Java control panel applet and check that only 4 tabs are available: General, Java, Security, Advanced. If updates were enabled, there would be a 5th tab for Updates.

5. Documentation and reporting

You should typically deliver at least one document describing what was changed in the installation. Some tools can autogenerate a report / document for you, to use as the basis.

WiLstXfm.vbs (from the Window SDK) documents the changes made in a transform (mst) file. If you pipe the output to a text file, you now have a record of what was done, and can include it in your documentation.

CScript //nologo "WiLstXfm.vbs" "mymsi.msi" "mymst.mst" >changes.txt

Java 8 Example (continued):

For our Java package, we changed 2 properties and added a 3rd. Document them.

If you have the Windows Installer SDK installed, you could use this command to document the changes:

CScript //nologo "WiLstXfm.vbs" "jre1.8.0_40.msi" "Java.mst" >Java-changes.txt

Full documentation could include

  • All changes made in the transform.
  • The reason for the changes (guidelines, moved shortcut, disabled updates, etc)
  • Any LaunchConditions that were kept.
  • Any Custom Actions you made.
  • Any ICE warnings or errors that were not fixed.
  • Any warnings or errors seen during install/test/uninstall.

5.1 Validations

You should run ICE Validations against your output, to make sure it doesn't result in errors or incorrect behavior. Most tools can do so (Orca: Tools | Validate). While you can fix some vendor errors, there may also be too many to fix in the given timeframe. Document any important ICE validation errors that you do not fix.

Any changes you make should not result in a validation error or warning. If it did, find out what caused it and fix it. While some are caused by the tools used, most are easily fixable.

For example, specifying a file as Compressed when the whole installation is marked as Compressed gives a warning. You can fix this by changing the highlighted record in the File table ; by subtracting 16384 from the Attribute field value, you are removing the msidbFileAttributesCompressed flag. See the File Table documentation for more information.

In Orca, use the Tools | Validate function to validate your output using the "Full MSI Validation suite". After you close the Validation Output window, the tables and records that contain errors will be color highlighted. You can then right-click on an entry to see the validation error.

You can automate validation using the Windows Installer SDK. Once installed, there will be an MsiVal2 installer package somewhere (e.g. "C:\Program Files (x86)\Windows Kits\8.0\bin\x86"). Find it and install it, the default is to "C:\Program Files (x86)\MsiVal2\". You can now run this and pipe the output to a another text file for easy documentation or reporting.

Example script to validate an msi using MsiVal2 from the Windows Installer SDK:

SET VALDIR=C:\Program Files (x86)\MsiVal2\
REM -f = no info messages, -l = write to this file
"%VALDIR%MsiVal2.exe" "my.msi" "%VALDIR%darice.cub" -f -l mylog.txt

There is no way to easily validate a transform only. What the tools do is: copy the base msi to a temporary one, apply the transform to the temporary msi, validate the new temporary msi, then remove the temporary msi.

Example script to validate transform My.mst using the base installer package My.msi via the temporary Temp.msi:

SET VALDIR=C:\Program Files (x86)\MsiVal2\
REM Creating temporary msi Temp.msi
COPY "My.msi" "Temp.msi"
REM Applying transform My.mst to Temp.msi
CScript "WiUseXfm.vbs" "Temp.msi" "My.mst"
REM Running validation..
REM -f = no info messages, -l = write to this file
"%VALDIR%MsiVal2.exe" "Temp.msi" "%VALDIR%darice.cub" -f -l "My-mst-validation.txt"
REM Removing temporary msi
DEL "Temp.msi"

6. Delivery

Most larger businesses use a deployment system like Microsoft SMS/SCCM, ManageSoft, uDeploy or RES. The basic purpose of these systems is to be a central database over what applications are available to a PC or user. You then put all software on a shared location, and grant access to it via Active Directory groups or Collections for SMS. All software is automatically installed, removed or updated based on entries in these databases.

In order to make MSI/MST packages available in such systems, an additional file is typically needed for automation purposes. This allows the server administrator to simply import all information about the new software into the main database. Most tools generate at least an SMS file that contains installation/removal instructions and product information.

Before you deliver your test scripts, you should remove any PAUSE statement in your batch files. Later tests are typically done via SCCM or similar, and they may run your script directly instead of entering the commands (or importing an sms file). A Pause statement in your script would then pause the installation, waiting for a non-existing user to press any key to continue.

Java 8 Example (continued):

We will create a small SMS Package Definition file, to facilitate deployment of our Java package. Create Java.sms with the following text:
[PDF]
Version=2.0

[Package Definition]
Publisher=Oracle
Name=Java (64bit)
Version=8.0.40
Language=English
Programs=Install, Uninstall

[Install]
Name=Install
CommandLine=MsiExec /I "jre1.8.0_40.msi" TRANSFORMS="Java.mst" /Qn
AdminRightsRequired=True
SupportedClients=Win NT (x64)
Win NT (x64) MinVersion1=5.10.0000.0
Win NT (x64) MaxVersion1=9.99.9999.9999

[Uninstall]
Name=Uninstall
CommandLine=MsiExec.exe /X{26A24AE4-039D-4CA4-87B4-2F86418040F0} /Qn
AdminRightsRequired=True
SupportedClients=Win NT (x64)
Win NT (x64) MinVersion1=5.10.0000.0
Win NT (x64) MaxVersion1=9.99.9999.9999
A full reference is available ->here.

Note that during deployment, MsiExec is run with /Qn instead of /Qb+. Since the installation is run as another (hidden) user, there is noone to read the message and click [OK] ! We therefore run the installation with no userinterface at all (/qn).

Java 8 Example (continued):

Your package is now ready to be delivered to testing / deployment, and should contain:
  • Install.cmd - script for quick install test
  • Uninstall.cmd - script for quick removal test
  • jre1.8.0_40.msi - the vendor msi
  • Java.mst - your customized transform
  • Java.sms - the SMS deployment file
  • Some form of documentation

7. Virtualization

Lately a lot of companies have switched to a combination of a few locally installed applications (such as MS Office) and the remainder being virtual applications.

Virtual applications are applications that essentially run in a "bubble", their own imaginary Windows system. They have next to no impact on the local PC. While they are a bit slower, it's rarely an issue for the most common applications on newer PCs.

The main vendors are Microsoft (Application Virtualization, or App-V), VMWare (ThinApp) and Citrix (XenApp).

App-V is a separate topic, but you can read my quick guide to it here.

8. Tools

There are some tools available to help in tuning, repackaging or deploying applications. Most companies pay to use InstallShield, which has a whole suite of applications and tools supposed to help you manage large scale repackaging. For a price, it can help.

Microsoft provides the free Orca editor and Windows Installer SDK, along with several tools, scripts and other helpful resources. WiX is a free XML based engine for creating MSI packages, by Microsoft. While strong, these are for advanced use only.

Another alternative is InstEd, an Orca-like editor but much more advanced. There is also the PACE suite, which is much cheaper than InstallShield, and includes snapshotting functionality.

If you are using Orca, you probably also have Msi.chm installed. This help file from Microsoft is very helpful, as it is an offline complete reference for Windows Installer.

I made a small Tuner to help me consistenly create perfect transforms for various clients. If you are making a lot of transforms for one client, consider creating a script to help you do it consistently. I included examples of how to do so (using the free Windows Installer SDK from Microsoft) in the orange boxes.

9. Finished

Please keep in mind that this was the simplified guide to our basic tasks - it's meant to give you a basic understanding of the tasks at hand, and how to perform them. For further reading, please check MSDN or your tool documentation.

Developing good installation packages can be a highly complex task, and almost anything not provided by default can be made in the built-in scripting engine for Custom Actions. If that's not powerful enough, you can always call your own DLL functions or execute external programs - before, during or after the installation.

I hope you enjoyed this quick guide ! For more advanced topics, see Packaging 201

(c) 2015 Nicolai Kjaer, Spinner Software B.V.