| Main Page -> Papers -> Package Management for System Administration - installp |
Note: This is the AIX / installp specific content of a larger paper. If you came here directly, the recommendation is to start at the beginning.
AIX / installp introduction.
The package format
The installp package file is actually a backup file with specially ordered content. Unlike other package formats, it does not have a "package" header that wraps the backup archive*, but instead a backup format that encapsulates all package information.
[* "backup archive" is used in the generic sense here. The point is that all installable files need to be captured in some sort of restorable archive. Technically, this format is a backup file that contains an archive file (created using the ar command).]
The AIX terminology for a "package" can be misleading to first time users. What most people would consider a package is actually an AIX "fileset". An AIX package is the encapsulating "wrapper" that contains one or more filesets. So, a fileset is an installable set of files that is delivered, possibly with other filesets, in a package.
To clarify a bit: A package is viewable from the filesystem - a fileset is not. Filesets are installed to the system via the package manager - packages are not.
This paper attempts to draw the distinction between a package and a fileset where it is most relevant, but may use the generic "package" when referring to an installable fileset when the distinction is clear or less important.
AIX maintains separate fileset databases for different parts of the filesystem. These parts are the USR (/usr), ROOT (/), and SHARE (/usr/share). Historically, these sections have been segregated for network boot and install issues where each part may need to deal with special access methods such as shared or read-only access. With the advent of WPARs, these separate parts are now even more relevant to administrators.
The resulting fileset format is required to segregate content by these filesystem types / locations so that the install of the files are properly tracked in each location.
The packaging tool
This (installp-specific section of the) paper will focus almost entirely on the mkinstallp tool and its specially formatted template file as a means to create packages. The mkinstallp tool represents a greatly simplified and automated means of packaging content over the more traditional means of manually building the components. That said, the limitations of the tool will become apparent by the later examples, and those looking to push the envelope of AIX packaging needs should consider another means and another source on the subject.
Alternative / complementary tools
IBM has recently introduced the AIX Runtime Expert to help set and validate configuration changes across systems. This tool can be used for many of the examples here. Most of the versioned / basic file distribution may be best done from packages but this new tool can offer a single framework for many of the configuration options on an AIX system. AIX has long shipped with the tun* commands that can be used to extract (get), set, and validate the kernel tunables.
The point here is to be aware that AIX ships with a rich set of tools that can be used specifically for configuration management or in conjunction with the tools listed here.
Example 1 - Pushing system config files
Example definition and challenges
The point of this package is to deploy version-locked config files that will have requirements much more like binary files than config files. This means that the package manager will be used to both deploy and validate that the current version is correct. This is not a standard (package) deployment method for config files - that are usually allowed to be modified without invalidating the package.*
[* The goal is to have lppchk -c fileset fail when a config file has been modified. (Typically it would only fail on the change of a non-config file - such as an application binary.)]
Preparing content
For this example files are collected into two sub-directories of a "working directory" called /tmp/custom-net. (The choice to use /tmp as a base was to reduce some of the excessive pathing noise that would occur if it was run out of a project directory.)* The two sub-directories are pkg_build and template_src.
[* See the Notes and commentary section at the end of the paper for a discussion on using a specific project directory. The examples here were staged (via scripts) for several reasons - one of which was that the content was maintained by a regular user and the builds were done as root. The scripted stage and build process allowed a manageable contamination of the user's project directory with root owned content.]
The pkg_build directory is designed to look like the / file system, but with just the files we will be packaging. It is important that this portion of the staging be done as root. This is because the files will need to be staged during this step with the final owner and permission configuration that they will have on the target systems. The files to be packaged are shown below in the listings with the owners and permissions.
The template_src directory contains the install / uninstall scripts used to backup and restore the files as mentioned in the previous section. It is not necessary to stage the files in this manner. This layout was chosen to keep the working directory clean and to separate content based on logical classifications. (The scripts could be accessed from any location on the system as long as they are properly referenced using a full path name in the template file.) The content of the scripts are included in the file boxes following the directory layout.
A way to think of the two sub-directories are as this: The build root is a staged version of the real filesystem that will be (selectively) backed up during package creation. The template source directory contains files that mkinstallp will archive into the backup file as part of metadata and active / scripting* content for the package.
[* "scripting" is used in the generic sense here. This active content can be in any executable format - such as a binary executable - but tends in most cases to be shell scripts.
The mkinstallp template file is placed in the root of the working directory. The location of this file is also a personal convention and not a requirement. How this file is accessed will be seen in the next section.
The .info directory is not necessary, as the build process (in the next section) will create it if it does not exist. It was created here as a convention for future examples that will require it to include files in the package that mkinstallp will not handle.
Note, once again, that the permissions set in the staging here are what the permissions will be on the target system. If these files are owned by a local / non-root user then they will be owned by that user when applied to the target systems.* mkinstallp will allow you to create a package as a user and will not complain if the owner/permissions are not right for the target systems.
[* And this is where the choice of maintaining as a non-privileged user, but then staging and building as root tends to make sense. Once again, a more thorough discussion can be found in the Notes and commentary section.]
The following two file boxes contain the contents of the install and uninstall files included from the template_src directory. The filenames are based on internal package conventions (although not a requirement at this point). See the table of script extensions at the bottom of the mkinstallp template page for a breakdown of when they run and the purpose of each.
Create a package manifest
mkinstallp can use a template file as a source for the build or be invoked interactively (and it will build a template file from user input). The examples here use a template file
| • | as a more appropriate means to script these builds, |
| • | provide a consistency between the builds, |
| • | communicate the points of interest in this documentation, |
| • | provide for a method of debugging, |
| • | and because the command line method can be a bit daunting for the first time user. |
The following file is the template file used for this package.
Some notes on the file contents:
| • | In AIX terms, this template file describes a package. The package contains a single installable fileset. In more generic terms, this template file describes a package-file that contains a single sub-package. |
| • | The package name is ordered in reverse specificity of the content. "acme" is the company, "custom" is a group of customizations, and "net" is a specific set of customizations. The idea is that ACME Corp. will have multiple packages, and more than one of them will be customizations like this "net" example. (Note: Later examples will deal with multiple location packages. The assumption here is that ACME has only one data center location, and therefore only one network configuration - otherwise it might have been named like the "acme.datacenter.nyc.rte" fileset in the mkinstallp template page.) |
| • | The package and (the single) fileset VRMF are the same, as the package consists only of the versioned files. As a result, we simply keep them in sync. The next version can be 1.1.0.0, then 1.2.0.0, etc. The second "octet" is versioned here - our expectation is that we will never overflow the possible revisions for that octet with edits to that file. It should be noted that the potential versions are not the same for each octet. A breakdown of the VRMF can be found in the mkinstallp template page. |
| • | The fileset name is the package name with the ".rte" (Run-Time Environment) extension. This is by common convention, not necessarily a rule. There are a number of recognized extensions, this is the most appropriate for these files. The package name is generally a common subset of all included filesets. With only a single fileset in the package, the names are nearly the same. |
| • | This is not an update package. Update in this context means that the package is effectively a diff between versions. As a result, the package tool will complain if we used this version number and set update to be "Y". (This is not to be confused with upgrading a package to a new major version. Here "update" is a statement on the structure of the package.) |
| • | No copyright / license was specified. |
| • | The ROOTLIBLPPFiles section is where we specify scripts that are used during package install / uninstall operations. This section has an equivalent for the /usr (USR) part called USRLIBLPPFiles. The /usr (USR) version was not used because the files we are touching are in the / (ROOT) filesystem. The file keywords are not well documented and were reverse engineered from the mkinstallp script. Additional info can be found in this breakout of the mkinstallp template format. Example four will utilize files not covered by the mkinstallp template. |
| • | This fileset does not require the system to re-generate a boot volume via bosboot. (None of these files are required for the system to boot or needed at that point of the system initialization.) |
| • | This fileset does not require any requisite filesets. (No other filesets need to be installed before or along with this install.) |
| • | There are no files to be placed in the /usr filesystem. For this reason the USRFiles section is empty. It is required in the mkinstallp template file even if it is empty. Because we do have / (ROOT) files we include the optional ROOTFiles section. |
| • | The naming of the template file uses a ".template" extension, and uses a project-specific naming convention of custom-net used throughout this project / paper. The template file name is not relevant, although the script files specified here and placed in the template_src directory in the previous section do have a naming convention. (Note: When copied by the mkinstallp process the naming convention will be automatically applied when inserted into the package. They are named with the fileset.file_purpose here not as a requirement, but following the internal convention of the package. When copied manually, as in example four, this naming convention must manually be applied by the staging script / method.) |
| • | Package files (content) are specified relative to the build root /tmp/custom-net/pkg_build as though it were the root filesystem. Script files are specified relative to the actual root file system. |
Build the package
In this example, mkinstallp is executed from the build directory (/tmp/custom-net/pkg_build). This is because the command places a special importance on the directory it runs from. The problem is that mkinstallp will create & use directories relative to a / (root) file system from its current working directory. So, by running it in the build root, it will create temp files and the package there - and not modify the actual root file system or the directory it is run from. It is possible to run with the -d option as a reference to the build directory. This would allow mkinstallp to run from another directory, but reference the build directory for the file content. Even if the -d option is used, it will create files relative to where it was run.
The template file is specified to mkinstallp via the -T option. The path to the file is relative to the current directory. (Absolute and leading / file paths are only required within the template file.)
The resulting package will be placed in the /tmp* directory within the build directory. (As mentioned earlier, if mkinstallp is not run in the build root then it will create a tmp relative to the directory it is run from and place the package file there.)
[* As with most references in the build (root) directory, the /tmp path is not really /tmp but /tmp/custom-net/pkg_build/tmp. This is a convention from the mkinstallp process and its template file that is carried forward here.]
mkinstallp will complain that the files in this example are already exist as system files. This is a warning that can be ignored. (Ignored because the impact of modifying existing files belonging to another fileset has been acknowledged and somewhat mitigated.)
Finally, a clarifying word on the directory structure. The template file was placed in the root of the staging directory (/tmp/custom-net) as a local convention that seemed "logical" because of the two sub-directores. The template, and even the template source files, could easily be placed into the build root and referenced from there. The idea here was to draw a distinction between how the files are referenced by the mkinstallp process.
Example 2 - Deploying the ACME ToolKit
Example definition and challenges
The primary purpose of this example is to distribute ACME local-ware. The secondary purpose is to deploy the software with a license / disclaimer authored by ACME IT management.
installp has a specific capability for handling licence requirements that will be utilized. How the license file is specified is not terribly intuitive and must be used with care. If the file is specified, but not properly included, the package build will succeed, but the install may fail. Properly packaging this file will be a focus of the following sections of this example.
Preparing content
Staging the files will look much like the previous example with a few exceptions. The package does not have script files for the installation so a template_src directory has not been included to contain these "helper" files. The package puts the config files to be distributed in a special config sub-directory in /etc. The staging directory also includes a /usr/swlag/en_US directory even though this (directory) will not be explicitly included in the file list within the template file.
The missing script files and support directory are fairly self explanatory - this package simply does not use scripting during this install or uninstall, and therefore does not have the active content as in the last example.
Finally, the /usr/swlag/en_US/acme.toolkit.la file is included as this is the standard location for license files (assuming the ACME locale). In short, the file is specific to this package, but the directory is not. If the package was installed with a different locale (other than "en_US"), then this path would be different. There are options to specify multiple locales and then use a locale variable when specifying the license. (More on this in the next section of this example.) Here, the assumption is that one locale file will be used, and it will be hard coded later to the path of that locale.*
[* There is a bit more expansion on this concept in the mkinstallp template page.]
The following is a set of OS commands to document the build location layout. Permissions are only shown for the /usr/bin directory but will be relevant for everything in /etc/acme, the /etc/acme directory itself, our license file, and our binaries in /usr/bin. As in the previous example, the permissions / ownership set here are what will be used when the files are installed to the target systems.
Create a package manifest
In comparison to the previous example, the ToolKit template file is slightly more complex in file content, but exchanges the new license file complexity for that of scripting in the previous example.
The following discussion points will center around the deviations from the previous example.
| • | The ToolKit license agreement file is specified three times. The first states that the acceptance of the license is required for installation. The second states that we should include the license file because it is not already on the (target) systems. The final mention of the license file in the USRFiles section is not so that it will be installed (that can be accomplished by the second mention alone), but so that it will be removed if/when the package is uninstalled. The second and third references would not be required if the license was known to be on the system. (One alternative would be to make the license file requisite fileset in the same package. Requisites and multiple filesets (per-package) will be introduced in later examples.) |
| • | If the license agreement is required for install (satisfied using the "-Y" option to installp) the license agreement must be included in the package, or exist on the system prior to package install. If these conditions are not met then the build will successfully finish, but the package install will fail because it is unable to present the license for the user to accept. |
| • | Note the ";" at the end of the License file path: entry. This is so that multiple files can be included. Here only one is used. The Name of license agreement: does not use this convention because it only references a single license. |
| • | The /etc/acme directory is included in the ROOTFiles section because it is installed (created) with this package. Once again, this directory was included only to make this point. By including this file in the manifest (file list), it will be owned by the fileset and removed when the fileset is uninstalled (provided that no additional content is added). |
| • | This example has a ROOT and USR section, while the last only had a ROOT section, but still included the USR. The ACME ToolKit will place files into both filesystems. |
| • | This install includes only "simple" / manifest-installable files, so no scripts were specified in the template file. |
The template allows for the specification of multiple license files to be included in the package. (Only one is specified here.) When multiple license files are included, a wildcard (%L) can be specified for the locale in the path. This will allow the user to view the appropriate locale when installing the package. As mentioned earlier, this package is for the ACME corporation who uses only a single locale, and has hard-coded this file to the "en_US" locale. More information on these items are included in the mkinstallp template page.
The configuration files specified in this file list are version locked to the fileset. This means that if the configuration files are changed, then the validation* of the fileset will fail. This is desired behavior for the first example, but may be inappropriate for the config files in the ToolKit. One method to allow configuration files to change is to mark them as "VOLATILE" in an override inventory file specified in the template file.
[* The fileset can be validated using "lppchk -c fileset-name".]
The following template file (with the majority of the content removed for brevity) shows how to specify an inventory override file for files in the ToolKit fileset. The override file is shown (in its entirety in the second file block).
The following file is the inventory override file for the config files in the ToolKit fileset. It is in stanza format with the stanza header being the file name and then all items being ignored except for the "size" and "checksum" items that are set to "VOLATILE". When files have indeterminate size and checksums then these checks will be ignored when validating the fileset. Note that only one override inventory file is used per fileset, it can contain both ROOT and USR parts.
Build the package
This package requires no special building instructions beyond the first example. That said, the recommendation is to test the install (specifically the license) prior to deployment. If this part is not properly specified in creation, the package install will fail on the license acceptance.
The following system output demonstrates how to view the license file of a fileset. Note that installp will automatically generate a .toc if one does not exist.
The following AIX screen is the process of extracting a single file (liblpp.a) from the package file, then extracting a single file (acme.toolkit.al) from the liblpp.a file. The process begins by using restore to list all files in the package and then extracting one of the listed files. Once the file (an archive) is extracted, ar is used to list all contents and extract a single file. Finally cat is used to display the contents of the final file. (Additional line feeds are used to delineate the commands.)
Example 3 - Collections of packages
Example definition and challenges
Two obvious methods of deploying groups of filesets in AIX is by declaring a bundle or by creating a grouping fileset through requisites. The method used here is to create a fileset that drops a single flag file* and has dependencies on a group of filesets. When installp is invoked with the "-g" flag it will resolve the dependencies and install the required filesets.
[* A bundle could not drop the flag file like the fileset does in this example.]
Preparing content
When compared to previous examples, the staged files are even more simplistic. The bulk of the "complexity" of this package will be seen in the template file in the next section.
Create a package manifest
Except for the "Requisites" line, this package template looks much like the previous examples. This line uses two potential formats; a file, or the somewhat complex line seen here. A break-down of this content is discussed after the template.
installp packages allow for multiple different types of requisites. (See the mkinstallp template page for an explanation of the different types.) This fileset requires that the required filesets be installed prior to this fileset's installation. The first "Requisites entry breaks down like this:
| * | Marks the beginning of a requisite entry. |
| prereq | This is the requisite type. (Install of this requisite fileset is required before the installation this "grouping" fileset in this package.) |
| acme.custom.net.rte | This is the fileset name (not the package name). |
| 1.0.0.0 | This is the minimal version of acme.custom.net.rte that is required. |
| ; | Marks the end of a requisite entry. |
Building the package
The package build works like the previous examples. The odds of it throwing errors are rather minimal (as long as the template file is properly formatted). But, like the last package with the license requirement, the real test is in deployment. The most common error here will be improperly specifying the required fileset names. mkinstallp will not provide build-time checking of the fileset names. So if the requisite names are wrong (ie using a package name instead of a fileset name or spelling error) then the group package will not be able to be installed based upon failed requisites.
Requisites for a fileset are viewable in the .toc file created from the package it resides in.
The following AIX terminal shows installation / testing of the group package on a target system.
Example 4 - "Non-standard" content
Example definition and challenges
This package breaks down into two filesets; one for the ODM description (creating the ODM (table) object) and another for the ODM data (inserting the "rows" into the object). This was useful for debugging of the package, and to demonstrate multiple filesets in a package, but also because it is likely that most people will be inserting new ODM data and NOT creating new ODM objects. (One example would be deploying the customized SMIT ODM entries as documented in the Extending SMIT For Common Localized Tasks page.)
The mkinstallp template file provides for a significant reduction in complexity of the packaging process. This comes at the expense of a reduced feature set. The ODM capabilities of the AIX package manager are not present in the mkinstallp tool(s).
In order for the ODM files to be used they need to be copied into the .info directory manually where they will be picked up by the packaging process.
Finally, the format of the ODM specific package files is not entirely clear from most of the public documentation from IBM. (This documentation is based upon reverse engineering of existing packages and testing of the results.) Hopefully, this document will serve as a clarification for those looking to do the same.
Preparing content
The content in this package does not follow previous conventions because there is now "active" content in the build root that was previously only used for installable files. In previous examples this content was referenced from the "template_src" (script) directory by the template file, and was copied into place by the mkinstallp process. But, because mkinstallp does not handle the ODM files used in this example, the staging process must do it.* Additional breakdown of the files follows the listing.
The question then becomes "Then why not copy them in this manner all along?". The answer is because mkinstallp will purge all filetypes it knows of from the archive build directory (.info) when it runs. We can do it here because mkinstallp ignores the files with *odm* extensions used here.
The purpose of the files are as follows:
| .info | |
| Files placed into this directory with the proper naming convention will be added to the liblpp.a archive. This archive is the heart of the active content and meta-info for the final package. Information on pulling this archive back out of a .bff file is included in the previous and "Notes and special notices" sections. Content for the ROOT part is placed in a sub directory called .info/root using the same naming conventions for the files. | |
| .info/acme.odmdb.data.rte.* | |
| The "odm" files placed into this directory all follow a convention of fileset_name.extension. These files are for the "data" fileset. (The data fileset has the ODM row entries, while the defs fileset creates the OBM object (table) that the data is inserted into.) The ODM "context" (See the $ODMDIR variable) is set automatically by the fact that this is in the USR context - so the entries will be applied to /usr/lib/objrepos. If the data should go in /etc/objrepos then these files should be placed in .info/root instead of .info. | |
| acme.odmdb.data.rte.odmadd | |
| The .odmadd file is in odmadd (the command) format. To clarify, this is not a script, it is a text file of ODM entries in stanza format that will be passed to the odmadd command. | |
| acme.odmdb.data.rte.odmdel | |
| This is a script of commands to remove the ODM entries created by the .odmadd file. It is run when the package is removed from the system. (Note the contrast with the next file.) | |
| acme.odmdb.data.rte.unodmadd | |
| The .unodmadd file has (in this case) the exact same content as the .odmdel script. The difference is that this script is run prior to an upgrade of the package. If the fileset were to be upgraded the odmadd command would to run again (with similar content) then the ODM would end up with multiple entries having the same values. The point of this script is to remove the appropriate entries before adding the new versions. | |
| odm-def.cre | |
| This is a odmcreate input file. It is placed in this directory as a "package convention" (meaning: I thought it was a good idea). The alternative is to generate the file from the script (using a "here" document. (Note: odmcreate does not accept stdin input.) The chosen method keeps a copy, along with the header files (that will be created when odmcreate runs) on the system for reference. The config script in the "defs" fileset will use this file as input to the odmcreate command to create the ODM objects. The .cre extension is an IBM convention for a file of this type. | |
| acme.odmdb.defs.rte.config | |
| This is a standard package script that will be run on install. It is in the template_src directory because it will be placed by the template file / mkinstallp process. It will use the previous file to create the ODM objects. Because this file is in the USR part of the package, and not the ROOT part, it will act upon the /usr/lib/objrepos location of the ODM and not the ROOT or SHARE parts. | |
| acme.odmdb.defs.rte.unconfig | |
| This is a package removal script that will delete the ODM class using odmdrop. | |
Also note that the template file is placed in the root of the build directory. This is a deviation from the previous convention, but it makes the build command a bit more straightforward. Either way, it works fine - it is a personal preferences thing on what works best - this is just one example of an alternative method.
Create a package manifest
This template file is most noticeably different from the previous examples because it contains two filesets.
The first fileset looks like a normal fileset with content files and scripts. The second fileset looks empty - with only a requisite on the first. The reason that the second fileset looks empty is because the content was copied directly into the .info directory during the stage part of the process.
Building the package
As mentioned earlier (in reference to the template file location) the build command is slightly different. The location of the template file is not really relavant to much other than referencing it to the mkinstallp command. All the files within the template are either fully pathed or pathed to the build root so the location of the file does not matter in this respect. As always, the most important part is the current working directory of mkinstallp when it runs.
Notes and special notices
Debuggery
The following is a list of commands and tips to assist in debugging a package.
Use the restore command to view contents of the package file
restore -Tqf acme.toolkit.1.0.0.0.bff
Restore a single file from the package file
restore -xqf acme.toolkit.1.0.0.0.bff <FILENAME_FROM_PREVIOUS_COMMAND>
View contents of a restored archive file
ar -t liblpp.a
Extract all files in an archive
ar -x liblpp.a
Create a .toc (Table of Contents) file for all packages/filesets in the current directory
inutoc .
List all filesets in the .toc
installp -l -d .
Install a specific fileset (First installs of a fileset are automatically committed.)
installp -a -d . acme.custom.net.rte
Install a fileset that has a license acceptance requirement
installp -aY -d . acme.toolkit.rte
Automatically install all requisite filesets (-g), extend filesystems as necessary (-X), and accept licenses (-Y)
installp -aYXg -d . acme.datacenter.nyc.rte
Remove an installed fileset
installp -u acme.toolkit.rte
Validate files in a fileset (see if any of the installable files have changed)
lslpp -c acme.custom.net.rte
Determine what (installed) fileset owns an (existing) file
lslpp -w /etc/hosts
MetaData
By: William Favorite <wfavorite@tablespace.net>
Version info in base document