| Main Page -> Papers -> Package Management for System Administration - rpm |
Note: This is the Red Hat / RPM specific content of a larger paper. If you came here directly, the recommendation is to start at the beginning.
Red Hat / RPM introduction.
The package format
RPM is a very popular format in the Linux world, with adoption outside of the Red Hat based distributions. (Much of this may apply beyond Red Hat, but the paper will be written in the context of just that distribution.)
The RPM package file is a wrapper around a cpio archive. Unlike the AIX / installp format, there is really only one method of building the package, and that is via the RPM tools.*
[* Originally this was the rpm command itself. This paper will use the newer rpmbuild command.]
Finally, the RPM build process is designed to integrate with the open-source model. This means that it is capable of working on source trees, and actually building the software during the package creation process. (This will be largely ignored in this document as the intent is to distribute configuration changes and minimal actual software.)
An opening point
The problem with writing a paper on RPM is making an impact when there is a significant amount of pre-existing literature on the subject.* This paper will attempt to focus on the administrators tasks and hopefully provide a quick introduction to the format that may be more difficult to obtain elsewhere.
[* Unlike much of the open source documentation, the RPM content tends to be much better. The problem is that, like other open source projects, the code has changed and much of the older documentation still survives.]
Example 1 - Pushing system config files
Example definition and challenges
When installing over other packages, RPM will recognize both packages as the file's owner. In this example, one file (/etc/nsswitch.conf) belongs to a package and the other (/etc/resolv.conf) does not. The dual ownership of one file is a bit problematic, but not so much that it will break the system. The view is that the value of validating the package will be more valuable with the files in the manifest.
The other RPM specific issue (at least for the first time user) will be the discarding of a large amount of the build processes capabilities. As mentioned earlier, the package build process offers a stage and build component that is related to converting source code into packages - (a well served need for the community). The examples in this paper will not be leveraging this part of the tool features, and will even take efforts to avoid those features.
Preparing content
RPM uses a very specific directory layout for the build process. There are a number of RPM macros that are designed to act within this structure. This example follows it closely but makes two departures from the "standard" Red Hat method.
The first difference is that a local build directory is used. Red Hat will default to /usr/src/redhat. This project wishes to maintain a staged project directory and has set the %_topdir macro in the users's .rpmmacros file*. All file entries in the RPM specification file will utilize relative pathing, macros, or environmental variables, so it is not necessary to dwell on the actual location other than to say that it can be modified from the default.
[* This was done in the stage script that is probably unique to this project (as discussed elsewhere). One solution may be to encapsulate this in a script that sets this value, then builds against the pre-staged content.]
[* This file also contains a "%_signature gpg" and "%_gpg_name ..." line required for signing the package. Signing of packages, while not terribly complex, was not included in this paper.]
The second, and more important, difference is that the SOURCES directory and the RPM stages to utilize it will not be used. Because the files in this package are not "built", but simply modified, they will be staged directly into the BUILD directory as though the source code specific %prep and %build steps (seen in the next section) had been accomplished.
Unlike the installp example, the permissions and ownership of files are not important. As long as the permissions / ownership are properly specified in the .spec file, it is not necessary (or even recommended) to run as a privileged user. (All of these examples were built as a "regular" user.)
The BUILD directory follows a specific layout here that mirrors how the files will be laid out in the target system. This is a local convention that makes the files easier to find / manage, and is not a requirement. The file locations in the BUILD directory are only relevant to the final staging depending upon how the files are copied to the directory that the package will be built from. In this case, the files are simply cp-d into the buildroot directory.*
[* The best practices recommended method is to stage from the BUILD to the buildroot directory (as is done in this example). See example three for an alternate method of building directly from the BUILD directory.]
Create a package manifest
The manifest file (RPM uses a .spec file) for this package is simplistic, yet includes active (scripting) content that will be unique to this project. Commentary on specific items follows the file.
Some notes on the file contents:
| • | Even though RPM does not have a highly structured install naming convention like other package managers, the choice here was to use a very specific name that included "acme", "custom", and "net" so that this package is: clearly from the ACME packages, is a customization, and customizes the network configuration. This example is based on a simplistic "ACME" network of only a single configuration / location. In a more realistic example the package would be network or location specific and the naming convention would reflect that (such as "acme-custom-nyc-net"). |
| • | RPM is not rigid on the format of the version, and allows for multiple dot-version levels with no (reasonable) limitations on the numbers. The choice here was to use two levels of versions, and start with "1.0" |
| • | Because of the simplistic nature of the package, it is not expected that additional releases at any version will be required. (The release is a concept that there may be multiple editions of the package at a single version of the software. While this is useful for someone packaging software that they did not author, it is not an issue for the configuration files in this package.) The "Release" will probably remain at 1 for the entire life of this project. |
| • | A BuildArch of noarch was chosen to demonstrate the ability of the package format. ACME deploys Linux to only a single architecture, so this is not necessary. |
| • | The Buildroot setting specifies where the files will be "installed" to so that the package can be built there. The number of times the content is copied during the RPM build process is one of the more complex concepts for the new user - this specifies the temporary / final destination that the RPM build process will stage to prior to building the final package. |
| • | The %prep step / section is not used. This stage is used to prepare the raw source prior to building it. The examples here are staged directly to the BUILD directory, and are not built from the SOURCES directory. |
| • | The %build step / section is also not used. If the files in this package were binaries, and were built from source, this could be used to actually compile them. In this case, the files are staged to the BUILD directory in their final state and do not require processing in the build phase. |
| • | The %install step / section is where content that has been staged in the BUILD directory (here referenced by $RPM_BUILD_DIR) is installed to the Buildroot directory (here referenced by $RPM_BUILD_ROOT). In this step the install command is sometimes used as though it was applied to the target system. The problem with the install method is that root (or equally privileged user access) is required to run the packaging process. The alternative is to use some form of a standard copy, then specify permissions and ownership in the %files section. When run in this manner, the privileged user access is not required. |
| • | The %install stage should begin with a clean and rebuild of the Buildroot directory ($RPM_BUILD_ROOT) in the event that a previous build did not finish and clean up appropriately. |
| • | The %clean section tells how clean up of the Buildroot directory is done after the package creation. The simple answer is to simply delete it through the $RPM_BUILD_ROOT reference. |
| • | The %files section is a listing of files that the package build process should capture from the Buildroot directory. This list of files should match those that were copied / installed in the %install step / section. Directories are not mentioned here unless they are specific to the package. In this case we are using /etc and will not be included as that is a very basic directory that is known to exist and is not specific to this package. |
| • | There are numerous macros that can be applied to files listed in the %files section. Because of the simplistic nature of this package (two files with the same permissions) a single default set of attributes will be applied to all files in the package using the %defattr(644,root,root) macro. One of the macros (that will be used in the next section) is to declare that these files are "config" files. This will establish a set of rules about how these files are handled once they are modified that is not desirable for the requirements of this package. For this reason, the files in this package will be treated much like binary files in that modification from the original packaged version is not acceptable. |
| • | There are four different timings for active (script) content: %pre (before install), %post (after install), %preun (before removal), and %postun (after removal). This install requires that we save existing files before install (using %pre) and restore them after package removal (using %postun). The script content in this section will handle that functionality. |
| • | All defined macros are available using the rpm --showrc and individual macros are viewable using rpm -E <macro-name>. |
Build the package
The RPM build process will run the %prep, %build, %install, and %clean steps from the .spec file. Because the permissions are specified in the %files section / phase and not the %install phase, the build can run as a normal* user.
[* Recall from the earlier section that files were not staged to /usr/src/redhat, but were placed in a local project directory that a normal user can create / access.]
[* --sign would be used as a paramater to rpmbuild to sign the package using GPG settings mentioned in the previous "Preparing content" section. Signing of packages is considered out of scope for this paper, but relevant in a deployment environment.]
When complete, the package will be placed in the RPMS directory seen in the "Preparing content" section.
The -bb option tells rpmbuild to only build the binary* package (and not the "source" package**).
[* Here "binary" refers to the source vs. binary classification of software. The configuration files, while simple text, will be considered a "binary" distribution.]
[** This is yet another example of how the design of RPM serves the open source community where distribution of source with a binary is frequently a license requirement. Because this is an internal distribution, is not GPL licensed, and is not derived from pre-compiled source, there will be no "source" rpm.]
As mentioned earlier, this package is built from a project directory set with a macro in the user's ~/.rpmmacros file. This can be the default /usr/src/redhat or any directory that is desired - provided the %_topdir macro is properly set. The point intended here (with the pwd command) is that this is run from the base of that directory - and for that matter can be run from anywhere (once again, as long as the %_topdir macro is properly set and the .spec file is properly referenced). The RPM build process will reference everything in regards to these two items.
Example 2 - Deploying the ACME ToolKit
Example definition and challenges
The original requirement for this example was that ACME wanted to insure a degree of non-liability for this code by requiring that a license be accepted prior to install. The idea for this "requirement" came from the strong feature capability in this regards in the AIX package manager also used in this white paper. Because RPM does not not have the same capability as installp, ACME could include some wrapper or requirement in the pre-install scripts but at a loss of un-attended installs.* Therefore a license "script" will not be included as the primary goal is to make this easily distributable rather than "commercially" packaged.
[* One compromise may be to put an "If you install you agree to..." statement in the %description of the RPM. While this is not as clear, it may provide sufficient legal notification of intent without breaking automation efforts.]
Preparing content
Like the last package, the ToolKit does not come from a tar-ball of source. It comes from a group of administrators using multiple different languages and packaging structure*. The solution here is to stage in the same manner with the intent of skipping the %prep and %build steps in the RPM packaging process.
[* In all honesty, there are plenty of really good (as in, much better) papers on packaging tar-ball structured content. The idea here is to cover more simplistic content that most admin types are inclined to create and distribute.]
One atypical part of the build is that the config files are placed into a sub-directory of /etc. While it may be a good idea for a large number of ACME-specific config files, the purpose here is to demonstrate that when directories are part of the install they must be accounted for in the package manifest.
Create a package manifest
The .spec file for the ToolKit is a very straightforward and simplistic file. It basically consists of a header, %install, %files, and %clean sections.
The content is copied into the Buildroot where it is captured according to the entries in the %files section. When the package build is done, the Buildroot ($RPM_BUILD_DIR) is deleted by the instructions in the %clean section.
Some notes on the file contents:
| • | The "URL" line is optional - and could be used to reference the ACME Unix department's web page or some relevant page of that nature. It is included here with a reference to this paper (only as an example of a URL). |
| • | Once again the %prep and %build sections are empty with the %install, %clean, and %files sections performing all the work of the package build. |
| • | %install performs the seemingly unnecessary re-staging of the content to the Buildroot. (Reasons for this step exist that are not quite so clear in these simplistic package builds. Some commentary on this subject may be found in example three.) |
| • | %clean simply undoes any work completed by the %install stage. |
| • | The %files section is used to specify what files should be copied into the package, and what permissions are required. (A breakdown of this section follows.) |
| • | The %defattr macro sets the default attributes for each file in the %files section. The alternative is to set individual permissions as seen in the /etc/acme/* line. (The use of %attr in this line sets multiple file attributes because the wildcard (*) references more than one file. The syntax is the same when used for one file.) |
| • | It is generally not recommended to explicitly include directories in the file list. In this case, the /etc/acme directory is part of, and specific to, this package. Inclusion here means that it will be owned by the package and removed during package removal. |
| • | The %config macro tells the package manager that these files are configuration files and are subject to change. With this option set, the package will continue to validate (using rpm -V) even if these files have been modified. |
| • | Setting permission and ownership using these macros in the %files section means that the files in the buildroot do not require special ownership or permissions. For this reason, the package build can be run by any user. |
Build the package
The package is built like the last by calling the rpmbuild command and passing the appropriate parameters. The parameters are -bb and the .spec file.
Once the package has been built, it can be found in the RPMS directory. The following is a post-build listing of the same directory structure seen in the pre-build section earlier.
Validating that the package works correctly is simpler than one with active / scripting content. The most important thing to check for is that files are installed with the proper permissions and ownership.
The following sequence of commands shows the method to extract a single file from a RPM package. The package is cataloged using rpm2cpio and cpio. (rpm -qlp is actually more straightforward, but the point here is to demonstrate rpm2cpio.) The same commands are then used to extract a single file by name.
Example 3 - Collections of packages
Example definition and challenges
The phrase "group" is used frequently in this example. It is not meant as an RPM package group*, but as a grouping function of all packages related to this parent package personality. The term group means that this package will require the install of a related group of packages for it to install.
[* Groups of packages (in this context) are not covered in this paper. Groups can be used as a reference to the yum command (as one example) to install a collection of packages. The package dependency method is a bit more flexible as it allows grouping with overlap of the sub-packages with a simple recursive flow of dependencies that could be much more complex if specified in a group (file) format. All packages in this paper use a common group (even though no group file or repository was specified). This could be used to categorize these packages using many of the high-level tools for package management.]
Preparing content
The staging of the package contents will be the most simplified yet. The following listing shows the package "flag file" and the .spec file.
Create a package manifest
The .spec file represents two key departures from previous versions. First it introduces explicit* requirements with the "Requires" keyword. Second, it has an empty %install step.
[* Modern versions of RPM will automatically scan input to the build for dependent libraries / interpreters and add the dependencies to the requirements for the package. These can be viewed using rpm -qp --requires <package_name>.rpm.]
The "Requires" keyword is actually very simple. It is a comma separated list of packages and version numbers. In this example, there are three package requirements that must be completed for this install to finish. Two of the requirements are previous examples in this paper, and another that has not been previously discussed. The concept is that for this group / personality to exist on the system, all of these packages, at these versions or greater must exist on the system.
The syntax of the requirements should be fairly straightforward. it is package name, a relational operator, and a version.
One alternative to the "Requires" keyword is the "PreReq" keyword. The PreReq option requires that listed packages be installed before the installation of this package begins. The difference here is that the Requires dependencies can be installed in the same operation - and may be somewhat out of order. If the package were to run binaries that it had installed in the post-install script, then the dependent packages (such as libraries the binary depends on) need to be installed for the package to complete successfully.*
[* This is most relevant in provisioning / initial deployment when many packages are installed at once. A package that calls useradd may behave normally when building the command and when run on an installed system, but fail miserably in a provisioning scenario because the package containing useradd may not have been installed when the script in this package is run. A PreReq on the package containing useradd would insure that it exists on the system when the package is installed (and the command is called).]
The next deviation is the modification of the Buildroot option and %install section. Instead of performing an "install" of a single file to a temporary build root just to package it, the packaging happens directly from the BUILD directory. This is a packaging "hack" that may not be approved by the RPM community, but it saves the complexity of yet another copy of the data by just grabbing it from where it was staged in the BUILD directory.
One potential problem with creating the installable file content from the BUILD directory instead of the Buildroot is that modern implementations of the RPM build tools will fail if all the files in the source directory are not included in the file list. Explicitly staging content to the Buildroot would (theoretically) resolve this problem - but with the simplistic content of this example it is not much of an issue.
Build the package
Building this package runs much like any other build. The debugging and testing becomes more relevant in the target systems and repositories. The RPM build process will not be able to properly validate the requirements at build time. The first post build test is to check to see if the dependencies look correct. The previously mentioned query (rpm -qp --requires <package_name>.rpm) is one method of testing, another is to attempt to install the group package when none of the dependent packages have been installed. The following is an example of attempting to install this package on a system without the dependent packages:
This document will not attempt to explain the yum tools / environment, but once the packages are placed into an ACME repository they can be accessed and installed through the single group package as in this example:
Example 4 - "Non-standard" content
Example definition and challenges
This package will have no (filelist) content, but will modify the runlevels of two services on the system. When it installs it will call the chkconfig command to alter service startup. If the package is removed, the services should be reset to the previous values.
This particular example does not have a "residual" problem of multiple applications of the package building up content / changes on the system. Even though it is not seen as a problem with this example, an uninstall script has been included to return all values to the default / original settings - as a general / best practice.
This package will leverage a feature in the RPM format that allows for custom package validation routines.
The KickStart syntax allows for both service modification and user creation during install. Additionally (as mentioned later in this example) package-based modification / install of these items carries some unforeseen problems when run during install-time. The expected trade off is the ability to group these changes in a top-down manner for a system "personality" and the ability to write validation code that can be distributed in an object-oriented fashion and run anytime post-install to validate the contents of the package.
Preparing content
There is basically no "traditional" / file content for this package. The package is entirely about active / scripting content that will be encapsulated in the .spec file.
Create a package manifest
The following manifest contains all the content of the package in the form of three scripts and the package metadata. The "%pre" and "%preun" scripts should be fairly straightforward.* The choice of pre or post-install is largely irrelevant as no installable files are ever applied to the system. The point is simply to explicitly turn off some services and to restore them to original values when the package is removed.
[* Also note that the scripting content is rather simplistic. This is an attempt to simplify for the example. More appropriate code would check input (ie: Does the service actually exist on the system?) and output (ie: stdout / stderr, return value) of any commands that it runs.]
The third script (in the "%verifyscript" section) is unique to this kind of package that has no regular file content. It is used to verify that the package is still "properly" installed. When the package is validated (after install using rpm -V) and one of the commands* in this script returns a non-zero value the validation will fail. Because this package is all script and has no file content to validate, the validation routines** must be provided as well.
[* Technically, the function "isoff()" will be returning the 0 (good) or 1 (fail) values.]
[** The validating code is also (intentionally) very simplistic. Here it just throws an error value, with no message. It may be more appropriate to print the specific error to stderr (echo "ERROR: Item X failed." >&2 ;) and continue to run the code, capturing error values until the end.]
Note: This example does not have dependencies on the packages that it is modifying. It may install without issue in a run-time environment, but may behave very differently in an install-time environment. For this package to work properly in a KickStart environment, every service listed should be listed as an "PreReq" requisite (as mentioned in example three). This will insure that the service to be modified will actually be installed before this package attempts to modify it. One unintended side effect of adding packages as prerequisites is that changes to the KickStart process (such as removing a service) may not have an effect as this requirement will simply pull it back in.*
[* Other package managers have a concept of a conditional requirement. Unfortunately RPM does not support this concept. The safest bet to avoid bringing in unintended packages in this manner is to test thoroughly and/or install this package post-install (of the OS) where a requirement will not be required and missing files can be handled in the script without forcing their install.]
Build the package
The building of this package should always complete successfully. Between the simplicity of most sections of the .spec file and the fact that the package build process will not validate the included active content, the chances for failure are really down to syntactical errors.
Testing and validation of the package must happen at install-time. If the package will be used in the KickStart process (at OS install-time) then some additional testing must take place as the OS install-time environment may be very different than the post install environment. The primary concern is that this package be installed after any package it modifies or tools it uses to modify them.* A key debugging aid is to look for errors that are written to the "install.log" that is dropped in root's home directory.
[* As mentioned earlier, this can happen through an explicit PreReq clause in the .spec file or by only installing when the required filesets are known to exist (such as after the initial OS install).]
The following query option can be used to view the scripting content of the package that was just built.
The following Linux terminal session shows the installation of the package and various validations of the package. The first validation happens after the install (that passes). The second validation happens after one of the services has been turned back on (that fails).
Notes and special notices
Debuggery
The following is a list of commands and tips to assist in debugging a package.
Install a package
rpm -Uvh my-package.rpm
Remove a package
rpm -e my-package
Validate an installed package
rpm -V my-package
Expand a macro
rpm -E %{_builddir}
Print all macros
rpm --showrc
Extract the cpio archive from an rpm file
rpm2cpio logparse-1.14.0-1.i386.rpm > logparse.cpio
List the files in the rpm package
rpm -qlp logparse-1.14.0-1.i386.rpm
Extract the files from an rpm file
rpm2cpio logparse-1.14.0-1.i386.rpm | cpio -id
MetaData
By: William Favorite <wfavorite@tablespace.net>
Version info in base document