RPM 4.6.0 Release Notes

Download information

Summary of changes from RPM 4.4.x

General bugfixes and enhancements

  • RPM now supports 64bit package sizes and larger files in general.
  • Support for file checksums other than MD5 (such as SHA256 and SHA512) has been added.
  • Verification uses chroot() if –root is specified to get user/group names right (RhBug:434150)
  • Verification filters out meaningless timestamp differences of shared files (RhBug:426672, RhBug:444661)
  • File conflicts on non-elf files are always enforced on multilib systems too (RhBug:190209)
  • rpmdb selinux contexts are now restored after db rebuild (RhBug:442149)
  • More informative error messages in many cases (RhBug:427064, RhBug:164743, RhBug:461362, …)
  • Cleanly handled errors instead of dying on assert (RhBug:448939, RhBug:456100)
  • Several exit-code corrections (RhBug:462631, RhBug:446202, …)
  • Always honor epoch nopromote/promote rules (RhBug:466503)
  • Handle attempt to install packages with identical NEVR correctly (RhBug:467822)
  • Allow non-pre scriptlets to fail (RhBug:216221, RhBug:465409, …)
  • Don’t corrupt packages when attempting to sign V3 packages (RhBug:463482)
  • Don’t crash when signing corrupted packages (RhBug:442761)
  • Don’t run %pre- and %posttrans scripts if –noscripts is specified (RhBug:475582)
  • Avoid floating point exception when time doesn’t seem to go forward (RhBug:435309)
  • Queryformat consistency between tag extensions and actual header tags.
  • Permit queries to be interrupted in reasonable time.
  • Improved performance on initial installation due to better hash algorithm.
  • Improved performance on array type queries.
  • Improved memory use by avoiding fragmentation (RhBug:473167)
  • Improved support for ARM and SH architectures
  • Automatic cleaning up of stale rpmdb locks on Berkeley DB >= 4.5.20
  • A number of memory leaks have been fixed.
  • Lots of missing failure checks added all over the codebase.
  • Several new and updated translations.
  • Documentation updates and corrections.
  • API documentation is now shipped in the release tarballs.

Package building

  • %{_topdir} defaults to $(HOME)/rpmbuild/ now, instead of former /usr/src/redhat/.
  • Rpm enforces BuildRoot for all packages and ignores the directive in spec.
  • Group tag in spec files is not considered mandatory anymore, “Unspecified” will be used unless specifically set.
  • Patches are applied with zero fuzz tolerance by default.
  • Macros for helper binaries such as %{__tar} and %{__gzip} are now honored everywhere in rpmbuild.
  • List of directories to be considered documentation is now macro configurable.
  • New macros %{patches} and %{sources} have been added, these can be used to address spec defined patches and sources in regular for loops.
  • rpmbuild automatically adds additional architecture information into package provides to make it possible to correctly express dependencies such as dlopen()’ed plugins and development packages.
  • Dependencies for pkg-config and libtool files are automatically generated by rpmbuild.
  • Various fixes to tar-build mode:
    • The specfile is extracted with sane permissions (RhBug:253648)
    • Attempting build on tarball with no spec included no longer causes ugly and bogus files to be created (RhBug:281391)
  • Stricter spec syntax checking
    • Multiple definitions of same patch- and source numbers are no longer permitted
    • %patch is no longer equal to %patch0
  • rpmbuild -bs and similar imply –nodeps automatically (RhBug:235507)
  • rpmbuild automatically creates the build directory structure if necessary.
  • Several whitespace-in-filenames fixes (RhBug:430428)
  • noarch sub-packages are permitted in otherwise arch-dependant packages (RhBug:227790)
  • –nodirtokens build option works again after several years (RhBug:462391)
  • Various ancient hardcoded limits and defaults have been eliminated / moved to configurable macros.
  • Debuginfo generation works correctly with latest “file” utility (RhBug:468129)
  • Package size is calculated correctly wrt %excludes and duplicate files (RhBug:247374)
  • SELinux contexts are no longer stored in package headers. The contexts vary wildly between different policies and policy-versions, whereas header data is static in nature. The contexts from headers were never used for anything anyway.
  • Experimental support for LZMA payloads has been added but this is unsupported as of 4.6.0 release, official support will be added in an update once a stable release of xz-utils (formerly lzma-utils) is available.

API changes, internal improvements and cleanups

  • The header tag data manipulation API has been revamped entirely.
    • Tag data (from headers and extensions) are now passed around in a container which knows how the memory was allocated, the data type of tag, supports iteration, type safe access methods, formatting to several styles and much more.
    • headerGet() can be used to retrieve extension data too.
    • headerGet() memory allocation behavior can be controlled with flags for deterministic behavior.
    • headerSprintf() has been deprecated in favor of a simpler headerFormat() call.
    • headerSprintf/Format() is several magnitudes of order faster for array types.
    • Support for 64bit integer types has been added.
    • the old headerGetEntry() family of calls is deprecated but still supported in compatibility mode
  • Signature checking has been mostly decoupled from the transaction set
    • A new abstract keyring API has been added and used by the low level signature checking routines.
    • Rpmdb is still used for permanent storage of public keys by default, but for the actual signature checking the in-memory keyring is used.
    • Public keys can now live outside the rpm database (but the feature is somewhat incomplete as of version 4.6.0)
  • A proper callback has been added to rpmlog infrastructure.
  • Scriptlet errors and warnings are now reported via the transaction callback.
  • Rpm problem sets have been made opaque, new methods for iterating over the sets and accessing the data have been added.
  • Successive calls to rpmTagGetName() no longer destroys results of former calls.
  • The internal OpenPGP parser has been made re-entrant.
  • 64bit rpm_loff_t type is used internally and externally everywhere for handling package and file sizes.
  • librpmdb has been merged with librpm. The split was largely artificial and caused more entirely unnecessary problems than it solved.
  • Several new string- and file helper functions have been added to librpmio.
  • The internal network transports and related “RPC” functions have been removed from librpmio. Transparent remote fetch capability is implemented via calling an external, macro-configurable helper application.
  • Use of the non-portable and unsafe alloca() call has been entirely eliminated in the source code.
  • All/most static-sized string-buffers have been eliminated from the source code and replaced with dynamically allocated buffers.
  • Most static variables/buffers have been eliminated from the source code and replaced with dynamically allocated memory / parameter passing etc.
  • A lot of code-duplication has been eliminated through rewriting/refactoring.
  • A lot of code has been rewritten/refactored more readable and simplified.
  • Exported API and ABI has been vastly cleaned up:
    • Lots of internal symbols (functions and global variables) have been eliminated from the public API/ABI, namespace cleaned up.
    • Internal functions are hidden from exported ABI if supported by compiler.
    • Several previously exported internal structures have been made opaque.
    • Several headers have been made private.
    • Private vs public headers are now properly separated, public headers are always accessed in <rpm/header.h> style.
    • Various definitions (enums etc) have been moved to more logical places, rpmlib.h is no longer a catch-all for everything.
  • Splint annotations have been removed from the source code, they were seriously hurting readability.
  • Variable, function return and parameter types have been audited to large extent and made more consistent, both in internal use and exported API. Standard POSIX and C99 types are used where appropriate. Bogus const on pointers to allocated memory have been removed from exported API and mostly internally too.
  • rpmlog() is used for logging everywhere in the source code, rpmError() and rpmMessage() have been deprecated and only available in compatibility mode.
  • All uses of mktemp() have been replaced by mkstemp() in the source code.
  • Rpm now includes pkg-config metadata (.pc) file to ease building and linking against librpm.

Python bindings

  • The in-rpm python bindings have been mostly left alone for maximum compatibility, the development focus is on creating a new (not entirely compatible)) bindings.
  • Python code tracebacking in rpm transaction callback now aborts the entire transaction (RhBug:463447)
  • Return type of header data is now always consistent.
  • rpmfi has new FDigest() method to replace old MD5() method (due to configurable file checksum algorithm)
  • rpmts.addInstall() no longer accepts “available” type packages
  • Header data access by tag names is likely to be faster due to bsearch() use instead of linear lookup
  • rpmds has two new methods: Rpmlib() to retrieve rpmlib provides, and Search() for searching in dependency sets.
  • rpmrc and rpmfts have been removed, both were incomplete, unused (and unusable)

Build process

  • RPM no longer ships with 3rd party libraries in the distribution tarballs, internal copies of Berkeley DB, Lua, popt and libmagic have been removed. Building with an internal Berkeley DB is however still supported, see INSTALL for instructions.
  • The build process has been cleaned up and auto-tool usage modernized.
  • RPM now uses NSS for low-level encryption, beecrypt is no longer supported.
  • A C99-compliant compiler is now required to build RPM.
  • A new automated testsuite (beginnings of) has been added.
  • Lua >= 5.1 is now required for internal Lua-script support.
  • ./configure now honors –libdir instead of trying to guess lib vs lib64

Compatibility notes

Large package support

  • Packages can now theoretically be up to 64bit sizes, and individual files within packages are limited to 4GB each due to cpio format limitation whereas they were previously limited to ~2GB. Large packages (over ~2GB in size) are incompatible and unreadable with previous versions of RPM due to requiring 64bit integer type support in headers, “normal” sized packages are fully compatible with older versions however.
  • Limitations on accepted header size can cause limit the practical package size when the number of files in a package is extremely high.

Strong file checksums

  • By default, MD5 is used for file checksums for backwards compatibility. Packages built with stronger checksums are incompatible with previous versions of RPM. Enabling stronger checksums can be done through %_source_filedigest_algorithm and %_binary_filedigest_algorithm macros at build time.
  • When file checksum algorithm changes in an upgrade, normal behavior with configuration files cannot be guaranteed. When correct handling cannot be done, a backup of the existing configuration file is taken to avoid data loss, but this can lead to seemingly unnecessary .rpmsave/.rpmnew files.
  • Files shared among several packages can cause file conflicts on cross-hash upgrades. Packages sharing files should be upgraded at the same time in a single transaction to avoid this.
  • Due to reverting an erronously introduced tag change, packages with strong checksums built with 4.6.0 release candidates are incompatible with the 4.6.0 final release and vice versa. Unless you have specifically enabled strong checksums in your build environment, this does not affect you in any way.

Package building and build environment setup

  • %patch macro now applies patches with zero fuzz tolerance by default, which can cause previously accepted patches to fail. It’s strongly recommend fixing the patches to apply cleanly, but if non-feasible for whatever reason this can be overridden with %_default_patch_fuzz macro (set to level 2 for former behavior), either in spec or build environment.
  • Rpm now enforces using a build root for all packages. The exact location and naming of package buildroots is an internal implementation detail that should have never been exposed to spec files, but to permit sharing of spec files with older versions of rpm, BuildRoot in spec files is silently ignored. By default rpmbuild now places the buildroot inside %{_topdir} but this can be changed through macro configuration.
  • %{_topdir} defaults to users home directory ($HOME/rpmbuild/) instead of a potentially read-only location in /usr/src/. This and automatically creating the build tree if it doesn’t exist was done to have a working default configuration and encourage building packages as a regular user instead of root. The default can of course still be changed by overriding the %{_topdir} macro.
  • Group in spec files is no longer considered mandatory, but if spec files are to be shared with older rpm versions, it should still be added for compatibility.
  • Rpm now permits sub-packages to be declared “noarch”, but such a spec is incompatible with older rpm versions. The resulting noarch binary packages are however fully compatible with older rpm versions (unless other new, incompatible features have been enabled/used).
  • As an early step towards phasing out rpmrc, rpm no longer processes “macrofiles:” directive from rpmrc, a built-in default list of macro file locations is used instead. The macrofiles directive is silently ignored to permit sharing an existing rpmrc with older versions of rpm. The built-in list can be overridden with –macrofiles= command line switch.

Removed features

  • Support for repackage and rollback have been removed as they’re seen too unreliable to be generally useful.
  • Internal network transports have been removed. The remote fetch capability is preserved by using an external, configurable helper (curl by default). For network related configuration such as proxy settings, environment variables like http_proxy are honored by curl, rever to helper documentation for futher information.
  • Solve database support has been removed as it has proven impractical, severely limited in functionality and hasn’t been used by any major distro in years. Plans are to replace the suggestion mechanism by a more flexible plugin architecture at some point.
  • Support for automatic package relocation for IA-32 emulation layer on IA-64 has been removed.

RPM API

  • This version introduces vast changes in the RPM API and ABI to overcome fundamental design flaws and limitations, and is incompatible with older versions in several areas. However many librpm-using programs can be built against RPM 4.6.0 with minimal or no chances by enabling compatibility mode with -D_RPM_4_4_COMPAT when building against RPM 4.6.0.
  • The python bindings are mostly compatible with older versions, python clients such as yum and smartpm can be expected to work with no modifications.

RPM database

  • The fundamental database format is unchanged from 4.4.x, but is subject to changes in the underlying Berkeley DB version.
  • Filemd5 index has been renamed to Filedigests, if necessary rpm –rebuilddb can be used to regenerate the indexes.
  • Some of the new features will render the headers in the database incompatible with older versions, such as large packages (due to 64bit integer types in headers)