RPM 4.14.0 Release Notes

Download information

  • rpm-4.14.0.tar.bz2
  • SHA256SUM: 06a0ad54600d3c42e42e02701697a8857dc4b639f6476edefffa714d9f496314

Summary of changes from RPM 4.13.x

Security

  • Restrict following symlinks to directories by ownership (CVE:2017-7500)
  • Don’t follow symlinks on file creation (CVE:2017-7501)

General bugfixes and enhancements

  • Numerous documentation enhancements
  • Add support for RISCV-64
  • Optional coloring of error messages and warnings (%_color_output yes/no/auto)

Command line

  • Add alias -P for –provides for symmetry with -R/–requires
  • Add –changes option to show changelog with full timestamps
  • Add –builtrpms query option to rpmspec (RhBug:961833)
  • Add –noconfig query/verify filter option (RhBug:1373022)
  • Add –trace option to enable macro %trace from the command line
  • Add –nocaps option to allow disabling file capabilities (RhBug:648654)
  • Add –target as a global option
  • Fix –noplugins to work with -V/–verify
  • Fix –noghost query/verify filter option (RhBug:1306438, RhBug:1395818))
  • Fix usage message getting printed in verbose mode
  • Fix source packages getting installed with –freshen (RhBug:1402856)
  • Fix first –define getting doubly defined
  • Fix inconsistent macro name transformations between –predefine and –define
  • Return error codes on invalid –define/–predefine/–eval
  • Make use of -p option optional (trac:28, RhBug:172821)
  • Simplify/clarify signature checking output in non-verbose mode
  • Don’t accept dash (‘-‘) in manifest files
  • Don’t write archive data to a terminal from rpm2cpio and rpm2archive
  • Use bash-style 128 + signum exit code when exiting on signals
  • Fail verification of directory symlinks on owner mismatch (related to CVE:2017-7500)

Transactions

  • Add a configurable mode to minimize writes (at expense of more reads) to conserve SSD’s (RhBug:783480)
  • Add a configurable mode to flush IO during transaction (useful on some server workloads)
  • Fix file triggers failing to match on some packages (MgBug:18797)
  • Fix support for old packages with missing digest for empty files (RhBug:1352222)
  • Backup and overwrite files when they stop being %config (RhBug:1263859)
  • Improve rename() failure message and handling (RhBug:1177479)
  • Do not permit excluding %license files via install policies (RhBug:1333509)
  • Normalize dubiously large fs blocksizes to 4096 (SuseBug:894610, SuseBug:829717, SuseBug:965322, RhBug:847960, RhBug:1397569, …)
  • Order by weak and reverse dependencies also (RhBug:1330668)
  • Block signals during write-transactions (RhBug:???, …)
  • Avoid redundant writes for unchanged %config files

Queries

  • Fix segfault on incorrect data type to shell escape formatting
  • Fix expand query format to work with all string types
  • Fix date formatting buffer too small for some locales (RhBug:1425231)
  • Fix queryformat with mismatching array sizes to behave as documented, use of ‘=’ is required to repeat the first element (eg %{=NAME}).

Package building

Spec

  • Add support for with/without rich dependencies (PR:299)
  • Add support for Ruby Gems in %setup
  • Add support for -o option in %patch
  • Add “gendiff” backend for %autosetup (PR:109 and PR:110)
  • Add back support for %_noPayloadPrefix (trac:900)
  • Add separate macros for main package definitions, fixing debuginfo etc
  • Add support for section end markers (%end) (RhBug:564613)
  • Add support for more QT translations in %find_lang (RhBug:729336)
  • Add support for KF5 in %find_lang –with-kde
  • Add support for date-style full timestamps in %changelog (trac:903)
  • Add support for running rpmbuild with debug verbosity
  • Add support for whitespaces in %include filename (GH:125)
  • Add some support for reproducible builds via SOURCE_DATE_EPOCH standard (https://reproducible-builds.org/specs/source-date-epoch):
    • Support setting build time from SOURCE_DATE_EPOCH
    • Support setting file mtime from SOURCE_DATE_EPOCH
    • Support setting buildhost via %_buildhost macro (RhBug:1309367)
    • Option to set SOURCE_DATE_EPOCH from %changelog
    • Do not store digests of %ghost files
    • Do not store directory sizes in packages
  • Fix %autosetup not applying patches if BuildArch is present (RhBug:1084309)
  • Fix %autosetup quilt backend to honor quiet/verbose
  • Fix %setup -q to be position independent
  • Fix %setup logic in presence of ‘-[cbaT]’ options
  • Fix classification of ELF ELF binaries with both setuid/setgid set
  • Fix debuginfo packages where sub-packages have different version/release (RhBug:1051407)
  • Leave build scriptlets on disk in debug mode (RhBug:1442015)
  • Main package tags are stored separately in uppercase macros (eg %{VERSION})
  • Build scriptlets exit with status of last command (PR:249)
  • Improve various error messages (RhBug:1374138, …)
  • Log a warning on possible unexpanded macros in NVR
  • Require whitespace around trigger – separators (RhBug:1352828)
  • Require triggers to have a condition
  • Require file trigger conditions to be paths
  • Preserve timestamps by default in %make_install (RhBug:959872)
  • Detect and error out on multiple BuildArch lines per package (RhBug:1385257)
  • Copy all all possible special %doc/%license files before checking errors
  • Warn on possible unexpanded macros in NVR (RhBug:547997)
  • Stricter validation of rich dependency syntax

Macros

  • Add macro %{shrink:…} to trim out excess white-space
  • Add macro %{quote:…} for quoting parametric macro arguments (empty arguments, arguments with whitespace)
  • Add macro %{_rpmmacrodir} which points to rpm’s macros.d directory (RhBug:846679)
  • Fix macro scoping level on re-entry from Lua rpm.expand()
  • Fix invalid memory access in %trace mode
  • Fix ancient off-by-one in macro scoping level
  • Change macro scoping to be global / local to parametric macros (RhBug:552944, RhBug:551971, …)
  • Enforce scope visibility for automatic macros
  • Expand parametric macro arguments before processing (GH:127, RhBug:1397209)
  • Raise actual errors on illegal macro names and unknown options
  • Improve error messages
  • Don’t ellipsize %trace output
  • %{error:…} now actually raises an error
  • %{echo:…} now outputs to stdout
  • %{echo:…} and %{warn:…} messages include a trailing newline
  • Maximum macro recursion/nesting level has been increased to 64

Buildroot policies

  • Fix brp-python-hardlink causing build failure if .pyo/.pyc files are missing
  • Fix brp-python-bytecompile behavior when /usr/bin/python missing (RhBug:1411588)
  • Handle Python 3.5’s *.opt-[12].pyc in brp-python-hardlink

Generators

  • Add Python distutils based dependency generator
  • Add AppStream metainfo dependency generator
  • Add support for generating rich dependencies from generators
  • Add support for new package declarations in Perl 5.12+
  • Log a deprecation warning on use of external dependency generator
  • Filter out provides and requires from %_docdir (RhBug:964126)
  • Fixes to perl generator (RhBug:1268021, RhBug:1275551)

Debuginfo

  • Add support for debuginfo subpackages
  • Add support for debugsource subpackages
  • Add support for build-id links in main package(s) (TODO: better description)
  • Add support for unique build-ids across build versions/releases
  • Add support for unique debug file names
  • Add support for minimal debuginfo (RhBug:834073, RhBug:1382394, RhBug:1052415)
  • Add support for DWZ debuginfo compression (RhBug:833311, RhBug:971119)
  • Add support for parallel processing to find-debuginfo.sh
  • Add configuration option for controlling GDB index addition
  • Add provides for build-ids
  • Add support for string/line table rewriting (RhBug:304121, RhBug:757089, …)
  • Add support for unique debug source directories
  • Add –keep-section and –remove-section options to find-debuginfo (RhBug:1465997)
  • Fix non-standard inherented modes of directories in debuginfo (RhBug:641022)
  • Fix numerous misc bugs
  • Debuginfo-packages are now parallel-installable (if configured)
  • TODO: documentation for macros etc

Package format

  • Add a digest on the compressed payload content, verify on –checksig (GH:163)
  • Add a SHA256 digest on the header, verify similar to SHA1
  • Add RPMTAG_SOURCEPACKAGE tag to source package headers
  • Extend maximum header size to 256MB (RhBug:913099, RhBug:1434656, …)
  • File digests now default to SHA256
  • Optional support for zstd compression in package payload and sources

Plugins

  • Add prioreset plugin to handle scriptlet priority reset on legacy systems (functionality moved into plugin)
  • Fix memleak in systemd_inhibit
  • Log errors on plugin hook failures (RhBug:1262424)
  • Better diagnostics for systemd inhibition problems (RhBug:1372925)

Python bindings

  • Add RPMTRANS_FLAG_DEPLOOPS transaction flag symbol
  • Add RPMSENSE_MISSINGOK dependency flag symbol
  • Add RPMVERIFY_* file verification symbols
  • Add .verify() method to rpm.files objects
  • Add .parsed getter for spec to retrieve parsed spec
  • Add blockSignals() for blocking and unblocking signals on librpm level
  • Fix signing extension build via setup.py
  • Fix file descriptor leak on hdrFromFdno() fail
  • Fix header memory leak (RhBug:1358467)
  • Fix spec object refcounting (GH:114)
  • Raise an exception on macro expansion errors
  • Documentation updates
  • Support for legacyHeader argument to header .unload() method removed
  • Dependency set .Notify() method removed

Lua interface

  • Add rpm.undefine() function
  • Force FD_CLOEXEC on open files before posix.exec() (RhBug:919801)
  • Raise Lua errors on rpm.define() and rpm.expand() errors

Signatures

  • Make gpg command configurable
  • Numerous fixes (memleaks, sanity, zero-length files, default hash algo, config files, correct hook…) to file signing (TODO: better description)
    • Fix file signing being advertised in –help even if not built in
    • Fix various crashes and memleaks
    • Fix typos in error messages etc

Removed features

  • %GNUconfigure macro

API changes

Added APIs

  • rpmfiVerify() and rpmfilesVerify()
  • rpmsqPoll(), rpmsqActivate(), rpmsqSetAction(), rpmsqBlock()
  • rpmDigestBundleAddID()
  • RPMTRANS_FLAG_NOCAPS flag to disable file capabilities
  • RPMVSF_NOPAYLOAD flag to disable payload digest verification
  • pgpPubkeyKeyID()
  • rpmPushMacro() and rpmPopMacro() (to replace addMacro() and delMacro())

Changed APIs

  • rpmtdFree() now implies rpmtdFreeData()
  • Header is now always available in RPMCALLBACK_* events (RhBug:1485389, …)
  • headerCopy() no longer reloads to legacy image
  • pgpPubkeyFingerprint() now returns actual fingerprint instead of key id
  • rpmPkgDelSign() now takes signing args struct as a second argument

Removed APIs

  • Internal helpers: headerVerifyInfo(), headerSort() and headerUnsort(), rpmfiDecideFate(), rpmfiConfigConflict(), rpmsqAction(), rpmsqEnable(), rpmdbCheckSignals(), rpmdsNotify(), pgpExtractPubkeyFingerprint()
  • Deprecated: headerNVR(), headerNEVRA(), headerGetNEVR(), headerGetNEVRA(), headerGetEVR(), headerGetColor(), rpmfiMD5(), expandMacros(), addMacro(), delMacro()
  • rpm 4.4.x API compatiblity (_RPM_4_4_COMPAT define): headerRemoveEntry(), headerFreeData(), headerFreeTag(), headerGetEntry(), headerGetEntryMinMemory(), headerGetRawEntry(), headerNextIterator(), headerModifyEntry(), headerAddOrAppendEntry(), headerAppendEntry(), headerAddEntry()
  • Symbols RPMVSF_NOMD5HEADER and RPMVSF_NOSHA1 for unimplemented and unplanned features

Internal improvements and cleanups

  • Add support for OpenSSL crypto backend
  • Add support for multithreaded XZ compression
  • Add support for LMDB database backend, EXPERIMENTAL for now
  • Fix off-by-one stack write in rpmGlob() (RhBug:1371914)
  • Fix various file trigger scriptlet diagnostics showing “unknown”
  • Fix Ftell() on > 2GB files on 32bit architectures (RhBug:1492587)
  • Fix pgpDigParamsCmp() to distinguish between signature and pubkey and different userid
  • Fix LFS in sepdebugcrcfix and systemd_inhibit plugin
  • Limit fallback to private BDB environment to read-only operations
  • Fallback to private BDB environment on DB_VERSION_MISMATCH (RhBug:1465809, RhBug:1397087)
  • Make static buffers thread local where beneficial
  • Numerous fixes to the new experimental “ndb” database format
  • Numerous fixes to harden against malformed headers
    • Unify all header/package reading paths to a new low-level API for operating with pre-imported header blobs
    • Always sanity-check entire header before importing
    • Validate string-lengths during sanity-checking
    • Enable data overlap changes for signature headers too
    • Disallow tags outside regions in package files
    • Perform signature and digest checks prior to importing headers
  • Improve detection of source/binary packages
  • Improve file trigger diagnostics
  • Major refactoring to signing and signature checking code to pave way for enhancements
  • Honor RPMVSF_NEEDPAYLOAD in signature checking
  • Make signature and digest verification messages more consistent
  • Clean up rpmdb directory on –rebuilddb
  • Simplify rpmdb cleanup by using standard atexit() mechanism
  • Make signal behavior consistent across crypto backends (NSPR vs SIGPIPE)
  • Major rewrite of signal handling code, allowing for custom handlers in the polled signal queue
  • Optimize RemovePathPostfixes processing
  • Make ASCII digest comparison case insensitive (trac:905)
  • Reduce external dependencies of rpm2cpio.sh script
  • Handle format extension type checking centrally
  • Numerous dead code removals
  • Make rpmlead structure opaque within rpm itself
  • New FA_TOUCH mode to only update file metadata

Build process

  • Fix dist and distcheck targets to work out of the box again
  • Fix build with PYTHON=python3 ./configure
  • Return error exit status if testsuite fails
  • The rpm binary is installed into $(bindir) instead of hardwired /bin/rpm
  • Support building with μClibc and musl
  • Portability fixes (BSD, OS X, fstat64, Darwin archs…)
  • Proper support for beecrypt in the build system
  • Use pkgconfig to locate python and NSS/NSPR if present
  • Automatically fall back to –with-external-db if internal db is not present
  • Support building with libdw/libelf (TODO: better description)
  • Check if make supports -O
  • Several fixes to gcc flags tests
  • Don’t mess with CFLAGS from configure
  • Support detecting gpg2 and prefer it over gpg
  • Crypto backend is now selected by –with-crypto=<beecrypt/nss/openssl>
  • Many new cases in test-suite
  • IMAEVM:
    • Fix excessive linking to libimaevm
    • Clean up conditional compilation
    • libimaevm version >= 1.0 is required now
  • Drop support for systems without SA_SIGINFO
  • Create $(rpmconfigdir)/macros.d directory on make install

Compatibility notes

  • The maximum supported header size was significantly raised in this release. This cannot be tracked with rpmlib() dependencies as that requires accessing the header, so attempting to access packages with larger than 32MB header will just abruptly fail with ALL older rpm versions.
  • File digest algorithm was changed to SHA256 from the legacy MD5. This is compatible with rpm >= 4.6.0 but to build packages compatible with older versions than that, %_source_filedigest_algorithm and %_binary_filedigest_algorithm need to be set to 1 for MD5.
  • Due to the changes and fixes to the macro subsystem, some macros will require adjusting to the new behavior.

Known issues in the 4.14.0 beta release