Dancing Kayak - FreeBSD - Full Release and Security Upgrade

Security Upgrade

Here's what to do to upgrade an existing FreeBSD installation to the latest security release of that same version.

Preparation

  1. Understand which value you will pass to the -j option in make. -j4 is good for a single CPU. -j6 to -j10 is good for multiple CPUs.
  2. Prepare /etc/make.conf as follows
  3. install the sysutils/fastest_cvsup port - this requires the devel/p5-Time-HiRes port.
  4. run fastest_cvsup with the appropriate arguments to find out which is the cvsup server with the shortest response time. For me, the command line is # fastest_cvsup -c us
  5. copy stable-supfile and ports-supfile from /usr/share/examples/cvsup to /usr/src
  6. mkdir /usr/sup
  7. copy /usr/share/examples/cvsup/refuse to /usr/sup
  8. edit refuse to remove sendmail file reference since haven't configured it yet
  9. cd /usr/src
  10. mv stable-supfile security-supfile
  11. edit security-supfile as follows
  12. Put the security-supfile under source code control, e.g., mkdir RCS; ci -u security-supfile - I have started a practice of storing OS and port upgrade supfiles within the root user's home directory (specifically, /root/supfiles) so that I keep personal configuration files in a directory I'm more likely to back up.

Upgrade Installation

  1. perform backups, especially of /etc
  2. cvsup security-supfile - if you are running X, the gui client will appear; click the green arrowhead at the lower left to start the process, and close the window once the process completes
  3. look at /usr/src/UPDATING
  4. check GENERIC, LINT for changes and integrate them into the configuration file for your machine as needed
  5. cd /usr/src; make buildworld - can use the -j6 flag on dual-processor machine - a build of 4.x takes about 2 hours on dual PPro200, about 90 minutes on dual P3/450 - builds of 5.x and 6.0 take (a rough guess) at least twice as long
  6. make buildkernel [KERNCONF=filename] to build the kernel; the KERNCONF setting is optional if you have already put a value for this in /etc/make.conf. This takes about 10-20 minutes (for 4.x) and (rough guess) an hour (5.x, 6.0) on the previously-mentioned machines.
  7. make installkernel [KERNCONF=filename] takes about a minute
  8. reboot
  9. check out the system for basic sanity. If the upgrade has changed sizes of kernel structures, you may get weird behavior doing stuff like ps.
  10. go to single-user mode by shutdown now
  11. run mergemaster -p to do a merge before running make installworld
  12. cd /usr/src; make installworld - takes about 4 minutes.
  13. Run mergemaster to review and choose changes to configuration files in /etc. Don't proceed without this step, even though you likely will want to decline some of the changes it points out. For example, I always decline the change to /etc/motd because the build entered the correct FreeBSD version information into that file, and the generic one does not have that information.
  14. reboot
  15. check SCSI devices - you may have to run MAKEDEV

Upgrading to the Latest Full Release

Upgrading to the latest full release (e.g., upgrading to 4.9 from 4.8) is similar to upgrading to a security release, but you will want to upgrade the ports tree by ensuring the appropriate lines exist in the upgrade supfile.

After upgrading the machine as instructed above, complete the update of the ports tree as stated in the section entitled After Upgrading the Ports Tree

If, instead, you updated your ports tree before updating source, building, and installing a major version upgrade to the operating system, (e.g., from 4.x to 5.x or from 5.x to 6.x) you will need to rebuild the ports indexes as specified below, since the name of the ports database file differs among major version numbers.

Notes on Upgrading 4.x to 5.x

This was a more complicated upgrade. Instead of performing a source code upgrade like for minor version number releases and security updates, I had to install 5.x from scratch (a clean install) and then restore my data from backups.

Notes on Upgrading 5.x to 6.x

6.x uses SHA256 as well as MD5 to validate the integrity of source code archives downloaded when building a port.

This was an easy upgrade, just like a minor version or security upgrade. There was no need to install from scratch and restore data from backups, like I had to do when upgrading 4.10 to 5.x.

Notes on Upgrading 6.3 to 7.0

Root Partition Size

I had made a 256 MB root partition - that size served me well for 4.x, 5.x, and 6.x. When I built 7.0 and tried to install it, it filled up the root partition. Here are various hints to reduce size:

Next time I build a FreeBSD box, it will have a 1 GB root partition now that disk is cheap.

Ports Upgrade

To do this right, run the following command:

FreeBSD 7.0 uses version 4 of the GNU compiler collection; FreeBSD 6.x uses version 3 of GCC. This means that every port must be rebuilt (actually, every port that uses that compiler to produce binaries or shared libraries, but not many of us want to spend the time to track that as opposed to having the ease-of-use of a single upgrade command. GCC version 4 uses different shared library numbers than does GCC version 3.

That said, I didn't do it that way! Since I had so many old ports to upgrade, I did portupgrade -aR. I regret it somewhat because I later had to:

  1. go through /usr/local/lib and look at the build dates of the shared libraries
  2. run pkg_info -W /usr/local/lib/sharedlibname to find out what port had installed that library
  3. run portupgrade -f thePortName to rebuild the port

However, I discovered aspects of my ports dependencies that might not have been upgraded even if I had done it the correct way. I had to rebuild the devel/gvfs port with CDDB support disabled so that libgnomeui would not depend on it, in order to get libgnomeui to build. I also had to hack into the tests for GraphicsMagick and disable the exceptions and attributes tests in order for the port to build and install. These tests in the GraphicsMagick port would fault and exit the program when an exception was thrown. I'll have to work on this later, now that I have my computer in a usable state again.

Updating the Ports Tree

Updating the ports tree is useful to keep up on the latest versions of third-party software.

Do not remove the following directories in /usr/ports, or any other directories there - I tried this to save space in /usr/ports, but this resulted in a broken make index result because other directories have dependencies within these directories

Here's how to update the ports tree

  1. portinstall -R portsnap
  2. portsnap fetch extract - read the man page - if you have any files within /usr/ports that you modified or created copy them elsewhere before running this command because this command will delete them (this doesn't apply to files within /usr/ports/distfiles)

Here's a much slower but still proper way to update the ports tree

  1. edit ports-supfile as follows
  2. Put the ports-supfile under source code control, e.g., mkdir RCS; ci -u ports-supfile
  3. cvsup [-h cvsup??.??.freebsd.org ] ports-supfile

After Updating the Ports Tree

Follow these instructions after updating the ports tree (either directly or as part of an operating system version upgrade) to complete the update of the ports tree.

  1. portsnap fetch update

If you used the much slower way above, do the following:

  1. cd /usr/ports
  2. portsdb -Uu - this rebuilds necessary indexes
  3. make readmes

You can also use the command make index && make readmes this takes a long time, but don't use the -j argument for make index - you can use the -j argument for make readmes since this part is amenable to parallel processing

Upgrading a Port

The big hint here is to use portupgrade to help you upgrade (and possibly portinstall to help you install something new). Otherwise, make install clean will often get you what you want, unless there are conflicts (which portupgrade will often resolve.

Run pkgdb -F after you run portupgrade or portinstall, (or if you can't run these programs because of inconsistencies in the index used by these programs).

Warnings

Read the following files in /usr/ports for special instructions and notes before building or upgrading a port. These files contain information you need in order to upgrade certain ports.

You can also read /usr/ports/LEGAL and /usr/ports/README(.html) if you are so inclined.

Apparently sometimes you will have to watch out for cases where a recent upgrade in a port is dependent on something that's in the STABLE or CURRENT branch.

Read the release notes for the FreeBSD ports of GNOME and KDE before upgrading either of these meta-ports

Upgrading KDE from 3.1.4 to 3.2

Here's what I did (the first steps enable portupgrade to work automatically - see FreeBSD KDE page for why:

  1. # pkg_delete kdebase-3.1.4
  2. # pkg_delete kdenetwork-3.1.4
  3. # pkgdb -F (answer yes to fixing up dependencies for the packages deleted in the previous two steps)
  4. # portupgrade -R kde (I had to restart this because one of the fetches failed - I had to manually download the archive being fetched before restarting)
  5. # pkgdb -F (detected nothing to fix)

Upgrading KDE from 3.2.1 to 3.2.3

This upgrade was lengthy (about 3 days), but had no problems. A simple portupgrade -R x11/kde3 did it.

Upgrading Gnome from 2.4 to 2.6

There is an FAQ and a shell script pointed at by the FreeBSD Gnome Project that you must follow when upgrading. Even with all that help, my upgrade broke. The problems related to step 4, which performs an upward-recursive portupgrade of glib2*; there were some dependent ports whose upgrade required a downward-recursive upgrade of other ports on which they depended.

Also, the upgrade of one port left a patch file in the files subdirectory, but the file had been obsoleted and was in the Attic subdirectory.

Here are the steps I took and what happened. All of the below is done as root

  1. copy the upgrade script from FreeBSD Gnome project (downloaded as a non-root user) to /root
  2. cd /usr/src
  3. cvsup ports-supfile
  4. cd /usr/ports
  5. make index && make -j4 readmes
  6. cd /root
  7. sh ./gnome_upgrade.sh - this failed after about a day and a half because of a single reason: what apparently is a bad patch in multimedia/gstreamer-plugins. There is a patch at files/patch-gst-libs_ext_ffmpeg_ffmpeg_libavcodec_alpha_simple_i that apparently doesn't correspond to any source file.
  8. cd /usr/ports/multimedia/gstreamer-plugins
  9. mv files/patch-gst-libs_ext_ffmpeg_ffmpeg_libavcodec_alpha_simple_i .
  10. portupgrade gstreamer-plugins - this failed because libmusicbrainz needed to be upgraded and I didn't use the -R flag to upgrade ports on which gstreamer-plugins depends
  11. portupgrade -wR gstreamer-plugins - this failed because the upgrade to libmusicbrainz wouldn't upgrade even with the correct flags, making me doubt that I entered the correct flags!
  12. portupgrade -wR libmusicbrainz - this succeeded
  13. portupgrade -wR gstreamer-plugins - this succeeded
  14. cd /root
  15. sh ./gnome_upgrade.sh - this is going through another day-and-a-half, because step 4 in the script is a huge portupgrade -r -f glib2* forced upward-recursive upgrade (forced rebuild of glib2* and everything that depends on it) which rebuilds just about everything on my machine including evolution, firefox, and KDE. This succeeded.

I also upgraded my work machine, and I had to manually update the port nas because an upgraded version was required by the port sdl12 as well as manually updating the libmusicbrainz port (although without having to do a downward-recursive update). I also had some problems with upgrading KDE to 3.2.1, but that is not relevant to the Gnome upgrade discussed here except that the KDE upgrade was a consequence of the upward-recursive Gnome upgrade.

Upgrading GNOME from 2.6 to 2.6.2

This upgrade was lengthy and also went smoothly, but broke some additional functionality. When launching GNOME after the upgrade, I got a number of error messages including for the clock display. This resulted in some missing items in the task bars, including the clock. When launching Evolution, I got four error messages involving CORBA of the following format:

Cannot active component DAFIID: GNOME_Evolution_Mail_Shellcomponent INV_OBJREF:1.0

The other three messages involved components with similar names, substituting Addressbook, Calendar, and Summary for Mail.

Rebuilding Evolution was no help. Killing the wombat server and the timer server did not help. What helped was killing the bonobo_activation_server, which apparently is a CORBA server, and restarting Evolution. Since the libbonobo package was apparently upgraded to 2.6.2 as part of the GNOME upgrade, it's likely I needed a new server instance.

Upgrading GNOME from 2.6.2 to 2.8 and KDE from 3.2.3 to 3.3.1

On November 18, 2004, I decided to upgrade the X desktops of my workstation. This includes upgrading GNOME, KDE, and windowmaker, as well as X. Here's what I had to do to complete the job.

  1. portupgrade -R x11/XFree86-4 - this went smoothly
  2. portupgrade -R x11-wm/windowmaker - this went smoothly
  3. download the GNOME 2.6 to 2.8 upgrade script - ok
  4. run the GNOME 2.6 to 2.8 upgrade script - failures as follows:
  5. I downloaded the devel/desktop-file-utils source code from www.marcuscom.com and saved it in /usr/ports/distfiles. I did this during the late portion of the above GNOME upgrade script run, so that my list of failed installs above may not be a complete list of all the installs that would fail.
  6. portupgrade --noclean games/gnomegames2 to resolve the missing devel/desktop-file-utils port source
  7. portupgrade --noclean editors/gedit2 to resolve the missing devel/desktop-file-utils port source
  8. portupgrade --noclean graphics/eog2 to resolve the missing devel/desktop-file-utils port source
  9. portupgrade --noclean mail/evolution to resolve the missing devel/desktop-file-utils port source
  10. portupgrade --noclean graphics/gpdf to resolve the missing devel/desktop-file-utils port source
  11. portupgrade --noclean archivers/fileroller to resolve the missing devel/desktop-file-utils port source
  12. portupgrade --noclean print/ggv2 to resolve the missing devel/desktop-file-utils port source
  13. portupgrade -R x11/kde to resolve the conflicts encountered updating KDE as part of the GNOME upgrade - this hung building graphics/sane-backends because the build presented the configuration screen, and I was redirecting stdout to a file, and I wasn't running the command with the batch-mode flag
  14. portupgrade graphics/sane-backends
  15. portupgrade -R x11/kde3 - upgrading x11-toolkits/open-motif from 2.2.2_2 to 2.3 failed because I have portaudit installed and 2.3 has a security problem (2.4 doesn't but it isn't in the ports tree that I have, and the FreeBSD ports system has 2.3 as of 7:30 AM PT November 21, 2004)
  16. Rerun the GNOME upgrade script, adding the -restart argument
  17. pkg_delete kdeutils-3.2.3_1 because kdebase-3.3.1 installs files into the same place
  18. portupgrade --noclean kdeutils to install the built kdebase
  19. Rerun the GNOME upgrade script, adding the -restart argument
  20. Rename /usr/ports/distfiles/libart_lgpl.tar.bz2 to libart_lgpl.tar.bz2.old to attempt to resolve a checksum mismatch and local modification time does not match remote error on this file
  21. Rerun the GNOME upgrade script, adding the -restart argument. This time it completed step 4 and partially completed step 5 (reinstalled gdesklets and gnomeapplets2, failing reinstallation of gnome2). The gnome2 build failed building ximian-connector-setup-2.0.2 for port mail/ximian-connector-setup. The error messages included
  22. Give up on GNOME upgrade for awhile (maybe I have to update security/krb5, which isn't installed on my system) and focus on KDE.
  23. portupgrade -R -x x11-toolkits/open-motif x11/kde3 - excluding open-motif because a version without the security hole is not yet in the ports tree. This succeeded.
  24. Run pkgdb -F three times to remove old versions of autoconf and automake.
  25. portupgrade kdegraphics-kamera kdegraphics-kooka kdegraphics-kuickshow
  26. portupgrade x11-clocks/kdetoys
  27. pkg_delete -f kdeaddons-kontact-plugins-3.2.3
  28. pkgdb -F to remove obsolete links between the kontact plugin and other ports
  29. portupgrade -R -x x11-toolkits/open-motif misc/kdeaddons3 - excluding open-motif because a version without the security hole is not yet in the ports tree. This succeeded.
  30. pkgdb -F just to check
  31. portupgrade -R -x x11-toolkits/open-motif kdebase-konqueror-nsplugins-3.2.3 kdeutils-klaptopdaemon-3.2.3 kdemultimedia-mpeglib_artsplugin-3.2.3
  32. At this point, KDE works, so I'm taking a break after a few days of console-only operation
  33. Back to GNOME - install security/krb5 in hopes of resolving the missing function for the gnome2 portupgrade
  34. Run portupgrade -f -m "GNOME_UPGRADE_SH_VER=280" -O gnome2 to manually resume step 5 of the GNOME upgrade script. This succeeded, building the ximian-connector-setup port without error.
  35. Run portupgrade -f -m "GNOME_UPGRADE_SH_VER=280" -O gnomevfs2 to manually resume step 5 of the GNOME upgrade script.
  36. Run portupgrade -f -m "GNOME_UPGRADE_SH_VER=280" -O libgnome to manually resume step 5 of the GNOME upgrade script.
  37. Run portupgrade -f -m "GNOME_UPGRADE_SH_VER=280" -O gconf-editor to manually resume step 5 of the GNOME upgrade script.
  38. Try it out - seems to work

At the end, upgrading KDE adds fonts to the system, so


Upgrading GNOME applications from 2.10 to 2.12

Due to the changes in gnomehier, I followed the instructions in the entry dated 20071024 in /usr/ports/UPDATING, namely:

  1. forcing the package database to be consistent (pkgdb -Ff) - I had to force the deinstall of old automake and autoconf because of duplicate entries in the package database.
  2. portupgrade -f -o textproc/rarian textproc/scrollkeeper - this was easy and quick
  3. portupgrade -a - this took a day and a half. I did the fetch of all archives first, then did the build. The build of multimedia/kdemultimedia3 failed during installation due to a known problem with the path_dps.m4 file (the location is obvious from the build errors); I had to deinstall the x11/dgs (display ghostscript system) package; no package was depending on that package, so that made it easy. Then, I resumed the build of kdemultimedia with the -w option to skip the clean before build to save all the work.

Back to FreeBSD main page