Upgrading 11.0 to 12.0

Summary

This section contains the sanitized summary of what I would do next time.

  1. pkg audit -F to see if there are any current ports that now have known security vulnerabilities
  2. freebsd-update -r 12.0-RELEASE upgrade
  3. freebsd-update install
  4. shutdown -r now
  5. freebsd-update install
  6. pkg-static upgrade -f
  7. pkg audit to see if there are any current ports that now have known security vulnerabilities (don't fetch new database this time)
  8. portsnap fetch update if need to collect any fixes to vulnerable ports
  9. portmaster -af
  10. freebsd-update install

Log

This section shows all the stumbles I made/encountered in the process of upgrading.

Based on instructions in the FreeBSD Handbook, here's the steps I'm taking:

  1. freebsd-update -r 12.0-RELEASE upgrade
  2. freebsd-update install
  3. shutdown -r now
  4. freebsd-update install
  5. pkg-static upgrade -f
  6. portmaster -af
  7. freebsd-update install

I had to modify what I did based on what I found during the process. When I ran pkg-static upgrade -f, it generated a message recommending that I run pkg-static install -f pkg due to a major OS version upgrade detected, so I stopped the run and ran the program as suggested. When I then ran portmaster -af, the run failed because devel/cmake-modules had been deleted and merged into devel/cmake. I tried running portmaster -a, which resulted in devel/cmake-modules appearing in the list of ports to be deleted, and devel/cmake appearing in the list of ports to be upgraded, which seemed good. The command completed, and I ran the last freebsd-update command to complete the set of commands.

Reran portmaster -af to see if I could rebuild everything. One nice enhancement is that config options for all packages are presented at the start of the operation, so they can be all answered separately from the build process, so the build process doesn't get stopped in the middle by having to answer config questions.

Unfortunately, there are lots of blocking messages asking if I want to delete old archive files, which has a similar effect.

The run failed because openjpeg-2.3.0_2 has security vulnerabilities. /tmp/portmaster.fail has the command line command that will resume the rebuild.

Review vulnerabilities in ports by running pkg audit -F.

OpenJPEG got fixed on 20190105, so resuming. /tmp/portmaster.fail has <flags> instead of the needed flags. I only used -af for flags, so maybe doesn't need any (or just -f). Running - working so far. Asked to confirm the changes, then builds as specified. However, I needed to update the ports tree first.

Trying pkg-static upgrade -f which has new packages, but maybe isn't what I want. Next, try portsnap fetch update Next, rerun the command saved in /tmp/portmaster.fail, using the -f flag.

Failed due to:

Installing check-0.12.0_1...

pkg-static: check-0.12.0_1 conflicts with libcheck-0.10.0 (installs files into the same place). Problematic file: /usr/local/bin/checkmk

*** Error code 70

Run pkg delete libcheck-0.10.0.

Then, rerun the command saved in /tmp/portmaster.fail, using the -f flag. This completed.

Next, run freebsd-update install.

At the end of this, see if there is a portmaster command to clean out any old build products, including those archives from older builds. Also, look for a command to check the installed ports to see if there are any mismatches.

Need to read /usr/ports/UPDATING after updating the ports tree and before rebuilding ports.

References

FreeBSD Handbook on updating/upgrading

See document on upgrading ports.

Also, see man page for portmaster.

See how to install and manage ports on FreeBSD 10.1 (Digital Ocean).

Warning: Old Stuff Ahead

The following sections are pretty old (for 4.x to 7.x releases).

Dancing Kayak - FreeBSD - Full Release and Security Upgrade

Security Upgrade

Here's what to do to upgrade an existing FreeBSD installation to the latest security release of that same version.

Preparation

  1. Understand which value you will pass to the -j option in make. -j4 is good for a single CPU. -j6 to -j10 is good for multiple CPUs.
  2. Prepare /etc/make.conf as follows
    • Set CPUTYPE variable to specify the machine's CPU.
    • Set KERNCONF to the name of the kernel configuration file (this is often the same name as the machine). This step is optional since you can set this value on the command line during the upgrade.
  3. install the sysutils/fastest_cvsup port - this requires the devel/p5-Time-HiRes port.
  4. run fastest_cvsup with the appropriate arguments to find out which is the cvsup server with the shortest response time. For me, the command line is # fastest_cvsup -c us
  5. copy stable-supfile and ports-supfile from /usr/share/examples/cvsup to /usr/src
  6. mkdir /usr/sup
  7. copy /usr/share/examples/cvsup/refuse to /usr/sup
  8. edit refuse to remove sendmail file reference since haven't configured it yet
  9. cd /usr/src
  10. mv stable-supfile security-supfile
  11. edit security-supfile as follows
    • s/stable/security/
    • tag=RELENG_#_#, where #_# is the release number of your release (e.g., 4_9).
    • set host to cvsup#.us.freebsd.org, where # is a number from 1 to 18 (as of February 2004) that references the cvsup site you identified when running fastest_cvsup.
  12. Put the security-supfile under source code control, e.g., mkdir RCS; ci -u security-supfile - I have started a practice of storing OS and port upgrade supfiles within the root user's home directory (specifically, /root/supfiles) so that I keep personal configuration files in a directory I'm more likely to back up.

Upgrade Installation

  1. perform backups, especially of /etc
  2. cvsup security-supfile - if you are running X, the gui client will appear; click the green arrowhead at the lower left to start the process, and close the window once the process completes
  3. look at /usr/src/UPDATING
  4. check GENERIC, LINT for changes and integrate them into the configuration file for your machine as needed
  5. cd /usr/src; make buildworld - can use the -j6 flag on dual-processor machine - a build of 4.x takes about 2 hours on dual PPro200, about 90 minutes on dual P3/450 - builds of 5.x and 6.0 take (a rough guess) at least twice as long
  6. make buildkernel [KERNCONF=filename] to build the kernel; the KERNCONF setting is optional if you have already put a value for this in /etc/make.conf. This takes about 10-20 minutes (for 4.x) and (rough guess) an hour (5.x, 6.0) on the previously-mentioned machines.
  7. make installkernel [KERNCONF=filename] takes about a minute
  8. reboot
  9. check out the system for basic sanity. If the upgrade has changed sizes of kernel structures, you may get weird behavior doing stuff like ps.
  10. go to single-user mode by shutdown now
  11. run mergemaster -p to do a merge before running make installworld
  12. cd /usr/src; make installworld - takes about 4 minutes.
  13. Run mergemaster to review and choose changes to configuration files in /etc. Don't proceed without this step, even though you likely will want to decline some of the changes it points out. For example, I always decline the change to /etc/motd because the build entered the correct FreeBSD version information into that file, and the generic one does not have that information.
  14. reboot
  15. check SCSI devices - you may have to run MAKEDEV

Upgrading to the Latest Full Release

Upgrading to the latest full release (e.g., upgrading to 4.9 from 4.8) is similar to upgrading to a security release, but you will want to upgrade the ports tree by ensuring the appropriate lines exist in the upgrade supfile.

After upgrading the machine as instructed above, complete the update of the ports tree as stated in the section entitled After Upgrading the Ports Tree

If, instead, you updated your ports tree before updating source, building, and installing a major version upgrade to the operating system, (e.g., from 4.x to 5.x or from 5.x to 6.x) you will need to rebuild the ports indexes as specified below, since the name of the ports database file differs among major version numbers.

Notes on Upgrading 4.x to 5.x

This was a more complicated upgrade. Instead of performing a source code upgrade like for minor version number releases and security updates, I had to install 5.x from scratch (a clean install) and then restore my data from backups.

Notes on Upgrading 5.x to 6.x

6.x uses SHA256 as well as MD5 to validate the integrity of source code archives downloaded when building a port.

This was an easy upgrade, just like a minor version or security upgrade. There was no need to install from scratch and restore data from backups, like I had to do when upgrading 4.10 to 5.x.

Notes on Upgrading 6.3 to 7.0

Root Partition Size

I had made a 256 MB root partition - that size served me well for 4.x, 5.x, and 6.x. When I built 7.0 and tried to install it, it filled up the root partition. Here are various hints to reduce size:

Next time I build a FreeBSD box, it will have a 1 GB root partition now that disk is cheap.

Ports Upgrade

To do this right, run the following command:

FreeBSD 7.0 uses version 4 of the GNU compiler collection; FreeBSD 6.x uses version 3 of GCC. This means that every port must be rebuilt (actually, every port that uses that compiler to produce binaries or shared libraries, but not many of us want to spend the time to track that as opposed to having the ease-of-use of a single upgrade command. GCC version 4 uses different shared library numbers than does GCC version 3.

That said, I didn't do it that way! Since I had so many old ports to upgrade, I did portupgrade -aR. I regret it somewhat because I later had to:

  1. go through /usr/local/lib and look at the build dates of the shared libraries
  2. run pkg_info -W /usr/local/lib/sharedlibname to find out what port had installed that library
  3. run portupgrade -f thePortName to rebuild the port

However, I discovered aspects of my ports dependencies that might not have been upgraded even if I had done it the correct way. I had to rebuild the devel/gvfs port with CDDB support disabled so that libgnomeui would not depend on it, in order to get libgnomeui to build. I also had to hack into the tests for GraphicsMagick and disable the exceptions and attributes tests in order for the port to build and install. These tests in the GraphicsMagick port would fault and exit the program when an exception was thrown. I'll have to work on this later, now that I have my computer in a usable state again.

Updating the Ports Tree

Updating the ports tree is useful to keep up on the latest versions of third-party software.

Do not remove the following directories in /usr/ports, or any other directories there - I tried this to save space in /usr/ports, but this resulted in a broken make index result because other directories have dependencies within these directories

Here's how to update the ports tree

  1. portinstall -R portsnap
  2. portsnap fetch extract - read the man page - if you have any files within /usr/ports that you modified or created copy them elsewhere before running this command because this command will delete them (this doesn't apply to files within /usr/ports/distfiles)

Here's a much slower but still proper way to update the ports tree

  1. edit ports-supfile as follows
    • ensure the value for the tag parameter is . - find and heed the warning in the file about setting the value of tag
    • Modify the host parameter to reference the best cvsup server for you (just like you did for the security-supfile above) as the value for the host parameter (or you can use the -h argument for cvsup to override what's in the ports-supfile)
  2. Put the ports-supfile under source code control, e.g., mkdir RCS; ci -u ports-supfile
  3. cvsup [-h cvsup??.??.freebsd.org ] ports-supfile

After Updating the Ports Tree

Follow these instructions after updating the ports tree (either directly or as part of an operating system version upgrade) to complete the update of the ports tree.

  1. portsnap fetch update

If you used the much slower way above, do the following:

  1. cd /usr/ports
  2. portsdb -Uu - this rebuilds necessary indexes
  3. make readmes

You can also use the command make index && make readmes this takes a long time, but don't use the -j argument for make index - you can use the -j argument for make readmes since this part is amenable to parallel processing

Upgrading a Port

The big hint here is to use portupgrade to help you upgrade (and possibly portinstall to help you install something new). Otherwise, make install clean will often get you what you want, unless there are conflicts (which portupgrade will often resolve.

Run pkgdb -F after you run portupgrade or portinstall, (or if you can't run these programs because of inconsistencies in the index used by these programs).

Warnings

Read the following files in /usr/ports for special instructions and notes before building or upgrading a port. These files contain information you need in order to upgrade certain ports.

You can also read /usr/ports/LEGAL and /usr/ports/README(.html) if you are so inclined.

Apparently sometimes you will have to watch out for cases where a recent upgrade in a port is dependent on something that's in the STABLE or CURRENT branch.

Read the release notes for the FreeBSD ports of GNOME and KDE before upgrading either of these meta-ports

Upgrading KDE from 3.1.4 to 3.2

Here's what I did (the first steps enable portupgrade to work automatically - see FreeBSD KDE page for why:

  1. # pkg_delete kdebase-3.1.4
  2. # pkg_delete kdenetwork-3.1.4
  3. # pkgdb -F (answer yes to fixing up dependencies for the packages deleted in the previous two steps)
  4. # portupgrade -R kde (I had to restart this because one of the fetches failed - I had to manually download the archive being fetched before restarting)
  5. # pkgdb -F (detected nothing to fix)

Upgrading KDE from 3.2.1 to 3.2.3

This upgrade was lengthy (about 3 days), but had no problems. A simple portupgrade -R x11/kde3 did it.

Upgrading Gnome from 2.4 to 2.6

There is an FAQ and a shell script pointed at by the FreeBSD Gnome Project that you must follow when upgrading. Even with all that help, my upgrade broke. The problems related to step 4, which performs an upward-recursive portupgrade of glib2*; there were some dependent ports whose upgrade required a downward-recursive upgrade of other ports on which they depended.

Also, the upgrade of one port left a patch file in the files subdirectory, but the file had been obsoleted and was in the Attic subdirectory.

Here are the steps I took and what happened. All of the below is done as root

  1. copy the upgrade script from FreeBSD Gnome project (downloaded as a non-root user) to /root
  2. cd /usr/src
  3. cvsup ports-supfile
  4. cd /usr/ports
  5. make index && make -j4 readmes
  6. cd /root
  7. sh ./gnome_upgrade.sh - this failed after about a day and a half because of a single reason: what apparently is a bad patch in multimedia/gstreamer-plugins. There is a patch at files/patch-gst-libs_ext_ffmpeg_ffmpeg_libavcodec_alpha_simple_i that apparently doesn't correspond to any source file.
  8. cd /usr/ports/multimedia/gstreamer-plugins
  9. mv files/patch-gst-libs_ext_ffmpeg_ffmpeg_libavcodec_alpha_simple_i .
  10. portupgrade gstreamer-plugins - this failed because libmusicbrainz needed to be upgraded and I didn't use the -R flag to upgrade ports on which gstreamer-plugins depends
  11. portupgrade -wR gstreamer-plugins - this failed because the upgrade to libmusicbrainz wouldn't upgrade even with the correct flags, making me doubt that I entered the correct flags!
  12. portupgrade -wR libmusicbrainz - this succeeded
  13. portupgrade -wR gstreamer-plugins - this succeeded
  14. cd /root
  15. sh ./gnome_upgrade.sh - this is going through another day-and-a-half, because step 4 in the script is a huge portupgrade -r -f glib2* forced upward-recursive upgrade (forced rebuild of glib2* and everything that depends on it) which rebuilds just about everything on my machine including evolution, firefox, and KDE. This succeeded.

I also upgraded my work machine, and I had to manually update the port nas because an upgraded version was required by the port sdl12 as well as manually updating the libmusicbrainz port (although without having to do a downward-recursive update). I also had some problems with upgrading KDE to 3.2.1, but that is not relevant to the Gnome upgrade discussed here except that the KDE upgrade was a consequence of the upward-recursive Gnome upgrade.

Upgrading GNOME from 2.6 to 2.6.2

This upgrade was lengthy and also went smoothly, but broke some additional functionality. When launching GNOME after the upgrade, I got a number of error messages including for the clock display. This resulted in some missing items in the task bars, including the clock. When launching Evolution, I got four error messages involving CORBA of the following format:

Cannot active component DAFIID: GNOME_Evolution_Mail_Shellcomponent INV_OBJREF:1.0

The other three messages involved components with similar names, substituting Addressbook, Calendar, and Summary for Mail.

Rebuilding Evolution was no help. Killing the wombat server and the timer server did not help. What helped was killing the bonobo_activation_server, which apparently is a CORBA server, and restarting Evolution. Since the libbonobo package was apparently upgraded to 2.6.2 as part of the GNOME upgrade, it's likely I needed a new server instance.

Upgrading GNOME from 2.6.2 to 2.8 and KDE from 3.2.3 to 3.3.1

On November 18, 2004, I decided to upgrade the X desktops of my workstation. This includes upgrading GNOME, KDE, and windowmaker, as well as X. Here's what I had to do to complete the job.

  1. portupgrade -R x11/XFree86-4 - this went smoothly
  2. portupgrade -R x11-wm/windowmaker - this went smoothly
  3. download the GNOME 2.6 to 2.8 upgrade script - ok
  4. run the GNOME 2.6 to 2.8 upgrade script - failures as follows:
    • My system didn't have devel/desktop=file-utils 0.9 and I couldn't build it because I couldn't download the source because the freedesktop.org web site was being rebuilt after suffering a hack. The latest version of devel/desktop-file-utils/Makefile has an alternate download location (www.marcuscom.com), but I didn't have that Makefile in my ports tree because it had been checked in just a couple of days prior, and I had updated my ports tree earlier than that. That resulted in a number of ports building but not installing, including
      • games/gnomegames2
      • editors/gedit2
      • graphics/eog2
      • mail/evolution
      • graphics/gpdf
      • archivers/fileroller
      • print/ggv2
    • x11-fm/nautilus2 failed to build due to an error detected by the compiler in the source code: nautilus-image-properties-view.c:134: macro 'exif_content_get_value' used with too many (4) args
    • x11/screensaver-kde failed to install due to an installed package conflict (kdepim-3.2.3_1), for which the likely solution was to upgrade KDE.
    • www/mozilla failed to build due to "fetch: libart_lgpl.tar.bz2: local modification time does not match remote"
    • other downstream errors caused by the above errors
  5. I downloaded the devel/desktop-file-utils source code from www.marcuscom.com and saved it in /usr/ports/distfiles. I did this during the late portion of the above GNOME upgrade script run, so that my list of failed installs above may not be a complete list of all the installs that would fail.
  6. portupgrade --noclean games/gnomegames2 to resolve the missing devel/desktop-file-utils port source
  7. portupgrade --noclean editors/gedit2 to resolve the missing devel/desktop-file-utils port source
  8. portupgrade --noclean graphics/eog2 to resolve the missing devel/desktop-file-utils port source
  9. portupgrade --noclean mail/evolution to resolve the missing devel/desktop-file-utils port source
  10. portupgrade --noclean graphics/gpdf to resolve the missing devel/desktop-file-utils port source
  11. portupgrade --noclean archivers/fileroller to resolve the missing devel/desktop-file-utils port source
  12. portupgrade --noclean print/ggv2 to resolve the missing devel/desktop-file-utils port source
  13. portupgrade -R x11/kde to resolve the conflicts encountered updating KDE as part of the GNOME upgrade - this hung building graphics/sane-backends because the build presented the configuration screen, and I was redirecting stdout to a file, and I wasn't running the command with the batch-mode flag
  14. portupgrade graphics/sane-backends
  15. portupgrade -R x11/kde3 - upgrading x11-toolkits/open-motif from 2.2.2_2 to 2.3 failed because I have portaudit installed and 2.3 has a security problem (2.4 doesn't but it isn't in the ports tree that I have, and the FreeBSD ports system has 2.3 as of 7:30 AM PT November 21, 2004)
  16. Rerun the GNOME upgrade script, adding the -restart argument
  17. pkg_delete kdeutils-3.2.3_1 because kdebase-3.3.1 installs files into the same place
  18. portupgrade --noclean kdeutils to install the built kdebase
  19. Rerun the GNOME upgrade script, adding the -restart argument
  20. Rename /usr/ports/distfiles/libart_lgpl.tar.bz2 to libart_lgpl.tar.bz2.old to attempt to resolve a checksum mismatch and local modification time does not match remote error on this file
  21. Rerun the GNOME upgrade script, adding the -restart argument. This time it completed step 4 and partially completed step 5 (reinstalled gdesklets and gnomeapplets2, failing reinstallation of gnome2). The gnome2 build failed building ximian-connector-setup-2.0.2 for port mail/ximian-connector-setup. The error messages included
    • libexchange.a(e2k-kerberos.o): In function 'e2k_kerberos_change_password':
    • e2k-kerberos.o(.text+0x231): undefined reference to 'krb5_free_data_contents'
    • e2k-kerberos.o(.text+0x241): undefined reference to 'krb5_free_data_contents'
    • gmake[3]: *** [cptest] Error 1
    • gmake[3]: Leaving directory '/usr/ports/mail/ximian-connector/work/ximian-connector-2.0.2/lib'
  22. Give up on GNOME upgrade for awhile (maybe I have to update security/krb5, which isn't installed on my system) and focus on KDE.
  23. portupgrade -R -x x11-toolkits/open-motif x11/kde3 - excluding open-motif because a version without the security hole is not yet in the ports tree. This succeeded.
  24. Run pkgdb -F three times to remove old versions of autoconf and automake.
  25. portupgrade kdegraphics-kamera kdegraphics-kooka kdegraphics-kuickshow
  26. portupgrade x11-clocks/kdetoys
  27. pkg_delete -f kdeaddons-kontact-plugins-3.2.3
  28. pkgdb -F to remove obsolete links between the kontact plugin and other ports
  29. portupgrade -R -x x11-toolkits/open-motif misc/kdeaddons3 - excluding open-motif because a version without the security hole is not yet in the ports tree. This succeeded.
  30. pkgdb -F just to check
  31. portupgrade -R -x x11-toolkits/open-motif kdebase-konqueror-nsplugins-3.2.3 kdeutils-klaptopdaemon-3.2.3 kdemultimedia-mpeglib_artsplugin-3.2.3
  32. At this point, KDE works, so I'm taking a break after a few days of console-only operation
  33. Back to GNOME - install security/krb5 in hopes of resolving the missing function for the gnome2 portupgrade
  34. Run portupgrade -f -m "GNOME_UPGRADE_SH_VER=280" -O gnome2 to manually resume step 5 of the GNOME upgrade script. This succeeded, building the ximian-connector-setup port without error.
  35. Run portupgrade -f -m "GNOME_UPGRADE_SH_VER=280" -O gnomevfs2 to manually resume step 5 of the GNOME upgrade script.
  36. Run portupgrade -f -m "GNOME_UPGRADE_SH_VER=280" -O libgnome to manually resume step 5 of the GNOME upgrade script.
  37. Run portupgrade -f -m "GNOME_UPGRADE_SH_VER=280" -O gconf-editor to manually resume step 5 of the GNOME upgrade script.
  38. Try it out - seems to work

At the end, upgrading KDE adds fonts to the system, so


Upgrading GNOME applications from 2.10 to 2.12

Due to the changes in gnomehier, I followed the instructions in the entry dated 20071024 in /usr/ports/UPDATING, namely:

  1. forcing the package database to be consistent (pkgdb -Ff) - I had to force the deinstall of old automake and autoconf because of duplicate entries in the package database.
  2. portupgrade -f -o textproc/rarian textproc/scrollkeeper - this was easy and quick
  3. portupgrade -a - this took a day and a half. I did the fetch of all archives first, then did the build. The build of multimedia/kdemultimedia3 failed during installation due to a known problem with the path_dps.m4 file (the location is obvious from the build errors); I had to deinstall the x11/dgs (display ghostscript system) package; no package was depending on that package, so that made it easy. Then, I resumed the build of kdemultimedia with the -w option to skip the clean before build to save all the work.

Back to FreeBSD main page