Live upgrade Fedora 15 to Fedora 16 using yum

Monday 14 Nov 2011 by Major Hayden | 1 Comment
Filed under: Blog Posts Tags: , , , , , , , , ,

Before we get started, I really ought to drop this here:

Upgrading Fedora via yum is not the recommended method. Your first choice for upgrading Fedora should be to use preupgrade. Seriously.

This begs the question: When should you use another method to upgrade Fedora? What other methods are there?

You have a few other methods to get the upgrade done:

  • Toss in a CD or DVD: You can upgrade via the anaconda installer provided on the CD, DVD or netinstall media. My experiences with this method for Fedora (as well as CentOS, Scientific Linux, and Red Hat) haven't been too positive, but your results may vary.
  • Download the newer release's fedora-release RPM, install it with rpm, and yum upgrade: This is the really old way of doing things. Don't try this (read the next bullet).
  • Use yum's distro-sync functionality: If you can't go the preupgrade route, I'd recommend giving this a try. However, leave plenty of time to fix small glitches after it's done (and after your first reboot).

Personal anecdote time (Keep scrolling for the meat and potatoes)
I have a dedicated server at Joe's Datacenter (love those folks) with IPMI and KVM-over-LAN access. The preupgrade method won't work for me because my /boot partition is on a software RAID volume. There's a rat's nest of a Bugzilla ticket over on Red Hat's site about this problem. I'm really only left with a live upgrade using yum.

Live yum upgrade process
Before even beginning the upgrade, I double-checked that I'd applied all of the available updates for my server. Once that was done, I realized I was one kernel revision behind and I rebooted to ensure I was in the latest Fedora 15 kernel.

A good practice here is to run package-cleanup --orphans (it's in the yum-utils package) to find any packages which don't exist on any Fedora mirrors. In my case, I had two old kernels and a JungleDisk package. I removed the two old kernels (probably wasn't necessary) and left JungleDisk alone (it worked fine after the upgrade). If you have any external repositories, such as Livna or RPMForge, you may want to disable those until the upgrade is done. Should the initial upgrade checks bomb out, try adding as few repositories back in as possible to see if it clears up the problem.

Once you make it this far, just follow the instructions available in Fedora's documentation: Upgrading Fedora using yum. I set SELinux to permissive mode during the upgrade just in case it caused problems.

I'd recommend skipping the grub2-install portion since your original grub installation will still be present after the upgrade. If your server has EFI (not BIOS), don't use grub2 yet. Keep an eye on the previously mentioned documentation page to see if the problems get ironed out between grub2 and EFI.

Before you reboot, be sure to get a list of your active processes and daemons. After your reboot, some old SysVinit scripts will be converted into Systemd service scripts. They might not start automatically and you might need to enable and/or start some services.

New to Systemd? This will be an extremely handy resource: SysVinit to Systemd Cheatsheet.

I haven't seen too many issues after cleaning up some daemons that didn't start properly. There is a problem between asterisk and SELinux that I haven't nailed down yet but it's not a showstopper.

Good luck during your upgrades. Keep in mind that Fedora 15 could be EOL'd as early as May or June 20102 when Fedora 17 is released.


Tracing a build through OpenStack Compute (Nova)

Monday 7 Nov 2011 by Major Hayden | No Comments
Filed under: Blog Posts Tags: , , ,

My work at Rackspace has changed a bit in the last few weeks and I've shifted from managing a team of engineers to a full technical focus on OpenStack Nova. Although it was difficult to leave my management position, I'm happy to get back to my roots and dig into the technical stuff again.

One of the first things I wanted to tackle was understanding how a build request flows through Nova to a XenServer hypervisor. Following this process through the code is a bit tricky (I'm still learning python, so that could explain it). Here are the basic steps:

  • Client requests a build via the API.
  • The API runs some checks (quotas, auth, etc) and hands the build off to the scheduler.
  • The scheduler figures out where the instance should go.
  • The scheduler drops a message in queue specific to one compute node (where the instance will be built).
  • The API responds to the client and the client is now unblocked and free to do other things.
  • The compute node updates the database with the instance details and calls to the hypervisor to assemble block devices for the instance.
  • A message is dropped into the network node's queue (from the compute node) to begin assembling networks for the instance. The compute node blocks and waits while this step completes.
  • Once the networking details come back (via the queue), the compute node does the remaining adjustments on the hypervisor and starts up the actual instance.
  • When the instance starts successfully (or fails to do so), the database is updated and a message is dropped onto another message queue as a notification that the build is complete.

Tracing an Instance Build Through NovaClick on the thumbnail on the right to see the flow chart I created to explain this process.

Please note: This information should be accurate to the Nova code as of November 1, 2011. There could be some refactoring of these build processes before Essex is released.



Installing irssi via MacPorts on OS X Lion 10.7.1

Friday 30 Sep 2011 by Major Hayden | 2 Comments
Filed under: Blog Posts Tags: , , , , ,

I've floated back and forth between graphical IRC clients and terminal-based clients for a long time. However, I was sad to see that irssi wouldn't build via MacPorts on OS X Lion. During the build, I saw quite a few errors from the compiler:

-E, -S, -save-temps and -M options are not allowed with multiple -arch flags

Sure enough, when I looked at the lines in the output, both x86_64 and i386 were passed to the compiler:

... -pipe -O2 -arch x86_64 -arch i386 -fno-common ...

I opened a ticket in trac and began looking for a workaround. Another trac ticket (from four years ago) on the MacPorts site gave some pointers on how to work around the bug for a previous version.

I changed up the instructions a bit since we're not dealing with the ppc architecture any longer:

sudo port -v clean irssi +perl
sudo port -v configure irssi +perl
cd /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_irc_irssi/irssi/work/
sudo find . -type f -exec sed -i "" -e "s/-arch i386//g" {} \;
cd
sudo port -v install irssi +perl

The build worked!

$ irssi -v
irssi 0.8.15 (20100403 1617)

Getting back to using eth0 in Fedora 15

Sunday 25 Sep 2011 by Major Hayden | 4 Comments
Filed under: Blog Posts Tags: , , , ,

Fedora 15 was released with some updates to allow for consistent network device names. Once it's installed, you'll end up with network devices that are named something other than eth0, eth1, and so on.

For example, all onboard ethernet adapters are labeled as emX (em1, em2...) and all PCI ethernet adapters are labeled as pXpX (p[slot]p[port], like p7p1 for port 1 on slot 7). Ethernet devices within Xen virtual machines aren't adjusted.

This may make sense to people who swap out the chassis on servers regularly and they don't want to mess with hard-coding MAC addresses in network configuration files. Also, it should give users predictable names even if a running system's drives are inserted into a newer hardware revision of the same server.

However, I don't like this on my personal dedicated servers and I prefer to revert back to the old way of doing things. Getting back to eth0 is pretty simple and it only requires a few configuration files to be edited followed by a reboot.

First, add biosdevname=0 to your grub.conf on the kernel line:

title Fedora (2.6.40.4-5.fc15.x86_64)
	root (hd0,0)
	kernel /boot/vmlinuz-2.6.40.4-5.fc15.x86_64 ro root=/dev/md0 SYSFONT=latarcyrheb-sun16 KEYTABLE=us biosdevname=0 quiet LANG=en_US.UTF-8
	initrd /boot/initramfs-2.6.40.4-5.fc15.x86_64.img

Open /etc/udev/rules.d/70-persistent-net.rules in your favorite text editor (create it if it doesn't exist) and add in the following:

# Be sure to put your MAC addresses in the fields below
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:11:22:33:44:10", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:11:22:33:44:11", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1"

Be sure to rename your ifcfg-* files in /etc/sysconfig/network-scripts/ to match the device names you've assigned. Just for good measure, I add in the MAC address in /etc/sysconfig/network-scripts/ifcfg-ethX:

...
HWADDR=00:11:22:33:44:10
...

Reboot the server and you should be back to eth0 and eth1 after a reboot.


Receive e-mail reports for SELinux AVC denials

Thursday 15 Sep 2011 by Major Hayden | 3 Comments
Filed under: Blog Posts Tags: , , , , , , , , , , , ,

SELinux isn't a technology that's easy to tackle for newcomers. However, there's been a lot of work to smooth out the rough edges while still keeping a tight grip on what applications and users are allowed to do on a Linux system. One of the biggest efforts has been around setroubleshoot.

The purpose behind setroubleshoot is to let users know when access has been denied, help them resolve it if necessary, and to reduce overall frustration while working through tight security restrictions in the default SELinux policies. The GUI frontend for setroubleshoot is great for users who run Linux desktops or those who run servers with a display attached. Don't worry, you can configure setroubleshoot on remote servers to send alerts elsewhere when a GUI alert isn't an option.

Install a few packages to get started:

yum install setroubleshoot{-server,-plugins,-doc}

Open /etc/setroubleshoot/setroubleshoot.conf in your favorite text editor and adjust the [email] section to fit your server:

recipients_filepath = /var/lib/setroubleshoot/email_alert_recipients
smtp_port = 25
smtp_host = localhost
from_address = selinux@myserver.com
subject = [MyServer] SELinux AVC Alert

You could probably see it coming, but you need to put the e-mail addresses for your recipients into /var/lib/setroubleshoot/email_alert_recipients:

echo "selinux@mycompany.com" >> /var/lib/setroubleshoot/email_alert_recipients

You'll notice that setroubleshoot doesn't have an init script and it doesn't exist in systemd in Fedora 15. It runs through the dbus-daemon and a quick bounce of the messagebus via its init script brings in the necessary components to run setroubleshoot:

service messagebus restart

A really easy (and safe) test is to ask sshd to bind to a non-standard port. Simply define an additional port on in your /etc/ssh/sshd_config like this:

Port 22
Port 222

When you restart sshd, it will bind to port 22 with success, but it won't be allowed to bind to port 222 (since that's blocked by SELinux as a non-standard port for the ssh_port_t port type). DON'T WORRY! Your sshd server will still be listening on port 22. If you wait a moment, you'll get an e-mail (perhaps two) that not only notify you of the denial, but they make suggestions for how to fix it:

SELinux is preventing /usr/sbin/sshd from name_bind access on the tcp_socket port 222.
 
*****  Plugin bind_ports (99.5 confidence) suggests  *************************
 
If you want to allow /usr/sbin/sshd to bind to network port 222
Then you need to modify the port type.
Do
# semanage port -a -t PORT_TYPE -p tcp 222
   where PORT_TYPE is one of the following: ...

For this particular example, the quick fix would be to run:

semanage port -a -t ssh_port_t -p tcp 222

Much of this post's information was gathered from the detailed documentation on Fedora's setroubleshoot User's FAQ as well as Dan Walsh's setroubleshoot blog post.


« Older Entries
Newer Entries »

  • Welcome! I started this blog as a way to give back to all of the other system administrators who have taught me something in the past. Writing these posts brings me a lot of enjoyment and I hope you find the information useful. If you spot something that's incorrect or confusing, please write a comment and let me know. Drop me a line if there's something you want to know more about and I'll do my best to write a post on the topic.
    -- Major Hayden

    Flattr this