Tag: Solaris

Replacing A Solaris EFI Disk Label

This is kind of an adjunct to part 4 of my “Server Upgrade Chronicles”.

ZFS root pools have some requirements and best-practices at variance to other ZFS pools. One of the most annoying is that you can’t use a whole disk, and you can’t use an EFI-labeled disk. This is annoying because for most ZFS uses using a whole disk is the best practice, and when you do that ZFS puts an EFI label on that disk.

So, when you try to use in a root pool a disk you’d previously used somewhere else in ZFS, you often see this:

bash-3.2$ pfexec zpool attach rpool c4t0d0s0 c9t0d0s0
cannot attach c9t0d0s0 to c4t0d0s0: EFI labeled devices are not supported
on root pools.

What do you do then? Well, you google, of course. And you find many sites explaining how to overwrite an EFI label on a disk. And every single one of them omits several things that seem to me to be key points (and which I had to play around with a lot to get any understanding of). The fact that ZFS is what drew me back into Solaris, and that I wasn’t ever really comfortable with their disk labeling scheme to begin with, is no doubt a contributing factor.

This is going to get long, so I’m putting in a cut here. Continue reading Replacing A Solaris EFI Disk Label

OpenSolaris ZFS Root Pool Mirroring

This has been a real pain, due to incomplete and misleading documentation. I believe I’ve finally gotten it to work again, and I need to write down in some detail what I did.

This is based on installing OpenSolaris 2008.11. The two disks I want to make my root pool are c5t0d0 and c5t1d0. The general process is to install on one of the disks, and attach the second as a mirror later. Most of this requires root, and I see I haven’t been too careful showing that. Sometimes the “pfexec” command is showing, which is the OpenSolaris roles-oriented equivalent of sudo for executing one command with the root role.

First point: while in general you want to give ZFS whole disks to work with, you cannot do that for a root pool that you intend to mirror. There’s a nice convenient “whole disk” option in the installer, too, and no warning that you shouldn’t use it if you want to mirror later.

So, let’s say you’ve gone ahead and installed with nearly all of c5t0d0 as your zfs root pool (it’s called rpool), and now you want to make it a mirror.

First, using the format tool, see what the partition structure is (what I expect on an x86 box is one fdisk partition occupying the whole disk, and a set of Solaris slices in that). In my case, I think what the installer always does, s0 is the root slice, and it occupied cylinders 1-9724 on a disk having cylinders 0-9725. In any case — duplicate this structure on the disk you intend to mirror with (c5t1d0 in my case).

At which point your second disk will look like this:

Specify disk (enter its number)[1]: 1
selecting c5t1d0
[disk formatted]
format> verify

Primary label contents:

Volume name = <        >
ascii name  = 
pcyl        = 9728
ncyl        = 9726
acyl        =    2
bcyl        =    0
nhead       =  255
nsect       =   63
Part      Tag    Flag     Cylinders        Size            Blocks
  0       root    wm       1 - 9724       74.49GB    (9724/0/0) 156216060
  1 unassigned    wm       0               0         (0/0/0)            0
  2     backup    wu       0 - 9725       74.50GB    (9726/0/0) 156248190
  3 unassigned    wm       0               0         (0/0/0)            0
  4 unassigned    wm       0               0         (0/0/0)            0
  5 unassigned    wm       0               0         (0/0/0)            0
  6 unassigned    wm       0               0         (0/0/0)            0
  7 unassigned    wm       0               0         (0/0/0)            0
  8       boot    wu       0 -    0        7.84MB    (1/0/0)        16065
  9 unassigned    wm       0               0         (0/0/0)            0

Now you can mirror it:

pfexec zpool attach -f rpool c5t0d0s0 c5t1d0s0

The “-f” is necessary because, in previous playing around, I’ve sometimes managed to get a recognizable part of a zpool on that slice, so I have to tell zpool to overwrite it. So be VERY careful using -f! Don’t do it at first, and if you get an error and you’re SURE you really want to overwrite the old data, then use -f.

So this was very successful:

localddb@fsfs:/boot/grub$ zpool status
  pool: rpool
 state: ONLINE
 scrub: resilver completed after 0h3m with 0 errors on Sun Jan 18 11:28:36 2009
config:

	NAME          STATE     READ WRITE CKSUM
	rpool         ONLINE       0     0     0
	  mirror      ONLINE       0     0     0
	    c5t0d0s0  ONLINE       0     0     0  25.6M resilvered
	    c5t1d0s0  ONLINE       0     0     0  3.57G resilvered

errors: No known data errors

Now, to do a really complete job, you need to install grub on the secondary disk as well. In fact zpool will tell you you should do that. If you don’t do this, you won’t be able to boot from just the second disk when something happens to the first one. (Yes, “when”. Murphy rules!)

So, to install grub:

localddb@fsfs:/$ cd /boot/grub
localddb@fsfs:/boot/grub$ pfexec installgrub stage1 stage2 /dev/rdsk/c5t1d0s0
stage1 written to partition 0 sector 0 (abs 16065)
stage2 written to partition 0, 267 sectors starting at 50 (abs 16115)

And that should be that.

A suppose a truly wise admin would play with various failure modes and recoveries before going on to install much on this base.