This small demo shows the basic ZFS operations. You will need a Solaris 10 or OpenSolaris host and 1,2GB of disk space.

First, go to a directory where you have enought space. We will need 1,2GB for 6 files of 200MB. The files are named like disks, but they are only files for this demo.

cd /var/tmp
mkfile 200m c0d0 c0d1 c0d2 c1d0 c1d1 c1d2

pfexec zpool create demo /var/tmp/c0d0 
zpool status

  pool: demo
 state: ONLINE
 scrub: none requested
config:
 
	NAME             STATE     READ WRITE CKSUM
	demo             ONLINE       0     0     0
	  /var/tmp/c0d0  ONLINE       0     0     0
 
errors: No known data errors

I will omit the errors: No known data errors in all following outputs.

Also, the rpool is also omitted.

This will attach a second disk to the existing, creating a mirror.

pfexec zpool attach demo /var/tmp/c0d0 /var/tmp/c0d1 
zpool status

  pool: demo
 state: ONLINE
 scrub: resilver completed after 0h0m with 0 errors on Tue Aug 11 15:45:59 2009
config:
 
	NAME               STATE     READ WRITE CKSUM
	demo               ONLINE       0     0     0
	  mirror           ONLINE       0     0     0
	    /var/tmp/c0d0  ONLINE       0     0     0
	    /var/tmp/c0d1  ONLINE       0     0     0  73K resilvered

Note that the zpool status output now shows a mirror over two files.

Also note the resilver completed message. This tells you that the mirror is synchronized.

In this example, I made the mistake of mirroring c0d0 to c0d1. I wanted to mirror c0d0 to c1d0, simulating mirroring two disks on different controllers. So, let's replace c0d1 with c1d0.

pfexec zpool replace demo /var/tmp/c0d1 /var/tmp/c1d0 
zpool status

  pool: demo
 state: ONLINE
 scrub: resilver completed after 0h0m with 0 errors on Tue Aug 11 15:52:24 2009
config:
 
	NAME               STATE     READ WRITE CKSUM
	demo               ONLINE       0     0     0
	  mirror           ONLINE       0     0     0
	    /var/tmp/c0d0  ONLINE       0     0     0
	    /var/tmp/c1d0  ONLINE       0     0     0  83,5K resilvered

Note that the new “disk” c1d0 also had to be resilvered.

Now we add the “disk” c0d1 to the stripe, expanding the capacity of the pool.

pfexec zpool add demo /var/tmp/c0d1

invalid vdev specification
use '-f' to override the following errors:
mismatched replication level: pool uses mirror and new vdev is file

You have to force adding a vdev with a different redundancy. It's recommendend to have the same type of redundancy (none, mirror or raidz) for all vdevs in a pool. For this demo, we will ignore this recommendation.

pfexec zpool add -f demo /var/tmp/c0d1
zpool status

  pool: demo
 state: ONLINE
 scrub: resilver completed after 0h0m with 0 errors on Tue Aug 11 15:52:24 2009
config:
 
	NAME               STATE     READ WRITE CKSUM
	demo               ONLINE       0     0     0
	  mirror           ONLINE       0     0     0
	    /var/tmp/c0d0  ONLINE       0     0     0
	    /var/tmp/c1d0  ONLINE       0     0     0  83,5K resilvered
	  /var/tmp/c0d1    ONLINE       0     0     0

You see that the pool demo consists of a mirror (c0d0 and c1d0) and a single file c1d0.

We now attach a mirror to the second “disk”, which will bring us back to what we should have done in the step before.

pfexec zpool attach demo /var/tmp/c0d1 /var/tmp/c1d1 
zpool status

  pool: demo
 state: ONLINE
 scrub: resilver completed after 0h0m with 0 errors on Tue Aug 11 15:57:30 2009
config:
 
	NAME               STATE     READ WRITE CKSUM
	demo               ONLINE       0     0     0
	  mirror           ONLINE       0     0     0
	    /var/tmp/c0d0  ONLINE       0     0     0
	    /var/tmp/c1d0  ONLINE       0     0     0
	  mirror           ONLINE       0     0     0
	    /var/tmp/c0d1  ONLINE       0     0     0
	    /var/tmp/c1d1  ONLINE       0     0     0  18,5K resilvered

The new disk also had to be resilvered.

This time, we will add two mirrored “disks” right away in one step.

pfexec zpool add demo mirror /var/tmp/c0d2 /var/tmp/c1d2 
zpool status

  pool: demo
 state: ONLINE
 scrub: resilver completed after 0h0m with 0 errors on Tue Aug 11 15:57:30 2009
config:
 
	NAME               STATE     READ WRITE CKSUM
	demo               ONLINE       0     0     0
	  mirror           ONLINE       0     0     0
	    /var/tmp/c0d0  ONLINE       0     0     0
	    /var/tmp/c1d0  ONLINE       0     0     0
	  mirror           ONLINE       0     0     0
	    /var/tmp/c0d1  ONLINE       0     0     0
	    /var/tmp/c1d1  ONLINE       0     0     0  18,5K resilvered
	  mirror           ONLINE       0     0     0
	    /var/tmp/c0d2  ONLINE       0     0     0
	    /var/tmp/c1d2  ONLINE       0     0     0

It's a good practice to run a scrub from time to time. The scrub will read all blocks of the pool and check the checksums. Any errors will be reported an corrected (if there is enough redundancy).

pfexec zpool scrub demo
zpool status

  pool: demo
 state: ONLINE
 scrub: scrub completed after 0h0m with 0 errors on Tue Aug 11 16:00:19 2009
config:
 
	NAME               STATE     READ WRITE CKSUM
	demo               ONLINE       0     0     0
	  mirror           ONLINE       0     0     0
	    /var/tmp/c0d0  ONLINE       0     0     0
	    /var/tmp/c1d0  ONLINE       0     0     0
	  mirror           ONLINE       0     0     0
	    /var/tmp/c0d1  ONLINE       0     0     0
	    /var/tmp/c1d1  ONLINE       0     0     0
	  mirror           ONLINE       0     0     0
	    /var/tmp/c0d2  ONLINE       0     0     0
	    /var/tmp/c1d2  ONLINE       0     0     0

In this case, the scrub didn't find any errors, so nothig needed to be repaired.

For now, we are finished. We destroy the pool and remove the files. Note that zpool status doesn't show the pool any more.

pfexec zpool destroy demo
zpool status
rm c[01]d[012]

to be continued…