====== Online resize on SAN DS4700 ======

<note warning>I tried this on my OCFS2 partition, but it didn't work well (indeed it didn't work at all). I have Xen domU with image file running on the top of OCFS2. It might be because of Xen, so I'll do some test without Xen running on OCFS2.</note>

When working on a [[wp>Storage_Area_Network|SAN]], like IBM DS4700, you don't want to add up layer on the operating system running on it. I mean that all the functionalities provided by a tools like LVM2 are already provided by the SAN. Also no partitioning (with ''fdisk'') is made on a Logical Drive, it just add a layer of complexity. The main idea is :

  * Create a logical drive on the SAN
  * Map it to your server
  * Create the filesystem directly on the whole Logical Drive.

I personally use [[wp>xfs|XFS]] which seems to me as a really good file system. I do not have benchmarking to provide, but I'm using it for over a year now, and it has nice functionalities, like freeze and grow. I'd like to see [[wp>zfs|ZFS]] on a SAN, but I didn't have the time yet to setup a [[wp>opensolaris|OpenSolaris]].

Now the problem with this is when you want to resize it online. On the [[wp>Storage_Area_Network|SAN]] side it's easy, right click on the Logical Drive and select "Increase", or something like that. You can still use the command line tools provided if you want. Now that your drive is bigger, how do we do to make "multipath" and all the underlying system knows about that.

===== Test procedure =====

To test online resizing I needed a simple stuff that would keep on writing to my Logical Drive while I was resizing my drive. So I did just this little command line (the case is that my drive is mounted as ''/mnt/tmp2'').
<cli prompt="# ">root@debian:/mnt# while [ 1 ]; do echo `date` >> /mnt/tmp2/test ; sleep 1 ; done</cli>
This command keeps adding the date to a file every second. So I could see if some data get lost during operation.

===== Before starting =====

The resizing on the SAN is already done, this is not a problem, so here is the state before operating the resizing (commented):
<cli prompt="# " comments="# ">
root@debian:/mnt/tmp2# date
Wed May  7 17:29:16 CEST 2008 # Start date
root@debian:/mnt/tmp2# multipath -l mpath4 # Logical drive to be resized
mpath4 (3600a0b80002684aa00000e49481ffd4b) dm-3 IBM,1814      FAStT
[size=7.0G][features=1 queue_if_no_path][hwhandler=0]
\_ round-robin 0 [prio=0][enabled]
 \_ 0:0:0:3 sdd 8:48  [active][undef] 
\_ round-robin 0 [prio=0][active]
 \_ 1:0:0:3 sdh 8:112 [active][undef] 
root@debian:/mnt/tmp2# mount # I cut off part not about mpath4
/dev/mapper/mpath4 on /mnt/tmp2 type xfs (rw)
root@debian:/mnt/tmp2# df -h # Same as for mount
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/mpath4    4.0G  260M  3.8G   7% /mnt/tmp2
</cli>

===== Let's resize =====

Ok now we go resize our drive. The idea is to remove is drive from the multipath and add it again. This could be done because we have this multipath stuff. During this operation, if the second path is failing, you will lose your drive, so this operation must be done while your SAN is not experiencing any problem and when both path are fully working.

==== Remove and add first path ====

We remove and add our first path :
<cli prompt="# " comments="# ">
root@debian:/mnt/tmp2# echo "scsi remove-single-device 0 0 0 3" > /proc/scsi/scsi # We remove the first path (0:0:0:3)
root@debian:/mnt/tmp2# echo "scsi add-single-device 0 0 0 3" > /proc/scsi/scsi # And add it again
</cli>
 
Sadly I forgot to run ''multipath -ll mpath4'' to show that the system was running on one path!

==== Reload multipath ====

This operation **must** be done. If not, you'll get ''I/O Error'' and you'll need to remount your drive :
<cli prompt="# " comments="# ">
root@debian:/mnt/tmp2# multipath # Reload multipath
sdd: checker msg is "tur checker reports path is down"
sdh: checker msg is "tur checker reports path is down"
8:112: size 14680064, expected 16777216. Discard
</cli>

We see that multipath check each path (report them as down, don't know why as they're not) and see that the size has changed, but did not take it into account. This is normal, the second disk has not been reloaded. But now we're running on both path again.

<cli prompt="# " comments="# ">
root@debian:/mnt/tmp2# multipath -l mpath4
mpath4 (3600a0b80002684aa00000e49481ffd4b) dm-3 IBM,1814      FAStT
[size=7.0G][features=1 queue_if_no_path][hwhandler=0]
\_ round-robin 0 [prio=0][enabled]
 \_ 1:0:0:3 sdh 8:112 [active][undef] 
\_ round-robin 0 [prio=0][enabled]
 \_ 0:0:0:3 sdd 8:48  [active][undef] 
</cli>

==== Remove and add second path ====

This is done the same way as the first path :
<cli prompt="# " comments="# ">
root@debian:/mnt/tmp2# echo "scsi remove-single-device 1 0 0 3" > /proc/scsi/scsi 
root@debian:/mnt/tmp2# echo "scsi add-single-device 1 0 0 3" > /proc/scsi/scsi 
</cli>

==== Reload multipath, again ====

We reload multipath and, ooh ! 
<cli prompt="# " comments="# ">
root@debian:/mnt/tmp2# multipath
sdd: checker msg is "tur checker reports path is down"
sdh: checker msg is "tur checker reports path is down"
reload: mpath4 (3600a0b80002684aa00000e49481ffd4b)  IBM,1814      FAStT
[size=8.0G][features=1 queue_if_no_path][hwhandler=0]
\_ round-robin 0 [prio=3][undef]
 \_ 0:0:0:3 sdd 8:48  [active][ready] 
\_ round-robin 0 [prio=4][undef]
 \_ 1:0:0:3 sdh 8:112 [undef][ready] 
</cli>

A new message came : ''reload: mpath4 (3600a0b80002684aa00000e49481ffd4b)  IBM,1814      FAStT'' instead of ''Discard'' ! And if we look at the size of mpath4, it's now **8.0G** !!! Our resizing seems to have worked !!!

===== After resizing =====

Now we want to be sure that everything is fine, first we resize our file system :

<cli prompt="# " comment="# ">
root@debian:/mnt/tmp2# cd ..
root@debian:/mnt# xfs_growfs tmp2/ # Resize with XFS
meta-data=/dev/mapper/mpath4     isize=256    agcount=11, agsize=98304 blks
         =                       sectsz=512   attr=0
data     =                       bsize=4096   blocks=1048576, imaxpct=25
         =                       sunit=0      swidth=0 blks, unwritten=1
naming   =version 2              bsize=4096  
log      =internal               bsize=4096   blocks=2560, version=1
         =                       sectsz=512   sunit=0 blks
realtime =none                   extsz=65536  blocks=0, rtextents=0
data blocks changed from 1048576 to 2097152
</cli>

Check if it has work :

<cli prompt="# " comments="# ">
root@debian:/mnt# mount # See actual mounts (cut off)
/dev/mapper/mpath4 on /mnt/tmp2 type xfs (rw)
root@debian:/mnt# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/mpath4    8.0G  260M  7.8G   4% /mnt/tmp2
root@debian:/mnt# multipath -l mpath4
mpath4 (3600a0b80002684aa00000e49481ffd4b) dm-3 IBM,1814      FAStT
[size=8.0G][features=1 queue_if_no_path][hwhandler=0]
\_ round-robin 0 [prio=0][enabled]
 \_ 0:0:0:3 sdd 8:48  [active][undef] 
\_ round-robin 0 [prio=0][active]
 \_ 1:0:0:3 sdh 8:112 [active][undef] 
</cli>

And the date we finished :

<cli prompt="# " comments="# ">
root@debian:/mnt# date
Wed May  7 17:32:27 CEST 2008
</cli>

So it took 3 minutes to resize our drive, but do we lost something ?

===== Control =====

We started a simple little script that write data onto the disk during our resizing, so here is the content of the file generated :
<code>
Wed May 7 17:29:14 CEST 2008
Wed May 7 17:29:15 CEST 2008
Wed May 7 17:29:16 CEST 2008
Wed May 7 17:29:17 CEST 2008
Wed May 7 17:29:18 CEST 2008
Wed May 7 17:29:19 CEST 2008
Wed May 7 17:29:20 CEST 2008
Wed May 7 17:29:21 CEST 2008
Wed May 7 17:29:22 CEST 2008
Wed May 7 17:29:23 CEST 2008
Wed May 7 17:29:24 CEST 2008
Wed May 7 17:29:25 CEST 2008
Wed May 7 17:29:26 CEST 2008
Wed May 7 17:29:27 CEST 2008
Wed May 7 17:29:28 CEST 2008
Wed May 7 17:29:29 CEST 2008
Wed May 7 17:29:30 CEST 2008
Wed May 7 17:29:31 CEST 2008
Wed May 7 17:29:32 CEST 2008
Wed May 7 17:29:33 CEST 2008
Wed May 7 17:29:34 CEST 2008
Wed May 7 17:29:35 CEST 2008
Wed May 7 17:29:36 CEST 2008
Wed May 7 17:29:37 CEST 2008
Wed May 7 17:29:38 CEST 2008
Wed May 7 17:29:39 CEST 2008
Wed May 7 17:29:40 CEST 2008
Wed May 7 17:29:41 CEST 2008
Wed May 7 17:29:42 CEST 2008
Wed May 7 17:29:43 CEST 2008
Wed May 7 17:29:44 CEST 2008
Wed May 7 17:29:45 CEST 2008
Wed May 7 17:29:46 CEST 2008
Wed May 7 17:29:47 CEST 2008
Wed May 7 17:29:48 CEST 2008
Wed May 7 17:29:49 CEST 2008
Wed May 7 17:29:50 CEST 2008
Wed May 7 17:29:51 CEST 2008
Wed May 7 17:29:52 CEST 2008
Wed May 7 17:29:53 CEST 2008
Wed May 7 17:29:54 CEST 2008
Wed May 7 17:29:55 CEST 2008
Wed May 7 17:29:56 CEST 2008
Wed May 7 17:29:57 CEST 2008
Wed May 7 17:29:58 CEST 2008
Wed May 7 17:29:59 CEST 2008
Wed May 7 17:30:00 CEST 2008
Wed May 7 17:30:01 CEST 2008
Wed May 7 17:30:02 CEST 2008
Wed May 7 17:30:03 CEST 2008
Wed May 7 17:30:04 CEST 2008
Wed May 7 17:30:05 CEST 2008
Wed May 7 17:30:06 CEST 2008
Wed May 7 17:30:07 CEST 2008
Wed May 7 17:30:08 CEST 2008
Wed May 7 17:30:09 CEST 2008
Wed May 7 17:30:10 CEST 2008
Wed May 7 17:30:11 CEST 2008
Wed May 7 17:30:12 CEST 2008
Wed May 7 17:30:13 CEST 2008
Wed May 7 17:30:14 CEST 2008
Wed May 7 17:30:15 CEST 2008
Wed May 7 17:30:16 CEST 2008
Wed May 7 17:30:17 CEST 2008
Wed May 7 17:30:18 CEST 2008
Wed May 7 17:30:19 CEST 2008
Wed May 7 17:30:20 CEST 2008
Wed May 7 17:30:21 CEST 2008
Wed May 7 17:30:22 CEST 2008
Wed May 7 17:30:23 CEST 2008
Wed May 7 17:30:24 CEST 2008
Wed May 7 17:30:26 CEST 2008
Wed May 7 17:30:27 CEST 2008
Wed May 7 17:30:28 CEST 2008
Wed May 7 17:30:29 CEST 2008
Wed May 7 17:30:30 CEST 2008
Wed May 7 17:30:31 CEST 2008
Wed May 7 17:30:32 CEST 2008
Wed May 7 17:30:33 CEST 2008
Wed May 7 17:30:34 CEST 2008
Wed May 7 17:30:35 CEST 2008
Wed May 7 17:30:36 CEST 2008
Wed May 7 17:30:37 CEST 2008
Wed May 7 17:30:38 CEST 2008
Wed May 7 17:30:39 CEST 2008
Wed May 7 17:30:40 CEST 2008
Wed May 7 17:30:41 CEST 2008
Wed May 7 17:30:42 CEST 2008
Wed May 7 17:30:43 CEST 2008
Wed May 7 17:30:44 CEST 2008
Wed May 7 17:30:45 CEST 2008
Wed May 7 17:30:46 CEST 2008
Wed May 7 17:30:51 CEST 2008
Wed May 7 17:30:52 CEST 2008
Wed May 7 17:30:53 CEST 2008
Wed May 7 17:30:54 CEST 2008
Wed May 7 17:30:55 CEST 2008
Wed May 7 17:30:56 CEST 2008
Wed May 7 17:30:57 CEST 2008
Wed May 7 17:30:58 CEST 2008
Wed May 7 17:30:59 CEST 2008
Wed May 7 17:31:00 CEST 2008
Wed May 7 17:31:01 CEST 2008
Wed May 7 17:31:02 CEST 2008
Wed May 7 17:31:03 CEST 2008
Wed May 7 17:31:04 CEST 2008
Wed May 7 17:31:05 CEST 2008
Wed May 7 17:31:06 CEST 2008
Wed May 7 17:31:07 CEST 2008
Wed May 7 17:31:08 CEST 2008
Wed May 7 17:31:09 CEST 2008
Wed May 7 17:31:10 CEST 2008
Wed May 7 17:31:11 CEST 2008
Wed May 7 17:31:12 CEST 2008
Wed May 7 17:31:13 CEST 2008
Wed May 7 17:31:14 CEST 2008
Wed May 7 17:31:15 CEST 2008
Wed May 7 17:31:16 CEST 2008
Wed May 7 17:31:17 CEST 2008
Wed May 7 17:31:18 CEST 2008
Wed May 7 17:31:19 CEST 2008
Wed May 7 17:31:20 CEST 2008
Wed May 7 17:31:21 CEST 2008
Wed May 7 17:31:22 CEST 2008
Wed May 7 17:31:24 CEST 2008
Wed May 7 17:31:25 CEST 2008
Wed May 7 17:31:26 CEST 2008
Wed May 7 17:31:27 CEST 2008
Wed May 7 17:31:28 CEST 2008
Wed May 7 17:31:32 CEST 2008
Wed May 7 17:31:33 CEST 2008
Wed May 7 17:31:34 CEST 2008
Wed May 7 17:31:35 CEST 2008
Wed May 7 17:31:36 CEST 2008
Wed May 7 17:31:37 CEST 2008
Wed May 7 17:31:38 CEST 2008
Wed May 7 17:31:39 CEST 2008
Wed May 7 17:31:40 CEST 2008
Wed May 7 17:31:41 CEST 2008
Wed May 7 17:31:42 CEST 2008
Wed May 7 17:31:43 CEST 2008
Wed May 7 17:31:44 CEST 2008
Wed May 7 17:31:45 CEST 2008
Wed May 7 17:31:46 CEST 2008
Wed May 7 17:31:52 CEST 2008
Wed May 7 17:31:55 CEST 2008
Wed May 7 17:31:56 CEST 2008
Wed May 7 17:31:57 CEST 2008
Wed May 7 17:31:58 CEST 2008
Wed May 7 17:31:59 CEST 2008
Wed May 7 17:32:00 CEST 2008
Wed May 7 17:32:02 CEST 2008
Wed May 7 17:32:03 CEST 2008
Wed May 7 17:32:04 CEST 2008
Wed May 7 17:32:05 CEST 2008
Wed May 7 17:32:06 CEST 2008
Wed May 7 17:32:07 CEST 2008
Wed May 7 17:32:08 CEST 2008
Wed May 7 17:32:09 CEST 2008
Wed May 7 17:32:10 CEST 2008
Wed May 7 17:32:11 CEST 2008
Wed May 7 17:32:12 CEST 2008
Wed May 7 17:32:13 CEST 2008
Wed May 7 17:32:14 CEST 2008
Wed May 7 17:32:15 CEST 2008
Wed May 7 17:32:16 CEST 2008
Wed May 7 17:32:17 CEST 2008
Wed May 7 17:32:18 CEST 2008
Wed May 7 17:32:19 CEST 2008
Wed May 7 17:32:20 CEST 2008
Wed May 7 17:32:21 CEST 2008
Wed May 7 17:32:22 CEST 2008
Wed May 7 17:32:23 CEST 2008
Wed May 7 17:32:24 CEST 2008
Wed May 7 17:32:25 CEST 2008
Wed May 7 17:32:26 CEST 2008
Wed May 7 17:32:27 CEST 2008
Wed May 7 17:32:28 CEST 2008
Wed May 7 17:32:29 CEST 2008
Wed May 7 17:32:30 CEST 2008
Wed May 7 17:32:31 CEST 2008
</code>

Our start date and end date are inside and there's no missing second in it, so we did not lost any data. Our online resizing was done well.