Dealing with Disks


Hard Disks (IDE)

IDE hard disk controllers can control 2 disks, a master and a slave. Usually, you'll find at least 1, usually 2 controllers on the motherboard, so that gives 4 disks (unless you add additional controllers). Linux assigns names as follows :

	Description			IDE controller:channel		Linux name
	-----------			----------------------		----------
	primary controller, master disk		0:0			/dev/hda
	primary controller, slave disk		0:1			/dev/hdb
	secondary controller, master disk	1:0			/dev/hdc
	secondary controller, slave disk	1:1			/dev/hdd
	

each disk in turn can hold 1 or more partitions. These are named by adding numbers to the disk name, eg 4 primary partitions : /dev/hda1 - /dev/hda2 - /dev/hda3 - /dev/hda4

In Linux, these partitions are mounted to directories. /dev/hda/ will often be used for / ("root of the filesystem"), while eg /dev/hdb

Moving or Replacing disks

It's obvious that moving a disk, eg from 1:0 to 0:0, will change its name. Yet it is quite easy to do that, and still have access to all data, with minimal changes to your system configuration.

recipe : moving some disks

- move cdrom from secundary master to secundary slave
- move disk from primary slave to secundary master

  1. shut down
  2. move/swap the disks. Check the jumpers for Master/Slave, Single, ..
  3. reboot

While you reboot, you'll get an error about "/dev/hdb not found", because the directories and files that the system expects to find there are now on /dev/hdc (secundary master). Continue to boot or enter maintenance mode. Edit /etc/fstab to replace /dev/hdb1 with /dev/hdc1, and /dev/hdc with /dev/hdd (cdrom).

now reboot (or just execute mount -a . See how all your files are still where they used to be, eg. how your home directory on the disk you moved, is still in /home/username and still contains all your files, as before. The filesystem (paths, filenames) is independent of the underlying disk and partition layout. Great !.

One small thing to fix : /dev/cdrom is a link to /dev/hdc. It now needs to be replaced by link to /dev/hdd

	test# rm /dev/cdrom
	test# ln -s /dev/hdd /dev/cdrom

	test# ls -al /dev/cdrom
	lrwxrwxrwx 1 root root 8 2007-10-01 12:37 /dev/cdrom -> /dev/hdd

	test# ls -al /dev/hdd
	brw-rw---- 1 root disk 22, 64 2005-02-26 07:38 /dev/hdd
	

recipe : Replace a disk but keep its data

case : you have a /home on a separate disk, but this disk has become to small and you want to replace it with a larger disk, but without losing the settings and data in your home directories.

  1. add a new disk, leave the old disks where they are. throw out the cd player if necessary
  2. create a partition and a filesystem on the disk (see further)
  3. mount it somewhere (eg to /mnt/newdisk)
  4. copy your files (eg cp -R -p /home/* /mount/newdisk : copy, including subdirectories, and preserve modes, ownership and timestamps )
  5. shutdown
  6. remove old disk, put new disk in its final location,
  7. boot
  8. correct /etc/fstab if necessary (probably not)

recipe : Moving subdirectories to a new disk

case : you have your entire directory tree (/) on 1 disk, and you're running out of disk space. You are planning to solve this by moving a top level directory (/e.g. /usr or /home) to a 2nd (newly added) disk.

  1. add a new disk
  2. create a partition and a filesystem on the disk (see further)
  3. mount it somewhere (eg to /mnt/newdisk)
  4. copy your files (eg cp -R -p /home/* /mount/newdisk
  5. unmount /mount/newdisk
  6. unmount /home. You need to be root (with homedir = /root) so you don't get affected by /home disappearing temporarily
  7. mount the new disk (partition) to /home
  8. check that everything's fine
  9. correct /etc/fstab so that the new disk is mounted to /home automatically when booting

note that the old disk will still contain (the original) files of the directory you moved, but that they are inaccessible because they are no longer part of your directory tree. You can avoid that by moving the files in stead of copying them to the new disk. Downside :

  1. you can't roll back
  2. moving files probably won't work if you're trying to move a system directory such as /usr, because it contains the binaries of the system you're running

Workaround : use a live CD and mount the disk (eg to /media/hda1), and remove the files from there.

recipe : adding disks for additional space

Say you have your /home on /dev/hdb1, but the partition is getting to small for your music collection. You get an additional lots-of-gigabytes disk, which will become /dev/hdd on your system. Now you'd like to have all that space inside your home directory. You can achieve that by mounting the disk to /home/joerandom/music

  1. add a disk (mind jumpers + location, eg primary slave = IDE 0:1 = hdb
  2. boot
  3. check presence of disks with fdisk:
    test:~# fdisk -l
    
    Disk /dev/hda: 1073 MB, 1073741824 bytes
    255 heads, 63 sectors/track, 130 cylinders
    Units = cylinders of 16065 * 512 = 8225280 bytes
    
       Device Boot      Start         End      Blocks   Id  System
    /dev/hda1               1          98      787153+  83  Linux
    /dev/hda2              99         130      257040   82  Linux swap / Solaris
    
    Disk /dev/hdb: 1073 MB, 1073741824 bytes
    16 heads, 63 sectors/track, 2080 cylinders
    Units = cylinders of 1008 * 512 = 516096 bytes
    
    Disk /dev/hdb doesn't contain a valid partition table
    
    Disk /dev/hdc: 1073 MB, 1073741824 bytes
    255 heads, 63 sectors/track, 130 cylinders
    Units = cylinders of 16065 * 512 = 8225280 bytes
    
       Device Boot      Start         End      Blocks   Id  System
    /dev/hdc1               1          65      522081   83  Linux
    
    

Note : "Disk /dev/hdb doesn't contain a valid partition table". So you have to create a partition, using fdisk as follows :

	fdisk /dev/hdb
	Command (m for help): m
		...
		n	add new partition
		o	create partition table
		w	write
		...
	Command: o
	Command : n

	Command Action
		e extended
		p primary
	p

	partition number '1-4):  1
	First Cylinder : default
	Last Cylinder : default

	command : w
	[Command : quit]	exit without saving
	

You have now created 1 partition, the size of the entire disk. Next, you need to format it (= add a filesystem)

checking that the partition is really there :

	lvmtest:~# fdisk -l /dev/hdb

		Disk /dev/hdb: 1073 MB, 1073741824 bytes
		16 heads, 63 sectors/track, 2080 cylinders
		Units = cylinders of 1008 * 512 = 516096 bytes
		
		Device Boot      Start         End      Blocks   Id  System
		/dev/hdb1               1        2080     1048288+  83  Linux
	

Note that the disk is called /dev/hdb, the partition on it /dev/hdb1. Now, create a filesystem (eg ext2 or ext3) in that partition

		# format the partition (ext 2) : 
		mkfs -t ext2 /dev/hdb1
	

And finally use it as /home/joerandom/music

	# create the directory
	mkdir /home/joerandom/music

	# modify /etc/fstab with a text editor, or like so :
	echo "/dev/hdb1		/home/joerandom/music	ext2 	defaults 0 2" >> /etc/fstab

	# mount the partition
	mount -a

	#check with 'mount' or 'df -h'

	lvmtest:~# mount
	/dev/hda1 on / type ext2 (rw,errors=remount-ro)
	/dev/hdc1 on /home type ext2 (rw)
	/dev/hdb1 on /home/joerandom/music type ext2 (rw)


	lvmtest:~# df -h
	Filesystem            Size  Used Avail Use% Mounted on
	/dev/hda1             721M  247M  436M  37% /
	/dev/hdc1             478M   18K  453M   1% /home
	/dev/hdb1            1008M  152M  805M  16% /home/joerandom/music
	

The entire new disk is now a subdirectory of /home/joerandom/
You can do something similar to add storage space to the /srv directory of a file server (or whatever), as shown here

replace system disk (/)

in stead of using hdb for extra space, we're going to replace the disk that holds the operating system (eg you're getting "imminent drive failure" warnings on the disk that holds your operating system and you want to replace it without having to re-install the operating system.

procedure : copy entire disk to new disk

You could follow the 'replace a disk' procedure mentioned earlier, but that would only allow you to copy your system's files. You also need to copy the MBR (Master Boot Record - not a part of the filesystem). Workaround : You can fix the MBR later (using a rescue CD, grub-update ...). It's a bit tricky but it may work. If parts of the system (/boot /usr /bin ... ) are on separate disks/partitions, you need to think this through. cp -R / will include these subdirectories so you'll have them both on their old partition and as a copy in a subdir of / on the new disk. This gets messy.

Fortunately , there's a better way - you can clone an entire disk with 1 simple command. It also works perfectly the new disk is larger than the old one ...

  1. attach new disk, eg
    - primary master = old disk with / (/dev/hda)
    -primary slave = new disk, no partition table, unformatted (no file system) (/dev/hdb)
  2. reboot. ignore warning about no filesystem on hdb - we know that.
  3. fdisk -l again says "Disk /dev/hdb doesn't contain a valid partition table"
  4. mirror hda to hdb
    dd if=/dev/hda of=/dev/hdb
  5. see what gives :
    			lvmtest:~# fdisk -l /dev/hdb
    
    			Disk /dev/hdb: 1932 MB, 1932734976 bytes
    			255 heads, 63 sectors/track, 234 cylinders
    			Units = cylinders of 16065 * 512 = 8225280 bytes
    
    			   Device Boot      Start         End      Blocks   Id  System
    			/dev/hdb1               1          98      787153+  83  Linux
    			/dev/hdb2              99         130      257040   82  Linux swap / Solaris
    			
    the new disk is now an exact copy of hda, partitions (swap) and all.
  6. shutdown
  7. remove old disk
  8. put new disk were old disk used to be (primary master, 0:0) so it becomes hda
  9. reboot
  10. rejoice, for lo and behold, your system boots off its new primary master disk as if nothing ever happened.

Because the new disk is bigger than the old disk, you seem to have lost some space - where to find it ? use fdisk :


	test:~#fdisk /dev/hda

	command: v	#verify
	835380 unallocated blocks

	## make a new partition (n) , primary (p) , partition#3 (3), write new partition table (w) 
	##		--> exit and reboot

	Command: n
	p
	3
	Start:default
	End:default

	command:w
	

after reboot, proceed as before : check partition table (fdisk -l), format the partition (mkfs), make a mount point for it and update /etc/fstab to mount the partition at boot. You now have

		lvmtest:~# fdisk -l /dev/hda

		Disk /dev/hda: 1932 MB, 1932734976 bytes
		255 heads, 63 sectors/track, 234 cylinders
		Units = cylinders of 16065 * 512 = 8225280 bytes

		   Device Boot      Start         End      Blocks   Id  System
		/dev/hda1               1          98      787153+  83  Linux
		/dev/hda2              99         130      257040   82  Linux swap / Solaris
		/dev/hda3             131         234      835380   83  Linux
	

And done. You've replaced the disk and the system hardly noticed it.

remark: SATA DISKS

SATA disks behave as normal ATA (IDE) disks, but because they have some scsi-like features, linux uses a scsi driver to approach them, and therefore the system represents them as sda, sdb, sdc, sdd, and likewise for partitions, eg sda1, ...

SCSI Disks

Where IDE controllers can have max 2 disks attached (master, slave), scsi controllers support up to 8 (or 16) disks without hierarchy (all equal, no masters or slaves). The disks get a scsi ID (configurable with scsi adapter config tools). Once the scsi controller is configured, disks appear in the OS as sda, sdb, etc.

You can swap them around and in and out as described before, once you've figured out how the scsi numbering on the adapter works. :-)

wikipedia on SCSI

RAID controllers

In a hardware raid, you attach multiple (scsi or ide, depends on the type of adapter) disks to a raid controller. The (hardware) raid controller creates an 'array' spanning multiple disks (usually with provisions for redundancy, better read performance, better write performance, ...). The sum of available space is then divided in logical units that present themselves to the operating system as disks.

The OS is unaware of the underlying array('s), it just sees 'disks' - but you can't play with it at OS level : the blocks that make up /dev/hda can be spread out over multiple physical disks, and this is known only to the raid adepter. So replacing or swapping disks can only be managed through raid adapter configuration tools. On the bright side : in all raids (except RAID-0) you can replace at least 1 disk, and the array will rebuild itself. Again, this is managed at the raid controller, not in the operating system (compare with editing BIOS settings)

wikipedia on RAID


Koen Noens
October 2007