Saturday, March 13, 2010

Linux Partitions

Creating Partition and Filesystem in Linux

Introduction
This post describes how to create a partitons in linux and use it. Creating a new partition for use is a 4 step process.
  1. You identify a disk and create a partition using fdisk
  2. You create a file system on that disk and assign a label
  3. You create an entry in /etc/fstab to make partition persistant accrose reboot
  4. You mount the partition for access to the user.
Lets start with creating a partition from a disk.
Creating Partition and filesystem
In my system I have /dev/sda as primary device and following are the different partitions
[root@10.176.87.179]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda2             2.9G  350M  2.4G  13% /
/dev/sda7             487M   18M  444M   4% /home
/dev/sda6             965M   18M  898M   2% /data
/dev/sda3             2.9G  1.7G  1.1G  62% /usr
/dev/sda1              99M   11M   83M  12% /boot
tmpfs                 252M     0  252M   0% /dev/shm
/dev/hdc              2.8G  2.8G     0 100% /cdrom

The total size for all these partitions (except cdrom) comes to 7.5G. Addition to that I have 1G swap partition created on /dev/sda5
[root@10.176.87.179]# swapon -s
Filename                                Type            Size    Used    Priority
/dev/sda5                               partition       1020088 0       -1

So that makes it 8.5G. Total size of /dev/sda is 10G. This leaves 1.5G free for creating another partition.
1) create a partition using fdisk
fdisk -l will give the list of existing partition
[root@10.176.87.179]# fdisk -l
Disk /dev/sda: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          13      104391   83  Linux
/dev/sda2              14         395     3068415   83  Linux
/dev/sda3             396         777     3068415   83  Linux
/dev/sda4             778        1305     4241160    5  Extended
/dev/sda5             778         904     1020096   82  Linux swap / Solaris
/dev/sda6             905        1031     1020096   83  Linux
/dev/sda7            1032        1095      514048+  83  Linux

Disk /dev/sdb: 5368 MB, 5368709120 bytes
255 heads, 63 sectors/track, 652 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Disk /dev/sdb doesn’t contain a valid partition table
For creating a partition, we can use fdisk followed by device name.
[root@10.176.87.179]# fdisk /dev/sda
The number of cylinders for this disk is set to 1305.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)

Command (m for help): n
First cylinder (1096-1305, default 1096):
Using default value 1096
Last cylinder or +size or +sizeM or +sizeK (1096-1305, default 1305): +1G

Command (m for help): p
Disk /dev/sda: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1          13      104391   83  Linux
/dev/sda2              14         395     3068415   83  Linux
/dev/sda3             396         777     3068415   83  Linux
/dev/sda4             778        1305     4241160    5  Extended
/dev/sda5             778         904     1020096   82  Linux swap / Solaris
/dev/sda6             905        1031     1020096   83  Linux
/dev/sda7            1032        1095      514048+  83  Linux
/dev/sda8            1096        1218      987966   83  Linux

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table.
The new table will be used at the next reboot.
Syncing disks.

The above step creates a raw partition. Here it prompted for
Command (m for help):
and we entered “n”. “n” here means new partition. Then it asked for starting cylinder. By default it takes the cylinder in continuation. Else if we leave some cylinders in between that slot will be empty and we wont be able to use those unless if we want to create the partition of exact that size. So better to accept the default and create a partition in continuous cylinders.
Next input it ask is the end cylinder number. Usually its hard to calculate the number of cylinders depending on the size of slot we need. So we can directly enter the size of slot as +1G or +500M. Here G, M and K can be used representing GB, MB and KB. Remember to use + at the start.
Once these inputs are given, we can use “p” to print the partition list and see if its going to create correct partition. Please note that till now nothing has happened. We are just checking our setting by typing “p”. After checking when we type “w” that time its going to start creating a partition. So we can backoff any time using “quit” until we type “w”.
2) create a file system on that partition and assign a label
For creating a partition, following command is used.
mkfs.ext3
Following are the main and important options
-b -> This represent the block size to be used.
-i -> This represent the block/inode ratio.

inode is a pointer to each and every file in linux. For every file there is an inode. usually its not a good idea to give same number of inodes as number of blocks.
Example:
Partition size = 1000M
block size = 2K
Number of blocks = 500K (1000M/2K)
For each file created, however small it is, its going to use atleast 1 block. And for each file we need 1 inode value.
Now if there are 500K blocks, its not a good idea to give 500K as inode value. Because we are not going to have 500K files. If you thing about it usually in a normal file system some of the file will be larger then 2K (block size). In that case single file will occupy more number of blocks and but still number of inode used for that file will be only 1.
Another disadvantage of giving more number of inode values is that, it will take more space for storing those many values in inode table.
For more options on mkfs.ext3 command, see the man pages.
Lets try creating a file system.
[root@10.176.87.179]# mkfs.ext3 -b 2048 -i 4096 /dev/sda8
mke2fs 1.39 (29-May-2006)
Could not stat /dev/sda8 — No such file or directory

The device apparently does not exist; did you specify it correctly?
OK. Now we got this error. The reason we got this error is because, even though we created a partition, our kernel does not know about it.
When we create a partition, an entry goes in the partition table of that device. This partition table is maintained in the 1st sector of that device. During booting kernel reads the partition table and loads it in memory.
So does that mean that we need to reboot out system? Apperantly reboot can solve our problem, but we can solve the same without reboot as well. We have a command called partprob which will ask kernel to read the partition table on the device and load into memory. Doing that, kernel will know about new partition.
You can list the current partition which kernel is aware of using /proc/partitions file. /proc is a virtual file system in memory. This contains all the information which kernel is aware of and which is used by kernel.
[root@10.176.87.179]# cat /proc/partitions
major minor  #blocks  name

8     0   10485760 sda
8     1     104391 sda1
8     2    3068415 sda2
8     3    3068415 sda3
8     4          1 sda4
8     5    1020096 sda5
8     6    1020096 sda6
8     7     514048 sda7
8    16    5242880 sdb

So we can see here that partition sda8 is not loaded in memory. Now lets run partprobe command.
[root@10.176.87.179]# partprobe
Now if we see /proc/partitions we see sda8
[root@10.176.87.179]# cat /proc/partitions
major minor  #blocks  name

8     0   10485760 sda
8     1     104391 sda1
8     2    3068415 sda2
8     3    3068415 sda3
8     4          1 sda4
8     5    1020096 sda5
8     6    1020096 sda6
8     7     514048 sda7
8     8     987966 sda8
8    16    5242880 sdb

Now try the mkfs.ext3 command.
[root@10.176.87.179]# mkfs.ext3 -b 2048 -i 4096 -L /oracle /dev/sda8
mke2fs 1.39 (29-May-2006)
Filesystem label=
OS type: Linux
Block size=2048 (log=1)
Fragment size=2048 (log=1)
247008 inodes, 493982 blocks
24699 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=506462208
31 block groups
16384 blocks per group, 16384 fragments per group
7968 inodes per group
Superblock backups stored on blocks:
16384, 49152, 81920, 114688, 147456, 409600, 442368

Writing inode tables: done
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 30 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.

So it has created the required file system.
-L is used to give label to that partition. This is not a mandatory option.
3) Create an entry in /etc/fstab to make partition persistant accrose reboot
Now to make this partition get mounted automatically when the system reboots, we should make entry in /etc/fstab file. /etc/fstab is the file which kernel reads during booting and mount the file system mentioned in this file.
This file has entry in following format
                   
Device Name -> Name of the partition which needs to be mounted
Mount Point -> Directory which is to be used as moint point
File system Type -> Type used for creating file system. ext3 in our case.
Mount Option -> Various options used during mount. Check man page for mkfs command to check various mount options.
Dump Frequency -> 0 – never dump, 1 – daily, 2 – every other day etc. This is a sort of taking  backup.
File System Check order -> Order in which file system is checked while the system boots. 0 – ignore, 1 – always for root etc.
So here is our entry will look like
[root@10.176.87.179]# cat /etc/fstab | grep oracle
LABEL=/oracle           /oracle                 ext3    defaults        0 0

4) mount the partition for access to the user.
[root@10.176.87.179]# mkdir /oracle
[root@10.176.87.179]# mount -a
[root@10.176.87.179]# cd /oracle
[root@10.176.87.179]# df -h .

Filesystem            Size  Used Avail Use% Mounted on
/dev/sda8             935M   24M  863M   3% /oracle

mount -a is going to mount all the devices present in /etc/fstab file. You can also mount /dev/sda8 using following command
[root@10.176.87.179]# mount /dev/sda8 /oracle

hope this helps !!

Integrate Jenkins with Azure Key Vault

Jenkins has been one of the most used CI/CD tools. For every tool which we are using in our daily life, it becomes really challenges when ...