• Home
• Blog
• Projects
• How-to

• « previous
• next »

Opensolaris server with COMSTAR and zfs

Nov. 8, 2009, 6:49 p.m.

I've being playing around with Sun's Opensolaris for a while and I'm quite pleased with it to say at least. Opensolaris 2009.06 comes with few nifty features and software like dtrace, zfs, COMSTAR, etc.

I tried it a while ago as a desktop system, and was quite presently surprised with ZFS and its snapshots so I played little more with it. I do plan to make another post on this subject alone but mine main intent of this post will be to help you setup an Opensolaris server, without GUI who will use its, I would say best quality, zfs and other set of tools like COMSTAR to provide scalable, high performance, low budget, storage server.

Since I intend to use this post for future reference in some setups I will try to keep it straightforward and just explain basic setup for get the non gui Opensolaris up & running with COMSTAR and additional zfs pool with single simple iSCSI target.

I will be using a Opensolaris 2009.06 (you can download LiveCD image from this link), a simple low budget hardware with three sata hard drives.

Installing Opensolaris

After you downloaded Opensolaris iso image burn it to a CD and boot your computer with it. When the cd boots you should get a screen like this:

 
just press enter and proceed to the next step at which you will chose keyboard layout and system language.

 
After that you should find yourself in gnome graphical environment just like with Ubuntu LiveCD installation.

 
 

If for some reason you end up with user/password prompt the default username is:jack and password is:opensolaris

Anyways, double-click on Install Opensolaris icon located on the Desktop. Setup wizard will show up which will guide you trough the setup process.

On the Disk selection step I will chose my smallest disk from 20Gb to be used as system disk, and leave those other two disks intact at the moment. Whole installation will take up about 3Gb of storage so the 20gb disk should be ok for now.

 
On the next two steps you will have to chose your time and locale settings, after which you will define a root password, create a new user and setup a host name for your box.

 
Clicking the next button will lead you to the installation review screen, if all of the parameters are correct click on the install button. At this moment you can see there is no way to deselect the gnome, but we will get rid of that in a while.

After the installation is done remove your installation CD and reboot the system. When system is booted you will find yourself in Gnome graphical environment. We won't remove it just yet since we need to make our new opensolaris box available on the network. If you might have a dhcp server somewhere on the network opensolaris will pick up an IP address and configuration from it. I don't usually use dhcp and I do prefer to use static IP addresses. So let's get started with that first.

Configuring static IP address

Since I'm doing a local lab installation my Ip addresses are non Internet routable, replace them with IP addresses suitable for your needs.

Login to your desktop as normal user, open up your terminal and follow these steps:

su -

enter the root password when asked.

svcadm disable network/physical:nwam
svcadm enable network/physical:default

Now let's find out our network interface names:

dladm show-phys

 
As you may see I have interface named e1000g0 so the following set of commands would be (adjust your comands for your interface and ip addresses):

echo 192.168.2.200 > /etc/hostname.e1000g0
echo opensolaris > /etc/nodename

Next we need to edit the hosts file:

nano /etc/hosts

Make it look like this:

192.168.2.200 opensolaris opensolaris.local loghost
::1 localhost
127.0.0.1 localhost

now edit a netmasks file:

nano /etc/netmasks

and append the line

192.168.2.0 255.255.255.0

Now run this:

ifconfig e1000g0 plumb
ifconfig e1000g0 192.168.2.200 netmask 255.255.255.0 up

Now, if you would like to use a DNS for name resolving you must configure resolv.conf file

nano /etc/resolv.conf

it should look something like this:

domain localdomain.com
       nameserver 192.168.2.1
       nameserver 192.168.2.2

now run:

cp /etc/nsswitch.dns /etc/nsswitch.conf

All we need to do is configure a default route:

echo 192.168.2.1 >> /etc/defaultrouter
route add default 192.168.2.1

Now reboot your server and verify the changes are persistent.

Removing unnecessary packages

Your server should be available trough ssh so use it for further configuration. To remove gui from our opensolaris installation we will be using minimization script bundled with http://kenai.com/projects/isc/pages/OpenSolaris To do so we would need mercurial so let's install it with:

pfexec pkg install SUNWmercurial

Next we will get the script with command:

hg clone ssh://anon@hg.opensolaris.org//hg/isc/src  isc

after script checkout run it with:

pfexec isc/opt/samples/minimization.ksh

Make sure you don't execute this command from graphical interface since it will be removed. As the matter of fact script will remove all non essential packages and disable service not needed for running opensolaris as a server.

Installing and configuring COMSTAR

By now we should have a clean, fresh, and minimal opensolaris box, now it's time to add some functionality to it.

To install it just enter:

pfexec pkg install storage-server

This will install all the required packages and services. First thing to do is enabling stmf service. By default the service is off, to enable it type:

pfexec svcadm enable stmf

now runing:

svcs stmf

should say the service is in maintenance mode. This is due to drivers not being loaded to kernel immediately after install, so we need to reboot the server in order to get this service running (still searching a way to circumvent the reboot process).

After reboot is complete verify that the service is running:

branko@opensolaris:~$ svcs stmf
STATE          STIME    FMRI
online         17:07:54 svc:/system/stmf:default
branko@opensolaris:~$ pfexec stmfadm list-state
Operational Status: online
Config Status     : initialized

Configuring zfs pool

Now for the rest of this post we will setup additional zfs pool, and configure one basic iscsi target on it.

Remember our two unused disks at the install? Well let's make another zfs pool on them. If you don't have additional disks, you can skip this step and configure iscsi target on already existing zfs pool.

Let's find out which disk we are already using:

su -
zpool status

You should get something like:

  pool: rpool
 state: ONLINE
 scrub: none requested
config:

NAME        STATE     READ WRITE CKSUM
        rpool       ONLINE       0     0     0
          c8t0d0s0  ONLINE       0     0     0

errors: No known data errors

Note the disk name already in use is c8t0d0s0. Now to find out what other disks are present on the system let's use format command:

format

Now program will launch interactive shell and should display something like this:

Searching for disks...done

AVAILABLE DISK SELECTIONS:
       0. c8t0d0 
          /pci@0,0/pci1000,8000@14/sd@0,0
       1. c8t1d0 
          /pci@0,0/pci1000,8000@14/sd@1,0
       2. c8t2d0 
          /pci@0,0/pci1000,8000@14/sd@2,0
Specify disk (enter its number):

If your disk doesn't show up try entering

devfsadm

prior to format command.

Hit the CTRL+C key combination since we actualy don't want to format any disks. Now we found additional two disks by lables c8t1d0 and c8t2d0

To make new zfs pool with them we will enter following command:

zpool create data mirror c8t1d0 c8t2d0

where "data" is the name of the new pool, mirror is type and c8t1d0 and c8t2d0 are devices for this pool.

Let's verify the pool existance:

zpool list

Should print out:

NAME    SIZE   USED  AVAIL    CAP  HEALTH  ALTROOT
data   99.5G    76K  99.5G     0%  ONLINE  -
rpool  19.9G  4.43G  15.4G    22%  ONLINE  -

or

zpool status data
pool: data
 state: ONLINE
 scrub: none requested
config:
    NAME        STATE     READ WRITE CKSUM
    data        ONLINE       0     0     0
      mirror    ONLINE       0     0     0
        c8t1d0  ONLINE       0     0     0
        c8t2d0  ONLINE       0     0     0

errors: No known data errors

And now we have a brand new zfs pool for storing iscsi targets.

Creating iscsi target

Let's start with creating 10Gb zvol which we will export as iscsi target. We will use -s flag for sparse so we can thin provision our targets.

zfs create -s -V 10g data/iscsitarget

Now let's see the volume:

# zfs list data/iscsitarget
NAME               USED  AVAIL  REFER  MOUNTPOINT
data/iscsitarget    16K  97.9G    16K  -

Now, let's create a logical unit using this volume:

sbdadm create-lu /dev/zvol/rdsk/data/iscsitarget

It will return:

Created the following LU:

GUID                    DATA SIZE           SOURCE
--------------------------------  -------------------  ----------------
600144f01cdf4f0000004af6f28b0001      10737352704      /dev/zvol/rdsk/data/iscsitarget

Verify the creation with:

sbdadm list-lu

To make a logical unit available to all hosts type:

stmfadm add-view 600144f01cdf4f0000004af6f28b0001

where 600144f01cdf4f0000004af6f28b0001 is your lu GUID

Enabling iscsit service

first we need to install SUNWiscsit

pkg install SUNWiscsit

Then check service states:

# svcs -a | grep -i iscsi
disabled       17:07:06 svc:/network/iscsi_initiator:default
disabled       18:07:47 svc:/network/iscsi/target:default
online         17:40:45 svc:/system/iscsitgt:default

Disable iscsitgt

svcadm disable iscsitgt

and enable the iscsi target service

svcadm enable -r svc:/network/iscsi/target:default

Make sure it's started:

svcs -a | grep iscsi

if it ends up in maintenance mode, reboot your machine and check again.

If the output is something like:

online  14:21:25 svc:/network/iscsi/target:default

Creating a target for discovery

If the service is started create a target with:

itadm create-target

Verify target creation:

itadm list-target

and link iscsi devices to local system:

devfsadm -i iscsi

you should now see this logical volume on your iscsi initiators.

And that would be all for now. In some future posts I will describe more usage examples based on this setup. Until next time...

• « previous
• next »

Commenting is disabled