May 132011

Well, it’s been a while since I wrote a new blogpost mostly due to the lack of time. If I look back at the last 6 months or so there where some heavily private issues, but also a lot of studying into new products. So what happened lately besides my personal problems….?

Starting from the beginning, last year I’ve studied and became certified for Cisco UCS implementation, which is a new compute hardware platform. Since my company has a certain preference for Cisco and with my background for Microsoft/compute technology I was asked to join this new “adventure”. I’ve to admit, while I manage some HP blades in our lab environment, UCS Is pretty cool. I love those service profiles and with just a few mouse clicks I can switch a profile and boot up a whole other OS from my SAN environment. Since we using UCS just recently in our lab for such I think this could greatly benefit us, in fact I truly believe many of our customers can benefit from such stateless computing systems. If a blade server fails, just replace it, set the profile and let it boot again. Or when you have a spare blade, UCS will automatically switch the service profile from the failed system to the spare system. For your end users there will be a small disruption, but it won’t take ages before it’s repaired.

I’m not going into to much details about UCS but I can surely recommend it to you. Else check out the great and continually improving simulator Cisco is offering free of charge.

Anyhow, besides of this I was also asked to do my VMware certification. This basically has to do with UC or Cisco Unified Communications product line (just like UCS by the way). Although I’m not a voice guy and not planning to become as such, Cisco did make it possible to virtualize the UC environment on VMware. So to support my colleagues I’ve followed the VMware training for VCP4 examination, which I passed a couple of months later. I’ve to admit, this was probably one of my toughest exams ever and as such I’m pleased I can call myself a VCP. Like I said we are currently setting up a UCS lab/demo environment with UCS, and of course VMware is one of the products we just set up. Besides this we also installed and configured Hyper-V but this was truly a pain in the ass to configure. Well at least for me it was a pain in the ass as an VMware engineer 🙂 Simple tasks like adding a shared LUN, takes different tools and and places to locations to complete the job. Maybe it’s because I haven’t a lot of experience in it yet, so right now I was actually reading a book about it.
In near future we also wanna to implement XenServer so we have multiple Virtualization products running on our storage, all on our 4 blades from UCS.
VDI will also be configured for all those platforms since our customers are asking for it.

If you think I’m done, well you might think again. Besides reading and studying (as such I still do) all the products above I’ve also done training for NetApp. IMHO if you know just VMware, you should also know how storage works, in the past I always thought it was just a bunch of disks with a form of connectivity like FC or Ethernet and I didn’t saw any fun into it. I didn’t care less about a bunch of disks, also I basically hate hardware especially when issues arrive. Hardware should just work, nothing more nothing less.
But after my recent NetApp certification path I actually can say I enjoyed it very much.
There is a lot of thoughts going through my head when I think about future possible implementations or configurations. What will I do to for a configuration with VMware. NFS or LUN?
Why choosing for FC is there’s no historical investment present for FC. In fact, with UCS 2.0 you can even boot from ISCSI whereby FC isn’t needed anymore for completely stateless computing. All this and many more are what I’ve thought about n the last months.. And every time I feel a little smile when I think about it. Where I previously enjoyed security I foresee that I gonna switch my love. Certainly I won’t give up my interests into security but virtualization from A to Z is IMHO the thing I want to do.

Since a couple of weeks from now, where working to win some important customers for our private cloud ideology. This might become a great start where I might blog more on it.

For now, I’m loving it 🙂

Nov 122010

While building some basic VMware Training I thought I needed a lab environment for my colleagues to test things out. For this I simply created an empty VMware Template in ESX and I imported this template into VMware Lab Manager (did I already said that I love this product?).

So After creating a basic environment containing a DC, a VCenter, an XP machine and 2 ESX servers I wanted to install ESX 4.0. However during this process I received the following error:

“Could not format a vmfs volume.” At first I thought, “What the hell???? It’s just a VMDK file… Just format that bloody…” So of course I started shouting at my computer like every technical engineer would do… Luckily I was working at home using a VPN connection to our lab so no one could hear me. 😉


I was a bit intrigued with it… what went wrong? I thought that this would become interesting. So after I found my senses I popped up a new browser tab and went to Google. After a while I found a blog post stating that it might have to do with NFS storage. However I’m running this on our FC SAN environment. So although this might be an issue, it wasn’t the exact issue I had. However it made me think about it.

VMware Lab Manager uses linked clones. What if… what if this was causing the issue? So I created a simple lab with just one ESX server inside of it, and I enabled the option “Full Clone”


Ok that seems to work. I could install ESX! But now what? Can I still use the “capture to  library” option to capture and share my setup for my colleagues? This is because there is no option to do a full clone for the ESX servers when I choose to clone to workspace. It states:  “Create a Linked Clone of All Virtual Machines or Selected Virtual Machines”

Nope, that didn’t work either.

Ok, But then what. How can i create a virtual test environment to teach my colleagues some VMware stuff without going to expensive training. They don’t need to certify themselves, they only need to know the basics about it…

But still there is an another option, what if I use Archive to Library instead of Capture to Library and then share it? That might work out since over here I do get an option to create full clones. Also I could share this one and in this case you also won’t have an issue with customizations and stuff.


So creating the archive is what I did. After a while (enough time to drink some coffee) it was finished. But now what, I still couldn’t use it?

I still need to deploy it to my workspace in order to get it work, so I choose from my library the option: “Clone to workspace”

And hey, now I get an Option to do full clones. That looks promising isn’t it?


So testing this setup brought me to a “Hurray!” moment because It passed the 10% error limit. 🙂 And yes it did finished the installation.


I took me about 2 hours to solve and test this out, shouting included of course. 🙂 It’s time for a nice cup of coffee.

Anyhow, to recap the issue. The problem that the vmfs volume couldn’t be formatted lies in the fact that that I was using linked clones or an original ESX configuration. Somehow ESX didn’t liked that and crashes. Full clones however are working fine, though you might understand that this can become an issue when you lack storage.

Nov 032010

Today I heard something about Project Onyx from VMware. Although I just started playing with it today, I thought it was so interesting that I needed to write something about it.

First of all, Project Onyx is a little tool which can help you to automate certain processes without having a large amount of PowerCLI or SOAP knowledge. Of course, if you want to edit the script you need some knowledge about those things.

Anyhow, first things first. VMware Project Onyx can be found here. A second URL of Project Onyx can be found here. It’s a new tool from VMware which can generate code based on the mouse clicks you make in the VMware vSphere Client. At the URL I just provided you can also find the download link to the product. You don’t need to register or login to get it. Also there is no installation required for Project Onyx. Just download it, unzip it and start it up.


Once launched you will see the above window. Just enter your vCenter server name, your ESX hostname or IP address. Before pressing the start button, it’s handy to the tick the checkmark by “Launch a client after connected”.  If you do you can choose to start the VMware PowerCLI or the VMware VI Client.

When starting the VMware VI Client or as in my case the VMware vSphere client, you will get a warning message that you are connecting to the vCenter Server without encryption. However the traffic isn’t encrypted but tunneled trough the Onyx application which will be connected on SSL or port 443. One of the other things you will notice is that you are connecting to a different port of your vCenter Server. This is port number 1545.

Launching the vSphere Client:


Warning message about the unencrypted traffic:


Once the client is connected to vCenter server you can browse to one of your VM’s. Once you’re there switch back to the “black screen” of Onyx.


The little play button on the top left starts the logging of your mouse clicks within the vSphere Client. The output mode is also interesting. Right now the output mode is PowerCLI.NET. This creates an output script in PowerCLI from your actions. However there are a couple of other possible actions including Raw SOAP Messages, C#.NET 2.0 and VCO JavaScript. Choose the language you need and press the little play button.

So what happens if I want to make sure that a VM boots into its BIOS at the next startup? This is a little and very handy option in the advanced properties of a VM, showed in the screenshot below.


If I enable the option: “The next time the Virtual Machine boots, force entry into the BIOS setup screen”, what is it that is actually happening? Well Check the screenshot below and this is how you could do it into PowerCLI:


Doesn’t this look awesome? Ok what happens then if I change the memory settings from my UDA machine from 512 MB to 1 GB?


At the third line from above you see the newly allocated memory to my Virtual Machine. Ok, I have to admit, I’m not such a scripter anymore as I used to be. However I truly believe that scripting is important and in the future scripting will become even more important then it currently is. In the meanwhile, this awesome application can help me out by creating scripts. 🙂

I think this tool is really awesome and I recommend that you check it out yourself! So check out the website and the YouTube videos that they have posted on their pages.

Nov 012010

In our lab environment we are testing VMware Lab Manager to help us design new environments, simulate customer issues and other things as well. I had used XP during the initial tests, however I noticed something pretty annoying when I switched to Windows 7.

If I used Internet Explorer 8 to open a console to one of my virtual machines in VMWare Lab Manager, I would receive the following error message:

“A website wants to open web content using this program on your computer. this program does not have a valid digital signature that verifies its publisher. This program will open outside of Protected mode, putting your computer at risk. You should only run programs from publishers you trust.”

If you read it carefully, it states that I want to open an application outside of my protected mode.


When I click on the “Allow”, button I receive the error “Bad Handle 0xb90, Then handle is invalid.”


The next error that comes up states: “C:\Program Files\Internet Explorer\PLUGINS\vmware-remotemks.exe: this executable should not be invoked directly.”


The first error (the one about a bad handle) doesn’t tell you a lot. However, the second one does. It’s actually telling you that you are not allowed to start the VMware browser plug-in to get a console.

I noticed that the issue went away when I ran Internet Explorer as Administrator. I was fine with that at first, but I needed to find a easy solution for my coworkers. So after a while I thought,”Let’s add the URL to my trusted sites,” and guess it worked. 🙂

However, since I wasn’t satisfied yet, I made some additional tests. In the end, I noticed that “Enable Protected mode” was enabled for my internet zone. Disabling this setting is not a good idea, so I recommend to simply add the website to your trusted sites.


In summary, the problem I had lies in the security features of Internet Explorer 7 and up. Also since VMware didn’t sign the application, Internet Explorer is not able to verify the publisher as trusted. Please note: our lab domain is a different domain than the one our clients belong to. If you have already added * to your trusted sites you might not get this error.

Apr 062010

So as promised I’ve copied the kickstart which I used to deploy our lab. Some options are default, others are not. Also I’ve added all the references that I used during the creation of the script. Options defined between [] are variables used in the sub template part of UDA. Before scrolling down to the deployment script I think you ought to look at the sub template part. This can save you loads and loads of time. Personally, I think creating such a script is loads of fun. You can configure almost anything you want and in the end all ESX servers will be identical. One of the benefits for me is that there is less change to make any mistakes during a manual installation 🙂

Back to the scripting part. Although you probably will notice that the sub template is just partial, I think you will get the idea.

The sub template:


And the Kickstart Script:

[sourcecode language=”plain”]

# VMware ESX4 template Kickstart file

# Installation Method
install url <a href="http://[UDA_IPADDR]/[OS]/[FLAVOR]">http://[UDA_IPADDR]/[OS]/[FLAVOR]</a>

# root Password
rootpw EnteryourPasswordHere

# Authconfig
auth –enableshadow –enablemd5

# BootLoader ( The user has to use grub by default )
bootloader –location=mbr

# Timezone
timezone Europe/Amsterdam

# Network install type
# This defines our final static IP on the vswif0 management interface of vmnic0 (aka the Service Console) and addvmportgroup=0 disables the default VM network creation.
network –device=vmnic0 –bootproto=static –ip=[IPADDR] –netmask=[SUBNET] –gateway=[GW] –nameserver=[DNS] –hostname=[FQDN] –addvmportgroup=0

# Keyboard
keyboard us

# Reboot after install ?

# Firewall from the installer itself will be disabled
firewall –disabled

# Clear Partitions
clearpart –overwritevmfs –firstdisk=local –all

# Either choose autopartitioning
# autopart –disk=[DISKTYPE]

# Or do the partitioning yourself
echo "Configuring Partitions"

part /boot –fstype=ext3 –size=250 –onfirstdisk=local
part local_[HOSTNAME] –fstype=vmfs3 –size=9000 –onfirstdisk=local –grow
part None –fstype=vmkcore –size=250 –onfirstdisk=local
virtualdisk cos –size=7500 –onvmfs=local_[HOSTNAME]
part / –fstype=ext3 –size=3000 –onvirtualdisk=cos –grow
part swap –fstype=swap –size=1000 –onvirtualdisk=cos
part /opt –fstype=ext3 –size=1000 –onvirtualdisk=cos
part /tmp –fstype=ext3 –size=1000 –onvirtualdisk=cos
part /home –fstype=ext3 –size=1000 –onvirtualdisk=cos

# VMware Specific Commands


%post –interpreter=bash

## —————- References —————————–
## <a href=""></a>
## <a href=""></a>
## <a href=""></a>
## <a href=""></a>
## <a href=""></a>
## <a href=""></a>
## <a href=""></a>
## <a href=""></a>
## ———————————————————

## Create one time only script which kicks in after reboot for networking part
touch /etc/default/
chmod 777 /etc/default/
cat > /etc/default/ <<EOF

# wait a couple of minutes to make sure that the
# hostd-daemon is started
sleep 2m

## Adding the VMkernel/VMotion port group to vSwitch0
echo "Configuring the VMkernel/VMotion port group"
/usr/sbin/esxcfg-vswitch -A VMotion vSwitch0
/usr/sbin/esxcfg-vmknic –add –ip [VMOTIONIP] –netmask [VMOTIONSUB] VMotion
/usr/sbin/esxcfg-route [VMOTIONGW]

## Wait a couple of seconds to give ESX the time to create the vmknic
sleep 10s
/usr/bin/vmware-vim-cmd hostsvc/vmotion/vnic_set vmk0
/usr/bin/vmware-vim-cmd internalsvc/refresh_network

# runonce script
cp /etc/rc.d/rc.local /etc/rc.d/rc.local.bak

cat >> /etc/rc.d/rc.local <<EOF3
mv -f /etc/rc.d/rc.local.bak /etc/rc.d/rc.local

## ——————————————————-
## —– the rest of the script which can run directly —
## ——————————————————-

## Give new accounts the path variables to run esxcfg commands without the need to find the correct path
cat >> /etc/skel/.bash_profile <<EOF
# .bash_profile

# Get the aliases and functions
if [ -f ~/.bashrc ]; then
        . ~/.bashrc

# User specific environment and startup programs

export PATH


## ——————————————————–
## ——————————————————–
## ——————————————————–
# Configure Active Directory authentication (change both domains to yours)
esxcfg-auth –enablead – –

# Add active directory users to the local database
/usr/sbin/useradd ADuser1 -c "Full user name" -m
/usr/sbin/useradd ADuser2 -c "Full user name" -m
/usr/sbin/useradd ADuser3 -c "Full user name" -m

# Additional DNS configuration when IP’s are known.
echo nameserver >> /etc/resolv.conf

## ——————————————————–
## ——————————————————–
## ——————————————————–

## Adding the local users to groups
/usr/sbin/usermod -G wheel ADuser1
/usr/sbin/usermod -G wheel ADuser2
/usr/sbin/usermod -G wheel ADuser3

## Adding SUDO rights to the users
cat >> /etc/sudoers << SUDO
root    ALL=(ALL)       ALL

## Allows people in group wheel to run all commands
%wheel        ALL=(ALL)       ALL

## Configuring NTP services
# Backup ntpd.conf and step-tickers file
mv /etc/ntpd.conf /etc/ntpd.conf.bak
mv /etc/ntpd/step-tickers /etc/ntpd/step-tickers.bak

# Add Servers to step-tickers
cat > /ect/ntp/step-tickers <<EOF
# create ntp.conf

cat > /etc/ntp.conf << EOF
restrict mask nomodify notrap noquery
restrict mask nomodify notrap noquery
driftfile /var/lib/ntp/drift

# Service restart
service ntpd restart

# Make ntp start at boot time
chkconfig –level 345 ntpd on

# Sync hardware clock
hwclock -–systohc

## Automatically enable and starts the webservices preventing 503 errors using the web browser
## This because the web service is per default disabled
/sbin/chkconfig –level 345 vmware-webAccess on
service vmware-webAccess restart

## Firewall configuration
/usr/sbin/esxcfg-firewall –enableService webAccess
/usr/sbin/esxcfg-firewall –enableService sshClient
/usr/sbin/esxcfg-firewall –enableService ntpClient
/usr/sbin/esxcfg-firewall –openport 88,tcp,out,KerberosClientTCP
/usr/sbin/esxcfg-firewall –openport 53,tcp,out,dns

## Adding illegal notice warning during SSH login
touch /etc/default/banner
chmod 777 /etc/default/banner

cat > /etc/default/banner << EOF
This is a private system.
Do not attempt to login unless you are an authorized user.
Any authorized or unauthorized access and use, may be monitored
and can result in criminal or civil prosecution under applicable
law of The Netherlands
chmod 755 /etc/default/banner
echo Banner /etc/default/banner >> /etc/ssh/sshd_config