netstat for ESXi

netstat for ESXi | Technodrone.

How do you check if an ESXi host has an open connection on a certain port, if it is listening on a certain port – for troubleshooting purposes.

Trying netstat on an ESXi host does not work – because that command is not there – see the screenshot below.

No netstat

Well that is not good – if the command is not in the busybox console then how would you go about getting that information? Well of course the clever people at VMware have already thought about this and have exposed all this information through esxcli. William Lam wrote a great set of posts on esxcli
esxcli Part1 – What is esxcli?, esxcli Part2 – Automating esxcli using vMA and esxcli Part3 – Automating esxcli using PowerShell

This is how you would go about getting the information from esxcli. (Be aware the command differ according to the different versions – 4.x is not the same as 5.x)

1
esxcli network ip connection list

esxcli1

That is fine and dandy – but to get that info you need to either:

  1. have access to the DCUI (and have it enabled of course)
    or
  2. access remotely with SSH (and also have it enabled of course)

But what if you do not want to enable neither of the above – that means you have to do it remotelyand for that you have two options, vCLI or PowerCLI.

The vCLI way

1
esxcli --server esx1.maishsk.local network ip connection list

vcli1

But me being more of PowerCLI guy I would do it like this.

The PowerCLI way

1
2
3
$esxcli = get-esxcli -vmhost esx1.maishsk.local
$esxcli.network.ip.connection.list() | ft

PowerCLI1

Output is almost identical – just that in the case of PowerCLI the values are returned as a set of objects – a  VMware.VimAutomation.ViCore.Impl.V1.EsxCli.EsxCliObjectImpl object to be precise. Once these presented as objects I can start to mold and dice my results to my liking.

For example – I would like to check if there is any connections open on port 80 (http) – with vCli – this is not so simple – because you are working essentially in a DOS window – so filtering is not the easiest with findstr. Using the console or SSH is easier – a simple grep will work as you can see below.

1
esxcli network ip connection list | grep :80

esxcli2

With PowerCLI

1
$esxcli.network.ip.connection.list() | where { $_.LocalAddress -like "*:80" } | ft

PowerCLI2

I hope you can see that the options this way are pretty much endless – like filtering all connections to show only those from a specific IP, or a complete subnet.

So that is how you netstat on ESXi….

How to allow root user to log in to ESX host with SSH?

When you try to SSH login to your ESX host using root account, it gives you “Access denied” message.

This is normal; ESX SSH login using the root account is disabled by default for security purpose. You can login to the host by using either of the below ways:

  1. Login directly to the ESX host using VI Client (not to the vCenter Server)
  2. Click Users & Groups tab
  3. Right-click on a blank area and click Add
  4. Enter a username and password as shown in the picture. Confirm your password. Note: Starting in  ESX 4.0 the password needs to be at least 8 characters in length.
  5. Select Grant shell access to this user and click OK.
  6. Open the SSH client (Putty for example).
  7. Complete the necessary fields. Ensure Port is set to 22 and Protocol is set to SSH. Press Enter or click Open.
  8. Log in as the new user you created in step 4.
  9. Type SU – and press enter (This command switches users to root access and provides the path to the root user commands).
  10. Enter the root password when prompted, press Enter and you are in.

Now, you know how to SSH login using root account but you will need to use the SSH login account (created on step 4 above). If you need to permanently login using root account directly without the need to use a temp account you will need to edit it the “ssh_config” file to permanently enable root account to login directly.

Here are the steps:

  1. Open a SSH session and login using steps mention above.
  2. Type command: nano /etc/ssh/sshd_config
  3. Find the line the says: PermitRootLogin and change the “no“ to “yes” .
  4. save the file and exit.
  5. Restart the sshd process by typing /etc/init.d/sshd restart .

Now, you can  permanently login to SSH using root account directly.

Troubleshooting a failed virtual machine conversion task

Conversions sometimes fail no matter how careful you are preparing the server. The failure can occur at various stages in the conversion process; these stages are based on the task bar percent and are estimated values.

1. Creation of the target virtual machine (VM) (0%-5%)
2. Preparing to Clone the Disk (5%-6%)
3. Cloning (6%-95%)
4. Post-cloning (95%-97%)
5. Customization/Reconfiguration (97%-99%)
6. Install Tools/Power On (99%-100%)

Converter creates a detailed log file during the conversion process which will contain exact errors pertaining to why the conversion failed. This log file is located on the server you are converting that is running the Converter agent, and is usually named vmware-converter-0.log and is located in the C:\Windows\temp\vmware-temp directory.

What is hostd and vpxa ?

hostd is an app that runs in the Service Console that is responsible for managing most of the operations on the ESX machine.  It knows about all the VMs that are registered on that host, the luns/vmfs volumes visible by the host, what the VMs are doing, etc.  Most all commands or operations come down from VC through it.  i.e, powering on a VM, VM vMotion, VM creation, etc.

vpxa also runs on the Service Console and talks to VC.  I believe it acts as an intermediary between VC and hostd. I think it also does some housekeeping on the ESX host, but not as much as hostd.

Courtesy : http://vmaze.wordpress.com/

 

Vmware hostd and vpxa on ESXi 5.X

HOSTD

The vmware-hostd management service is the main communication channel between ESX/ESXi hosts and VMkernel. If vmware-hostd fails, ESX/ESXi hosts disconnects from vCenter Server/VirtualCenter and cannot be managed, even if you try to connect to the ESX/ESXi host directly. It knows about all the VMs that are registered on that host, the luns/vmfs volumes visible by the host, what the VMs are doing, etc. Most all commands or operations come down from VC through it. i.e, powering on a VM, VM vMotion, VM creation, etc.

Restart the management agent

/etc/init.d/hostd restart

VPXA

It acts as an intermediary between VC and hostd. The vCenter Server Agent, also referred to as vpxa or the vmware-vpxa service, is what allows a vCenter Server to connect to a ESX host. Specifically, vpxa is the communication conduit to the hostd, which in turn communicates to the ESX kernel.

Restart the vpxa service

/etc/init.d/vpxa restart

 
Note:- If you have SSH enabled on your ESXi server these services can also be restarted and even if these are restarted by you then also your SSH session will not be impacted.

VPXD-It is Vcenter Server Service. If this service is stopped then we will not able to connect to Vcenter Server via Vsphere client.

VPXA-It is the agent of Vcenter server. also known as mini vcenter server which is installed on the each esx server which is managed by Vcenter server. What are the management action we are performing on top of the vcenter server. (Like:- Increasing/Decreasing RAM & HDD, Making any type of changes in cluster, doing vmotion. This agent collects all information from the vcenter server and pass this information to the kernal of the esx server.

HOSTD- This is the agent of ESX server, here VPXA pass the information to the HOSTD and hostd pass the information to ESX server.

In ESX, you have only hostd and (if you have vCenter) vpxa.

These are daemon (services) for remote management:

  • hostd is used to remote management using VIC
  • vpxa is used by vCenter (the vpxd part of vCenter) to remote manament

hostd is the daemon for direct VIC connection (when you use Virtual Infra Client (VIC) to connect to your ESX).

Also,

  • vpxa is the VC agent (ESX side)
  • vpxd is the VC daemon (VC side)

What do ESX and ESXi stand for?

After a little web searching, I found this article on acronyms on Yellow-Bricks which reveals the little known/cared about story of many terms in VMware’s world of virtualization. There is also this link to a video interview with Mike DiPetrillo which goes more in to this matter.

ESX and GSX

As VMware old-timer Mike Di Petrillo tells us, the first hypervisor versions were named by a marketing firm as:

Elastic Sky

Ground Storm

The engineers did not seem to like it and after a vote from the engineers they dropped the long names and retained the initials and added the X to make it sound technical. Hence, the next versions of hypervisors were named:

ESX stands for Elastic Sky  X

GSX stands for Ground Storm X

ESXi

Apparently, the “i” in ESXi stands for Integrated, probably coming from the fact that this version of ESX can be embedded in a small bit of flash memory on the server hardware.  In this manner, we can treat server hardware almost like a hot-swap commodity; just another node.

VMware vSphere Hypervisor

To confuse things further, VMware now offers the free VMware vSphere Hypervisor  (intended to replace the “free-ESXi”, and separate from ESX/ESXi, which are Enterprise-class hypervisors). For more info, see this FAQ from the VMware website for  VMware vSphere Hypervisor.

What is the difference between VMware ESXi and VMware vSphere Hypervisor?
VMware vSphere Hypervisor is the new name for what was formerly known as VMware ESXi Single Server or free ESXi (often abbreviated to simply “VMware ESXi”). VMware vSphere Hypervisor is the free edition of the vSphere production line. It is licensed to only unlock the hypervisor functionality of vSphere, but it can be seamlessly upgraded to more advanced offerings of VMware vSphere. VMware vSphere is available in multiple editions, including several options specifically designed for small businesses.