I saw this error before in establishing many failover clusters , In the validation phase of the cluster nodes everything went fine except the IP configurations part.

I made sure that all network drivers are static and the binding is right. But the error keep appearing every time I validate the cluster.

I found that one of my network drivers has APIPA IP although I haven’t

I run ‘ipconfig /all’ from the command prompt. You might have noticed a few extra interfaces, but not really understand what they are. I suspected that those NICs are the problem so open the server manager to see what I get. I saw that there is many drivers under Network adaptors

There I found driver called Teredo tunneling pseudo-interface

Teredo is an IPv6 transition technology that provides address assignment and host-to-host automatic tunneling for unicast IPv6 traffic when IPv6/IPv4 hosts are located behind one or multiple IPv4 network address translators (NATs). To traverse IPv4 NATs, IPv6 packets are sent as IPv4-based User Datagram Protocol (UDP) messages. For more information check Microsoft network part here

Teredo is enabled by default in Windows Server 2008.

To pass the validation wizard just disable Teredo tunneling pseudo-interface on the cluster nodes

What are host reserves?

 Host Reserves define how much of a host’s resources are to reserved for the host operating system. Once these reserves are configured, a virtual machine cannot be deployed on that host if doing so would require the use of those reserved resources. The host resources that may be reserved are as follows:

·         CPU Percentage

·         Memory

·         Disk Space

·         Maximum Disk I/O Per Second (IOPS)

·         Network Capacity Percentage

 Host reserves are specified on a host group basis. In addition, the group reserve settings may also be overridden on a per host basis. To specify the host group reserve settings, right click on the host group name in the Hosts pane, select Properties from the menu and click the Host Reserve Tabs in the Host Group Properties dialog as illustrated in the following figure:

What is cluster reserve ?

Depending on your needs, you can configure a cluster reserve for each host cluster that specifies the number of node failures a cluster must be able to sustain while still supporting all virtual machines deployed on the host cluster. If the cluster cannot withstand the specified number of node failures and still keep all of the virtual machines running, the cluster is placed in an Over-Committed state, and the clustered hosts receive a zero rating during virtual machine placement. The administrator can, during a manual placement, override the rating and place an HAVM on an over-committed cluster.

For example, if you specify a node failure reserve of 2 for an 8-node cluster, the rule is applied in the following ways:

·         If all 8 nodes of the cluster are functioning, the host cluster is marked Over-committed if any combination of 6 nodes (8-2) in the cluster lacks the capacity to accommodate existing virtual machines.

·         If only 5 nodes in the cluster are functioning, the cluster is marked Overcommitted if any combination of 3 (5-2) nodes in the cluster lacks the capacity to accommodate existing virtual machines.

VMM’s cluster refresher updates the host cluster’s Over-committed status after each of the following events:

·         A change in the cluster reserve value

·         The failure or removal of nodes from the host cluster

·         The addition of nodes to the host cluster

·         The discovery of new virtual machines on nodes in the host cluster

The cluster reserve is set on the General tab of the host cluster properties.

View the status of the cluster, and adjust the cluster reserve.

·         In the Cluster reserve field, specify the maximum number of node failures the cluster must be able to sustain but still keep all existing virtual machines running. If the rule is violated, the host cluster is marked Overcommitted.

 How VMM calculate Over-Committed Cluster?!!

 The VMM calculate depending on the highest VM RAM, i.e.. If you have 10 VMs with different RAMs then VMM calculate the amount of RAM by this equation (Highest RAM in VM * N) Where N is the no of VMs in the cluster and equal 10 in our example.

Error (13803)

The cluster node failure reserve equals or exceeds the number of nodes in cluster <Cluster Name>

Recommended Action

Specify a cluster node failure reserve less than the number of nodes in the cluster and then try the operation again.

this is another thing.. due to the fact that building Hyper-V cluster using one physical and one virtual node is not valid (Sure this make no sense). As the VMs cannot failover to the virtual node.

Microsoft did an important change for Windows 2008 R2 Hyper-V Clustering limitations.

To effectively plan for and deploy Hyper-V in Windows Server 2008 R2, you should understand the requirements and maximum configurations for the physical and virtual hardware that will comprise the virtualized server computing environment. You will need to consider the maximums that apply to each virtual machine as well as those that apply to the physical computer that runs the Hyper-V role.

Failover Clusters and Hyper-V

Source

One of my customers faced this problem.. he created Hyper-V cluster with CSV volume and start placing VMs on it.. after a while CSV start running out of space and he is planning to expand CSV volume using SAN tools. Is LUN expansion from the SAN level, is it support from MS ? What are the risks with doing that?

After some help from Microsoft support group I got this answer

According to:

Volume expansion is also a regular requirement since data growth is not often considered up-front. Cluster disks can be extended without rebooting if the controller supports dynamic LUN expansion. This feature allows for the physical expansion to be implemented without disruption and users can use tools (diskpart) provided by Microsoft to allow for the change to be seamlessly applied at the logical level as well.

Storage Topologies

http://technet.microsoft.com/en-us/library/cc738154(WS.10).aspx

We can see LUN expansion is supported by Microsoft Operation System.

Also, from the following information:

The ability to add storage while the system is operating enables better storage efficiency as it does not require any downtime for storage management. The storage can be expanded either by adding an additional LUN or by expanding the existing LUN on the CLARiiON array.

Optimized Storage Solution for Enterprise Scale Hyper-V Deployments

http://download.microsoft.com/download/D/E/6/DE619C29-BBCA-468F-960C-93B8113F612B/EMC_Sanbolic_MS_POC-Final.pdf

We can see the storage can be expanded either by adding an additional LUN or by expanding the existing LUN.

Regarding the risks, as far as I know, there should be no risk if we follow the proper steps to expand the LUN.

Here, I list following link for your reference:

Best Practices – SAN LUN Creation and Size Expansion for SAP Architectures

http://blogs.technet.com/b/lobapps/archive/2010/09/14/lun-creation-and-size-expansion.aspx

This week I got a call from one of my team mates (Nashaat Sorial) telling me that he is facing a problem with his Hyper-V cluster..

He is running 3 Hyper-V hosts with Windows 2008 R2 SP1 running multiple applications. One of the hosts keep failing and He can not Live migrate any VM to the other hosts until restart.

We went through all well known workarounds for such cases like

1- if you have AV please disable

2- Also check http://technet.microsoft.com/en-us/library/cc773533%28WS.10%29.aspx

3- Please if you have teaming..Break it and check again

4- Please check http://support.microsoft.com/kb/981618

5- Make sure that you enable the "file and printer sharing" as well as the "client for MS networks" on the heartbeat networks cards on all nodes.  http://support.microsoft.com/kb/2008795

So we have to start reading more in the cluster logs. You can generate the cluster logs as per

http://blogs.msdn.com/b/clustering/archive/2008/09/24/8962934.aspx

We got some interesting data and errors in the logs just like:

In Win2008 the Physical Disk resource type private property that stores the disk signature change from "Signature" and it is now "DiskSignature" in Win2008.  The lack of the DiskSignature property not being populated was resulting in the resource failing to come online

ERROR_CLUSTER_GROUP_MOVING(5908)’ because of ”Virtual Machine Configuration R-Web2003′ is owned by node 2, not 1.’

mm.. So it looks like something from the Hardware level. Searching HP blades errors come with more useful information

http://cb-net.co.uk/index.php?option=com_content&view=article&id=84:hp-bl460-asr-hpqilo2-issues&catid=3:windows-server-2003&Itemid=3

he HP Integrated Management Log shows ‘ASR Detected by System ROM‘ along with the following events in the System event log on an affected machine:

Event Type:    Warning
Event Source:    hpqilo2
Event Category:    None
Event ID:    57
Failed GET SENSOR READING, sensor 16
Event Type:    Warning
Event Source:    hpqilo2
Event Category:    None
Event ID:    57
NetFN 0x4, command 0x2D timed out

The solution for this was to perform the following:

• Install the latest ILO Firmware Update v. 1.81
• Install the HP iLO Management Channel Interface Driver v. 1.15.0.0
• Install the HP ProLiant iLO2 Management Controller Driver v. 1.12.0.0

All of these drivers can be downloaded form the following location, selecting your operating system:

http://h20000.www2.hp.com/bizsupport/TechSupport/DriverDownload.jsp?prodNameId=3288156?=en&cc=us&prodTypeId=3709945&prodSeriesId=1842750&taskId=135