Unable to remove failed DAG member from a two-node DAG group

Status
Not open for further replies.
M

MosesMusa

I have a simple test lab with two DAG members, each with CAS/MBX/HT roles. The FSW is installed on a NON-Exchange server. I am testing restore of a failed DAG member using Setup /m:RecoverServer. I am following MS TechNet library notes titled " Recover a Database Availability Group Member Server" . I successfully removed all replicated DB copies from the failed server using " Remove-MailboxDatabaseCopy" . The next step was to remove the failed DAG member from the DAG group using " Remove-DatabaseAvailabilityGroupServer" . This cmdlet failed with the following error:

There was a problem changing the quorum model for database availability group. Error: Cluster API SetClusterQuorumResource() failed with 0x1725. Error: A quorum of cluster nodes was not present to form a cluster

Question: Does the above error mean I cannot remove a DAG member from a DAG group consisting of a two-node cluster, even though the FSW is online??? Currently the DAG cluster is operating fine with only one DAG member + FSW. I am using the " Node and File Share Majority" quorum model. This was automatically configured when I created the DAG. Any inputs greatly appreciated as I am in a deployment time crunch.

Additional Info: I was even considering simply building a new Exchange server and add it to the DAG, then remove the currently failed DAG member. I figure this will take care of the quorum error.
 
B

Bharani.Billapati

This looks to me like some cluster error. Check in cluster administrator about the status of cluster. You can try by removing the member from there and then remove the member server from DAG object in adsiedit.

Thanks

Bharani
 
M

Michel de Rooij

So, your 1-node is running ok? Have you tried (forcefully) evicting the failed node (DAG member) and reconfiguring the DAG? Instructions in here (partial failed cluster section):
http://technet.microsoft.com/en-us/library/dd351049.aspx

PS: Try to use Exchange EMC/EMS to reconfigure DAGs/Failover clustering, try to avoid cluster admin - its not Exchange aware.
Michel de Rooij,
MCITP Ent.Msg 2007+2010| MCTS W2008, Ex2007+2010 Conf, OCS2007 Conf | MCSE+Msg2k3 | MCSE+Inet2k3 | Prince2 Fnd | ITIL
I blog on http://eightwone.wordpress.com/ and tweet on http://twitter.com/mderooij
 
G

Gulab Mallah

Michael is right, you will have to remove it from Failover Cluster Management.

-Gulab
 
M

MosesMusa

Michel, yes, the 1-node is working fine together with the non-Exchange FSW. As a rule, I don"t use Cluster Admin to manage Exchange DAGs. Occasionally I use Failover Cluster Manager to view the status of cluster core resources.

I am now working with MS tech support to try and resolve the issue. They decided I use " Remove-DatabaseAvailabilityGroupServer" with " -ConfigurationOnly" option to remove the failed server i.e.

Remove-DatabaseAvailabilityGroupServer -Identity DAG_NAME -MailboxServer SERVER_NAME -ConfigurationOnly

This seems to work - i.e. removes the failed server from DAG group. Not sure if there are any gotchas yet, as I am in process of rebuilding the failed server using Setup /m:RecoverServer. Will try to rejoin it to the DAG once Exch is installed and see what happens.
 
M

MosesMusa

After removing the failed DAG member from the DAG group using ConfigurationOnly option, I was able to successfully re-install Exchange 2010 SP1 using /m:RecoverServer. However when i tried to re-add the server back to the DAG using " Add-DatabaseAvailabilityGroupServer " , it failed with the following error:

A server-side database availability group administrative operation failed. Error: The operation failed. CreateCluster errors may result from incorrectly configured static addresses. Error: An error occurred while attempting a cluster operation. Error: Node exch02 is already joined to a cluster. [Server: EXCH01.exchlab.local]

FYI: When I ran the Get-DatabaseAvailabilityGroup cmdlet, it showed only 1 DAG member (the surviving DAG member server). It does not show the DAG member I removed using the ConfigurationOnly parameter (i.e. server exch02 referenced in the above error ).
 
G

Gulab Mallah

Evict this node from cluster (Failover Cluster Management) than add it

-Gulab
 
M

MosesMusa

I checked in Failover Cluster Manager and confirmed that exch02 node still showed. I am somewhat wary of using Failover Cluster Manager to manage Exchange DAGs so I used the following cmdlet to force removal of the failed node:

Get-ClusterNode exch02 | Remove-ClusterNode –Force

It worked OK and I was able to add back the rebuilt Exchange DAG member to the DAG group, then created DB copies. In summary the failed DAG member has now been successfully restored. Thanks to everyone who contributed.

TIP: For newbies like me, to run “Get-ClusterNode” in EMS, you may need to first import the Clusters module using “import-module failoverclusters”.
 
Status
Not open for further replies.
Top