What is DFSR cloning ??
DFSR cloning is a process of exporting a DFSR database from upstream partner and importing it to the downstream partner in order to avoid initial synchronization process (State: 2). Performing a DFSR cloning saves time and network bandwidth. Instead of replicating the files between the DFSR member servers only the file hash are compared as the data is already copied to the downstream partner using Robocopy. DFSR is a state based management i.e. it works in different states listed below.
DFSR cloning steps listed as below:
a. Create a replication group and a replicated folder, then add a upstream server as a member of that topology.
b. Let initial build complete. The DFSR member server should be in state 4(normal state)
Command to check the state of Replicated Folder:
wmic /namespace:\\root\microsoftdfs path DfsrReplicatedFolderInfo get ReplicationGroupName, ReplicatedFolderName, State
c. Export the cloned database from the upstream server
d. Preseed the files to the downstream server and copy in the exported clone Database files
e. Import the cloned database on the downstream server
f. Add the downstream server to the Replication Group and Replicated Folder membership using the DFSR console.
g. Let the initial sync validation complete and again check the state of RF on both the servers. It should be in state 4(normal state)
Pre-Seeding DFSR Data between 2012 & 2012R2 using Wbadmin.
Assuming The DFSR “RF:CMS” & “RG:production-application-services” with the DFSR Replication Members :DFS-1 & DFS-4(New Member Where Pre-Seeding has to be done) .The Drives where we have the existing “CMS” folder on “DFS-1” is “U:\CMS” . Assuming The Drive Where Pre-Seeding has to be done for “CMS” folder on “DFS-4” is “U:\CMS”.
The Actual steps for Pre-Seeding:
1.On the DFS-1 server Install the "Windows Server Backup" Feature from the server Manager.
2.Assuming that the drive that we are going to back-up the folder is “V:” Drive on “DFS-1” Server.
3.Run the command :
wbadmin start backup -backuptarget:V: -include:U:\CMS
4.Type “Y” to Accept
5.Once the Backup is complete Share the “V:” Drive on the Network with the appropriate Permissions.
6.Logon to the server “DFS-4”.
7.Open Command Prompt as an Administrator.
8.Run the command:
wbadmin get versions -backupTarget:\\DFS-1\V
9.You Should get an Output similar as below:
Backup time: 9/20/2017 11:59 AM
Backup target: Fixed Disk labeled New Volume(U:)
Version identifier: 09/20/2017-15:59
Can recover: Volume(s), File(s)
Snapshot ID: {5d219d13-30b6-48e7-b322-a5d1dcb72582}
10.In the above output we are interested in "Version identifier:".
11.Now on the Server run the following command on DFS-4
wbadmin start recovery -version:09/20/2017-15:59 -itemtype:File -backuptarget:\\DFS-1\V -items:U:\CMS -recursive -overwrite:createcopy -recoverytarget:U:\
§ You should see the similar Output as below :
wbadmin 1.0 - Backup command-line tool
(C) Copyright 2013 Microsoft Corporation. All rights reserved.
Retrieving volume information...
You have chosen to recover the file(s) U:\CMS from the backup created on 9/20/2017 11:59 AM to U:\.
Preparing to recover files...
Do you want to continue?
[Y] Yes [N] No
12.Type “Y” and hit Enter
13.Once the Restore is complete , Install the DFSN & DFSR Role (Do not Add the Member to the RG/RF).
14.Once the Restore is done we need to Validating Pre-seeding Data.
a. Note the path of six files within the source data server. These should be scattered throughout various nested folder trees.
b. For one of those test files, use DFSRDIAG.EXE to get a hash from the source computer and the matching file on the pre-seeded computer:
DFSRDIAG.exe filehash /path:"U:\CMS\cms-customer-data\Surrounding\cache\2876_5smdrqlf.gxo.tmp\25-b-1-.html" ==> Run on DFS-1
DFSRDIAG.exe filehash /path:"U:\CMS\cms-customer-data\Surrounding\cache\2876_5smdrqlf.gxo.tmp\25-b-1-.html" ==> Run on DFS-4
Example Output:
§ DFSR Member:USE1-DFS-1
DFSRDIAG.exe filehash /Path:"U:\CMS\cms-customer-data\Surrounding\cache\2876_5smdrqlf.gxo.tmp\25-b-1-.html"
File Hash: 08539D25-D698638B-AFB0E51A-8EE27C4B
Operation Succeeded
§ DFSR Member:USE1-DFS-4
DFSRDIAG.exe filehash /Path:"U:\CMS\cms-customer-data\Surrounding\cache\2876_5smdrqlf.gxo.tmp\25-b-1-.html"
File Hash: 08539D25-D698638B-AFB0E51A-8EE27C4B
Operation Succeeded
15.If DFSRDIAG shows the same hash value for both copies of the file, it has been pre-seeded correctly and matches in all file aspects.
Note: Keep in mind that unless your data is 100% static or users are not allowed to modify files during pre-seeding and DFSR initial sync, some file conflicts are to be expected. These will be visible in the form of DFSR Event Log 4412 entries on the server that was pre-seeded. The point of pre-seeding is to minimize the amount of data to be replicated initially during the non-authoritative replication phase on the downstream server; unless data never changes there will always be a delta that DFSR will have to catch up after pre-seeding.
16.Add the Member DFS-4 to the Replication Group "production-application-services" & the Replication Folder "CMS".
17.Once Added initially it will be at state "0" , Then it will move to the state "2" for initial Sync & then to the normal state "4".
DFSR CLONING:
Environment :
DSM1 : DFSR Primary Member.
DSM2 : DFSR Secondary Member.
1) To create an example replication environment for testing :
$DfsrComputerName = "DSM1"
$DfsrReplicatedFolderPath = "E:\MyDFSR"
$DfsrReplicationGroupName = "MyDFSR01"
$DfsrReplicatedFolderName = "MyDFSR01"
New-DfsReplicationGroup -GroupName $DfsrReplicationGroupName
New-DfsReplicatedFolder -GroupName $DfsrReplicationGroupName -FolderName $DfsrReplicatedFolderName
Add-DfsrMember -GroupName $DfsrReplicationGroupName -ComputerName $DFSRComputername
Set-DfsrMembership -GroupName $DfsrReplicationGroupName -FolderName $DfsrReplicatedFolderName -ContentPath $DfsrReplicatedFolderPath -ComputerName $DFSRComputername -PrimaryMember $True
Example :
4) Export the cloned database and volume configuration XML :
$DfsrCloneVolume = "E:"
$DfsrCloneDir = "\DfsrClone"
New-Item -Path $DfsrCloneVolume\$DfsrCloneDir -Type Directory
Export-DfsrClone -Volume $DfsrCloneVolume -Path $DfsrCloneVolume\$DfsrCloneDir
5) After completion, make a note of the Robocopy sample commands that are displayed by the Export-DfsrClone cmdlet.
You can later use these commands to copy the data to the destination server. You can also use Export-DfsrClone with the -WhatIf parameter to see the Robocopy samples without performing cloning, for example:
Robocopy.exe "E:\DfsrClone" "\\DSM2\E$\DfsrClone" /B
Robocopy.exe "H:\RF01" "" /E /B /COPYALL /R:6 /W:5 /MT:64 /XD DfsrPrivate /TEE /LOG+:preseed.log
Example :
Robocopy.exe "E:\DfsrClone" "\\DSM2\E$\DfsrClone" /B
-------------------------------------------------------------------------------
ROBOCOPY :: Robust File Copy for Windows
-------------------------------------------------------------------------------
Started : Tuesday, July 18, 2017 1:58:34 PM
Source : E:\DfsrClone\
Dest : \\DSM2\E$\DfsrClone\
Files : *.*
Options : *.* /DCOPY:DA /COPY:DAT /B /R:1000000 /W:30
------------------------------------------------------------------------------
New Dir 2 E:\DfsrClone\
100% New File 2097 config.xml
100% New File 8.0 m dfsr.db.clone
------------------------------------------------------------------------------
Total Copied Skipped Mismatch FAILED Extras
Dirs : 1 1 0 0 0 0
Files : 2 2 0 0 0 0
Bytes : 8.01 m 8.01 m 0 0 0 0
Times : 0:00:00 0:00:00 0:00:00 0:00:00
Speed : 41414231 Bytes/sec.
Speed : 2369.741 MegaBytes/min.
Ended : Tuesday, July 18, 2017 1:58:35 PM
6) Wait for a DFS Replication Event 2402 in the DFS Replication event log, which indicates that the export completed successfully.
7) On the destination server DSM2, ensure that a DFS Replication database does not already exist on the volume into which you will import data. To do so, use the following procedure:
Get-ChildItem -path "E:\System Volume Information\dfsr" -hidden
8) Import a clone of a DFS Replication database
Import-DfsrClone -Volume h: -Path "h:\Dfsrclone"
9) Add the Destination Server as a DFSR member
$DfsrSourceComputerName = "DSM1"
$DfsrDestinationComputerName = "DSM2"
$DfsrReplicationGroupName = "MyDFSR01"
$DfsrReplicatedFolderName = "MyDFSR01"
$DfsrReplicatedFolderPath = "E:\MyDFSR"
Add-DfsrMember -GroupName $DfsrReplicationGroupName -ComputerName $DfsrDestinationComputerName
Add-DfsrConnection -GroupName $DfsrReplicationGroupName -SourceComputerName $DfsrSourceComputerName –DestinationComputerName $DfsrDestinationComputerName
Set-DfsrMembership -GroupName $DfsrReplicationGroupName -FolderName $DfsrReplicatedFolderName -ContentPath $DfsrReplicatedFolderPath -ComputerName $DfsrDestinationComputerName
10) The RG State on the destination server should be at state 4.
11) End
DFSR cloning is a process of exporting a DFSR database from upstream partner and importing it to the downstream partner in order to avoid initial synchronization process (State: 2). Performing a DFSR cloning saves time and network bandwidth. Instead of replicating the files between the DFSR member servers only the file hash are compared as the data is already copied to the downstream partner using Robocopy. DFSR is a state based management i.e. it works in different states listed below.
0 = Uninitialized State
1 = Initialized State
2 = Initial Sync State
3 = Auto Recovery State
4 = Normal State
5 = Error State
DFSR cloning steps listed as below:
a. Create a replication group and a replicated folder, then add a upstream server as a member of that topology.
b. Let initial build complete. The DFSR member server should be in state 4(normal state)
Command to check the state of Replicated Folder:
wmic /namespace:\\root\microsoftdfs path DfsrReplicatedFolderInfo get ReplicationGroupName, ReplicatedFolderName, State
c. Export the cloned database from the upstream server
d. Preseed the files to the downstream server and copy in the exported clone Database files
e. Import the cloned database on the downstream server
f. Add the downstream server to the Replication Group and Replicated Folder membership using the DFSR console.
g. Let the initial sync validation complete and again check the state of RF on both the servers. It should be in state 4(normal state)
Pre-Seeding DFSR Data between 2012 & 2012R2 using Wbadmin.
Assuming The DFSR “RF:CMS” & “RG:production-application-services” with the DFSR Replication Members :DFS-1 & DFS-4(New Member Where Pre-Seeding has to be done) .The Drives where we have the existing “CMS” folder on “DFS-1” is “U:\CMS” . Assuming The Drive Where Pre-Seeding has to be done for “CMS” folder on “DFS-4” is “U:\CMS”.
The Actual steps for Pre-Seeding:
1.On the DFS-1 server Install the "Windows Server Backup" Feature from the server Manager.
2.Assuming that the drive that we are going to back-up the folder is “V:” Drive on “DFS-1” Server.
3.Run the command :
wbadmin start backup -backuptarget:V: -include:U:\CMS
4.Type “Y” to Accept
5.Once the Backup is complete Share the “V:” Drive on the Network with the appropriate Permissions.
6.Logon to the server “DFS-4”.
7.Open Command Prompt as an Administrator.
8.Run the command:
wbadmin get versions -backupTarget:\\DFS-1\V
9.You Should get an Output similar as below:
Backup time: 9/20/2017 11:59 AM
Backup target: Fixed Disk labeled New Volume(U:)
Version identifier: 09/20/2017-15:59
Can recover: Volume(s), File(s)
Snapshot ID: {5d219d13-30b6-48e7-b322-a5d1dcb72582}
10.In the above output we are interested in "Version identifier:".
11.Now on the Server run the following command on DFS-4
wbadmin start recovery -version:09/20/2017-15:59 -itemtype:File -backuptarget:\\DFS-1\V -items:U:\CMS -recursive -overwrite:createcopy -recoverytarget:U:\
§ You should see the similar Output as below :
wbadmin 1.0 - Backup command-line tool
(C) Copyright 2013 Microsoft Corporation. All rights reserved.
Retrieving volume information...
You have chosen to recover the file(s) U:\CMS from the backup created on 9/20/2017 11:59 AM to U:\.
Preparing to recover files...
Do you want to continue?
[Y] Yes [N] No
12.Type “Y” and hit Enter
13.Once the Restore is complete , Install the DFSN & DFSR Role (Do not Add the Member to the RG/RF).
14.Once the Restore is done we need to Validating Pre-seeding Data.
a. Note the path of six files within the source data server. These should be scattered throughout various nested folder trees.
b. For one of those test files, use DFSRDIAG.EXE to get a hash from the source computer and the matching file on the pre-seeded computer:
DFSRDIAG.exe filehash /path:"U:\CMS\cms-customer-data\Surrounding\cache\2876_5smdrqlf.gxo.tmp\25-b-1-.html" ==> Run on DFS-1
DFSRDIAG.exe filehash /path:"U:\CMS\cms-customer-data\Surrounding\cache\2876_5smdrqlf.gxo.tmp\25-b-1-.html" ==> Run on DFS-4
Example Output:
§ DFSR Member:USE1-DFS-1
DFSRDIAG.exe filehash /Path:"U:\CMS\cms-customer-data\Surrounding\cache\2876_5smdrqlf.gxo.tmp\25-b-1-.html"
File Hash: 08539D25-D698638B-AFB0E51A-8EE27C4B
Operation Succeeded
§ DFSR Member:USE1-DFS-4
DFSRDIAG.exe filehash /Path:"U:\CMS\cms-customer-data\Surrounding\cache\2876_5smdrqlf.gxo.tmp\25-b-1-.html"
File Hash: 08539D25-D698638B-AFB0E51A-8EE27C4B
Operation Succeeded
15.If DFSRDIAG shows the same hash value for both copies of the file, it has been pre-seeded correctly and matches in all file aspects.
Note: Keep in mind that unless your data is 100% static or users are not allowed to modify files during pre-seeding and DFSR initial sync, some file conflicts are to be expected. These will be visible in the form of DFSR Event Log 4412 entries on the server that was pre-seeded. The point of pre-seeding is to minimize the amount of data to be replicated initially during the non-authoritative replication phase on the downstream server; unless data never changes there will always be a delta that DFSR will have to catch up after pre-seeding.
16.Add the Member DFS-4 to the Replication Group "production-application-services" & the Replication Folder "CMS".
17.Once Added initially it will be at state "0" , Then it will move to the state "2" for initial Sync & then to the normal state "4".
DFSR CLONING:
Environment :
DSM1 : DFSR Primary Member.
DSM2 : DFSR Secondary Member.
1) To create an example replication environment for testing :
$DfsrComputerName = "DSM1"
$DfsrReplicatedFolderPath = "E:\MyDFSR"
$DfsrReplicationGroupName = "MyDFSR01"
$DfsrReplicatedFolderName = "MyDFSR01"
New-DfsReplicationGroup -GroupName $DfsrReplicationGroupName
New-DfsReplicatedFolder -GroupName $DfsrReplicationGroupName -FolderName $DfsrReplicatedFolderName
Add-DfsrMember -GroupName $DfsrReplicationGroupName -ComputerName $DFSRComputername
Set-DfsrMembership -GroupName $DfsrReplicationGroupName -FolderName $DfsrReplicatedFolderName -ContentPath $DfsrReplicatedFolderPath -ComputerName $DFSRComputername -PrimaryMember $True
Example :
2) Wait for DFS Replication event 4112 in the DFS Replication event log, which indicates that the replication folder initialized successfully
as the primary member.
Example :
3) Validate that all the existing replicated folders on the volume that stores the replicated folder that you want to pre-seed are in the
Normal, non-initial sync state. Replicated folders in other states are skipped during cloning.
wmic /namespace:\\root\microsoftdfs path DfsrReplicatedFolderInfo get replicatedFolderName,ReplicationGroupName,State
4) Export the cloned database and volume configuration XML :
$DfsrCloneVolume = "E:"
$DfsrCloneDir = "\DfsrClone"
New-Item -Path $DfsrCloneVolume\$DfsrCloneDir -Type Directory
Export-DfsrClone -Volume $DfsrCloneVolume -Path $DfsrCloneVolume\$DfsrCloneDir
5) After completion, make a note of the Robocopy sample commands that are displayed by the Export-DfsrClone cmdlet.
You can later use these commands to copy the data to the destination server. You can also use Export-DfsrClone with the -WhatIf parameter to see the Robocopy samples without performing cloning, for example:
Robocopy.exe "E:\DfsrClone" "\\DSM2\E$\DfsrClone" /B
Robocopy.exe "H:\RF01" "
Example :
Robocopy.exe "E:\DfsrClone" "\\DSM2\E$\DfsrClone" /B
-------------------------------------------------------------------------------
ROBOCOPY :: Robust File Copy for Windows
-------------------------------------------------------------------------------
Started : Tuesday, July 18, 2017 1:58:34 PM
Source : E:\DfsrClone\
Dest : \\DSM2\E$\DfsrClone\
Files : *.*
Options : *.* /DCOPY:DA /COPY:DAT /B /R:1000000 /W:30
------------------------------------------------------------------------------
New Dir 2 E:\DfsrClone\
100% New File 2097 config.xml
100% New File 8.0 m dfsr.db.clone
------------------------------------------------------------------------------
Total Copied Skipped Mismatch FAILED Extras
Dirs : 1 1 0 0 0 0
Files : 2 2 0 0 0 0
Bytes : 8.01 m 8.01 m 0 0 0 0
Times : 0:00:00 0:00:00 0:00:00 0:00:00
Speed : 41414231 Bytes/sec.
Speed : 2369.741 MegaBytes/min.
Ended : Tuesday, July 18, 2017 1:58:35 PM
6) Wait for a DFS Replication Event 2402 in the DFS Replication event log, which indicates that the export completed successfully.
7) On the destination server DSM2, ensure that a DFS Replication database does not already exist on the volume into which you will import data. To do so, use the following procedure:
Get-ChildItem -path "E:\System Volume Information\dfsr" -hidden
8) Import a clone of a DFS Replication database
Import-DfsrClone -Volume h: -Path "h:\Dfsrclone"
9) Add the Destination Server as a DFSR member
$DfsrSourceComputerName = "DSM1"
$DfsrDestinationComputerName = "DSM2"
$DfsrReplicationGroupName = "MyDFSR01"
$DfsrReplicatedFolderName = "MyDFSR01"
$DfsrReplicatedFolderPath = "E:\MyDFSR"
Add-DfsrMember -GroupName $DfsrReplicationGroupName -ComputerName $DfsrDestinationComputerName
Add-DfsrConnection -GroupName $DfsrReplicationGroupName -SourceComputerName $DfsrSourceComputerName –DestinationComputerName $DfsrDestinationComputerName
Set-DfsrMembership -GroupName $DfsrReplicationGroupName -FolderName $DfsrReplicatedFolderName -ContentPath $DfsrReplicatedFolderPath -ComputerName $DfsrDestinationComputerName
10) The RG State on the destination server should be at state 4.
11) End
