Building resilient IT systems – File Replication
Some history about the File Replication found in Windows Servers
Since the days of Windows Server 2000, Microsoft has been providing distributed file system as an add on to the server systems.
This service known as File Replication System (FRS) basically detects changed files and copies the entire file to all the servers associated with the FRS.
In Windows Server 2003 R2, Microsoft shipped an updated version of FRS known as Distributed File System Replication (DFSR) which detects the changes in the files and only copies these changes to the other servers, using a technology called Remote Differential Compression (RDC), along with giving you the ability to schedule network usage.
One major caveat to using this technology is the fact that it only works with closed files It is not a suitable candidate for any files which are constantly opened e.g database files.
What is the purpose of DFS and DFSR
Distributed File System aka DFS is the technology where you access a share via a domain name. So instead of the traditional \\server\fileshare, you will instead type \\domain\fileshare.
There are several purposes of DFS
- Load balance usage: There is a limit to how many shares can be created and maintained by a single server, with DFS you can effectively load balance multiple servers to increase the number of available connections.
- Reduce network bandwidth (2003 R2 and above): Since only the changes are compressed and sent, it drastically reduces the amount of data that is being sent. Do note though that this comes at the expense of transmission speed and CPU clock cycles (Windows first needs to find the changes and then compress them before sending it over the pipe)
- Added redundancies: Althought DFS works with at least one server, adding an additional server effectively provides a basic form of redundancy to guard against single server failure. The cost to this then is that the storage requirement of any file is doubled or tripled depending on the number of servers participating in this share.
- Virtualize shares: Since you can effectively remap any share to a main domain share, you are able to then create your own share directory structure and DFS will automatically map it to the appropriate file share for you
- Ease of adding new hosts: Utilizing both DFS and DFSR, adding and synchronizing the new host is done with just a few clicks. DFSR will automatically synchronize the file contents for you in the backgroud
- Ease of recovery: Should you have a host that went down unexpectedly, once it is up, DFSR will automatically synchronize the changes to the host
The main role of DFSR is purely to replicate the data for the various DFS shares. One of the main things to note is that although remote differential is good in a WAN setup, if your servers are all connected in a LAN (at least 1gbps), then it becomes a hindrance and you will do better to turn it off. The steps can be found here.
Guides to setup DFS
There are already detailed steps for setting up DFS and DFSR for the various Windows Server editions at technet, just click on the server edition which you are using to get the guide.
Important links you should take a look at before using DFS