Deploy Clustered Storage Spaces

Applies To: Windows Server 2012

This topic describes how to deploy clustered storage spaces by using Storage Spaces and Failover Clustering in Windows Server. Together, these technologies provide a resilient, highly available, and cost-efficient solution that you can scale from simple deployments to the needs of a large datacenter. You can build a failover cluster for your physical workloads or for virtual workloads that are available through the Hyper-V role.
The basic building block of a clustered storage spaces deployment is a small collection of servers, typically two to four, and a set of shared Serial Attached SCSI (SAS) just-a-bunch-of-disks (JBOD) enclosures. The JBOD enclosures should be connected to all the servers, where each server has redundant paths to all the disks in each JBOD enclosure. The following figure shows an example of the basic building block.
Figure 1: Example of clustered storage spaces
By using Cluster Shared Volumes (CSVs), you can unify storage access into a single namespace for ease of management. A common namespace folder is created at the path C:\ClusterStorage\ with all the CSVs in the failover cluster. All cluster nodes can access a CSV at the same time, regardless of the number of servers, the number of JBOD enclosures, or the number of provisioned virtual disks. This unified namespace enables highly available workloads to transparently fail over to another server if a server failure occurs. It also enables you to easily take a server offline for maintenance.
Clustered storage spaces can help protect against the following risks:

• Physical disk failures   When you deploy a clustered storage space, protection against physical disk failures is provided by creating storage spaces with the mirror or parity resiliency types. Additionally, mirror spaces use dirty region tracking (DRT) to track modifications to the disks in the pool. When the system resumes from a power fault or a hard reset event and the spaces are brought back online, DRT makes disks in the pool consistent with each other.
  
• Data access failures   If you have redundancy at all levels, you can protect against failed components, such as a failed cable from the enclosure to the server, a failed SAS adapter, power faults or failure of a JBOD backplane. For example, in an enterprise deployment, you should have redundant SAS adapters, SAS I/O modules, and power supplies. To protect against complete disk enclosure failure, you can use redundant JBOD enclosures.
  
• Data corruptions and volume unavailability   Both the NTFS file system and the Resilient File System (ReFS) help protect against corruption. For NTFS, improvements to the Chkdsk tool in Windows Server 2012 can greatly improve availability. For more information, see NTFS Health and Chkdsk. If you deploy highly available file servers (without using CSVs), you can use ReFS to enable high levels of scalability, high availability, and data integrity regardless of hardware or software failures. For more information about ReFS, see Resilient File System Overview.
  
• Server node failures   Through the Failover Clustering feature, you can provide high availability for the underlying storage and workloads. This helps protect against server failure and enables you to take a server offline for maintenance without service interruption.
  

Note
You cannot use a storage space to host the Windows system drive.

In this topic

• Prerequisites
  
  
• Step 1: Enable MPIO on each server
  
  
• Step 2: Verify that all servers can see the shared disks
  
  
• Step 3: (optional) Create storage spaces through File and Storage Services
  
  
• Step 4: Create a failover cluster
  
  
• Step 5: (optional) Create clustered storage spaces in Failover Cluster Manager
  
  
• Step 6: (optional) Add a cluster disk to a CSV
  
  

This topic includes two methods that you can use to create a clustered storage space. You can do either of the following:

• Configure storage spaces in File and Storage Services before you create the failover cluster, and then add the storage spaces as eligible storage during failover cluster creation. If you want to use this method, follow steps 1 through 4 in this topic. Optionally, you can complete step 6 to add the volume to a CSV.
  
• Create a clustered storage space by using Failover Cluster Manager after you create a failover cluster. If you want to use this method, skip step 3 in this topic. Optionally, you can complete step 6 to add the volume to a CSV.
  

Note
This topic includes sample Windows PowerShell cmdlets that you can use to automate some of the procedures described. For more information, see Using Cmdlets.

Prerequisites

To support clustered storage spaces, your configuration must meet the prerequisites that are outlined in the following table.

 

Area	Requirement
Operating system	Storage Spaces and the Failover Clustering feature are supported in all editions of Windows Server 2012 R2 and Windows Server 2012. This includes Server Core installations.
Disk bus type	The disk bus type must be SAS. Note We recommend dual-port SAS drives for redundancy. Storage Spaces does not support iSCSI and Fibre Channel controllers.
Resiliency type	The simple, mirror, and parity resiliency types are supported for a failover cluster. However, to protect against physical disk failure, you must use the mirror resiliency type, and the parity resiliency type is not supported on failover clusters running Windows Server 2012. Warning Do not use a simple space to host irreplaceable data.
Disk configuration	For a failover cluster, you must have at least three physical disks in addition to any hot spare disks. Also, realize the following: For two-way mirroring on failover clusters, you must have at least three physical disks. For three-way mirroring on failover clusters, you must have at least five physical disks. The disks must be blank and not formatted. Do not create volumes. All disks must be at least 4 GB. The physical disks that are used for a clustered storage pool must be dedicated to that pool. You cannot add startup disks.
HBA considerations	We recommend that you use simple host bus adapters (HBAs) that do not support RAID functionality. If RAID capable, HBAs must be in non-RAID mode with all RAID functionality disabled. Adapters must not abstract the physical disks, cache data, or obscure any attached devices. This includes enclosure services that are provided by attached JBOD devices. Storage Spaces is compatible only with HBAs where you can completely disable all RAID functionality.
JBOD enclosures	For full Storage Spaces functionality, verify with your storage vendor that the JBOD enclosure supports Storage Spaces. Disks must provide persistent reservations and pass cluster validation. To determine whether the JBOD enclosure supports enclosure and slot identification, run the following Windows PowerShell cmdlet: Get-PhysicalDisk | ? {$_.BusType –eq “SAS”} | fc If the EnclosureNumber and SlotNumber fields have values, this indicates that the enclosure supports these features.

Step 1: Enable MPIO on each server

With highly available storage, where there are multiple paths from the physical disks in the storage enclosure to the server, you must use multipath I/O (MPIO) software. The Windows Multipath I/O feature combines the multiple physical paths to physical disks as a single logical path that is used for data access. Also, it provides resiliency to a failure of a physical path and load balancing across physical paths.
For multipath support, you must enable MPIO on each server that can access the shared storage.
Do this step using Windows PowerShell

To enable MPIO

1. In Server Manager, click the server on which you want to install MPIO.
   
2. On the Manage menu, click Add Roles and Features.
   
3. On the Select features page of the Add Roles and Features Wizard, select the Multipath I/O check box.
   
4. Complete the Add Roles and Features Wizard.
   
5. Repeat this procedure for each server that you want to add as a failover cluster node.
   

Windows PowerShell equivalent commands
The following Windows PowerShell cmdlet or cmdlets perform the same function as the preceding procedure. Enter each cmdlet on a single line, even though they may appear word-wrapped across several lines here because of formatting constraints.
The following example installs the MPIO feature on a server that is named Server1.

Install-WindowsFeature –ComputerName Server1 –Name Multipath-IO

Step 2: Verify that all servers can see the shared disks

Before you create the failover cluster, verify that all servers that you want to add as cluster nodes can see the shared disks. You can do this by using Server Manager, the Disk Management tool, or through Windows PowerShell.
Do this step using Windows PowerShell

To verify that a server can see the shared disks

1. To verify from one console that all servers can see the shared disks, make sure that you add all computers that you want to add as cluster nodes to Server Manager.
   
2. In Server Manager, click File and Storage Services, and then under Volumes, click Disks.
   
3. Under each server, verify that the shared disks are listed. Also, verify the following:
   
   • If you want to create storage spaces before you create the failover cluster, the disks do not have to be online or initialized. The disks must not be partitioned.
     
   • If you want to create a clustered storage space by using Failover Cluster Manager after you create a failover cluster, the disks must be online and initialized without partitions.
     

Windows PowerShell equivalent commands
The following Windows PowerShell cmdlet or cmdlets perform the same function as the preceding procedure. Enter each cmdlet on a single line, even though they may appear word-wrapped across several lines here because of formatting constraints.
The following example lists the disks that are visible to the server. This includes their online or offline status and excludes the system disks.

Get-Disk | Where-Object IsSystem –eq $False

The following example lists the disks that are available for the storage pool, where the bus type is SAS.

Get-PhysicalDisk –CanPool $true | Where-Object BusType –eq “SAS”

The following example initializes the disks that you specify. You can obtain the disk numbers from the output of the Get-Disk cmdlet.

Initialize-Disk 1, 2, 3

Step 3: (optional) Create storage spaces through File and Storage Services

This step shows how to create storage spaces before you create the failover cluster by using the shared physical disks. When you create the failover cluster, if you designate that you want to add all eligible storage, the storage pools are automatically added as clustered storage pools with the associated virtual disks and volumes.
Skip this step if you want to create a clustered storage space by using Failover Cluster Manager after you create a failover cluster.
Do this step using Windows PowerShell

To create storage spaces through File and Storage Services

1. On one of the servers that you want to add as a cluster node, configure storage spaces as described in Deploy Storage Spaces on a Stand-Alone Server. However, to use with failover clustering, make sure that you do the following:
   
   • Verify that the disks meet the requirements that are outlined in the Prerequisites section of this topic. Only SAS disks are supported in a clustered storage space.
     
   • When you create a storage pool, you must select at least three physical disks. If you want to use the mirror resiliency type to protect against disk failure, make sure that you select at least three physical disks for a two-way mirror or at least five physical disks for a three-way mirror.
     
   • When you create virtual disks, you must do the following:
     
     
     • Specify the storage layout.
       
     • Specify Fixed as the provisioning type.
       
     • If you have multiple JBOD enclosures, and you want Storage Spaces to try to construct a storage space across them to help protect against complete JBOD failure, you must use Windows PowerShell to create virtual disks. For example, the following cmdlet shows how to create a virtual disk with the IsEnclosureAware $True parameter.
       
       
       New-VirtualDisk –StoragePoolFriendlyName StoragePool –FriendlyName VirtualDisk1 –ResiliencySettingName Mirror –ProvisioningType Fixed –Size 10TB –IsEnclosureAware $True
       

   • When you create volumes, you must format the volumes with NTFS if you want to add the volumes to CSVs.
     

Windows PowerShell equivalent commands
The Deploy Storage Spaces on a Stand-Alone Server topic includes the associated Windows PowerShell examples.

Step 4: Create a failover cluster

This step shows how to install a failover cluster that you can use to host clustered storage spaces.

Note
For more information about failover clustering, see Failover Clustering Overview.

Do this step using Windows PowerShell

To create the failover cluster

1. Install the Failover Clustering feature on each node. For more information, see Install or Uninstall Roles, Role Services, or Features.
   
2. On one of the nodes, open Failover Cluster Manager, and then under Management, click Validate Configuration to validate the cluster. When you run the validation tests, make sure that you include the storage tests.
   
3. When you create the cluster, do either of the following:
   
   • If you have already created storage spaces in File and Storage Services, on the Confirmation page of the Create Cluster Wizard, verify that the Add all eligible storage to the cluster check box is selected.
     
   • If you have not yet created any storage spaces, make sure that you do not add the physical disks as cluster disks. Therefore, on the Confirmation page of the Create Cluster Wizard, clear the Add all eligible storage to the cluster check box.
     
4. If you had existing storage pools that you added to the failover cluster as eligible storage, after you create the failover cluster, verify that the storage appears in Failover Cluster Manager. To do this, follow these steps:
   
   1. In Failover Cluster Manager, expand ClusterName, expand Storage, and then click Pools.
      
   2. In the Pools pane, verify that any storage pools that you created in File and Storage Services are listed.
      
      

      Tip
      For detailed information, click a storage pool. In the results pane, you can view information on the Summary, Virtual Disks, or the Physical Disks tabs.

   3. To verify that virtual disks were added as cluster disks, under the Storage node, click Disks. To add a cluster disk to a CSV, continue to Step 6: (optional) Add a cluster disk to a CSV.
      

   Note
   To create clustered storage spaces through Failover Cluster Manager, continue to step 5.

   
   

Windows PowerShell equivalent commands
The following Windows PowerShell cmdlet or cmdlets perform the same function as the preceding procedure. Enter each cmdlet on a single line, even though they may appear word-wrapped across several lines here because of formatting constraints.
The following example installs the Failover Clustering feature.

Install-WindowsFeature –Name Failover-Clustering –IncludeManagementTools

The following example runs all cluster validation tests on computers that are named Server1 and Server2.

Test-Cluster –Node Server1, Server2

Note
The Test-Cluster cmdlet outputs the results to a log file in the current working directory, for example C:\Users\username\AppData\Local\Temp.

The following example creates a failover cluster that is named MyCluster with nodes Server1 and Server2, assigns the static IP address 192.168.1.12, and adds all eligible storage to the failover cluster. If you have already created storage spaces in File and Storage Services, add all eligible storage.

New-Cluster –Name MyCluster –Node Server1, Server2 –StaticAddress 192.168.1.12

The following example creates the same failover cluster as in the previous example, but it does not add eligible storage to the failover cluster. If you have not yet created any storage spaces, make sure that you do not add eligible storage.

New-Cluster –Name MyCluster –Node Server1, Server2 –StaticAddress 192.168.1.12 -NoStorage

Step 5: (optional) Create clustered storage spaces in Failover Cluster Manager

If you have an existing failover cluster, you can create clustered storage spaces through Failover Cluster Manager.

Tip

For an existing failover cluster, you can also create storage spaces in File and Storage Services, and then add the storage space in Failover Cluster Manager. To do this, in Failover Cluster Manager, expand ClusterName, expand Storage, right-click Pools, and then click Add Storage Pool. In the Add Pools to a Cluster dialog box, select the check box next to the pool, and then click OK. To add virtual disks and associated volumes, right-click Disks, and then click Add Disk. In the Add Disks to a Cluster dialog box, select the check box next to each cluster disk that you want to add, and then click OK.

Important
To have Storage Spaces try to construct a mirror access across multiple supported JBOD enclosures, you must use Windows PowerShell to create the virtual disk by using the IsEnclosure $True parameter. For more information, see the second example in the Create a virtual disk section of this step.

Do this step using Windows PowerShell

To create clustered storage spaces in Failover Cluster Manager

1. In Failover Cluster Manager, expand ClusterName, and then expand Storage.
   
2. Right-click Pools, and then click New Storage Pool.
   The New Storage Pool Wizard opens.
   
3. On the Before you begin page, click Next.
   
4. On the Specify a storage pool name and subsystem page, enter a name and optional description for the storage pool, select the group of available physical disks that you want to use, and then click Next.
   

   Note
   In the Available to column, verify that the cluster name is listed.

   
   
5. On the Select physical disks for the storage pool page, do the following, and then click Next:
   
   1. Select the check box next to each physical disk that you want to include in the storage pool. For a failover cluster, you must select at least three physical disks.
      
   2. If you want to designate one or more disks as hot spares, under Allocation, click the drop-down arrow, and then click Hot Spare.
      
6. On the Confirm selections page, verify that the settings are correct, and then click Create.
   
7. On the View results page, verify that all tasks are completed, select the Create a virtual disk when this wizard closes check box, and then click Close.
   The New Virtual Disk Wizard opens.
   
8. On the Before you begin page of the New Virtual Disk Wizard, click Next.
   
9. On the Select the storage pool page, click the desired storage pool, and then click Next.
   
10. On the Specify the virtual disk name page, enter a name and optional description for the virtual disk, and then click Next.
    
11. On the Select the storage layout page, click the desired layout, and then click Next.
    

    Note
    For a failover cluster, you can select either Simple or Mirror. For resiliency to recover from disk failure, you must select Mirror.

    
    
12. If you selected Mirror in the previous step, and you have at least five physical disks, the Configure the resiliency settings page appears. Select one of the following options:
    
    • Two-way mirror
      
      
    • Three-way mirror
      
      
13. On the Specify the size of the virtual disk page, do either of the following, and then click Next:
    
    • To specify a size, click Specify size, enter a value in the Virtual disk size box, and then select the units (MB, GB or TB).
      
      If you use a mirror storage layout, the virtual disk uses more free space than the size that you specify. To avoid a potential error where the size of the volume exceeds the storage pool free space, you can select the Create the largest virtual disk possible, up to the specified size check box.
      
    • To create a virtual disk that uses the maximum capacity of the storage pool, click Maximum size.
      
14. On the Confirm selections page, verify that the settings are correct, and then click Create.
    
15. On the View results page, verify that all tasks completed, make sure that the Create a volume when this wizard closes check box is selected, and then click Close.
    

    Important
    By default, the Create a volume when this wizard closes check box is selected. If you clear the check box, you must create a volume through File and Storage Services in Server Manager or through Windows PowerShell. You cannot create the volume through Failover Cluster Manager.

    
    
16. On the Before you begin page of the New Volume Wizard, click Next.
    
17. On the Select the server and disk page, do the following, and then click Next.
    
    1. In the Server area, click the failover cluster on which you want to provision the volume.
       
    2. In the Disk area, click the virtual disk on which you want to create the volume.
       
18. On the Specify the size of the volume page, enter a volume size, specify the units (MB, GB, or TB), and then click Next.
    
19. On the Assign to a drive letter or folder page, configure the desired option, and then click Next.
    
20. On the Select file system settings page, do the following, and then click Next.
    
    1. In the File system list, click NTFS or ReFS.
       
       

       Important
       If you want to add the virtual disk to a CSV, you must use NTFS. In Windows Server 2012, ReFS does not support CSVs, data deduplication, or File Server Resource Manager (FSRM).

    2. In the Allocation unit size list, either leave the setting at Default or set the allocation unit size.
       
       

       Note
       For more information about allocation unit size, see Default cluster size for NTFS, FAT, and exFAT.

    3. Optionally, in the Volume label box, enter a volume label name, for example HR Data.
       
21. On the Confirm selections page, verify that the settings are correct, and then click Create.
    
22. On the View results page, verify that all tasks completed, and then click Close.
    
23. To verify that the storage pool with associated virtual disks and volumes was added to the cluster, follow these steps:
    
    1. In Failover Cluster Manager, expand ClusterName, expand Storage, and then click Pools.
       
    2. In the Pools pane, verify that the storage pool is listed.
       
       

       Tip
       For more detailed information, click the storage pool. In the results pane, you can view information on the Summary, Virtual Disks, or the Physical Disks tabs.

    3. To verify that virtual disks were added as cluster disks, under the Storage node, click Disks. Make sure that cluster disks are listed, for example, Cluster Disk 1. Click a cluster disk, and then view the information in the results pane. The results pane shows information such as the virtual disk name, the health, the resiliency type, and volume information.
       
       

       Tip
       You can also view the volumes on the Volumes page of File and Storage Services in Server Manager. Or, in Windows Explorer, you can view the volumes on the node that owns the cluster disk.

24. To add a cluster disk to a CSV, continue to Step 6: (optional) Add a cluster disk to a CSV.
    

Windows PowerShell equivalent commands
The following Windows PowerShell cmdlet or cmdlets perform the same function as the preceding procedure. Enter each cmdlet on a single line, even though they may appear word-wrapped across several lines here because of formatting constraints.
1. Create a storage pool
The following example creates a storage pool StoragePool1 that uses all available SAS disks. It also sets the fixed provisioning type as the default.

$physdisk = Get-PhysicalDisk –CanPool $true | Where-Object BusType –eq “SAS”
$stsubsys = Get-StorageSubsystem -FriendlyName "Storage Spaces*"
$stsubsys | New-StoragePool –FriendlyName StoragePool1 –PhysicalDisks $physdisk –ProvisioningTypeDefault Fixed

The following example sequence of cmdlets creates a storage pool StoragePool1 that uses three physical disks: PhysicalDisk1, PhysicalDisk2, and PhysicalDisk3. It also sets the fixed provisioning type as the default.

$physdisk = Get-StorageSubSystem -FriendlyName "Storage Spaces*" | Get-PhysicalDisk PhysicalDisk1, PhysicalDisk2, PhysicalDisk3
New-StoragePool –FriendlyName StoragePool1 –StorageSubsystemFriendlyName "Storage Spaces*" –PhysicalDisks $physdisk –ProvisioningTypeDefault Fixed

The following example adds the hot spare disk that you select to StoragePool1.

$hotsparedisk = Get-PhysicalDisk –CanPool $true | Out-GridView –PassThru
Add-PhysicalDisk –StoragePoolFriendlyName StoragePool1 –PhysicalDisks $hotsparedisk –Usage HotSpare

2. Create a virtual disk
The following example creates a 10 GB virtual disk VirtualDisk1 with the mirror resiliency type in the storage pool StoragePool1.

$newspace = New-VirtualDisk –StoragePoolFriendlyName StoragePool1 –FriendlyName VirtualDisk1 –ResiliencySettingName Mirror –Size (10GB)

The following example is the same as the previous example, but it adds the IsEnclosureAware $True parameter. Specify this parameter if you want Storage Spaces to try to construct a mirror across multiple supported JBOD enclosures.

$newspace = New-VirtualDisk –StoragePoolFriendlyName StoragePool1 –FriendlyName VirtualDisk1 –ResiliencySettingName Mirror –Size (10GB) –IsEnclosureAware $True

3. Create a volume
The following example sequence of cmdlets initializes the virtual disk that you created in the previous step, creates a partition with the default partition style of GUID partition table (GPT), assigns a drive letter, and then formats the volume with the default of NTFS.

$newvol = $newspace | Get-Disk
Initialize-Disk –Number $newvol.Number
$partition = New-Partition –DiskNumber $newvol.Number –AssignDriveLetter –UseMaximumSize
Format-Volume –Partition $partition

4. Add the virtual disk to the cluster
The following example sequence of cmdlets adds the virtual disk VirtualDisk1 to the failover cluster.

$space = Get-VirtualDisk –FriendlyName VirtualDisk1
Add-ClusterDisk $space

Step 6: (optional) Add a cluster disk to a CSV

This step shows how to add a cluster disk to a CSV. By doing this, the clustered storage is available across a uniform namespace on all nodes. By default, this namespace is C:\ClusterStorage, where C:\ represents the system disk.
Do this step using Windows PowerShell

To add a cluster disk to a CSV

1. In Failover Cluster Manager, expand ClusterName, expand Storage, and then click Disks.
   
2. Right-click a cluster disk, and then click Add to Cluster Shared Volumes.
   The Assigned To column changes to Cluster Shared Volume.
   

Windows PowerShell equivalent commands
The following Windows PowerShell cmdlet or cmdlets perform the same function as the preceding procedure. Enter each cmdlet on a single line, even though they may appear word-wrapped across several lines here because of formatting constraints.
The following example adds the cluster disk Cluster Disk 1 to a CSV on the local failover cluster.

Add-ClusterSharedVolume –Name “Cluster Disk 1”

See also

• Storage Spaces Overview
  
  
• Deploy Storage Spaces on a Stand-Alone Server
  
  
• Storage Cmdlets in Windows PowerShell
  
  
• Use Cluster Shared Volumes in a Windows Server 2012 Failover Cluster
  
  
• Storage Spaces Frequently Asked Questions (FAQ)