Moving forward in the cloud world with software-defined storage

[Guest Microsoft Windows Server]

Over the coming weeks, we’ll be publishing more on Windows Server 2016 and the key capabilities coming in the next wave of Microsoft datacenter solutions. To start off, we’re looking at storage and its evolution. Stay tuned for more on software-defined datacenter and the modern application platform.

 

 

Storage is a foundational component of the datacenter fabric and is an intrinsic part of Microsoft’s software-defined datacenter solution. Our storage investments are centered on bringing value to customers in terms of increasing cloud scale, availability, performance, and reliability, while lowering acquisition and operational costs - with Windows Server, and now also with Microsoft Azure Stack.

 

Microsoft storage strategy

 

 

Microsoft provides customers with a consistent storage platform across the entire cloud spectrum – private, hybrid and public. This allows customers to seamlessly move their data based on their platform requirements.

 

• On-Premises Storage: With Storage Spaces (Software Defined Storage) and traditional SAN/NAS connectivity to support Private Clouds and Virtualized environments. In addition, the Microsoft Azure Stack will be available in the future.
  
• Hybrid Cloud Storage: With StorSimple and third party solutions providing on-premises storage integrated with Azure Storage. Azure Site Recovery and Backup provides availability on demand.
  
• Public Cloud Storage: With Microsoft Azure Storage for scalable, reliable, and economical cloud storage for data big and small.
  

Storage choice on-premises

 

 

Microsoft offers an industry leading portfolio for building on-premises clouds. We embrace your choice of storage for your cloud – be it traditional SAN/NAS or the more cost-effective Software-defined Storage Solutions using Storage Spaces Direct and Storage Spaces with Shared JBODs.

 

 

Hyper-converged infrastructure

 

 

In Windows Server 2016, we provide Hyper-converged infrastructure with Storage Spaces Direct (discussed in detail later in this post). The Microsoft Hyper-converged solution offers the following advantages:

 

• Cloud design points and management with standard servers and local storage. It introduces new device types such as SATA and NVMe SSD. Once deployed management tools are available through System Center Virtual Machine Manager (SCVMM), System Center Operations Manager (SCOM) and PowerShell.
  
• Reliability, scalability and flexibility: This solution is fault tolerant to disk, enclosure and node failures. It scales pools to a large number of drives with simple and fine grained expansion available. VM creation performance and snapshotting has been optimized.
  
• Simplifies the datacenter by collapsing storage and compute. The storage area network is no longer necessary with a software service acting as a storage controller.
  

 

 

Converged software defined storage

 

 

Microsoft envisions a low cost commodity scale-out compute layer. It is cost prohibitive to put expensive HBA’s in each compute node. Ethernet is quickly becoming the fabric of the datacenter. With the support of offloads (like RDMA), SMB over Ethernet has proven to be an on par, if not a better, alternative to SAN in terms of performance ($/IOPS and latency). It is superior in terms of cost savings, ease of use, and scalability. Back end storage in this configuration is NAS, which can be either third party NAS or a Software-defined Storage NAS solution, consisting of a Scale-out File Server as a NAS head which is backed by Storage Spaces Direct.

 

 

Choice of converged infrastructure

 

 

Microsoft’s on-premises strategy is for a converged/disaggregated architecture running Hyper-V over SMB. That could be either to a 3rd party NAS device, or to a Software-defined Storage solution with a Scale-out File Server as a NAS head which is backed by either traditional SANs, Storage Spaces Direct or Storage Spaces with shared JBOD’s.

 

 

Innovations for on-premises storage in Windows Server 2016

 

 

Here are some of the exiting storage innovations available for you to test in the Windows Server 2016 Technical Preview 4 release:

 

Storage Quality of Service (QoS) for more control and performance - We began the Storage QoS journey in 2012 R2. Customers can scale storage QoS across unified storage and Hyper-V clusters. We are providing flexible ways for IT Pros to define min and max IOPs for storage resources (could be a VHD on a VM, a single VM, or a group of VMs in a service). You can put IO caps on rogue VMs and enable IO min, max boundaries on VMs or specific VHDs. For example, a Log VHD in your SQL Server VM may need to have a higher IOPs bar. Storage QoS delivers IO traffic shaping to the datacenter.

 

Storage Replica enables storage-agnostic, block-level, synchronous replication between servers or clusters for disaster recovery, as well as stretching of a failover cluster between sites. Synchronous replication enables mirroring of data in physical sites with crash-consistent volumes to ensure zero data loss at the file-system level. Asynchronous replication allows site extension beyond metropolitan ranges with the possibility of data loss. With Storage Replica, customers can now have mission critical business continuance and disaster recovery at affordable price points.

 

Azure consistent object storage enables Azure Object storage (Tables and Blobs) in the private cloud. If a developer writes a cloud born application to Azure storage APIs, the same application can run in the private cloud with minimal changes.

 

VM Storage resiliency for protecting virtual machines from underlying transient storage failures. VM storage resiliency will monitor the state of storage and gracefully pause VMs and then resume them when storage is available again. With the VMs gracefully responding to the state of storage, it will reduce impact and increase availability of workloads running in virtual machines when storage is unreliable.

 

Deduplication: We have redesigned the optimization processing to be fully parallelized for individual volumes, allowing for simplified deployment using up to 64TB volumes. In addition, performance improvements allow usage of up to 1TB files without restrictions. The use of dedup to save storage space with virtualized backup solutions has been greatly simplified with the addition of the new "Backup" usage type. Dedup is also fully supported in Nano Server (the new installation option in Windows Server Technical Preview that provides a cloud-optimized Windows Server environment).

 

Storage Health Service for easier health monitoring and more streamlined operations. New cmdlets make collecting aggregated performance and capacity metrics simple and fast. Faults and health information bubble up to a single monitoring point per cluster. New in-box intelligence determines the root cause of faults to reduce chattiness, understand severity, and recommend next steps, including providing helpful physical location and part information for disk replacement. New automation retires failed physical disks, removes them from their pool, and adds their replacements to the same pool, all while kicking off the requisite repair and rebuild jobs.

 

Rolling upgrades empowers you to upgrade your Storage clusters (and Hyper-V compute clusters) seamlessly to the latest OS versions without incurring any downtime. This mechanism can also be used to upgrade your cluster nodes from Windows Server 2012 R2 Full SKU to Windows Server 2016 Nano SKU. Rolling upgrades can also be orchestrated through Virtual Machine Manager (VMM).

 

Storage Spaces Direct enables service providers and enterprises to use industry standard servers with local storage to build highly available and scalable software defined storage. Using servers with local storage decreases complexity, increases scalability, and enables use of storage devices that were not previously possible, such as SATA solid state disks for lower cost flash storage, or NVMe solid state disks for better performance. Storage Spaces Direct removes the need for a shared SAS fabric, simplifying deployment and configuration. Instead it uses the network as a storage fabric, leveraging our investments in SMB3 and SMB Direct (RDMA) for high speed and low latency storage. To scale out, simply add more servers to increase storage capacity and IO performance. Storage Spaces Direct supports both converged and hyper-converged deployment modes enabling customer choice. Converged, with storage and compute in separate tiers, for independent scaling and management and hyper-converged, with compute and storage collocated on the same servers, for simple deployment. With Windows Server 2016 Technical Preview 4, Storage Spaces Direct adds supports for all-flash configurations with NVMe SSD and SATA SSD devices, and Erasure Coding for increased storage efficiency.

 

 

The following are the development partners who will offer solutions with Storage Spaces Direct.

 

 

 

System Center and PowerShell for managing Microsoft SDS, SAN, NAS devices and FC fabric switches in a private cloud. At the heart of management is the Storage Management API (SMAPI), a common management interface used by System Center. In VMM, administrators can provision storage, classify storage, and present storage to Hyper-V hosts, clusters, and VM guests. Operations Manager (OM) can monitor storage devices managed by VMM.

 

We are announcing System Center support for Spaces Direct storage. System Center can bare-metal provision a Scale-out File Server configured with Spaces Direct, manage storage pools, create scale-out file shares, deploy virtual machines using rapid provisioning, manage Storage QoS policies, update and service cluster nodes, monitor operational and health status of storage, alerting with actionable alerts.

 

Wrapping up and useful links

 

 

We are offering customers choice for their private cloud storage needs - our rich ecosystem of SAN, NAS partner storage or Microsoft Software Defined Storage.

 

 

Here are a few links to deployment and experience guides – Storage QoS, Storage Replica, Rolling Upgrades, and Storage Spaces Direct.

 

Try our scenarios, we would love to hear your feedback in the Windows Server Preview discussion forum or our new Windows Server – Storage user voice.

 

 

Continue reading...