Next-generation storage for the software-defined datacenter

[Guest Microsoft Windows Server]

This is a post from Siddhartha Roy, Group PM Manager, High Availability and Storage and Paul Luber, Group PM Manager, Storage and File Systems

 

Storage is a foundational component of the datacenter fabric and is an intrinsic part of Microsoft’s software-defined datacenter solution. Our storage investments are centered on bringing value to customers in terms of increasing cloud scale, availability, performance, and reliability, while lowering acquisition and operational costs - with Windows Server, and now also with Microsoft Azure Stack.

 

Storage Choice

 

 

Customers can have more than one type of storage, based on their need. We have storage choices across the entire cloud spectrum. Microsoft has a rich ecosystem of Cloud OS Network partners who offer a spectrum of cloud solutions to customers. Specifically, we focus on the following scenarios to enable customer choice:

 

• Private Cloud with Microsoft Software Defined Storage - more on this later
  
• Hybrid Cloud with StorSimple - extend storage beyond the datacenter by tiering cold data to Azure
  
• Microsoft Azure Public Cloud - get instantaneous access to large scale storage on demand
  
• Private Cloud with SAN and NAS storage from our rich partner ecosystem
  

 

 

Let’s explore one of the storage categories - Private Cloud with Software Defined Storage.

 

What is Software Defined Storage or SDS ?

 

The storage industry is going through interesting shifts driven by various factors - large-scale cloud services influencing design points and enabling the use of standard volume hardware by putting more intelligence into storage software. Virtualization is driving the need for mobility and density – containers will push that envelope further. Large levels of scale out ensure that “pay-as-you grow” models are seamless, elastic and fluid.

 

Simply put – SDS is cloud scale storage and cost economics on standard volume hardware.

 

Our SDS journey

 

Our SDS solution was initially released in Windows Server 2012, bringing our public cloud experience to private clouds, and we have been continually evolving since then. Storage Spaces and Scale Out File Server provide a private cloud storage solution on standard volume hardware. We’ve provided storage efficiency capabilities like tiering and de-duplication while maintaining the principles of ease-of-management and operational simplicity. We delivered scale out storage that is load balanced with high performance enabled by SMB Direct. Features like SMB Multichannel, Storage Spaces provide resiliency to storage path, disk failures and more. ReFS file system enables detection and correction of latent bit rot. And much more.

 

Partners - We vet these Windows Server 2012 R2 SDS private cloud scenarios at scale with our private cloud integrated hardware and software solution named Cloud Platform System. We also provide Server 2012 R2 shared JBOD storage solutions from our partners DataOn, Dell, HP, RAID Inc. Storage Spaces solutions using shared SAS JBODs will continue to be supported.

 

Customers - Here are some customer case studies with Storage Spaces, Scale-Out File Servers with SMB3 or both.

 

Azure - Microsoft Azure public cloud storage is shaping and inspiring our private cloud storage journey for service providers and enterprises. Microsoft Azure is a huge listening system for our private cloud storage scenarios. Alongside customer feedback, we are bringing Azure cloud scale and cost economics design points to our private cloud customers.

 

Storage innovation for the software-defined datacenter

 

Now let’s fast forward to Windows Server 2016 and the newly announced Azure Stack. We are continuing the journey to embed more Azure design points in private cloud storage – thereby lowering costs further; not just acquisition costs, but operating costs as well. Let us highlight a few scenarios - unless mentioned otherwise, you can test these scenarios in Windows Server 2016 Technical Preview 2.

 

Storage QoS for more control and performance - We began the Storage QoS journey in 2012 R2. With Technical Preview 2, customers can scale storage QoS across unified storage and Hyper-V clusters. We are providing flexible ways for IT Pros to define min and max IOPs for storage resources (could be a VHD on a VM, a single VM, or a group of VMs in a service). You can put IO caps on rogue VMs and enable IO min, max boundaries on VMs or specific VHDs. For example, a Log VHD in your SQL Server VM may need to have a higher IOPs bar. Storage QoS delivers IO traffic shaping to the datacenter.

 

Storage Replica for protecting mission critical data and workloads. Storage Replica enables synchronous volume replication (with zero data loss) at the volume level. Not just Storage Spaces volumes but any SAN or NAS volume attached to the Windows Server host. You can deploy Storage Replica in multiple modes - stretch cluster across two sites with their own shared storage, between two clusters, or between two standalone servers. With Storage Replica, customers can now have mission critical business continuance and disaster recovery at affordable price points.

 

Azure consistent object storage enables Azure Object storage (Tables and Blobs) in the private cloud. If a developer writes a cloud born application to Azure storage APIs, the same application can run in the private cloud with minimal changes. This scenario will be available in a future preview.

 

VM Storage resiliency for protecting virtual machines from underlying transient storage failures. VM storage resiliency will monitor the state of storage and gracefully pause VMs and then resume them when storage is available again. With the VMs gracefully responding to the state of storage, it will reduce impact and increase availability of workloads running in virtual machines when storage is unreliable.

 

Deduplication - we have redesigned the optimization processing to be fully parallelized for individual volumes, allowing for simplified deployment using up to 64TB volumes. In addition, performance improvements allow usage of up to 1TB files without restrictions. The use of dedup to save storage space with virtualized backup solutions has been greatly simplified with the addition of the new "Backup" usage type.

 

Rolling upgrades empowers you to upgrade your Storage clusters (and Hyper-V compute clusters) seamlessly to the latest OS versions without incurring any downtime.

 

Storage Spaces Direct - we are introducing support for shared nothing storage, which enables more device choice, lowers costs, and increases scale. You can now use Storage Spaces to build private cloud solutions using the internal storage in standard servers, increasing density and rack efficiency. Storage Spaces Direct removes the need for a shared SAS fabric, simplifying configuration and management, and instead uses the network for storage traffic. With Storage Spaces Direct, you can get a large virtualized pool of reliable storage across disks on multiple servers connected by high speed, low cost storage networking. Our investments in RDMA and SMB Direct enable highly available clusters connected by high speed, low cost networks. To scale-out, simply add more servers to increase capacity or IO performance – no SAS cables required.

 

Storage Spaces Direct has two deployment options – converged (disaggregated) with storage and compute in different server tiers for independent scaling, or hyper-converged with storage and compute co-located on the same set of servers. Converged mode is available for evaluation in Technical Preview 2. We will offer hyper-converged mode for evaluation in subsequent previews.

 

 

We are also announcing our preliminary development partners who will offer solutions with Storage Spaces Direct.

 

 

System Center and PowerShell for managing Microsoft SDS, SAN, NAS devices and FC fabric switches in a private cloud. At the heart of management is the Storage Management API (SMAPI), a common management interface used by System Center. In Virtual Machine Manager (VMM), administrators can provision storage, classify storage, and present storage to Hyper-V hosts, clusters, and VM guests. Operations Manager (OM) can monitor storage devices managed by VMM.

 

We are announcing System Center support for Spaces Direct storage. System Center can bare-metal provision a Scale-out File Server configured with Spaces Direct, manage storage pools, create scale-out file shares, deploy virtual machines using rapid provisioning, manage Storage QoS policies, update and service cluster nodes, monitor operational and health status of storage, alerting with actionable alerts.

 

Wrapping up and useful links

 

We are offering customers choice for their private cloud storage needs - our rich ecosystem of SAN, NAS partner storage or Microsoft Software Defined Storage.

 

 

Here are a few links to deployment and experience guides – Storage QoS, Storage Replica, Rolling Upgrades, and Storage Spaces Direct.

 

Please join us at the Ignite conference or visit back here to hear much more. Try our scenarios, we would love to hear your feedback in the Windows Server Preview discussion forum or our new Windows Server – Storage user voice.

 

 

Continue reading...