Category Archives: Uncategorized

Uncategorized

Why HCI Matters in the Datacenter and Beyond

Leave a comment

Technology is changing and evolving at an ever-increasing pace, whether you are a consumer of electronics, or you are a CEO of a large organisation with a large IT infrastructure, the changes in technology affect us all in different ways.  An example of this is CPUs and Flash Storage, we’re now at an era of constantly increasing CPU Core densities, and Flash Storage is becoming bigger and faster, these technology transformations are not only changing the way we operate as human beings in our own personal IT bubbles at home, but also within organisations too.

As organisations large and small take on the whole business transformation, a key element of the business transformation is their IT, whilst the last 15 years IT was more focused around being IT centric with traditional applications and the wide adoption of the internet.  The next 15 years poses some challenges as IT becomes more business centric along with cloud applications and the Internet of Everything.

A key enabler to the whole IT transformation is the Software Defined Data Center, many of you would have heard me talk about the Software Defined Data Center not as an object, but more as an Operating System that runs your IT infrastructure, if you are asked what three things are required to run an operating system?   You’ll find yourself answering storage, compute and networking for connectivity, which is essentially the three key elements that make up the Software Defined Data Center.

Hyper-Converged Infrastructure allows you to deliver capabilities that underpin the whole Software Defined Data Center based on a standard x86 architecture and offers a building block approach, it also brings the storage closer to the CPU and Memory which in a virtualised environment is highly benefitial and it is more VM centric rather than being storage centric.

So why is HCI being adopted by the masses?

There are a number of reasons for this, we’ve already outlined the fact that having the storage closer to the compute delivers a much more efficient platform, but outside of that there is a Harware Evolution which is driving the changes in infrastructure, rather much like an Infrastructure Revolution.

Higher CPU Core densities means you can run much more dense workloads, in conjunction with this, RAM has become much comoditized, affordable and available in larger capacity.  From a storage aspect Flash has evolved in such a way that is has enabled the delivery of high capacity and high performing devices that only a few years ago would have took a whole refrigerator sized array to produce but now can be delivered by a device that you can hold in the palm of your hand.  Another aspect from the storage side of things is that traditional storage is unable to keep up with the demands of applications and IT, this resulted in a new approach to storage and infrastructure….HCI

What is required from your storage platform?

I have met with many customers in various meetings or events, and depending on who you talk to in the organisation you will get a different answer to that question

  • Application Owner – Performance and Scalability
    They need to deliver an application that performs well as well as offers scalability, so the storage has to be able to offer this.
  • Infrastructure Owner – Simplicity and Reliability
    They need the platform to be simple to deploy, simple to manage but also offer reliability, they don’t want to be getting calls in the middle of the night from the Application Owner!
  • CFO / Finance Team – Lower Cost and Operational Efficiency
    There’s always somebody looking at the numbers and it’s usually this side of the organisation, reducing TCO, CAPEX, OPEX and making IT more cost effective is the biggest driver here.

Everyone is aiming for that sweet spot where all three circles converge the only problem is, with traditional infrastructure, you can never satisfy all three of the above requirements, there’s usually one requirement that has to be sacrificed, and that’s usually the Finance Team or CFO that has to back down in order to deliver the requirements for the Application Owner and the Infrastructure Owner,  this is where HCI is different, HCI brings everyone to that central convergence and meets the goals of all the requirements, so now everyone is happy, lets take a closer look at how HCI powered by vSAN meets these requirements


vSAN HCI delivers an architecture that not only delivers on performance but it is scalable simply by adding more nodes, or by adding more storage, it also allows for linear scaling of performance.  This means as your IT or business applications scale and demand more capacity or performance, then this is easily delivered in whatever increments meet the requirements at that point in time.


vSAN HCI allows the infrastructure team to deploy and manage environments at a simple management plane in a single interface, no separate management tools are required which means there’s no extensive retraining of staff required.  Reliability and Resiliency are built in with the ability to protect from a Disk Level all the way up to a Site level.


We’ve already talked about how HCI offers a building block approach, this means environments can be built to meet your requirements now and be grown as and when required.   Because there’s a much simpler management plane, this means operational efficiencies come into play as well, offering a more streamlined approach to IT

At this point we have met all of the criteria set by the three key stakeholders, but the benefits of HCI don’t just stop there there are other positive impacts that HCI brings to your organisation:


vSAN HCI offers a much wider of choice in the hardware that can be used along with different hardware vendors to choose from, there is also the range of different deployment options, this allows organisations to have a lot more flexibility on how they adopt HCI as well as having choices for newer hardware technology at their fingertips, this includes:

  • vSAN Ready nodes from all major server OEM vendors to suit all performance and capacity requirements
  • Turnkey appliance solution from Dell EMC which is VxRAIL
  • VMware Cloud Foundation which incorporates a full SDDC Stack

For deployment options, vSAN HCI offers the following:

  • Standard clusters up to 64 Nodes
  • Remote Office / Branch Office (ROBO) Solutions for customers with multiple sites
  • Stretched Cluster Solutions
  • Disater Recover Solutions
  • Rack Level Solutions
  • Same Site “Server Room” configurations


vSAN HCI allows organisations to become more agile by allowing  faster deployments, faster procurement and giving more control back to the business, which in a competitive world is a key enabler to success

As you can see, no matter what the size of your IT infrastructure is, HCI brings a wealth of benefits, from large scale data center deployments, to multi site ROBO deployments, there’s a perfect fit for HCI

Uncategorized

Sizing for your workloads

Leave a comment

When sizing a vSAN environment there are many considerations to take into account, and with the launch of the new vSAN Sizing tool I thought I would take time and write up what questions I commonly ask people in order to get an understanding of what they want to run on vSAN as well as a scope of requirements that meet that workload.

Capacity
Obviously capacity is going to be our baseline for any sizing activity, no matter what we achieve with the other requirements, we have to meet a usable capacity, remember we should always work off a usable capacity for any sizing, a RAW capacity does not take into account any Failure Tolerance Methodology, Erasure Coding or Dedupe/Compression, this is something we will cover a bit later in this article.

Capacity should also include the required Swap File space for each of the VMs that the environment is being scoped for.

IOPS
I have been involved in many discussions where it is totally unknown what the performance requirements are going to be, so many times I have been told “We want the fastest performance possible” without being told what the current IOPS requirement is, to put it into context what is the point in buying a 200mph sports car when the requirement is to drive at 70mph max!

IOPS requirement plays a key part in determining what level of vSAN Ready Node specification is required, for example if a total IOPS requirement is 300,000 IOPS out of a 10 Node cluster, is there much point spending more money on an All-Flash configuration that delivers 150,000 IOPS Per node?  Simple answer…No!  You could opt for a lower vSAN All-Flash Ready Node config that meets the requirements a lot closer and still offer
room for expansion in the future.

Workload Type
This is a pretty important requirement, for example if your workload is more of a write intensive workload then this would change the cache requirements, it may also require a more write intensive flash technology such as NVMe for example.  If you have different workload types going onto the same cluster it would be worthwhile categorizing those workloads into four categories:-

  • 70/30 Read/Write
  • 80/20 Read/Write
  • 90/10 Read/Write
  • 50/50 Read/Write

Having the VMs in categories will allow you to specify the workload types in the sizing tool (in the advanced options).

vCPU to Physical Core count
This is something that gets overlooked not from a requirement perspective, but people are so used to sizing based on a “VM Per Host” scenario which with the increasing CPU core counts does not fit that model any more, even the new sizing tool bases it on vCPU to Physical Core ratio which makes things a lot easier, most customers I Talk to who are performing a refresh of servers with either 12 or 14 core processors can lower the amount of servers required by increasing the core count on the new servers, thus allowing you to run more vCPUs on a single host.

List of questions for requirements for each workload type

  • Average VMDK Size per VM
  • Average number of VMDKs per VM
  • Average number of vCPU per VM
  • Average vRAM per VM
  • Average IOPS Requirement per VM
  • Number of VMs
  • vCPU to Physical Core Ratio

RAW Capacity versus Usable Capacity, how much do I actually need?
The new sizing tool takes all your requirements into account, even the RAID levels, Dedupe/Compression ratios etc and returns with a RAW Capacity requirement based on the data you enter, if you are like me and prefer to do it quick and dirty, below is table showing you how to work out based on a requirement of 100TB of usable (Including Swap File Space), based on a standard cluster with no stretched capacilities it looks like this:

FTT LevelFTT MethodMin number of hostsMultiplication FactorRAW Capacity Based on 100TB Usable
FTT=0NoneN/A1x100TB
FTT=1Mirror32x200TB
FTT=2Mirror53x300TB
FTT=3Mirror74x400TB
FTT=1RAID541.33x133TB
FTT=2RAID661.5x150TB

Now in vSAN 6.6 VMware introduced localized protection (Secondary FTT) and the ability to include or not include specific objects from the stretched cluster (Primary FTT), below is a table showing what the RAW Capacity requirements are based on the two FTT levels

Primary FTT LevelSecondary FTT LevelSecondary FTT MethodMin Number of hosts per siteRAW Capacity Based on 100TB Usable
PFTT=1SFTT=0RAID01200TB
PFTT=1SFTT=1RAID13400TB
PFTT=1SFTT=1RAID54266TB
PFTT=1SFTT=2RAID66300TB

Mixed FTT levels and FTT Methods
Because vSAN is truly a Software Defined Storage Platform, this means that you can have a mixture of VMs/Objects with varying levels of protection and FTT Methods, for example for Read intensive workloads you may choose to have RAID 5 in the storage policy, and for more write intensive workloads a RAID 1 policy, they can all co-exist on the same vSAN Cluster/Datastore perfectly well, and the new sizing tool allows you to specify different Protection Levels and Methods for each workload type.

 

Uncategorized

Managing Storage Policies using PowerCLI

Leave a comment

It has been a personal project due to a few people asking me in working out how to create and manipulate vSAN Storage Policies from the new PowerCLI commandlets introduced in PowerCLi 6.5, the tasks I will cover here will be

  1. Creating a Brand New Storage Policy
  2. Changing an existing storage policy

For the purpose of this excersise I am using PowerCLI 6.5 Release 1 build 4624819 I owuld recommend you use that build or newer as some of the commands do not work as intended on previous PowerCLI builds.

 

Creating a Storage Policy

Before we proceed with creating a storage policy, you need to ensure you are logged into your vCenter server via PowerCLI and get a list of capabilities from the storage provider, for this we run the command Get-SpbmCapability and filter out the VSAN Policy capabilities

C:\> Get-SpbmCapability VSAN*
Name ValueCollectionType ValueType AllowedValue
---- ------------------- --------- ------------
VSAN.cacheReservation System.Int32
VSAN.checksumDisabled System.Boolean
VSAN.forceProvisioning System.Boolean
VSAN.hostFailuresToTolerate System.Int32
VSAN.iopsLimit System.Int32
VSAN.proportionalCapacity System.Int32
VSAN.replicaPreference System.String
VSAN.stripeWidth System.Int32

For the definitions of the values associated with each definition:-
System.Int32 = Numerical Value
System.Boolean = True or False
System.String = “RAID-1 (Mirroring) – Performance” or “RAID-5/6 (Erasure Coding) – Capacity”

So I want to create a Policy with the following Definitions

Policy Name: PowerCLI-RAID5
Policy Description: RAID5 Policy Created with PowerCLI
Failures to Tolerate: 1
Failures to Tolerate Method: RAID5
Stripe Width: 2
IOPS Limit: 1000

To create a policy with the above specification we need to run the following command:

New-SpbmStoragePolicy -Name PowerCLI-RAID5 -Description "RAID5 Policy Created with PowerCLI" `
-AnyOfRuleSets `
(New-SpbmRuleSet `
(New-SpbmRule -Capability (Get-SpbmCapability -Name "VSAN.hostFailuresToTolerate" ) -Value 1),`
(New-SpbmRule -Capability (Get-SpbmCapability -Name "VSAN.stripeWidth" ) -Value 2),`
(New-SpbmRule -Capability (Get-SpbmCapability -Name "VSAN.replicaPreference" ) -Value "RAID-5/6 (Erasure Coding) - Capacity"),`
(New-SpbmRule -Capability (Get-SpbmCapability -Name "VSAN.iopsLimit" ) -Value 1000)`
)

We can then check for our policy by running the following command:

C:\> Get-SpbmStoragePolicy -Name "PowerCLi-RAID5" |Select-Object -Property *

CreatedBy : Temporary user handle
CreationTime : 11/04/2017 13:42:36
Description : RAID5 Policy Created with PowerCLI
LastUpdatedBy : Temporary user handle
LastUpdatedTime : 11/04/2017 13:42:36
Version : 0
PolicyCategory : REQUIREMENT
AnyOfRuleSets : {(VSAN.hostFailuresToTolerate=1) AND (VSAN.stripeWidth=2) AND (VSAN.replicaPreference=RAID-5/6 (Erasure Coding) - Capacity) AND (VSAN.iopsLimit=1000)}
CommonRule : {}
Name : PowerCLI-RAID5
Id : 9f8f61f4-24db-4513-b48f-a48c584f0eac
Client : VMware.VimAutomation.Storage.Impl.V1.StorageClientImpl

And from the UI We can see our storage policy:

That’s our policy created, now we’ll move onto changing an existing policy.

Changing an existing storage policy

The policy we created earlier, if we suddenly decide that we want RAID 6 rather than RAID 5, we have to change the Failure To Tolerate number to 2, and we will also want to change our policy name and description as it will no longer be RAID5. In order to change a policy, we have to ensure we specify the other values for the other policy definitions, if we do not specify the policy definitions that we do not want to be changed, they will be removed from the storage policy, so first we run the command that tells us what our policy definition is:

C:\> Get-SpbmStoragePolicy -Name "PowerCLi-RAID5" |Select-Object -Property *

CreatedBy : Temporary user handle
CreationTime : 11/04/2017 13:42:36
Description : RAID5 Policy Created with PowerCLI
LastUpdatedBy : Temporary user handle
LastUpdatedTime : 11/04/2017 13:42:36
Version : 0
PolicyCategory : REQUIREMENT
AnyOfRuleSets : {(VSAN.hostFailuresToTolerate=1) AND (VSAN.stripeWidth=2) AND (VSAN.replicaPreference=RAID-5/6 (Erasure Coding) - Capacity) AND (VSAN.iopsLimit=1000)}
CommonRule : {}
Name : PowerCLI-RAID5
Id : 9f8f61f4-24db-4513-b48f-a48c584f0eac
Client : VMware.VimAutomation.Storage.Impl.V1.StorageClientImpl

So we need to make the following changes to the policy

  • Change the Failure to Tolerate value from 1 to 2
  • Change the Name of the Policy from PowerCLI-RAID5 to PowerCLI-RAID6
  • Change the Description of the policy from “RAID5 Policy Created with PowerCLI” to “RAID6 Policy Changed with PowerCLI”

We run the following command which specified the new values, and also the values we do not want to change:

Set-SpbmStoragePolicy -StoragePolicy PowerCLI-RAID5 -Name PowerCLI-RAID6 -Description "RAID6 Policy Changed with PowerCLI" `
-AnyOfRuleSets `
(New-SpbmRuleSet `
(New-SpbmRule -Capability (Get-SpbmCapability -Name "VSAN.hostFailuresToTolerate" ) -Value 2),`
(New-SpbmRule -Capability (Get-SpbmCapability -Name "VSAN.stripeWidth" ) -Value 2),`
(New-SpbmRule -Capability (Get-SpbmCapability -Name "VSAN.replicaPreference" ) -Value "RAID-5/6 (Erasure Coding) - Capacity"),`
(New-SpbmRule -Capability (Get-SpbmCapability -Name "VSAN.iopsLimit" ) -Value 1000)`
)

We can verify the policy has been changed by running the command with the new policy name:

C:\> Get-SpbmStoragePolicy -Name "PowerCLi-RAID6" |Select-Object -Property *

CreatedBy : Temporary user handle
CreationTime : 11/04/2017 13:42:36
Description : RAID6 Policy Changed with PowerCLI
LastUpdatedBy : Temporary user handle
LastUpdatedTime : 11/04/2017 14:03:51
Version : 2
PolicyCategory : REQUIREMENT
AnyOfRuleSets : {(VSAN.hostFailuresToTolerate=2) AND (VSAN.stripeWidth=2) AND (VSAN.replicaPreference=RAID-5/6 (Erasure Coding) - Capacity) AND (VSAN.iopsLimit=1000)}
CommonRule : {}
Name : PowerCLI-RAID6
Id : 9f8f61f4-24db-4513-b48f-a48c584f0eac
Client : VMware.VimAutomation.Storage.Impl.V1.StorageClientImpl

And again in the UI we can see the changes made

That’s how you change an existing storage policy, I hope this blog helps you manage your storage policies via PowerCLI, thanks go out to Pushpesh in VMware for showing me the error of my ways when trying to figure this out myself.