Calculating Standard Deviation using PowerShell

I have been reading a book on data science recently, and as a result I have had to revisit my high school maths!
One interesting thing I found is that there is more than one way to calculate the end result, and that the standard taught method may not be the most accurate!
The most accurate (according to the experts) is a method created by B.P.Welford, which is detailed in-depth in Donald Knuth’s ‘Art of computer programming’.

My brain thought “I wonder if anyone has implemented this in PowerShell”?

I found it in many difference languages, but not PowerShell, so I thought I would take someone else’s hard work and translate!

The site I choose is https://blog.logentries.com/2016/10/overview-of-online-algorithm-using-standard-deviation-example/#Welford

My version does not yet work as a class where you may ‘pop’ on another value dynamically, it assumes instead that you have the figures, and you would like to work out the Standard Deviation using either a Sample or a Population variance.

The code is as follows:


function Get-StandardDeviation {
    [CmdletBinding()]
    Param (
    # Array of double values
    [Parameter(Mandatory=$true, ValueFromPipelineByPropertyName=$true,ValueFromPipeline=$true)]
    [ValidateNotNullOrEmpty()]
    [double[]]$Values
    )
    Begin {
        $count=0.0
        $mean=0.0
        $sum=0.0
    }#begin

    Process {
        foreach ($value in $values) {
            ++$count
            $delta = $mean + (($value - $mean) / $count)
            $sum += ($value - $mean) * ($value - $delta)
            $mean = $delta
        }#foreach
    } # process

    End {
        $VariancePopulation = $sum/($count)
        $VarianceSample = $sum/($count-1)
        $obj=[PSCustomObject]@{
            "VariancePopulation" = $VariancePopulation
            "VarianceSample" = $VarianceSample
            "STDEVPopulation" = [Math]::Sqrt($VariancePopulation)
            "STDEVSample" = [Math]::Sqrt($VarianceSample)
            "Mean" = $mean
            "Count" = $count
        }#obj
        Write-Output $obj
    } #end

}#function

To test this, create an array of doubles, and then use either method shown below to get the results.

$data = (50.0, 45.0, 55.0, 58.0, 43.0, 49.0, 50.0)</code>

Get-StandardDeviation -Values $data
$data | Get-StandardDeviation

Advertisements

Connecting Virtual Networks in different Azure Subscriptions to an ExpressRoute circuit in Resource Manager

Scenario

I have an ExpressRoute circuit configured with a connection to a Version 2 (Resource Manager) Azure Virtual Network. I also have an additional Virtual Network in a different Subscription which I need to connect to the same circuit.

The Issue

The instructions on connecting another Virtual Network in a different Subscription are a little confusing. It’s also worth noting that some of the parameters are different now with the latest version of the PowerShell Azure cmdlets. The original instructions may be found at this URL: https://azure.microsoft.com/en-us/documentation/articles/expressroute-howto-linkvnet-arm/#connect-a-virtual-network-in-a-different-azure-subscription-to-an-expressroute-circuit

Continue reading

Automatically Naming Resources in an Azure Resource Management Template

Why?

Resource templates are a great concept, but are fraught with danger. Badly named resources and naming inconsistency across resources in different subscriptions can make it difficult to determine purpose.

If a resource such as a Virtual Network or a Storage Account is badly named in a resource template file, you may find that you have to rely on the icon pictures in the Azure portal to indicate what type of resources you are looking at.

What if you need to see at a glance:

  • What region the resource belongs to i.e. Australia East or Australia Southeast?
  • What environment the resource belongs to i.e. Production or Testing?
  • What type of resource it is?

The problem comes from trying to keep the names meaningful and consistent. How do you ensure that your naming standards are adhered to?

Scenario

In my case I would like to name resources as follows:

<Resource Prefix>_<Environment>_<Location>

For instance:

"Vnet_Prod_auSoutheast"
 Continue reading 

Scripting the Migration of Applications from MDT to Configuration Manager 2012

The need to migrate applications from the Microsoft Deployment Toolkit 2013 (MDT) to Configuration Manager (ConfigMgr) 2012 without MDT integration is not uncommon. Companies use MDT as a stand alone proving ground, then decide to migrate the task sequence and applications to a production ConfigMgr system.

Manually migrating the packages however is a time consuming task. It involves:

  • Creating a Package (Including Data Source, Name, Version etc.)
  • Creating a Programme (Including command line, run time parameters etc.)

Enter Powershell.

Two scripts are used in this example:

  • Script one: Exports the application information from MDT.
  • Script two: Creates packages in ConfigMgr based on this information, complete with install command lines.

After this simply copy the application source from the Application folder of the MDT deployment share, to the location you are planning to store package source in for ConfigMgr.

Script 1 : Export the list of applications from MDT to a CSV file

Note: Update variables for:

  • Location of MDT Deployment Share.
  • CSV file location (The file containing application information exported from MDT.
$MDTDeploymentShare = "C:\DeploymentShare"
$ExportPath = "c:\temp\MDTApplication.csv"

Add-PSSnapin Microsoft.BDD.PSSnapIn New-PSDrive -Name MDT -PSProvider mdtprovider -Root $MDTDeploymentShare
Get-ChildItem "mdt:\applications" | selectName,Version,Publisher,source,Commandline | Export-Csv -Path $exportPath 

Script 2:  Create new application packages based on the exported CSV file

Note: Update variables for:

  • Location of the ConfigMgr Powershell Module.
  •  CSV file location (The file created by the first script).
  • Your site code (Mine is called ABC).
  • Location of the new directory you wish to store the application source in.
$ConfigMgrModulePath = "c:\Program Files\Microsoft Configuration Manager\AdminConsole\bin\ConfigurationManager.psd1"
$ImportPath = "c:\temp\MDTApplication.csv"
$NewSorcePathLocation = "\\ServerName\Sources$\Applications\"

Import-Module $ConfigMgrModulePath

Set-Location ABC:

$csv = Import-Csv $ImportPath

ForEach ($line in $csv){

    $name = $line.name

    $version = $line.version

    $manufacturer = $line.Publisher

    $path = $line.Source -replace ".\\Applications\\", $NewSorcePathLocation

    $command = $line.CommandLine

    New-CMPackage-Name $name -Version $version -Manufacturer $manufacturer -Path $path

    New-CMProgram-Package -Name $name -StandardProgramName "Install" -CommandLine $command -RunType Normal -ProgramRunType WhetherOrNotUserIsLoggedOn -RunMode RunWithAdministrativeRights -DriveMode RenameWithUnc -Duration 120

}

Replacing GUIDS in a file using PowerShell

A quick line of PowerShell code that replaces existing GUIDS in a file with completely new randomly generated GUIDS (Should all be on one line):

(get-content .\Input.xml) | 
ForEach-Object {$_ -replace "\{[0-9a-z\-]*\}",
("{" + [guid]::newguid()  + "}")} |
 Set-Content output.xml -Encoding UTF8

The code basically does a find and replace using a regular expression, then outputs the new content to the output.xml file. The encoding output is completely up to you, mine is set to UTF-8 as it matched the input file.

Richard