Blog

For the most part I am a self taught web developer and, for the most part, this is OK, because I’m capable of learning what I need to, when I need to.

Just-in-time knowledge.

Today I’ve been showing my son how to write HTML and he asked me how I learned to type so fast and our conversation led us to https://www.typingclub.com

What an eye opener. In the very first minute of the very first tutorial I learned something new. Something fundamental to the use of every QWERTY keyboard in existence…

See those little tabs on the F & J keys…

They are for finger alignment, so you can feel your way back to the right location on the keyboard.

Boom. Mind blown.

Tell me I’m not the only one that didn’t know this?… anyone?

Throughout my career I’ve always loved my job. But recently I’ve struggled. I’ve been working on a lot of legacy projects, I’ve not been empowered to implement change and I’ve not been proud of the products I’ve been creating.

This bothers me. It bothers me because I care.

I need to reboot.

I need to reengage with the development community.

I need to deep dive into Azure and Firebase.

I need to build an Angular application. And a probably a React.js application too.

I need to break out the RaspberryPis and get them doing something. Anything.

In my previous article about collecting performance data from remote-servers we looked at a script for extracting perf-data from multiple servers.

This script is used to convert the extracted CSV file into an HTML report.

clear

# define the first field in the file - this will be the x-axis for most graphs
# and can also be eliminated from the report
$timeField = "(PDH-CSV 4.0) (GMT Standard Time)(0)"

$testId = 201611301041
$inFilePath = "C:\jmeter\results\$testId"

# add some start / end filters
$startDate = Get-Date "2016-11-30 10:30:00"
$endDate = Get-Date "2016-11-30 10:45:00"

$headers = @()
$counterNames = @()
$dataSet = @()

# Open the PerfMon CSV file
$csv = Import-Csv $inFilePath\$testId.csv

# extract a list of the headers
$headers = $csv | Get-Member -MemberType NoteProperty | select  name
$headers

# for each of the headers extract a unique list of the servers being monitored
ForEach($header in ($headers | Where { $_ -notlike "*network*" }))
{
    $counterName = ($header -split "\\\\")[2].Trim('}')
    if (-not $counterNames.Contains($counterName)){
        $counterNames += $counterName
        $counterName
    }
}

$counterNames = @("system\processor queue length",
".net clr exceptions(_global_)\# of exceps thrown / sec")


# date/values "[date, value]" for use on the Google Chart

# reuse the date strings if they haven't changed
[datetime]$previousTime = Get-Date;

# the date must be a JavaScript date, so some manipulation will be required.
$jsDate = ""

# loop through the CSV file and create a list of counters and 
$csv | Foreach-Object { 
    foreach ($property in $_.PSObject.Properties)
    {
        $counterDate = [datetime]::ParseExact($_.$timeField, "MM/dd/yyyy HH:mm:ss.fff", $null)

        # if the record is before or after the threshold, ignore the data
        if ($counterDate -lt $startDate -or $counterDate -gt $endDate){
            "This row is out of the date range being processed."
            continue
        }

        # if the time hasn't changed since the last time, we don't need to process this again
        if ($previousTime -ne $counterDate){
            $jsDate = Get-Date $counterDate -format "yyyy,MM,dd,HH,mm,ss"
            $previousTime = $counterDate
        }
        That 
        # if the value is empty, or property name is the same as the time field, don't include
        if (-not [String]::IsNullOrWhiteSpace($property.Value) -and $propertyName -ne $timeField) {
            
            $objDataRow = New-Object System.Object
            $objDataRow | Add-Member -type NoteProperty -name Name -value $property.Name
            $objDataRow | Add-Member -type NoteProperty -name Value -value ("[new Date({0}),{1}]" -f $jsDate, $property.Value)
            $dataSet += $objDataRow 

            $objDataRow.Name.Substring(0,30) + ": " + $objDataRow.Value 
        }
    } 
}

[string] $headerName = ""

# build html
$html = "<html>`n"
$html += "<head>`n"
$html += "<script type='text/javascript' src='https://www.gstatic.com/charts/loader.js'></script>`n"
$html += "
<script type='text/javascript'>
    // Load the Visualization API and the corechart package.
    google.charts.load('current', {'packages':['corechart']});
</script>`n"

$html += "</head>`n"
$html += "<body>`n"

$html += "<h1>Performance report</h1>`n"

ForEach($counterName in $counterNames){
    
    if ($counterName.Length -gt 0){
        $html += "<h2>{0}</h2>`n" -f $counterName

        # tidy up the name, so it can be used as a unique JS reference
        $jsCounterName = $counterName.Replace(" ","").Replace("/","").Replace("\","").Replace("#","").Replace(":","").Replace("(","").Replace(")","").Replace("%","").Replace("[","").Replace("]","").Replace(".","").Replace("-","").Replace("_","").Replace("{","").Replace("}","")
        $jsCounterName

        # loop through each header to find the like-for-like counters
        ForEach($header in $headers | Where { $_.Name -like "*$counterName*"}) {

            # if there aren't any records, don't output it
            if (($dataSet | Where { $_.Name -eq $header.Name } | Measure ) -eq 0){
                continue
            }

            $chartData = [string]::Join(",",($dataSet | Where { $_.Name -eq $header.Name } | Select -ExpandProperty  Value ))
        }

        $html += ("<div id='{0}_chart_div'></div>`n" -f $jsCounterName)

        $html += "<script type='text/javascript'>
            // Set a callback to run when the Google Visualization API is loaded.
            google.charts.setOnLoadCallback(drawChart" + $jsCounterName + ");

            function drawChart" + $jsCounterName + "() {
                // Create the data table.
                var data = new google.visualization.DataTable();
                data.addColumn('date', 'Time');

                data.addColumn('number');
                data.addRows([" + $chartData + "]);

                data.addColumn('number');
                data.addRows([" + $chartData + "]);


                // Set chart options
                var options = {'title':title,
                               'width':600,
                               'height':300};

                // Instantiate and draw our chart, passing in some options.
                var chart = new google.visualization.LineChart(document.getElementById('" + $jsCounterName + "_chart_div'));
                chart.draw(data, options);
            }
            </script>`n"
        $html += "<hr>`n"
    }
}

$html += "</body>`n"
$html += "</html>`n"

$html | out-file -force -Encoding ascii "$inFilePath\$testId.html"

This will extract data from the saved CSV file, and create an HTML file using the rich Google Chart API to generate the graphs.