Tuesday, May 14, 2013

Groovy - Saving CouchDB runtime statistics json file to excel file

Last year, I wrote a blog on how to get CouchDB Runtime Statistics as well as saving it as a json file. However, I wanted to learn Groovy so I wrote a Groovy script to read Runtime Statistics json file and save all the Key values into an excel file.

Below is the prerequisite to execute the script as well as the script.

NOTE: Their already exists a python script to capture the CouchDB runtime statistics.

Prerequisite:
  • Download and Install Groovy Console application.
  • Download and copy  jxl.jar file into the Groovy lib folder.
Script:
import groovy.json.*
import jxl.*
import jxl.write.*


Parameter = ["Group","Key","current","max", "mean", "min", "stddev","sum"]; //couchDB worksheet headers

exlfile = "C:\\Harinder\\Groovy\\couchDB.xls"; //excel file path

if   (new File(exlfile).exists())  //check if file exists esle create a new one, write label and close the file
{
    println "File already exists";
}
else
{
    WritableWorkbook workbook1 = Workbook.createWorkbook(new File(exlfile))
    WritableSheet sheet1 = workbook1.createSheet("couchDB", 0);
    //Label label = new Label(column, row, "Text input in Excel");
    for (int iheader=0; iheader<8;iheader++)
    {
        Label label = new Label(iheader, 0, Parameter[iheader]);
        sheet1.addCell(label);
    }
    workbook1.write();
    workbook1.close();
}

def reader =new BufferedReader(new FileReader("C:\\Harinder\\Groovy\\couchDB.json")); //create a json file into a buffer
def jparsedData =new JsonSlurper().parse(reader);


/*open an exisiting excel file, write Key values and close the file*/
Workbook workbook = Workbook.getWorkbook(new File(exlfile)); 
WritableWorkbook copy = Workbook.createWorkbook(new File(exlfile),workbook);

Groups = jparsedData.collect{a,b->a}.reverse(); //Groups=["couchdb","httpd_request_methods"...]

WritableSheet sheet = copy.getSheet(0);
int groupCount=1;
int rowCount=1;
for (int gCount=0;gCount<Groups.size();gCount++)  //iterate through the Groups
 {   
    tGroups=Groups[gCount]; //assign Groups[gCount] value to a temporary variable tGroups
    sheet.addCell(new Label(0,groupCount,  tGroups)) //save tGroups into the sheet
    Keys = jparsedData."$tGroups".collect{a,b->a}.reverse();  //collect all the Keys associated to Group[gGroup]
    for (int kCount=0;kCount<Keys.size();kCount++) // iterate through all the Keys and save their min,max,count,mean,stddev,sum values into the sheet
     {
       tKeys=Keys[kCount];
       rowCount=kCount+groupCount; 
       sheet.addCell(new Label(1,rowCount,  tKeys));
       sheet.addCell(new Label(2,rowCount,  jparsedData."$tGroups"."$tKeys".current.toString()));
       sheet.addCell(new Label(3,rowCount,  jparsedData."$tGroups"."$tKeys".max.toString()));
       sheet.addCell(new Label(4,rowCount,  jparsedData."$tGroups"."$tKeys".mean.toString()));
       sheet.addCell(new Label(5,rowCount,  jparsedData."$tGroups"."$tKeys".min.toString()));
       sheet.addCell(new Label(6,rowCount,  jparsedData."$tGroups"."$tKeys".stddev.toString()));
       sheet.addCell(new Label(7,rowCount,  jparsedData."$tGroups"."$tKeys".sum.toString()));
     }
     groupCount=rowCount;
 }

copy.write()
copy.close()

Script steps:
  1. Check couchDB excel file exists. If it does not, create it and add all the necessary headers to a worksheet and close the file.
  2. Read and parse the json file.
  3. Open the couchDB file for writing.
  4. Navigate through the parsed json data and save all the Group names.
  5. Navigate through each group name in step 4 and save all the Keys associated to it.
  6. For each Keys saved in step 5, navigate through it and save all the associated values into excel file.
  7. Once done, close the worksheet.

Result:
 

Monday, April 8, 2013

SOASTA CloudTest Lite VM - Switching to graphics mode

Today, I downloaded CloudTest Lite VM from SOASTA website so I could have a look. After starting it up with VMWare Player, I got the following screen.

To switch to graphic mode, which is what we want, you need to hit
CTRL+ALT+F7

After hitting the keys, you will see the Welcome CloudTest Lite Screen.

Sunday, April 7, 2013

How to compare two heap dumps and view result using jhat

The jhat is a Java Heap Analysis Tool that comes as a part of the JDK. This tool can be found in the JDK bin directory. For more information on jhat refer to Java SE documentation.













Following is the step to compare two heap files using jhat:

jhat -baseline baseline.hprof newbaseline.hprof

If successful, jhat will start an http server on default 7000 port. To view the result, navigate to
http://localhost:7000/ 


















Classes
Some of the classes you might see
http://localhost:7000/allClassesWithPlatform/
http://localhost:7000/showRoots/
http://localhost:7000/showInstanceCounts/includePlatform/
http://localhost:7000/showInstanceCounts/
http://localhost:7000/histo/
http://localhost:7000/finalizerSummary/
http://localhost:7000/oql/


Tuesday, April 2, 2013

Generating Websphere verboseGC Graph in LoadRunner Analysis tool

Few weeks ago, I was working for a client and I wanted to analyze Websphere Application server verboseGC logs. I could have used tools such as IBM PMAT but what I really wanted was to merge verboseGC graph with response time graph in Load Runner and this required a lot of manual work. Therefore, I modified existing Silkperformer vbscript to Websphere verboseGC vbscript.

For the blogging purpose, this script saves only handful of verboseGC attributes into CSV file. You can then import the csv file as an external monitor in Load Runner Analysis tool.

NOTE: You can modify this script to suit your requirements. Also make sure everything is contained within verbosegc tag in your verboseGC log file. See the example below.

Please read the following blog on how to generate the CSV file.

Websphere verboseGC Vbscript
Option Explicit
 
Dim xmlDoc
Dim af, gc, timestamp, gcIntervalms, DateTimeArray, afDate, afTime, afIntervalms,minimum,requestedBytes,Totalms,gcTotalms,gcTotal,Time
Dim reportFile, outputFile
Dim myFSO, fileHandle
 

'Murray Wardle code
Set xmlDoc = CreateObject("Microsoft.XMLDOM")
 
Const ForReading = 1, ForWriting = 2, ForAppending = 8
 
If Wscript.Arguments.Count = 0 Then
    msgbox "Please specify the overview Report file to process"
Else
 
 ' Get report file name & set output filename
    reportFile = Wscript.Arguments(0)
 outputFile = Left(reportFile, Len(reportFile)-3) + "csv"
 
 xmlDoc.async = false
 xmlDoc.SetProperty "SelectionLanguage", "XPath"
 xmlDoc.SetProperty "ServerHTTPRequest", True
 xmlDoc.validateOnParse = False
 xmlDoc.resolveExternals = False
 
 'load overview report
 xmlDoc.load(reportFile)
 xmlDoc.setProperty "SelectionLanguage", "XPath"
 
 'open csv file to dump results into
 Set myFSO = CreateObject("Scripting.FileSystemObject")
 Set fileHandle = myFSO.OpenTextFile(outputFile, ForWriting, True)

'Modified code for verboseGC
fileHandle.WriteLine("date,time,afIntervalms,requestedBytes,gcIntervalms,gcTotalms,Totalms")
 
For Each af In xmlDoc.SelectNodes("//af")
  timestamp = af.getAttribute("timestamp")'get af timestamp attribute value
  DateTimeArray=Split(timestamp," ",-1,1) 'split the date time into array
  afDate=FormatDateTime(DateTimeArray(1)+"/"+DateTimeArray(0)+"/"++DateTimeArray(3),vbShortDate) 'format into date
  afTime=FormatDateTime(DateTimeArray(2),vbLongTime) 'format into time
 
  afIntervalms=af.getAttribute("intervalms") 'get af intervalms attribute value
 
  For Each minimum In af.SelectNodes("./minimum") 'get minimum requested bytes value
    requestedBytes = minimum.getAttribute("requested_bytes")
  Next
  For Each gc In af.SelectNodes("./gc") 'get gc intervalms value
    gcIntervalms = gc.getAttribute("intervalms")
  Next
  For Each gcTotal In af.SelectNodes("./gc/time") 'get total gc time value
    gcTotalms = gcTotal.getAttribute("totalms")
 Next
 For Each Time In af.SelectNodes("./time")
    Totalms = Time.getAttribute("totalms") 'get total time value
 Next   
 fileHandle.WriteLine(afDate+","+afTime+","+afIntervalms+","+ requestedBytes+","+ gcIntervalms+","+gcTotalms+","+Totalms)
Next
fileHandle.Close
end if
Websphere Application verboseGC log Example


  
  


  
  

CSV Output



VerboseGC graph in LoadRunner

Saturday, February 2, 2013

Failed to initialize dtrace message

Recently, I wanted to learn DTrace programming, so I installed OpenSolaris 10 virtual machine on my laptop. After installation, I tried to execute a simple dtrace command but got the following message:

"dtrace: failed to initialize dtrace: DTrace require additional previleges"


Solution:
By default, when you install OpenSolaris 10, your primary profile is set as 'Primary Administrator' and role as 'root' but you are logged in as a normal user without root privileges. Therefore, to solve my issue I had to 'su' as 'root' and run DTrace commands.


You can check what role and profile you are assigned after installation by running following command:    cat /etc/user_attr


NOTE: If you don't want to log in as a root each time you want to run the DTrace command, you can read the following article on how to give DTrace privilege to a normal user.