Showing posts with label CouchDB. Show all posts
Showing posts with label CouchDB. Show all posts

Tuesday, May 14, 2013

Groovy - Saving CouchDB runtime statistics json file to excel file

Last year, I wrote a blog on how to get CouchDB Runtime Statistics and saving it as a json file. However, want I really wanted was to save and open  the file in excel. Therefore, I wrote a Groovy script to read Runtime Statistics data from json file and save all the Key values into an excel file.

Below is the prerequisite to execute the script as well as the script.

NOTE: Their already exists a python script to capture the CouchDB runtime statistics.

Prerequisite:
  • Download and Install Groovy Console application.
  • Download and copy  jxl.jar file into the Groovy lib folder.
Script:
import groovy.json.*
import jxl.*
import jxl.write.*


Parameter = ["Group","Key","current","max", "mean", "min", "stddev","sum"]; //couchDB worksheet headers

exlfile = "C:\\Harinder\\Groovy\\couchDB.xls"; //excel file path

if   (new File(exlfile).exists())  //check if file exists esle create a new one, write label and close the file
{
    println "File already exists";
}
else
{
    WritableWorkbook workbook1 = Workbook.createWorkbook(new File(exlfile))
    WritableSheet sheet1 = workbook1.createSheet("couchDB", 0);
    //Label label = new Label(column, row, "Text input in Excel");
    for (int iheader=0; iheader<8;iheader++)
    {
        Label label = new Label(iheader, 0, Parameter[iheader]);
        sheet1.addCell(label);
    }
    workbook1.write();
    workbook1.close();
}

def reader =new BufferedReader(new FileReader("C:\\Harinder\\Groovy\\couchDB.json")); //create a json file into a buffer
def jparsedData =new JsonSlurper().parse(reader);


/*open an exisiting excel file, write Key values and close the file*/
Workbook workbook = Workbook.getWorkbook(new File(exlfile)); 
WritableWorkbook copy = Workbook.createWorkbook(new File(exlfile),workbook);

Groups = jparsedData.collect{a,b->a}.reverse(); //Groups=["couchdb","httpd_request_methods"...]

WritableSheet sheet = copy.getSheet(0);
int groupCount=1;
int rowCount=1;
for (int gCount=0;gCount<Groups.size();gCount++)  //iterate through the Groups
 {   
    tGroups=Groups[gCount]; //assign Groups[gCount] value to a temporary variable tGroups
    sheet.addCell(new Label(0,groupCount,  tGroups)) //save tGroups into the sheet
    Keys = jparsedData."$tGroups".collect{a,b->a}.reverse();  //collect all the Keys associated to Group[gGroup]
    for (int kCount=0;kCount<Keys.size();kCount++) // iterate through all the Keys and save their min,max,count,mean,stddev,sum values into the sheet
     {
       tKeys=Keys[kCount];
       rowCount=kCount+groupCount; 
       sheet.addCell(new Label(1,rowCount,  tKeys));
       sheet.addCell(new Label(2,rowCount,  jparsedData."$tGroups"."$tKeys".current.toString()));
       sheet.addCell(new Label(3,rowCount,  jparsedData."$tGroups"."$tKeys".max.toString()));
       sheet.addCell(new Label(4,rowCount,  jparsedData."$tGroups"."$tKeys".mean.toString()));
       sheet.addCell(new Label(5,rowCount,  jparsedData."$tGroups"."$tKeys".min.toString()));
       sheet.addCell(new Label(6,rowCount,  jparsedData."$tGroups"."$tKeys".stddev.toString()));
       sheet.addCell(new Label(7,rowCount,  jparsedData."$tGroups"."$tKeys".sum.toString()));
     }
     groupCount=rowCount;
 }

copy.write()
copy.close()

Script steps:
  1. Check couchDB excel file exists. If it does not, create it and add all the necessary headers to a worksheet and close the file.
  2. Read and parse the json file.
  3. Open the couchDB file for writing.
  4. Navigate through the parsed json data and save all the Group names.
  5. Navigate through each group name in step 4 and save all the Keys associated to it.
  6. For each Keys saved in step 5, navigate through it and save all the associated values into excel file.
  7. Once done, close the worksheet.

Result:
 

Saturday, March 3, 2012

Runtime Statistics in CouchDB


CouchDB  is an open source document-oriented database. More information on CouchDB is available here. 

CouchDB 0.9 and above comes with a list of counters that lets you inspect how CouchDB performs. On windows (local machine where CouchDB is installed), you can view these counters by navigating to the following URL: http://localhost:5984/_stats. More information is available here.

The information returned is in JSON format. There are free tools and websites available to display the stats in a more user friendly format.


GroupKeyDescription
Couchdbdatabase_writesNumber  of times a database was changed
open_databasesNumber  of open databases
auth_cache_hitsNumber  of authentication cache hits
auth_cache_missesNumber  of authentication cache misses
database_readsNumber  of times a document was read from a database
request_timeLength of a request inside CouchDB without MochiWeb
open_os_filesNumber  of file descriptors CouchDB has open
httpdRequestsNumber  of HTTP requests
bulk_requestsNumber  of bulk requests
view_readsNumber  of view reads
clients_requesting_changesNumber  of clients for continuous _changes
temporary_view_readsNumber  of temporary view reads
httpd_request_methodsDELETENumber  of HTTP DELETE requests
HEADNumber  of HTTP HEAD requests
POSTNumber  of HTTP POST requests
PUTNumber  of HTTP PUT requests
GETNumber  of HTTP GET requests
COPYNumber  of HTTP COPY requests
httpd_status_codes400Number  of HTTP 400 Bad Request responses
201Number  of HTTP 201 Created responses
403Number  of HTTP 403 Forbidden responses
409Number  of HTTP 409 Conflict responses
200Number  of HTTP 200 OK responses
202Number  of HTTP 202 Accepted responses
404Number  of HTTP 404 Not Found responses
301Number  of HTTP 301 Moved Permanently responses
405Number  of HTTP 405 Method Not Allowed responses
500Number  of HTTP 500 Internal Server Error responses
401Number  of HTTP 401 Unauthorized responses
304Number  of HTTP 304 Not Modified responses
412Number  of HTTP 412 Precondition Failed responses

Below is a formated JSON output example:

{ "couchdb" : { "auth_cache_hits" : { "current" : 2089.0,
          "description" : "number of authentication cache hits",
          "max" : 222,
          "mean" : 0.83199999999999996,
          "min" : 0,
          "stddev" : 10.222,
          "sum" : 2089.0
        },
      "auth_cache_misses" : { "current" : 12.0,
          "description" : "number of authentication cache misses",
          "max" : 3,
          "mean" : 0.0050000000000000001,
          "min" : 0,
          "stddev" : 0.113,
          "sum" : 12.0
        },
      "database_reads" : { "current" : 643.0,
          "description" : "number of times a document was read from a database",
          "max" : 131,
          "mean" : 0.25700000000000001,
          "min" : 0,
          "stddev" : 4.1050000000000004,
          "sum" : 643.0
        },
      "database_writes" : { "current" : 205.0,
          "description" : "number of times a database was changed",
          "max" : 28,
          "mean" : 0.082000000000000003,
          "min" : 0,
          "stddev" : 1.056,
          "sum" : 205.0
        },
      "open_databases" : { "current" : 10.0,
          "description" : "number of open databases",
          "max" : 7,
          "mean" : 0.0040000000000000001,
          "min" : -5,
          "stddev" : 0.21099999999999999,
          "sum" : 10.0
        },
      "open_os_files" : { "current" : 10.0,
          "description" : "number of file descriptors CouchDB has open",
          "max" : 7,
          "mean" : 0.0040000000000000001,
          "min" : -5,
          "stddev" : 0.28699999999999998,
          "sum" : 10.0
        },
      "request_time" : { "current" : 13881.477999999999,
          "description" : "length of a request inside CouchDB without MochiWeb",
          "max" : 12043.0,
          "mean" : 115.679,
          "min" : 0.0,
          "stddev" : 1098.511,
          "sum" : 13881.477999999999
        }
    },
  "httpd" : { "bulk_requests" : { "current" : 27.0,
          "description" : "number of bulk requests",
          "max" : 10,
          "mean" : 0.010999999999999999,
          "min" : 0,
          "stddev" : 0.253,
          "sum" : 27.0
        },
      "clients_requesting_changes" : { "current" : null,
          "description" : "number of clients for continuous _changes",
          "max" : null,
          "mean" : null,
          "min" : null,
          "stddev" : null,
          "sum" : null
        },
      "requests" : { "current" : 2395.0,
          "description" : "number of HTTP requests",
          "max" : 221,
          "mean" : 0.95299999999999996,
          "min" : 0,
          "stddev" : 10.359999999999999,
          "sum" : 2395.0
        },
      "temporary_view_reads" : { "current" : 92.0,
          "description" : "number of temporary view reads",
          "max" : 33,
          "mean" : 0.036999999999999998,
          "min" : 0,
          "stddev" : 0.94199999999999995,
          "sum" : 92.0
        },
      "view_reads" : { "current" : 144.0,
          "description" : "number of view reads",
          "max" : 37,
          "mean" : 0.058000000000000003,
          "min" : 0,
          "stddev" : 1.1579999999999999,
          "sum" : 144.0
        }
    },
  "httpd_request_methods" : { "COPY" : { "current" : 1.0,
          "description" : "number of HTTP COPY requests",
          "max" : 1,
          "mean" : 0.0,
          "min" : 0,
          "stddev" : 0.02,
          "sum" : 1.0
        },
      "DELETE" : { "current" : 187.0,
          "description" : "number of HTTP DELETE requests",
          "max" : 81,
          "mean" : 0.074999999999999997,
          "min" : 0,
          "stddev" : 1.6919999999999999,
          "sum" : 187.0
        },
      "GET" : { "current" : 1672.0,
          "description" : "number of HTTP GET requests",
          "max" : 222,
          "mean" : 0.66600000000000004,
          "min" : 0,
          "stddev" : 9.1310000000000002,
          "sum" : 1672.0
        },
      "HEAD" : { "current" : null,
          "description" : "number of HTTP HEAD requests",
          "max" : null,
          "mean" : null,
          "min" : null,
          "stddev" : null,
          "sum" : null
        },
      "POST" : { "current" : 190.0,
          "description" : "number of HTTP POST requests",
          "max" : 35,
          "mean" : 0.075999999999999998,
          "min" : 0,
          "stddev" : 1.1719999999999999,
          "sum" : 190.0
        },
      "PUT" : { "current" : 345.0,
          "description" : "number of HTTP PUT requests",
          "max" : 85,
          "mean" : 0.13700000000000001,
          "min" : 0,
          "stddev" : 2.1110000000000002,
          "sum" : 345.0
        }
    },
  "httpd_status_codes" : { "200" : { "current" : 1734.0,
          "description" : "number of HTTP 200 OK responses",
          "max" : 221,
          "mean" : 0.68999999999999995,
          "min" : 0,
          "stddev" : 9.2690000000000001,
          "sum" : 1734.0
        },
      "201" : { "current" : 245.0,
          "description" : "number of HTTP 201 Created responses",
          "max" : 31,
          "mean" : 0.098000000000000004,
          "min" : 0,
          "stddev" : 1.2170000000000001,
          "sum" : 245.0
        },
      "202" : { "current" : 3.0,
          "description" : "number of HTTP 202 Accepted responses",
          "max" : 1,
          "mean" : 0.001,
          "min" : 0,
          "stddev" : 0.035000000000000003,
          "sum" : 3.0
        },
      "301" : { "current" : 3.0,
          "description" : "number of HTTP 301 Moved Permanently responses",
          "max" : 1,
          "mean" : 0.001,
          "min" : 0,
          "stddev" : 0.035000000000000003,
          "sum" : 3.0
        },
      "304" : { "current" : 25.0,
          "description" : "number of HTTP 304 Not Modified responses",
          "max" : 8,
          "mean" : 0.01,
          "min" : 0,
          "stddev" : 0.215,
          "sum" : 25.0
        },
      "400" : { "current" : 23.0,
          "description" : "number of HTTP 400 Bad Request responses",
          "max" : 8,
          "mean" : 0.0089999999999999993,
          "min" : 0,
          "stddev" : 0.22900000000000001,
          "sum" : 23.0
        },
      "401" : { "current" : 5.0,
          "description" : "number of HTTP 401 Unauthorized responses",
          "max" : 2,
          "mean" : 0.002,
          "min" : 0,
          "stddev" : 0.059999999999999998,
          "sum" : 5.0
        },
      "403" : { "current" : 14.0,
          "description" : "number of HTTP 403 Forbidden responses",
          "max" : 6,
          "mean" : 0.0060000000000000001,
          "min" : 0,
          "stddev" : 0.156,
          "sum" : 14.0
        },
      "404" : { "current" : 100.0,
          "description" : "number of HTTP 404 Not Found responses",
          "max" : 80,
          "mean" : 0.040000000000000001,
          "min" : 0,
          "stddev" : 1.6220000000000001,
          "sum" : 100.0
        },
      "405" : { "current" : 3.0,
          "description" : "number of HTTP 405 Method Not Allowed responses",
          "max" : 1,
          "mean" : 0.001,
          "min" : 0,
          "stddev" : 0.035000000000000003,
          "sum" : 3.0
        },
      "409" : { "current" : 3.0,
          "description" : "number of HTTP 409 Conflict responses",
          "max" : 1,
          "mean" : 0.001,
          "min" : 0,
          "stddev" : 0.035000000000000003,
          "sum" : 3.0
        },
      "412" : { "current" : null,
          "description" : "number of HTTP 412 Precondition Failed responses",
          "max" : null,
          "mean" : null,
          "min" : null,
          "stddev" : null,
          "sum" : null
        },
      "500" : { "current" : 87.0,
          "description" : "number of HTTP 500 Internal Server Error responses",
          "max" : 75,
          "mean" : 0.035000000000000003,
          "min" : 0,
          "stddev" : 1.508,
          "sum" : 87.0
        }
    }
}