Sunday, September 17, 2017

JMeter: How to Verify JSON Response?


JSON (Javascript object notation) is a serialization format (in key-value pairs) of data structures .  For REST API, it is widely used for data transfer from server to client.  For example, a client sends an HTTP request with below header:

  • Accept: application/json

The server can respond with below sample JSON data:

  {
    "result": [],
    "ccapiInfo": {
      "createdOn": "2017-09-07T15:25:29.000Z",
      "cachedOn": "2017-09-07T15:21:49.513Z",
      "origin": "cache",
      "canonicalLink": "http://www.myServer.com:9885/computeConsoleApi/infra1626compute1/api/v1/instance/Compute-infra1626compute1/"
    }
  }

with a response header of:
Content-Type: application/json

In this article, we will discuss how to achieve two tasks in Apache JMeter:


JSON Extractor / JSONPath


One of the advantages of XML is the availability of numerous tools to analyse, transform and selectively extract data out of XML documents. XPath is one of these powerful tools.  For JSON, we have a similar tool called JSONPath.

JSONPath is the XPath for JSON.  Since a JSON structure is normally anonymous, JSONPath assigns symbol $ as the root object.

Below is a side-by-side comparison of the JSONPath syntax elements with its XPath counterparts.[9]

XPathJSONPathDescription
/$the root object/element
.@the current object/element
/. or []child operator
..n/aparent operator
//..recursive descent. JSONPath borrows this syntax from E4X.
**wildcard. All objects/elements regardless their names.
@n/aattribute access. JSON structures don't have attributes.
[][]subscript operator. XPath uses it to iterate over element collections and for predicates. In Javascript and JSON it is the native array operator.
|[,]Union operator in XPath results in a combination of node sets. JSONPath allows alternate names or array indices as a set.
n/a[start:end:step]array slice operator borrowed from ES4.
[]?()applies a filter (script) expression.
n/a()script expression, using the underlying script engine.
()n/agrouping in Xpath

JSONPath expressions can use the dot–notation

$.ccapiInfo.canonicalLink

or the bracket–notation

$['ccapiInfo']['canonicalLink']

for input paths. For the internal or output paths, they will always be converted to the more general bracket–notation.  Below diagram shows the evaluation result using a JSONPath Online Evaluator with the input and JSONPath Expression as given in this article.


JSR223 Assertion


Assertion in JMeter help verify that your server under test returns the expected results. JMeter includes quite a few assertion elements for validating the sampler’s response, yet sometimes your validation decision might follow complex logic, and can’t be configured using the out-of-the-box JMeter assertions - scripting is then required.

If you need to write scripting assertion code to extend baseline JMeter functionality, JSR223, in combination with Groovy language is a good choice performance-wise—especially when its compilation caching is enabled.



Groovy Script

String jsonString =  vars.get("myCanonicalLink");
String userNameString = vars.get("user_name");

log.info ("The canonicalLink is " + jsonString);

if ( jsonString != "http://myserver.com:9885/computeConsoleApi/" + 
      userNameString + "/api/v1/instance/Compute-" + userNameString + "/") 
{
AssertionResult.setFailureMessage("The canonicalLink is wrong");
    AssertionResult.setFailure(true); 
}


However, every test element including assertion added to the test plan will increase the total CPU and memory requirements.  So, plan your use of assertions sparingly.

Friday, September 1, 2017

Linux: How to Setup and Get Started wtih cron

When you need to run maintenance jobs routinely in Linux, cron comes in handy. cron is a job scheduler which will automatically perform tasks according to a set schedule. The schedule is called the crontab, which is also the name of the program used to edit that schedule.

cron — Daemon to execute scheduled commands
crontab — Schedule a command to run at a later time

In this article, we will show you how to setup and get started with cron in Oracle Linux Server 6.7.

Commands


The cron service (daemon) runs in the background and constantly checks the following file/directories:
  • /etc/crontab file
  • /etc/cron.*/ directories
  • /var/spool/cron/ directory
    • Each user can have their own crontab, and though these are files in /var/spool/ , they are not intended to be edited directly.
Crontab is the program used to install, deinstall or list the tables used to drive the cron. For example, to display the current crontab, you can do:

# crontab -l

# HEADER: This file was autogenerated at Wed Jan 13 22:49:06 +0000 2016 by puppet.
# HEADER: While it can still be managed manually, it is definitely not recommended.
# HEADER: Note particularly that the comments starting with 'Puppet Name' should
# HEADER: not be deleted, as doing so could cause duplicate cron jobs.
# Puppet Name: cron.puppet.apply
48 * * * * /usr/local/pdit/bin/puppet-apply > /dev/null 2>&1
00 0 * * * /etc/cron.daily.random/at_daily_random.sh

Configuration Files:

You can control access to the crontab command by using two files in the /etc directory:[2]
  • cron.deny
  • cron.allow
These files permit only specified users to perform crontab command tasks such as creating, editing, displaying, or removing their own crontabfiles. Read [2] for more details.


Who can access to
crontab command?
cron.allow
Exists
Does Not Exists
cron.deny ExistsOnly users listed in
cron.allow
All users except those listed in
cron.deny
Does Not ExistsOnly users with superuser privilege


How to Edit Crontab Entries?


To edit a crontab entries, use
crontab -e
By default this will edit the current logged-in user's crontab.

After changing the crontable file, you don't need to restart cron. Cron will examine the modification time on all crontabs and reload those which have changed. Thus cron need not be restarted whenever a crontab file is modified.

[ramesh@user1 ~] $ crontab -e
# clean up Monitoring Tables weekly
0 0 * * 5 /scratch/user1/scripts/db/cleanMonitor.sh > /dev/null 2>&1 
~
"/tmp/crontab.XXXXSERJLH" 2L, 112C

[Note: This will open the crontab file in Vim editor for editing.
Please note cron created a temporary /tmp/crontab.XX... ]
When you save the above temporary file with :wq, it will save the crontab and display the following message indicating the crontab is successfully modified.

~
"crontab.XXXXSERJLH" 2L, 112C written
crontab: installing new crontab
To edit crontab entries of other Linux users, login to root and use:
crontab -u {username} -e

Syntax of crontab (Field Description)


The syntax is:

1 2 3 4 5 /path/to/command arg1 arg2
OR

1 2 3 4 5 /root/backup.sh

Where,
1: Minute (0-59)
2: Hours (0-23)
3: Day (0-31)
4: Month (0-12 [12 == December])
5: Day of the week(0-7 [7 or 0 == sunday])
/path/to/command – Script or command name to schedule
cron also provides a number of operators that allow you to specify more complex repetition intervals. You can read [9] for more details.


Triggering JFR from Cron job

Below crontab entry will trigger jfr in every 45 minutes for 900 seconds interval.
*/45 * * * * jfr.sh

jfr.sh :



BACKUP_DIR="/opt/app/oracle/backup"
SERVER="OSB"
NODE="MS1"
LOG_DIR="${BACKUP_DIR}/${SERVER}/${NODE}/JFRs"
LOG_FILE="${LOG_DIR}/PRODOSB_${NODE}_`date '+%Y%m%d%H%M%S'`.jfr"
JDK_HOME="/opt/app/oracle/jdk"

PID=`ps -ef | grep ${SERVER}_${NODE} |grep 'Dweblogic' | grep -v grep | awk '{print $2}'`

if [ ! -z "${PID}" ];then

${JDK_HOME}/bin/jcmd ${PID} JFR.start duration=900s filename=${LOG_FILE}

fi

Auditing


Auditing collects data at the kernel level that you can analyze to identify unauthorized activity. The entries in the audit rules file, /etc/audit/audit.rules, determine which events are audited. In the below example, we have set up a rule to audit crontab activities.
# cat /etc/audit/audit.rules
# This file contains the auditctl rules that are loaded
# whenever the audit daemon is started via the initscripts.
# The rules are simply the parameters that would be passed
# to auditctl.

-a always,exit -F path=/usr/bin/crontab -F perm=x -F auid>=500 -F auid!=4294967295 -k privileged

Each rule is a command-line option that is passed to the auditctl command. You should typically configure this file to match your site's security policy.

Logging


Rsyslogd is a system utility providing support for message logging. It is configured via the rsyslog.conf file, typically found in /etc. For example, in the below statement, it directs all cron messages to the file /var/log/cron.

rsyslog.conf

# Log cron stuff
cron.* /var/log/cron

How to Debug?


If you suspect that your cron job was not executed correctly, here are the steps that you could take to debug:
  • Check the local user's email which will contain the output of cron jobs
    • Read [10] to find out where the email is and how to open and  read it
  • Add the following at the top of your bash script:
    • #!/bin/bash -x 
    • Next time when your script runs, it will show all the commands it executes
  • Check if there are mail messages in /var/spool/mail/root that indicate that mail to your user isn't getting delivered
    • Consider restarting sendmail after fixing your issues by doing:[14]
      • /etc/init.d/sendmail restart

References

  1. HowTo: Add Jobs To cron Under Linux or UNIX?
  2. Controlling Access to the crontab Command
  3. Configuring and Using Auditing
  4. Linux Crontab: 15 Awesome Cron Job Examples
  5. /usr/local : Local hierarchy
  6. How to schedule a biweekly cronjob?
  7. Configuring and auditing Linux systems with Audit daemon
  8. auditctl - Unix, Linux Command
  9. Schedule Tasks with Cron
  10. What is the “You have new mail” message in Linux/UNIX?
  11. How to check if a cron job ran
  12. 25 simple examples of Linux find command
  13. Stop Cron Daemon from Sending Email for Each Job
  14. How to stop and restart sendmail daemon