Tuesday, August 8, 2017

Jenkins Script Console and some Groovy Scripts

I am one of those jenkins users who use Jenkins Script Console  [1] extensively. I have been maintaining some of those scripts in a private repository, lately I came across this gist [2] by Damien Nozay which uses a much simpler way to share these scripts i.e. using gists :D
After forking it I have started adding my scripts in gists as well   https://gist.github.com/mubbashir/484903fda934aeea9f30 [3].

I found it quite useful e.g. lately I need to know the results of all the downstream jobs triggered by a certain job:
/ author : Ahmed Mubbashir Khan
// Licensed under MIT
// ---------------------------------------------------------
// This script prints out information of last dwonstream job of Upstream job
// e.g. printInformationOfDownstreamJobs("ChangeListner", 11, "All Tests")
// will print all the downstream jobs invodked by ChangeListner build 11 in the view "All Tests"
// ---------------------------------------------------------

import hudson.model.*
//printInformationOfDownstreamJobs("ChangeListner", 11, "All Tests")

def printInformationOfDownstreamJobs(jobName, buildnumber, viewName){
  def upStreamBuild = Jenkins.getInstance().getItemByFullName(jobName).getBuildByNumber(buildnumber)
  println "${upStreamBuild.fullDisplayName}" +
  def cause_pattern = /.*${jobName}.*${buildnumber}.*/
  println "Cause pattern: ${cause_pattern}"
  // Upated for any builds of the upstream  job in a view from only the last build
  def view = Hudson.instance.getView(viewName)
  def buildsByCause = []
   // For each item in the view 
    def jobBuilds=it.getBuilds() // get all the builds 
    jobsBuildsByCause = jobBuilds.findAll { build ->    
    build != null &&
    build.getCause(hudson.model.Cause.UpstreamCause)!= null &&
   // printing information 
  buildsByCause.each{ d_build->
   // def d_build = job.lastBuild
    println("Build: ${d_build.fullDisplayName}->"+
     "result:${d_build.result}->${d_build.buildStatusSummary.message}, " +
     "(was triggered by:${d_build.getCause(hudson.model.Cause.UpstreamCause).upstreamRun})" )

Or information about the builds in a new e.g. node on  which they were executed along with the time they took:

// author : Ahmed Mubbashir Khan
// ---------------------------------------------------------
// This script goes through all the jobs in a view, filters succesful and failed jobs seprately
// Then prints outs them along with the time they took
// ---------------------------------------------------------
import hudson.model.*
def str_view = "Pipeline Tests"
def view = Hudson.instance.getView(str_view)
def successfulJobs = view.getItems().findAll{job -> job.lastBuild != null && job.lastBuild.result == hudson.model.Result.SUCCESS}
def faildJobs = view.getItems().findAll{job -> job.lastBuild != null && job.lastBuild.result == hudson.model.Result.FAILURE}
def disabledJob = view.getItems().findAll{job -> job.disabled == true}
def enabledJob = view.getItems().findAll{job -> job.disabled != true}
println "Total jobs: " + view.getItems().size +" Successful: " +successfulJobs.size+
  " Failed: " + faildJobs.size + " Enabled jobs: " +enabledJob.size + " Disabled jobs: " +disabledJob.size 
println "Current Successful job:"
successfulJobs.each{job -> printInfo(job)}
println "Current Fail job:"
faildJobs.each{job -> printInfo(job)}
println "Current disabled job:"
disabledJob.each{job -> printInfo(job)}
println "Current enabled job:"
enabledJob.each{job -> printInfo(job)}

def printInfo(job){
  println "Job: ${job.name} build on ${job.getAssignedLabelString()}, "+
    "took ${job.lastBuild.getDurationString()} to build, is disabled : ${job.disabled}"

[1] https://wiki.jenkins.io/display/JENKINS/Jenkins+Script+Console
[2] https://gist.github.com/dnozay/e7afcf7a7dd8f73a4e05
[3] https://gist.github.com/mubbashir/484903fda934aeea9f30 

Updated https://gist.github.com/mubbashir/#file-jobstatustriggeredbyupstreamcause to list any downstream job which matches and upstream cause

Tuesday, January 26, 2016

Interacitve shell for php-webdriver to debug selenium

I do most of my selenium (actually locator’s) debugging in chrome console, $() for css selectors and and $x() for xpath selectors but when it actually comes to debugging selenium/webdriver api calls I prefer interactive shell of programming language (python, ruby and php and pretty decent built in shells)
Bellow is how you can configure php shell to play around with selenium


  • get composer Composer (If you don’t already use Composer)
curl -sS https://getcomposer.org/installer | php
mv composer.phar /usr/local/bin/composer
  • get psysh A runtime developer console, interactive debugger and REPL for PHP
wget psysh.org/psysh
chmod +x psysh
Then install the library:
php composer.phar require facebook/webdriver
All you need as the server for this client is the selenium-server-standalone-#.jar file provided here: http://selenium-release.storage.googleapis.com/index.html
Download and run that file, replacing # with the current server version.
java -jar selenium-server-standalone-#.jar

// author : Ahmed Mubbashir Khan
require_once 'vendor/autoload.php'; //composer generated autoload file
use Facebook\WebDriver\Remote\DesiredCapabilities;
use Facebook\WebDriver\Remote\RemoteWebDriver;
use Facebook\WebDriver\WebDriverBy;

$host = '';
$driver = RemoteWebDriver::create($host, DesiredCapabilities::firefox());
Or  if you wan't to modify $capabilities
use Facebook\WebDriver\Remote\WebDriverCapabilityType;
$capabilities = array(\WebDriverCapabilityType::BROWSER_NAME => 'firefox');
$driver = RemoteWebDriver::create($host, $capabilities);
//e.g. Mouse over on an element
$element = $driver->findElement(WebDriverBy::id('some_id'));
$driver->getMouse()->mouseMove( $element->getCoordinates() );
 - [1]: http://selenium-release.storage.googleapis.com/index.html
 - [2]: https://github.com/facebook/php-webdriver
 - [3]: https://getcomposer.org/
-  [4]: http://psysh.org/

Monday, January 4, 2016

Jenkins: List all the jobs for which SCM is not currently configured

Put the following in  Jenkins Script Console  to  list all the jobs for which SCM is not currently configured:

// Licensed under MIT
// author : Ahmed Mubbashir Khan
// ---------------------------------------------------------
// This script goes through all the jobs and checks if they configured SCM is hudson.scm.NullSCM
// if they are, then prints it's info
// ---------------------------------------------------------

counter = 0
jobs = Jenkins.instance.getAllItems()
for (job in jobs) {
  if (job.scm instanceof hudson.scm.NullSCM){
    println "Job= '${counter++}' '${job.name}' scm '${job.scm}'"

Sunday, October 25, 2015

Selenium, Jbehave , gradle and Serenity - Perfect combination for automating BDD styled checks in Java

I have just created this demo project using Selenium, gradle, Jbehave and Serenity


Why add Serenity?

Serenity adds plugins to Jbehave and makes it a lot easier to write acceptance tests as mentioned on it site Serenity BDD helps you write cleaner and more maintainable automated acceptance and regression tests faster.
The other things I loved about Serenity is the it has wonderful reports  for BDD "Serenity also uses the test results to produce illustrated, narrative reports that document and describe what your application does and how it works."


├── ReadMe.md  ## ReadME file of cource ;)
├── build.gradle ## gradle build file 
├── gradle
│   └── wrapper
│       ├── gradle-wrapper.jar
│       └── gradle-wrapper.properties
├── gradlew ## grdadle warapper
├── src
│   └── test
│       ├── java
│       │   └── test
│       │       ├── WebTest.java ## This is where the tests resides which add all the steps using net.serenitybdd.jbehave.SerenityStepFactory
│       │       └── selenium
│       │           ├── page ## This where all the pages extending net.serenitybdd.core.pages.PageObject
│       │           │   ├── google
│       │           │   │   └── GoogleLanding.java
│       │           │   └── interfaces
│       │           │       └── Landing.java
│       │           └── steps ## Pages are used in steps 
│       │               └── GoogleSteps.java
│       └── resources
│           └── stories
│               └── google
│                   ├── Search2.story
│                   └── search.story


Tuesday, October 20, 2015

Adding pre-commit (package) to git repo of your python projects

Git pre commits  are quite useful to analyze  the code before you make a commit and pre-commit makes it quite easy to mange these hooks and rules to run via these hooks.
  • Install pre-commit, pip install pre-commit ## detailed instruction  on it's website pre-commit
  • Run pre-commit install ## Usage instruction  on it's website pre-commit
  • Create .pre-commit-config.yaml file in your project root
pre-commit install will install pre-commit into your git hooks, now on words it will be run on every commit. 
Every time you clone a  project running pre-commit install should always be the first thing you do.
The first time pre-commit runs on a file it will automatically download, install, and run the hook. Note that running a hook for the first time may be slow. 
  • To Run all pre-commit manually on all file type pre-commit run --all-files
  • To run specific hook (via the id in .pre-commit-config.yaml) on specific file(s) pre-commit run autopep8-wrapper --files features/steps/google_steps.py

We can also run all the pre commit hook as a CI step e.g. pre-commit run --all-files 

Individual tools can configure in accordance to according to there configuration. For example flake8 can be configured by adding a setup.cfg file 

Tuesday, October 6, 2015

Parallel execution of automated check via selenium, behave and docker

We were looking for a solution for executing selenium bases scenarios written in behave in parallel to speed up the execution time. Unluckily behave doesn't have this in build like cucumber  fortunately we came across https://github.com/hugeinc/behave-parallel  in which  upstemsync was just couple of weeks old :)

I already had a picture in mind how would the entire step would work so creating  a demo project wasn't too difficult.

Check out https://github.com/mubbashir/behave-selenium 
Following is screen recoding where I had connected to both nodes via VNC and checks were executed in Parallel  on features level


Following is what the readme says at the moment:


Selenium's example using behave
A quick example of python, behave, selenium, webdriver and docker to run prallel tests
  • Install docker and create a selenium grid with say 2 nodes
    • docker run -d -p 4444:4444 --name selenium-hub selenium/hub:2.47.1 ## To Run the Hub
    • docker run -d -P --link selenium-hub:hub selenium/node-firefox-debug:2.47.1 ## Run it twice to create 2 nodes, Docker will pick a random name
    • Check the running docker containers
$ docker ps
CONTAINER ID        IMAGE                                COMMAND                  CREATED             STATUS              PORTS                     NAMES
34911106818f        selenium/node-firefox-debug:2.47.1   "/opt/bin/entry_point"   2 hours ago         Up 2 hours>5900/tcp   condescending_elion
14971ef95408        selenium/node-firefox-debug:2.47.1   "/opt/bin/entry_point"   2 hours ago         Up 2 hours>5900/tcp   trusting_mahavira
10688bf744d4        selenium/hub:2.47.1                  "/opt/bin/entry_point"   3 hours ago         Up 3 hours>4444/tcp    selenium-hub
  • Selenium gird console to have two nodes # if you are on mac or on windows get the ip of the running docker vm e.g. docker-machine ip default
  • Get port of VNC if you want to connect to a node
docker port <container-name|container-id> 5900
docker port 34911106818f
  • Clone the repo
  • Create virtulenv $virtualenv env
  • Activate the virtualenv and run the requirements file via pip source env/bin/activate; pip install -r requirements.txt
  • Export GRID_HUB_URL export GRID_HUB_URL=''
  • Run behave --processes 2 --parallel-element feature


  • Run the requiremetns file in the virtual environment
  • Run pre-commit install ## To install pre-commit
Run pre-commit install to install pre-commit into your git hooks. pre-commit will now run on every commit. Every time you clone this project running pre-commit install should always be the first thing you do.
If you want to manually run all pre-commit hooks on a repository, run pre-commit run --all-files. To run individual hooks use pre-commit run <hook_id>.
The first time pre-commit runs on a file it will automatically download, install, and run the hook. Note that running a hook for the first time may be slow. For example: If the machine does not have node installed, pre-commit will download and build a copy of node.
  • To Run all pre-commit manually on all file type pre-commit run --all-files
  • To run specific hook (via the id in .pre-commit-config.yaml) on specific file(s) pre-commit run flake8 --files bdd_feature_tests/features/steps/dubizzle_homepage_steps.py
  • To run autopep 8 manually (which will run automatically once pre-commit is configured) type pre-commit run 'autopep8-wrapper' --files bdd_feature_tests/features/steps/dubizzle_homepage_steps.py