Road to integration of calabash android test with Jenkins

In this post I am going to show you how to integrate calabash android test with Jenkins continuous integration for build. Also I will show you how to publish cucumber execution report in Jenkins.

 Installation:
1. Java: java should be installed
2. Jenkins: If you are new to Jenkins then click link "click here” of my post how to setup Jenkins in window before continue.
3. Install the”cucumber-jvm-reports-java” plugin: following are the steps to install plugin.
  • Go to dashboard of Jenkins.
  • Click on “Manage Jenkins”
  • Click on “manage plugins”
  • Click on “Available” tab
  • Search “cucumber-reports” and install plugin
  • Restart Jenkins server.

 Calabash android with ant: for the integration of calabash android test with Jenkins we need to create ant build file for the test execution. Below is the ant build file for the calabash android test.

Here we are using junit json report. cucumber-jvm runs and publish json format report into html format
--format json -o cucumber.json  


Jenkins Project: following are the steps to create calabash android Jenkins project.
  • Go to Jenkins dashboard.
  • Click on “New Jobs”
  • Enter Jobs name and check “Build a free-style software project” check box.
  • Click on "OK" button. 
  • Click on “Add build step” options under build and click “Invoke Ant” option.
  • Enter your target name in “Targets” filed and enter build file name with path into “Build File” field.
  • Click on “Add post build action” and click on “Publish cucumber results as a report” 
  • Click on save button.
  • Leave Json reports path blank to find automatically report. If automatically not read json file then you enter path name.

Reports:

After build execution click on “Cucumber Reports” link your execution report is generated as below.





39 comments:

  1. Michal31 July 2013 at 00:52

    Thanks really this is very nice post for calabash integration with Jenkins I was looking from couple of days. Now i am able integrate Calabash with Jenkins.

    Thanks again for this post...

    ReplyDelete
  2. Unknown3 August 2013 at 14:17

    thanks, Nice post for integration .
    keep it up .....


    ReplyDelete
  3. Tanu Bansal8 August 2013 at 15:55

    I added the above code in my build.xml ant script as follows:








    But when I build my project in Jenkins, I get the following message:
    BUILD SUCCESSFUL
    Total time: 1 minute 8 seconds
    $ D:\adt-bundle-windows-x86\sdk/platform-tools/adb.exe disconnect localhost:2535
    [android] Stopping Android emulator
    [android] Archiving emulator log
    $ D:\adt-bundle-windows-x86\sdk/platform-tools/adb.exe kill-server
    [CucumberReportPublisher] Compiling Cucumber Html Reports ...
    [CucumberReportPublisher] detected this build is running on the master
    [CucumberReportPublisher] there were no json results found in: C:\.jenkins\jobs\igLogisticsTest\workspace
    [CucumberReportPublisher] there were no json results found in: C:\.jenkins\jobs\igLogisticsTest\builds\2013-08-08_14-13-44\cucumber-html-reports
    Finished: SUCCESS


    Please help me running the calabash-android scripts through Jenkins and generating the report for the same.

    ReplyDelete
  4. Admin12 August 2013 at 07:44

    What error you getting in report file. Did your calabash text executed or not ?

    ReplyDelete
  5. Tanu Bansal12 August 2013 at 10:45

    No error is shown. Apk gets installed and then the emulator is stopped. I can't see my feature files getting executed. Am I missing something?

    ReplyDelete
  6. Admin13 August 2013 at 00:57

    Are you able to run using command line without build.xml file. Just using calabash-android command ?

    ReplyDelete
  7. Tanu Bansal13 August 2013 at 09:49

    Yes, am able to run calabash-android using command prompt (without build.xml)

    ReplyDelete
    Replies
    1. Admin13 August 2013 at 10:35

      Make sure you are passing your scenario name instead of @LoginApp after --tags

      Delete
    2. Tanu Bansal13 August 2013 at 10:40

      I did not pass any scenario name. I am running all my features as:

      Delete
    3. Admin13 August 2013 at 10:50

      Can you share your build.xml file?

      Delete
    4. Tanu Bansal13 August 2013 at 11:01

      This comment has been removed by the author.

      Delete
  8. Tanu Bansal13 August 2013 at 11:08

    This comment has been removed by the author.

    ReplyDelete
  9. Admin13 August 2013 at 11:11

    Please email me on roadtoautomation@gmail.com

    ReplyDelete
  10. View Higher26 August 2013 at 17:28

    Even i am facing the same issue

    ReplyDelete
  11. Unknown10 October 2013 at 17:41

    Hi,
    I need your help to integrated the calabash-android with jenkins.
    I have made the build.xml file as shown above and have placed it in the workspace folder. But I believe I need to change few parameters according to my project.
    Could you please advice what parameters should be changed.

    Kaushal

    ReplyDelete
    Replies
    1. Admin10 October 2013 at 20:52

      What type of parameter you want change ?

      First run your test scenario by command using ant. After that put all your project under Jenkins workspace and create job as above.

      Delete
  12. Unknown11 October 2013 at 04:35

    Ok let me explain my situation.
    I have created the my calabash-android tests and when I try to run in the command prompt, it runs correctly. In the command prompt, I use the following command
    calabash-android run C:\Mobile\MobileAutomation\Calabash-Tests\SanUK.apk -p none features\my_first.feature --format json -o cucumber.json
    Now I want to run these calabash tests in Jenkins. I read your blog but got a bit confused as this is my first attempt to work with Jenkins.
    I have installed Jenkins and the cucumber-reports plugin.
    Now as per your blog, I tried to create the ant build file by copying all the contents shown in your blog. I have put this build.xml file in the below location
    C:\Users\c0209693\.jenkins\jobs\Android_Native\workspace
    But in the build.xml file could you please explain what should I change the value of these tags?
    Project name
    Default
    Basedir
    Property name & location  location where the apk file is kept
    Target name
    Exec dir
    Executable = cmd
    Arg line

    As of now I have copied all the values as mentioned in your blog and configured the job as per your instructions. Now when I build it, I get the following output

    Started by user anonymous
    Building in workspace C:\Users\c0209693\.jenkins\jobs\Android_Native\workspace
    [workspace] $ cmd.exe /C '"ant.bat -file build.xml Login1 && exit %%ERRORLEVEL%%"'
    Buildfile: C:\Users\c0209693\.jenkins\jobs\Android_Native\workspace\build.xml

    Login1:
    [exec] *** WARNING: You must use ANSICON 1.31 or higher (https://github.com/adoxa/ansicon/) to get coloured output on Windows
    [exec] cucumber.yml was not found. Current directory is C:/Users/c0209693/.jenkins/jobs/Android_Native/workspace. Please refer to cucumber's documentation on defining profiles in cucumber.yml. You must define a 'default' profile to use the cucumber command without any arguments.
    [exec] Type 'cucumber --help' for usage.

    BUILD SUCCESSFUL
    Total time: 4 seconds
    [CucumberReportPublisher] Compiling Cucumber Html Reports ...
    [CucumberReportPublisher] detected this build is running on the master
    [CucumberReportPublisher] there were no json results found in: C:\Users\c0209693\.jenkins\jobs\Android_Native\workspace
    [CucumberReportPublisher] there were no json results found in: C:\Users\c0209693\.jenkins\jobs\Android_Native\builds\2013-10-11_00-01-32\cucumber-html-reports
    Finished: SUCCESS

    my build.xml file is this:










    Right now in my workspace folder there is only one file - build.xml

    Please guide me step step how to resolve this issue

    ReplyDelete
    Replies
    1. Admin11 October 2013 at 23:35

      did you put only build.xml file or all project into workspace ?
      As per your error I observed that you only put build.xml file into workspace.

      Please put all your project in workspace with build.xml file.

      Delete
  13. Unknown12 October 2013 at 00:47

    Thank you,
    It Worked :)

    ReplyDelete
  14. Unknown14 October 2013 at 21:23

    Hi,
    I have one more query.
    How can I link Jenkins with ALM 11 to run the calabash scripts.

    I have installed the "HP Application Automation Tools" and have configured the job accordingly. But I am getting the below error when running the job.

    Failed login to ALM Server URL: http://alm3........gs.corp/qcbin/start_a.jsp. Exception: java.io.FileNotFoundException: http://alm3........gs.corp/qcbin/start_a.jsp/rest/is-authenticated
    Empty Results
    Result Status: UNSTABLE
    Build step 'Execute HP tests using HP ALM Lab Management' changed build result to UNSTABLE
    Report not found
    Build step 'Publish HP tests result' changed build result to FAILURE
    Finished: FAILURE


    Please help.

    ReplyDelete
    Replies
    1. Admin14 October 2013 at 22:38

      I have no experience on Jenkins with ALM 11. I will reply if I get any solution.

      Delete
  15. TestEverything2 January 2014 at 16:13

    Is anyy way to use cucumber report plugin if i pass the json report file and it will generate the html report using cmd prompt

    ReplyDelete
    Replies
    1. Admin5 January 2014 at 12:35

      No need to generate report in JSON format, You can pass flag "-f html -o test-result.html" with calabash run command to generate report in HTML format.

      For more detail go below post:
      http://roadtoautomation.blogspot.in/2013/06/road-to-create-calabash-android-test.html

      Delete
  16. Priya3 January 2014 at 16:38

    Hi,

    I tried to configure as given above. The build is successful but the report gives 404 error.

    Can you help to resolve the issue.

    I got the following console output:

    BUILD SUCCESSFUL
    Total time: 3 seconds
    [CucumberReportPublisher] Compiling Cucumber Html Reports ...
    [CucumberReportPublisher] detected this build is running on a slave
    [CucumberReportPublisher] copying json from: file:/D:/Jenkins/connections/workspace/Calabash_Android/to reports directory: file:/D:/Jenkins/jobs/Calabash_Android/builds/2014-01-03_16-30-25/cucumber-html-reports/
    [CucumberReportPublisher] there were no json results found in: D:\Jenkins\jobs\Calabash_Android\builds\2014-01-03_16-30-25\cucumber-html-reports
    Finished: SUCCESS

    ReplyDelete
    Replies
    1. Admin5 January 2014 at 12:37

      are your project and build.xml file into workspace ?

      Delete
    2. Priya7 January 2014 at 11:14

      yes

      Delete
    3. Unknown11 May 2014 at 15:49

      Hi,
      Are some one how have got this kind of issues?

      Below show you which fails I usually get when I run a job in Jenkins:
      1: ant file

      [exec] Could not find an Android SDK please make sure it is installed.
      [exec] You can read about how Calabash-Android is searching for an Android SDK and how you can help here:
      [exec] https://github.com/calabash/calabash-android/blob/master/documentation/installation.md#prerequisites
      [exec] Result: 1

      2: shell command

      + calabash-android run /Users/ReinaldoB/calabash-samle-proj/magine/aliceoid_internal_0_36.apk
      Could not find an Android SDK please make sure it is installed.

      If I run above in a terminal all working well. I have got an explication that this issue I got is due to latest version of calabash like 0.4.21.

      Have some of you resolved this issue?

      Delete
    4. Admin11 May 2014 at 17:10

      I have updated calabash 0.4.21 and getting same issue and tried multiple machine but getting same issue for 0.4.21.
      After downgrading Calabash android my issue resolved

      Delete
  17. Unknown12 May 2014 at 15:23

    Hi,
    I have downgraded Calabash android and the issue is still left. See below.

    [exec] /Library/Ruby/Gems/2.0.0/gems/calabash-android-0.4.9/bin/calabash-android:53:in `raise_if_android_home_not_set': Please set the ANDROID_HOME environment variable (RuntimeError)
    [exec] from /Library/Ruby/Gems/2.0.0/gems/calabash-android-0.4.9/bin/calabash-android:72:in `'
    [exec] from /usr/bin/calabash-android:23:in `load'
    [exec] from /usr/bin/calabash-android:23:in `'
    [exec] Result: 1

    Do you know where in jenkins I need to set the ANDROID_HOME path because in my .profile in mac i have it.

    ReplyDelete
    Replies
    1. Admin12 May 2014 at 21:42

      I am using calabash-android version 0.4.16 and it is working at my end..

      Delete
  18. Unknown13 July 2014 at 09:46

    Hi,

    I've been struggling with a problem, I don't know how to get over it. Any helps would be much appreciated.
    I'd started with my build.xml configured as shown above on the post. When I started to build, an error occurred like below:

    Started by user anonymous
    Building in workspace C:\Program Files (x86)\Jenkins\jobs\Test\workspace
    [workspace] $ cmd.exe /C '"ant.bat -file build.xml login && exit %%ERRORLEVEL%%"'
    Buildfile: C:\Program Files (x86)\Jenkins\jobs\Test\workspace\build.xml

    login:
    [exec] 'calabash-android' is not recognized as an internal or external command,
    [exec] operable program or batch file.
    [exec] Result: 1

    BUILD SUCCESSFUL
    Total time: 0 seconds
    [CucumberReportPublisher] Compiling Cucumber Html Reports ...
    [CucumberReportPublisher] copying all json files from: C:\Program Files (x86)\Jenkins\jobs\eMobiz\workspace to reports directory: C:\Program Files (x86)\Jenkins\jobs\eMobiz\builds\2014-07-13_10-57-21\cucumber-html-reports
    [CucumberReportPublisher] there were no json results found in: C:\Program Files (x86)\Jenkins\jobs\eMobiz\builds\2014-07-13_10-57-21\cucumber-html-reports
    Finished: SUCCESS

    Thanks for your help,

    ReplyDelete
    Replies
    1. Admin13 July 2014 at 13:00

      Are you able to run test scripts using CMD, do you have setup ruby bin path in system variable ?

      Delete
    2. Unknown13 July 2014 at 17:07

      Hi,

      Thanks for your quick response.

      My script runs well on CMD, and I also added ruby path into the PATH environment variable.
      I've cloned your build.xml. But I don't know why the system doesn't recognize calabash-android command.
      (In additional, I'm using apache ant 1.9.4, I also added ANT_HOME variable, and ANT_HOME/bin into path env ).
      And I also tried to run cucumber script with the command: cmd c/ cucumber , but it didn't work either.

      Do you have any ideas of this situation?

      Deeply thanks for you help,


      Delete
    3. Admin13 July 2014 at 21:48

      There might me some thing wrong in setup environments variables. In Environments Variables there are two section one for User variables and other for System variables if you setup path in user variables then remove from there and setup all path in system variables section.

      Delete
    4. Unknown14 July 2014 at 10:09

      Hi,

      I've already placed all environment variables in system variables section.

      I've tried to configure on 2 computers, one works perfectly and another falls into that stuff.

      I think maybe I'm using a higher version of calabash-android. I'll try to downgrade its version and let you know results then. :)

      Thanks for your help,

      Delete
    5. Unknown23 November 2015 at 16:17

      In finally, How you can resolve in this problem please tell

      Thank you.

      Delete
  19. Unknown14 July 2014 at 10:02

    This comment has been removed by the author.

    ReplyDelete

Leave your comments, queries, suggestion I will try to provide solution

Subscribe to: Post Comments (Atom)