In this page

What is Zephyr?

(supported since Better Excel Exporter 3.2.0)

Zephyr is the most popular test management solution for the Jira platform.

Zephyr integration features

  • You can export the test steps for any test list. As Zephyr tests are regular Jira issues, this enables creating Excel files from fields (test details) plus test steps (instructions) with a single click.
  • You can export the test executions for any test list. Executions are exported with their details for a comprehensive report: ID, test cycle, version, execution status, assignee, executor user, execution date, execution defects, step defects and comments.
  • You can export the test step results of the test executions for detailed test reports. The following details are exported for each step: ID, step instructions, test data, expected result, execution status, and comments.
  • Better Excel Exporter also offers a reporting template that includes several pivot table and pivot chart worksheets to calculate test results by projects, test cycles, versions, testers or find defects by test cycles or individual tests. Although this is useful as is, this can also be used as a starting point to create custom Excel reports from Zephyr data.

(Need more features? Tell us in a support ticket!)

Zephyr integration vs. the Zephyr built-in Excel exports

You may want to ask: if Zephyr has a built-in Excel export feature, why would you use another app for the same?

While the Zephyr built-in Excel exports may be sufficient for basic use cases, the Better Excel Exporter integration is more powerful in, at least, these:

If any of these are important for you, give it a try.

Tutorial video

Watch this short video to get the gist of exporting Zephyr tests to Excel reports:

Export samples

Zephyr test steps

Zephyr test definitions are issues of the "Test" type. This example generated from the issue-navigator-with-test-steps.xlsx template shows a mix of regular issue fields in the left (key, summary, priority) and test step details in the right (instructions, test data, expected result).

jira-zephyr-test-steps.xlsx

Zephyr test executions

Easy and flexible: run a JQL search to find any arbitrary set of tests, then generate a filterable, searchable, shareable, grouped list of all test executions.

jira-zephyr-test-executions.xlsx

Zephyr test step results

This highly detailed Excel export captures the test results in three logical levels: tests, test executions for each test, and test step results for each execution. Being a complete snapshot of testing data, you can search this, import this to external systems, use it for archiving purposes, etc.

jira-zephyr-test-step-results.xlsx

Zephyr test report

You can also create powerful reports from Zephyr test data. The zephyr-report.xlsx template gives a valuable overview of test executions per version, test cycle and tester, shows the test execution history, and shows which tests and executions discover the most defects. Build your own Zephyr Excel report by copying this template.

jira-zephyr-report.xlsx

Configuration

Installing ZAPI

Please note that the Zephyr app itself does not expose any public API. Instead, there exists a separate app called ZAPI (also developed by the Zephyr team) to provide a REST API for Zephyr tests, test executions and so.

The Zephyr and Better Excel Exporter integration relies on the REST API provided by ZAPI. Therefore, please install ZAPI as the starting step. (Note: ZAPI is free since version 2.7.1.27107787. Before that, it was a paid app, but even if you need to use an older ZAPI version, a free trial license will be sufficient for evaluation purposes.)

Configuring the Zephyr REST API access

As this integration relies on the Zephyr REST API, you need to configure the login credentials of a valid Jira user account for the REST API calls in the templates:

  1. Go to AdministrationAdd-onsExcel Templates (under Better Excel Exporter).
  2. Open the zephyr-tool.groovy template for editing, and set the username and password to these configuration variables in the top part (don't remove the quotation marks around the string!):
    /* Zephyr REST API request parameters. */
    def restUserName = "admin"
    def restPassword = "admin"
    
  3. Save the changes. (Don't worry about storing passwords here: this file is visible only for Jira administrators, who would have super-user permissions any way.)

Notes:

  • As a good practice, this is safer to avoid usernames and passwords that contain non-English characters. Albeit our Zephyr script correctly encodes international usernames and passwords, their handling also depends on the configuration of the container that hosts your Jira web application (typically Tomcat). If you're having difficulties, just replace those characters in your username or password with English letters or numbers.

Finally, the following section describes a confusing situation when all REST API calls result in "Server returned HTTP response code: 403" or "401" errors. If you are not affected, you can skip this section.

Why does this happen? As its default behavior, Jira will lock your account and present a CAPTCHA on the login form after a few unsuccessful login attempts. Since the Zephyr integration may send several REST API requests per export, all using the same username and password, the unsuccessful login attempts limit can be reached very quickly if the password is wrong.

How to fix it? You just have to pass the CAPTHCA verification, that's it. First off, be 101% sure that the REST username and password are correct. If those are, but you are still receiving 403's, please open Jira in your browser, logout and login with the same user account which you configured for the REST calls. You will be asked for a CAPTCHA verification, but after answering that and logging in to Jira, the REST calls will also be successfully executed!

Please note that it will never happen again, unless you change the password of the corresponding Jira user account.

If you are an administrator, there is also a quicker solution: go to Administration → User management and clear the failed login attempts counter with one click. Or, you can increase the allowed login attempts or even disable this feature altogether in Jira.

Troubleshooting

Only the first 1000 Zephyr tests and executions are exported

This is a limit exposed by the Zephyr API for performance reasons. (Their documentation does not mention about lifting this limit, so you are encouraged to contact their support team.)

"Executed On" does not include the time part (only the date)

The Zephyr API returns this field as date-only (ex: "29/Aug/17"), albeit this is a complete date-time in the Zephyr web UI (ex: "2017/08/29 10:56").

We consider this a design issue in the Zephyr API and reported this to Zephyr team here. (Please vote that topic to increase its importance for the Zephyr team.)

When exporting step results, "Step", "Test Data", "Expected Result" and "Comment" texts are joined to a single-line string

The Zephyr API returns these fields without line-breaks.

We consider this a design issue in the Zephyr API and reported this to Zephyr team here. (Please vote that topic to increase its importance for the Zephyr team.)

When exporting step results, step defects are not associated with the steps, but with the executions

The Zephyr API returns the defects (including both execution-level and step-level defects) in the format "BUG-1, BUG-2 | BUG-3, BUG-4". In this string the issue keys before the pipe character are execution-level defects, while the ones after the pipe are step-level ones. From this information, one can separate execution-level defects from step-level defects, but cannot know which step the step-level defects are associated with.

We consider this a design issue in the Zephyr API and reported this to Zephyr team here. (Please vote that topic to increase its importance for the Zephyr team.)

I am using some older Zephyr version and the export does not work

Our integration party relies on the /execution/export API end-point provided by the Zephyr API. The XML document format generated by this API end-point has been changed multiple times before it reached its current shape. Therefore, our integration works with 3.2.0.32002611 and newer Zephyr versions.

Please note that this is possible to re-configure the zephyr-tool.groovy script for older Zephyr versions if you are restricted to one of those. In that case, just contact us.

Learn more about Zephyr