Xray Integration

In this page

What is Xray?

(supported since Better Excel Exporter 5.1.0)

Xray is a complete, end-to-end test management app for Jira. It supports the entire testing and quality assurance lifecycle: test planning, test design, test execution and test reporting.

Xray integration features

• You can export the Xray specific custom field types like Automated Test Type Select List (single choice), Cucumber Test Steps Editor, Generic Test Editor, Issue Link Count, Manual Test Steps, Manual Test Steps (Export View), Manual Test Steps Count, Pre-Condition Association with a Test, Pre-Condition Type Select List (single choice), Pre-Conditions Editor, Requirement Status, Requirement Status Custom Field, Test Association with a Pre-Condition, Test Association with a Test Plan, Test Environments Custom Field, Test Execution Association with Tests, Test Execution Defects, Test Execution Status Custom Field, Test Plan, Test Plan Association with a Test, Test Plan Root Folders Custom Field, Test Plan Status Custom Field, Test Repository Path, Test Run Status Custom Field, Test Set Status Custom Field, Test Sets Picker, Test Type Select List (single choice) and Tests to Excel.
• You can export the test runs for detailed test reports. The following details are exported for each run: Test Execution Key, Test Run Fix Versions, Test Run Test Environments, Test Run Assignee, Test Run Executed By, Test Run Started On, Test Run Finished On, Test Run Duration, Test Run Defects, Test Run Comment, Test Run Evidences, Test Run Status, Test Step Index, Test Step, Test Step Data, Test Step Expected Result, Test Step Attachments, Test Step Defects, Test Step Comment, Test Step Evidences and Test Step Status. (supported since Xray 2.0.0)
• Better Excel Exporter also offers two report templates: one focusing on tests and their relations (with test sets, test plans, test executions), while the other focusing on test runs and test results. Both include several pivot tables and pivot charts, which are super-useful as they are, but can also be used as a starting point to create custom Excel reports from Xray data.

(Need more features? Tell us in a support ticket!)

Xray integration vs. the Xray built-in Excel exports

You may want to ask: if Xray has a built-in Excel export feature, why would you use another app for the same?

While the Xray built-in Excel exports may be sufficient for basic use cases, the Better Excel Exporter integration is more powerful in, at least, these:

• It supports Excel automation.
  You can export, email and attach Excel files using Automation for Jira (visual automation rules), ScriptRunner (Groovy scripts), or the REST API (use your favorite language!).
• It is extremely customizable.
  Instead of accepting the pre-defined export types in Xray, you can define your own types via Excel templates.
• It supports all the powerful Excel features.
  You can use Excel formulas and functions, custom calculations in Groovy (sorting, complex math, data integration, etc.), charts, pivot tables, pivot charts, Visual Basic macros, among others.
• It is integrated with all the popular Jira apps.
  You can freely combine Xray data and other apps' data in the same Excel file.
• It comes with powerful templates optimized for Xray.
  You can use those "as is", or customize them to your needs.
• It is well-documented with the customization guide, the Expression Reference Manual, the recipes, and such.

If at least one of these is important for you, give the app a try.

Tutorial video

Better Excel Exporter exports all test-related information, including all Xray-specific entities and all Xray-specific custom fields, to Excel:

Xray Excel export samples

Configuration

Configuring the Xray Excel views

Note that the Xray custom fields will be available in every Excel view. There are additional views that are specific to Xray and are disabled by default. Don't forget to activate those!

Steps:

1. Login to Jira as administrator.
2. Go to Administration → Add-ons → Excel Views (under Better Excel Exporter).
3. Click the view Xray Test Runs (All f.).
4. Check the contexts (screens) in which you want this integration be available. (If unsure, check Issue Navigator at least.)
5. Click Save.

Repeat this also for the following views:

• Xray Test Runs (Current f.)
• Xray Test Report (Excel)
• Xray Test Run Report (Excel)

Configuring the Xray custom fields

There is nothing to do. Better Excel Exporter will automatically recognize the Xray managed fields and export them accordingly.

Configuring the export format for the Xray "associated with" custom fields

Xray is using several custom field types that associate an Xray-managed issue type with another, e.g. associate tests with test sets. In the Xray web interface, these values are displayed in the format "FOO-123 FOO-456" where each item is a link. (For some unknown reason, Xray does not even separate the items with commas.)

When exporting to Excel, Better Excel Exporter resolves the issue keys to summaries, and expands the values to the longer, but externally usable format "[FOO-123] My first test, [FOO-456] My second test". If you want to turn this off and return to "FOO-123, FOO-456" (with commas!), set this configuration variable to false in the top of the field-helper-tool.groovy script:

def xrayIssuesWithSummaries = true

These values are comma-separated, by default. If you want to break them to new lines within the cell, to make it easier to read at the cost of the increased cell height, it is simple. Just change the join() method's argument from ", " to "\n" in the field-helper-tool.groovy script in the following way:

case "com.xpandit.plugins.xray:precondition-test-custom-field":
case "com.xpandit.plugins.xray:test-plan-custom-field":
case "com.xpandit.plugins.xray:test-plans-associated-with-test-custom-field":
case "com.xpandit.plugins.xray:test-precondition-custom-field":
case "com.xpandit.plugins.xray:test-sets-custom-field":
case "com.xpandit.plugins.xray:test-sets-tests-custom-field":
case "com.xpandit.plugins.xray:tests-associated-with-test-plan-custom-field":
	value ? value.collect { xrayIssueKeyValueToString(it) }.join("\n") : null
	break

// ...

case "com.xpandit.plugins.xray:testexec-tests-custom-field":
	value ? value.collect { xrayIssueKeyValueToString(!it.metaClass.respondsTo(it, "a") ? it : it.a()) }.join("\n") : null
	break

Configuring the Xray REST API access

As this integration relies on the Xray REST API, you need to configure the login credentials of a valid Jira user account for the REST API calls in the templates:

1. Go to Administration → Add-ons → Excel Templates (under Better Excel Exporter).
2. Open the xray-tool.groovy template for editing, and set the username and password to these configuration variables in the top part (don't remove the quotation marks around the string!):

   // xray-tool.groovy
   
   /* Xray REST API request parameters. */
   def restUserName = "admin"
   def restPassword = "admin"
   

3. Save the changes. (Don't worry about storing passwords here: this file is visible only for Jira administrators, who would have super-user permissions anyway.)

Notes:

• It is safer to avoid usernames and passwords that contain non-English characters. Albeit our Xray script correctly encodes international usernames and passwords, their handling also depends on the configuration of the container that hosts your Jira web application (typically Tomcat). If you're having difficulties, just replace those characters in your username or password with English letters or numbers.
• Similarly, it is safer to avoid usernames and password that contain special characters (most typically "#" and "$"). These are special control characters in Groovy strings. If you can't avoid those, escape the characters according to the language rules.

Finally, the following section describes a confusing situation when all REST API calls result in "Server returned HTTP response code: 403" or "401" errors. If you are not affected, you can skip this section.

Why does this happen? As its default behavior, Jira will lock your account and present a CAPTCHA on the login form after a few unsuccessful login attempts. Since the Xray integration may send several REST API requests per export, all using the same username and password, the unsuccessful login attempts limit can be reached very quickly if the password is wrong.

How to fix it? You just have to pass the CAPTHCA verification, that's it. First off, be 101% sure that the REST username and password are correct. If those are, but you are still receiving 403's, please open Jira in your browser, logout and login with the same user account which you configured for the REST calls. You will be asked for a CAPTCHA verification, but after answering that and logging in to Jira, the REST calls will also be successfully executed!

Please note that it will never happen again, unless you change the password of the corresponding Jira user account.

If you are an administrator, there is also a quicker solution: go to Administration → User management and clear the failed login attempts counter with one click. Or, you can increase the allowed login attempts or even disable this feature altogether in Jira.

Troubleshooting

I cannot export test runs from pre-2.0.0 Xray versions

Exporting the test runs with the issue-navigator-with-xray-test-runs.xlsx and xray-test-run-report.xlsx templates relies on the /rest/raven/1.0/api/test/{key}/testruns endpoint provided by the Xray REST API. This endpoint is available since Xray 2.0.0. Therefore, Better Excel Exporter exports test runs only with Xray 2.0.0 or newer versions.

Note that other information, stored in Xray-specific custom fields, can be correctly exported even with older Xray versions.

Learn more about Xray

• Xray product information (at its own vendor)
• Xray documentation
• Xray app page (on the Atlassian Marketplace)