Xray Integration

In this page

What is Xray?

(supported since Better PDF Exporter 8.1.0)

Xray is a complete, end-to-end test management app for Jira. It supports the entire testing and quality assurance lifecycle: test planning, test design, test execution and test reporting.

Xray integration features

• You can export the Xray specific custom field types like Automated Test Type Select List (single choice), Cucumber Test Steps Editor, Generic Test Editor, Issue Link Count, Manual Test Steps, Manual Test Steps (Export View), Manual Test Steps Count, Pre-Condition Association with a Test, Pre-Condition Type Select List (single choice), Pre-Conditions Editor, Requirement Status, Requirement Status Custom Field, Test Association with a Pre-Condition, Test Association with a Test Plan, Test Environments Custom Field, Test Execution Association with Tests, Test Execution Defects, Test Execution Status Custom Field, Test Plan, Test Plan Association with a Test, Test Plan Root Folders Custom Field, Test Plan Status Custom Field, Test Repository Path, Test Run Status Custom Field, Test Set Status Custom Field, Test Sets Picker, Test Type Select List (single choice) and Tests to PDF.
• You can export Xray-managed information using the highly-configurable sections in the issue-fo.vm template:
  • for tests:
    • for manual tests: test repository path, test type, test steps (instructions, input/data and expected result), pre-condition list, test set list, test plan list and test run list
    • for Cucumber tests: test type, scenario type, scenario, pre-condition list, test set list, test plan list and test run list
    • for generic tests: test repository path, test type, test definition, pre-condition list, test set list, test plan list and test run list
  • for pre-conditions: pre-condition type, condition and test list
  • for test sets: overall execution status and test list
  • for test plans: begin date, end date, overall execution status, test list and test execution list
  • for test executions: test plan, test environments, begin date, end date, revision, overall execution status, test list

(Need more features? Tell us in a support ticket!)

Xray integration vs. the Xray built-in PDF exports

You may want to ask: if Xray has a built-in PDF export feature, why would you use another app for the same?

While the Xray built-in PDF exports may be sufficient for basic use cases, the Better PDF Exporter integration is more powerful in, at least, these:

• It supports PDF automation.
  You can export, email and attach PDF files using Automation for Jira (visual automation rules), ScriptRunner (Groovy scripts), or the REST API (use your favorite language!).
• It is extremely customizable.
  Instead of accepting the pre-defined export types in Xray, you can define your own types via PDF templates.
• It supports all the powerful PDF features.
  You can use custom calculations in Groovy (sorting, complex math, data integration, etc.), embedded attachments, PDF bookmarks, charts, graphics (in Groovy, SVG, QR codes and barcodes, etc.), among others.
• It is integrated with all the popular Jira apps.
  You can freely combine Xray data and other apps' data in the same PDF file.
• It comes with powerful templates optimized for Xray.
  You can use those "as is", or customize them to your needs.
• It is well-documented with the customization guide, the Expression Reference Manual, the recipes, and such.

If at least one of these is important for you, give the app a try.

Tutorial video

Better PDF Exporter exports all test-related information, including all Xray-specific entities and all Xray-specific custom fields, to PDF:

Xray PDF export samples

Configuration

Configuring the Xray custom fields

There is nothing to do. Better PDF Exporter will automatically recognize the Xray managed fields and export them accordingly.

Configuring the Xray sections in PDF templates

The template issue-fo.vm contains the following configuration parameters in its top part, to enable these features:

## Xray
#set($exportXrayTestExecutions = true) ## set to "true" to export Xray test executions
#set($exportXrayTestRuns = true) ## set to "true" to export Xray test runs

Note that the issue-fo.vm template shows the Xray-specific sections only for the issues that are actual Xray entities. It decides whether an issue is an Xray entity by checking its issue type name (e.g. if it is a "Test Plan") and its custom fields (e.g. if it has the "Test Type" custom field). Therefore, if you rename issue types, you also need to adapt the logic around this part in the template code:

## test management data
#set($testManagementIssueTypeNames = ['Pre-Condition', 'Test', 'Test Execution', 'Test Plan', ...])
#if($testManagementIssueTypeNames.contains($issue.issueTypeObject.name))
	## ...

Configuring the export format for the Xray "associated with" custom fields

Xray is using several custom field types that associate an Xray-managed issue type with another, e.g. associate tests with test sets. In the Xray web interface, these values are displayed in the format "FOO-123 FOO-456" where each item is a link. (For some unknown reason, Xray does not even separate the items with commas.)

When exporting to PDF, Better PDF Exporter resolves the issue keys to summaries, and expands the values to the longer, but externally usable format "[FOO-123] My first test (line break) [FOO-456] My second test". If you want to turn this off and return to "FOO-123, FOO-456" (with commas!), set this configuration variable to false in the top of the issue-fo.vm and issue-navigator-fo.vm templates:

#set($exportDetailedValues = true)

Note that the same variable also controls the format of some other fields, as explained in the next section.

Configuring the export format for the Xray "Test Plan Status" and "Test Set Status" custom fields

When you have 4 tests and all those were executed successfully, the status overview fields will be exported in the "full" format that includes all statuses (even those with zero test counts):

PASS: 4 (100%), FAIL: 0 (0%), ABORTED: 0 (0%), EXECUTING: 0 (0%), TODO: 0 (0%)

For the easier reading, you may want to use the compact format that shows only the non-zero statuses:

PASS: 4 (100%)

You can switch to the compact format by setting this configuration variable to false in the top of the issue-fo.vm and issue-navigator-fo.vm templates:

#set($exportDetailedValues = true)

Note that the same variable also controls the format of some other fields, as explained in the previous section.

Also note that the "Test Execution Status" custom field is implemented differently in Xray (for an unknown reason), and as a consequence, it is always exported in the full format.

Configuring the export format for the Xray manual test step custom fields

Xray has two separate custom fields that display the test steps of manual tests, both being supported by Better PDF Exporter. Note that their display format is slightly different in the web UI and also in the PDF documents.

This table helps to choose the one that is more suitable for your exports:

Custom field name	Format in the PDF document
"Manual Test Steps"	"Step", "Data" and "Expected Result" are exported to separate lines, with an empty line between steps. Tip: choose this if the space in the paper is limited. (This format is using less paper space.)
"Manual Test Steps (Export)"	Table with "Step", "Input/Data" and "Expected Results" columns. Tip: choose this for most use cases. (This format is more human-readable.)

Note that the "Manual Test Steps (Export)" custom field was removed in Xray 4.0. Since Better PDF Exporter 9.3.0, there is a configuration option in the issue-fo.vm and issue-navigator-fo.vm templates to control the output format of the "Manual Test Steps" custom field:

#set($exportManualTestStepsAsTable = true) ## set to "true" to export "Manual Test Steps" as table (for Xray 4.0 or later)

Exporting associated Xray test entities

In many use cases, you want to export multiple types of associated, Xray-managed issues with their full details to the same document. For example, you want to export a test set (with its details) plus the tests included in the test set (with their details) to the same document, for a comprehensive test report or test archive.

It is perfectly doable, because Better PDF Exporter allows freely mixing issues of various types within the same PDF document. The question is how to search for the input issues that will result in the expected document? In the above example, how to search for the test set and its tests with proper ordering, using the Issue Navigator?

Luckily, the JQL query language and the Xray custom JQL functions offer flexible searching for associated Xray issues. In the next sections, we will give working examples for typical use cases. While these are great to get the base idea, we strongly recommend reading the Xray queries page for all possibilities.

Exporting an Xray test entity and its associations

These examples show how to find the result merged from:

1. a single "source" test entity identified by its issue key (e.g. "key = FOO-123")
2. the associated issues returned by an Xray-specific JQL function which is parametrized with the source entity's key (e.g. "testPlanTests(FOO-123)" returns the tests in the FOO-123 test plan)

Our preferred ordering is exporting the source entity in the first position, then the associated issues ordered by their keys. For this, the examples use the "ORDER BY issuetype ASC, key ASC" clause, which works as expected for English issue type names. It may happen that in your language, the issue type name of the source entity and the issue type name of the associated issues are in the reverse order. If so, just use "ORDER BY issuetype DESC, key ASC".

Based on this recipe, you can easily craft your own JQL, but here are some examples with no further explanation.

Exporting a test plan and its associated tests

key = FOO-123 OR key IN testPlanTests(FOO-123) ORDER BY issuetype DESC, key ASC

Exporting a test and its associated pre-conditions

key = FOO-234 OR key IN testPreConditions(FOO-234) ORDER BY issuetype DESC, key ASC

Exporting a test and its associated test executions

key = FOO-345 OR key IN testTestExecutions(FOO-345) ORDER BY issuetype ASC, key ASC

Exporting a requirement and its associated tests

key = FOO-456 OR key IN requirementTests(FOO-456) ORDER BY issuetype ASC, key ASC

Exporting multiple Xray test entities and their associations

These examples are similar to the ones in the previous section, but in this case we have multiple source entities. For example, you want to export multiple test plans and all their associated tests.

To accomplish that, create a saved filter that returns the source entities. Then refer that twice in the final JQL query to get the merged result, totally similarly to what you did in the previous examples with the source entity's issue key.

Also in this case, our preferred ordering is the source entities first, then all the associated issues after those. For that, just use an "ORDER BY issuetype ASC, key ASC" or "ORDER BY issuetype DESC, key ASC". (Of course, if an issue is associated with more than one source, it will be exported only once.)

Exporting multiple test plans and their associated tests

filter = "My Tests Plans" OR key IN testPlanTests("My Tests Plans") ORDER BY issuetype DESC, key ASC

Configuring the Xray REST API access

As this integration relies on the Xray REST API, you need to configure the login credentials of a valid Jira user account for the REST API calls in the templates:

1. Go to Administration → Add-ons → PDF Templates (under Better PDF Exporter).
2. Open the issue-fo.vm template for editing, and set the username and password to these configuration variables in the top part (don't remove the quotation marks around the string!):

   ## issue-fo.vm
   
   ## Jira user credentials for REST API calls
   #set($restUserName = "admin") ## Jira username for REST authentication
   #set($restPassword = "admin") ## Jira password for REST authentication
   

3. Save the changes. (Don't worry about storing passwords here: this file is visible only for Jira administrators, who would have super-user permissions anyway.)

Notes:

• If you are using multiple REST API based integrations in the same template (ex: Git Integration, Gliffy, Zephyr Squad), then all REST API calls will use with the same user credentials.
• It is safer to avoid usernames and passwords that contain non-English characters. Albeit our Xray script correctly encodes international usernames and passwords, their handling also depends on the configuration of the container that hosts your Jira web application (typically Tomcat). If you're having difficulties, just replace those characters in your username or password with English letters or numbers.
• Similarly, it is safer to avoid usernames and password that contain special characters (most typically "#" and "$"). These are special control characters in Velocity strings. If you can't avoid those, escape the characters according to the language rules.

Finally, the following section describes a confusing situation when all REST API calls result in "Server returned HTTP response code: 403" or "401" errors. If you are not affected, you can skip this section.

Why does this happen? As its default behavior, Jira will lock your account and present a CAPTCHA on the login form after a few unsuccessful login attempts. Since the Xray integration may send several REST API requests per export, all using the same username and password, the unsuccessful login attempts limit can be reached very quickly if the password is wrong.

How to fix it? You just have to pass the CAPTHCA verification, that's it. First off, be 101% sure that the REST username and password are correct. If those are, but you are still receiving 403's, please open Jira in your browser, logout and login with the same user account which you configured for the REST calls. You will be asked for a CAPTCHA verification, but after answering that and logging in to Jira, the REST calls will also be successfully executed!

Please note that it will never happen again, unless you change the password of the corresponding Jira user account.

If you are an administrator, there is also a quicker solution: go to Administration → User management and clear the failed login attempts counter with one click. Or, you can increase the allowed login attempts or even disable this feature altogether in Jira.

Troubleshooting

I cannot export test runs from pre-2.0.0 Xray versions

Exporting the test runs with the issue-fo.vm template relies on the /rest/raven/1.0/api/test/{key}/testruns endpoint provided by the Xray REST API. This endpoint is available since Xray 2.0.0. Therefore, Better PDF Exporter exports test runs only with Xray 2.0.0 or newer versions.

Note that other information, stored in Xray-specific custom fields, can be correctly exported even with older Xray versions.

Learn more about Xray

• Xray product information (at its own vendor)
• Xray documentation
• Xray app page (on the Atlassian Marketplace)