Skip to main content

Format test reports

Results on the Tests tab are parsed from test reports specified in the Report Paths setting in Run and Run Tests steps. Test reports must be in JUnit XML format to appear on the Tests tab, because Harness parses test reports that are in JUnit XML format only.

For information about code coverage reports and viewing reports on the Artifacts tab, go to Code Coverage.

JUnit XML format resources

Use these resources to learn about JUnit XML formatting.

Tools with built-in JUnit XML output

Here are some Harness YAML examples for tools that produce JUnit XML output by default.

C, C++

You can use the --output-junit command with CTest.

              - step:
type: Run
identifier: test
name: Test
spec:
shell: Sh
command: |-
mkdir build
cmake -S . -B build
ctest --test-dir build --output-junit out.xml
reports:
type: JUnit
spec:
paths:
- /harness/build/out.xml

Java - Gradle

This example runs Gradle tests with Test Intelligence.

              - step:
type: RunTests
identifier: Run_Tests_with_Intelligence
name: Run Tests with Intelligence
spec:
args: test --tests
buildTool: Gradle
enableTestSplitting: true
language: Java
reports:
spec:
paths:
- "/harness/results.xml"
type: JUnit
runOnlySelectedTests: true

PHP

              - step:
type: Run
identifier: test
name: Test
spec:
shell: Sh
command: |-
mkdir -p /harness/phpunit
phpunit --log-junit /harness/phpunit/junit.xml tests
reports:
type: JUnit
spec:
paths:
- /harness/phpunit/junit.xml

Python

              - step:
type: Run
identifier: test
name: Test
spec:
shell: Sh
command: |-
. venv/bin/activate
mkdir /harness/test-results
pytest --junitxml=harness/test-results/junit.xml
reports:
type: JUnit
spec:
paths:
- /harness/test-results/junit.xml
info
  • You can use pytest to run unittest.
  • If you use test splitting with pytest, you must set junit_family=xunit1 in your code repo's pytest.ini file or include -o junit_family="xunit1" in the step's command.

Ruby - Cucumber

              - step:
type: Run
identifier: test
name: Test
spec:
shell: Sh
command: |-
bundle check || bundle install
mkdir -p /harness/cucumber
bundle exec cucumber --format junit --out /harness/cucumber/junit.xml
reports:
type: JUnit
spec:
paths:
- /harness/cucumber/junit.xml

JUnit converters, formatters, and plugins

If your test tool doesn't automatically produce test results in JUnit XML format, there are JUnit converters, formatters, and plugins available for all major languages. Some examples of conversion tools and corresponding Harness YAML are provided below.

C# - .NET Core, NUnit

              - step:
type: Run
identifier: test
name: Test
spec:
shell: Powershell
command: |-
cd dotnet-agent/TestProject1
wget -UseBasicParsing https://dot.net/v1/dotnet-install.ps1 -o dotnet-install.ps1
.\dotnet-install.ps1
dotnet build

wget https://raw.githubusercontent.com/nunit/nunit-transforms/master/nunit3-junit/nunit3-junit.xslt -o nunit3-junit.xslt

"C:/Program Files (x86)/NUnit.org/nunit-console/nunit3-console.exe" dotnet-agent/TestProject1/bin/Debug/net48/TestProject1.dll --result="UnitTestResults.xml;transform=nunit3-junit.xslt"
reports:
type: JUnit
spec:
paths:
- UnitTestResults.xml

Clojure

Go

You can use the go-junit-report tool.

              - step:
type: Run
identifier: test
name: Test
spec:
shell: Sh
command: |-
go install github.com/jstemmer/go-junit-report/v2@latest
go test -v ./... | tee report.out
cat report.out | $HOME/go/bin/go-junit-report -set-exit-code > report.xml
reports:
type: JUnit
spec:
paths:
- report.xml

Java - Maven

This example uses the Maven Surefire Plugin and runs tests with Maven and Test Intelligence.

              - step:
type: RunTests
identifier: Run_Tests_with_Intelligence
name: Run Tests with Intelligence
spec:
args: test -Dmaven.test.failure.ignore=true -DfailIfNoTests=false
buildTool: Maven
enableTestSplitting: true
language: Java
reports:
spec:
paths:
- "target/surefire-reports/*.xml"
type: JUnit
runOnlySelectedTests: true

JavaScript

ESLint

ESLint JUnit Formatter

  - step:
type: Run
name: Run ESLint Tests
identifier: run_eslint_tests
spec:
shell: Sh
command: |
mkdir -p /harness/reports
eslint ./src/ --format junit --output-file /harness/reports/eslint.xml
reports:
type: JUnit
spec:
paths:
- "/harness/reports/eslint.xml"
Jest

Jest JUnit Reporter

  - step:
type: Run
name: Run Jest Tests
identifier: run_jest_tests
spec:
shell: Sh
command: |
yarn add --dev jest-junit
jest --ci --runInBand --reporters=default --reporters=jest-junit
envVariables:
JEST_JUNIT_OUTPUT_DIR: "/harness/reports"
reports:
type: JUnit
spec:
paths:
- "/harness/reports/*.xml"
Karma

Karma JUnit Reporter

  - step:
type: Run
name: Run Karma Tests
identifier: run_karma_tests
spec:
shell: Sh
command: |
npm install
mkdir /harness/junit
karma start ./karma.conf.js
envVariables:
JUNIT_REPORT_PATH: /harness/junit/
JUNIT_REPORT_NAME: test-results.xml
reports:
type: JUnit
spec:
paths:
- "/harness/junit/test-results.xml"
Mocha

Mocha JUnit Reporter

  - step:
type: Run
name: Run Mocha Tests
identifier: run_mocha_tests
spec:
shell: Sh
command: |
npm install
mkdir /harness/junit
mocha test --reporter mocha-junit-reporter --reporter-options mochaFile=./path_to_your/file.xml
reports:
type: JUnit
spec:
paths:
- "/harness/junit/test-results.xml"

Ruby

Minitest

Add the Minitest Junit Formatter to your Gemfile.

  - step:
type: Run
name: Run Ruby Tests
identifier: run_ruby_tests
spec:
shell: Sh
command: |
bundle check || bundle install
bundle exec rake test --junit
reports:
type: JUnit
spec:
paths:
- "/harness/report.xml"
RSpec

Add the RSpec JUnit formatter to your Gemfile.

  - step:
type: Run
name: Run RSpec Tests
identifier: run_rspec_tests
spec:
shell: Sh
command: |
bundle check --path=vendor/bundle || bundle install --path=vendor/bundle --jobs=4 --retry=3
mkdir /harness/rspec
bundle exec rspec --format progress --format RspecJunitFormatter -o /harness/rspec/rspec.xml
reports:
type: JUnit
spec:
paths:
- "/harness/rspec/rspec.xml"