Format test reports
Results on the Tests tab are parsed from test reports specified in the Report Paths setting in Run and Run Tests steps. Test reports must be in JUnit XML format to appear on the Tests tab, because Harness parses test reports that are in JUnit XML format only.
For information about code coverage reports and viewing reports on the Artifacts tab, go to Code Coverage.
JUnit XML format resources
Use these resources to learn about JUnit XML formatting.
- Common JUnit XML format and examples from Testmo
- Examples of JUnit XML format mapping from IBM
- Apache Ant JUnit schema
Tools with built-in JUnit XML output
Here are some Harness YAML examples for tools that produce JUnit XML output by default.
C, C++
You can use the --output-junit command with CTest.
- step:
type: Run
identifier: test
name: Test
spec:
shell: Sh
command: |-
mkdir build
cmake -S . -B build
ctest --test-dir build --output-junit out.xml
reports:
type: JUnit
spec:
paths:
- /harness/build/out.xml
Java - Gradle
This example runs Gradle tests with Test Intelligence.
- step:
type: RunTests
identifier: Run_Tests_with_Intelligence
name: Run Tests with Intelligence
spec:
args: test --tests
buildTool: Gradle
enableTestSplitting: true
language: Java
reports:
spec:
paths:
- "/harness/results.xml"
type: JUnit
runOnlySelectedTests: true
PHP
- step:
type: Run
identifier: test
name: Test
spec:
shell: Sh
command: |-
mkdir -p /harness/phpunit
phpunit --log-junit /harness/phpunit/junit.xml tests
reports:
type: JUnit
spec:
paths:
- /harness/phpunit/junit.xml
Python
- step:
type: Run
identifier: test
name: Test
spec:
shell: Sh
command: |-
. venv/bin/activate
mkdir /harness/test-results
pytest --junitxml=harness/test-results/junit.xml
reports:
type: JUnit
spec:
paths:
- /harness/test-results/junit.xml
info
- You can use pytest to run unittest.
- If you use test splitting with pytest, you must set
junit_family=xunit1in your code repo'spytest.inifile or include-o junit_family="xunit1"in the step'scommand.
Ruby - Cucumber
- step:
type: Run
identifier: test
name: Test
spec:
shell: Sh
command: |-
bundle check || bundle install
mkdir -p /harness/cucumber
bundle exec cucumber --format junit --out /harness/cucumber/junit.xml
reports:
type: JUnit
spec:
paths:
- /harness/cucumber/junit.xml
JUnit converters, formatters, and plugins
If your test tool doesn't automatically produce test results in JUnit XML format, there are JUnit converters, formatters, and plugins available for all major languages. Some examples of conversion tools and corresponding Harness YAML are provided below.
C# - .NET Core, NUnit
- step:
type: Run
identifier: test
name: Test
spec:
shell: Powershell
command: |-
cd dotnet-agent/TestProject1
wget -UseBasicParsing https://dot.net/v1/dotnet-install.ps1 -o dotnet-install.ps1
.\dotnet-install.ps1
dotnet build
wget https://raw.githubusercontent.com/nunit/nunit-transforms/master/nunit3-junit/nunit3-junit.xslt -o nunit3-junit.xslt
"C:/Program Files (x86)/NUnit.org/nunit-console/nunit3-console.exe" dotnet-agent/TestProject1/bin/Debug/net48/TestProject1.dll --result="UnitTestResults.xml;transform=nunit3-junit.xslt"
reports:
type: JUnit
spec:
paths:
- UnitTestResults.xml
Clojure
Go
You can use the go-junit-report tool.
- step:
type: Run
identifier: test
name: Test
spec:
shell: Sh
command: |-
go install github.com/jstemmer/go-junit-report/v2@latest
go test -v ./... | tee report.out
cat report.out | $HOME/go/bin/go-junit-report -set-exit-code > report.xml
reports:
type: JUnit
spec:
paths:
- report.xml
Java - Maven
This example uses the Maven Surefire Plugin and runs tests with Maven and Test Intelligence.
- step:
type: RunTests
identifier: Run_Tests_with_Intelligence
name: Run Tests with Intelligence
spec:
args: test -Dmaven.test.failure.ignore=true -DfailIfNoTests=false
buildTool: Maven
enableTestSplitting: true
language: Java
reports:
spec:
paths:
- "target/surefire-reports/*.xml"
type: JUnit
runOnlySelectedTests: true
JavaScript
ESLint
- step:
type: Run
name: Run ESLint Tests
identifier: run_eslint_tests
spec:
shell: Sh
command: |
mkdir -p /harness/reports
eslint ./src/ --format junit --output-file /harness/reports/eslint.xml
reports:
type: JUnit
spec:
paths:
- "/harness/reports/eslint.xml"
Jest
- step:
type: Run
name: Run Jest Tests
identifier: run_jest_tests
spec:
shell: Sh
command: |
yarn add --dev jest-junit
jest --ci --runInBand --reporters=default --reporters=jest-junit
envVariables:
JEST_JUNIT_OUTPUT_DIR: "/harness/reports"
reports:
type: JUnit
spec:
paths:
- "/harness/reports/*.xml"
Karma
- step:
type: Run
name: Run Karma Tests
identifier: run_karma_tests
spec:
shell: Sh
command: |
npm install
mkdir /harness/junit
karma start ./karma.conf.js
envVariables:
JUNIT_REPORT_PATH: /harness/junit/
JUNIT_REPORT_NAME: test-results.xml
reports:
type: JUnit
spec:
paths:
- "/harness/junit/test-results.xml"
Mocha
- step:
type: Run
name: Run Mocha Tests
identifier: run_mocha_tests
spec:
shell: Sh
command: |
npm install
mkdir /harness/junit
mocha test --reporter mocha-junit-reporter --reporter-options mochaFile=./path_to_your/file.xml
reports:
type: JUnit
spec:
paths:
- "/harness/junit/test-results.xml"
Ruby
Minitest
Add the Minitest Junit Formatter to your Gemfile.
- step:
type: Run
name: Run Ruby Tests
identifier: run_ruby_tests
spec:
shell: Sh
command: |
bundle check || bundle install
bundle exec rake test --junit
reports:
type: JUnit
spec:
paths:
- "/harness/report.xml"
RSpec
Add the RSpec JUnit formatter to your Gemfile.
- step:
type: Run
name: Run RSpec Tests
identifier: run_rspec_tests
spec:
shell: Sh
command: |
bundle check --path=vendor/bundle || bundle install --path=vendor/bundle --jobs=4 --retry=3
mkdir /harness/rspec
bundle exec rspec --format progress --format RspecJunitFormatter -o /harness/rspec/rspec.xml
reports:
type: JUnit
spec:
paths:
- "/harness/rspec/rspec.xml"