Repository navigation
ci: capture pytest exit codes and build test-status.json per label - #913
CagriYonca wants to merge 1 commit into
Conversation
68fc532 to
0a145f1
Compare
0a145f1 to
4390282
Compare
fbf25af to
1c31418
Compare
649d208 to
ad98e78
Compare
| mkdir -p coverage_results | ||
| cp -R .coverage coverage_results/.coverage.${CIRCLE_BUILD_NUM} | ||
| cd coverage_results | ||
| exit 0 |
There was a problem hiding this comment.
Could you please explain why is an exit 0 needed here? I can see that you're capturing the $TEST_EXIT_CODE safely.
There was a problem hiding this comment.
In CircleCI, if a step exits with a non-zero code, subsequent steps like saving coverage files and pytest results are skipped.
Ending this step with exit 0 ensures that all coverage reports and artifact files (pip-freeze and test-status.json) are safely generated and saved to the workspace even when tests fail.
The job is still properly failed at the very end by the fail-if-tests-failed step using the captured $TEST_EXIT_CODE. This allows final_job (SonarQube) and update-currency-versions to run and report the failing packages to the currency reporter pipeline.
Signed-off-by: Cagri Yonca <cagri@ibm.com>
ad98e78 to
37ccf96
Compare
Why
When pytest failed in CI, the test job stopped right away, before coverage,
pip freezeoutput and test metadata were saved to the workspace. Downstream jobs then had no artifacts on failed runs, sorupy-currency-reporternever got updated dependency or test-status data.How
run-tests-with-coverage-reportnow saves the pytest exit code and lets the remaining steps run instead of stopping the job.fail-if-tests-failedcommand runs last in every test job (when: always). It fails the job only after all artifacts and workspace files are saved, and it prints the pytest short summary.tests/frameworks/test_fastapi.py→fastapi) and saved per job aspytest_failed_<label>.txt.capture-installed-versionsnow runs withwhen: always, and theautowraptjob also saves itspip freezeoutput.update-currency-versionscombines the failed modules from all jobs into a package-leveltest-status.json({"<package>": "fail"}; packages that aren't listed passed). The file is published next toinstalled_*.txt.final_jobandupdate-currency-versionsnow userequires: <job>: terminal, so they run whether the test jobs pass or fail.Verification
pip freezefiles are saved on both passing and failing runs.test-status.jsonworks withrupy-currency-reporter(instana/rupy-currency-reporter#9).