Skip to content

feat: add blobstore performance benchmarking - #216

Merged
serdarozerr merged 34 commits into
cloudfoundry:mainfrom
sap-contributions:feature/benchmark
Nov 16, 2026
Merged

serdarozerr merged 34 commits into
cloudfoundry:mainfrom
sap-contributions:feature/benchmark

Conversation

@serdarozerr

@serdarozerr serdarozerr commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

Adding benchmark harness under benchmark/ that measures wall-clock put/get performance of the storage-cli binary.
Each operation is timed independently including process startup and SDK behaviour. SHA-256 integrity is verified outside the timer after each get. Two profiles are supported: quick (4 sizes, 3 samples) and full (5 sizes,
5 samples). Results are written as results.json and summary.md.

Workflow for benchmarking is created. There are 2 ways run; Manual run(benchmark-manual) and the other way is in release-cron workflow as last step. Each run creates 2 artifacts: summary.md which reports median values of runs as result and results.json which reports the raw result of runs.

Add a subprocess-based benchmark harness under benchmark/ that measures
wall-clock put/get performance of the storage-cli binary without requiring
a CF deployment, BOSH director, or CC database.

Each operation is timed independently including process startup and SDK
behaviour. SHA-256 integrity is verified outside the timer after each get.
Two profiles are supported: quick (4 sizes, 3 samples) and full (5 sizes,
5 samples). Results are reported.

The package is excluded from the main build via //go:build benchmark and
tested with Ginkgo using injectable mock runners, no credentials required.
@serdarozerr
serdarozerr had a problem deploying to fork-pr-integration-test November 6, 2026 13:31 — with GitHub Actions Error
@serdarozerr
serdarozerr had a problem deploying to fork-pr-integration-test November 6, 2026 13:31 — with GitHub Actions Error
@serdarozerr
serdarozerr had a problem deploying to fork-pr-integration-test November 6, 2026 13:31 — with GitHub Actions Error
@serdarozerr
serdarozerr had a problem deploying to fork-pr-integration-test November 6, 2026 13:31 — with GitHub Actions Error
@serdarozerr
serdarozerr had a problem deploying to fork-pr-integration-test November 6, 2026 13:31 — with GitHub Actions Error
@serdarozerr
serdarozerr had a problem deploying to fork-pr-integration-test November 6, 2026 13:37 — with GitHub Actions Error
@serdarozerr
serdarozerr had a problem deploying to fork-pr-integration-test November 6, 2026 13:37 — with GitHub Actions Error
@serdarozerr
serdarozerr had a problem deploying to fork-pr-integration-test November 6, 2026 13:37 — with GitHub Actions Error
@serdarozerr
serdarozerr had a problem deploying to fork-pr-integration-test November 6, 2026 13:37 — with GitHub Actions Error
@serdarozerr
serdarozerr had a problem deploying to fork-pr-integration-test November 6, 2026 13:37 — with GitHub Actions Error
@serdarozerr
serdarozerr had a problem deploying to fork-pr-integration-test November 6, 2026 13:42 — with GitHub Actions Error
@serdarozerr
serdarozerr had a problem deploying to fork-pr-integration-test November 6, 2026 16:14 — with GitHub Actions Error
@serdarozerr
serdarozerr had a problem deploying to fork-pr-integration-test November 6, 2026 16:29 — with GitHub Actions Error
@serdarozerr
serdarozerr had a problem deploying to fork-pr-integration-test November 6, 2026 16:29 — with GitHub Actions Error
Comment thread benchmark/report.go Resolved
Comment thread benchmark/runner_test.go Resolved
Comment thread benchmark/README.md Resolved
Comment thread .github/workflows/release-cron.yml Resolved
- Failure calculatio for both get and put refactored
- aliyun tool instalation guarded
- benchmark tests are protected with build tag
Comment thread .github/workflows/release-cron.yml Resolved
Comment thread benchmark/runner.go Resolved
Comment thread benchmark/runner.go Resolved
Comment thread benchmark/README.md Resolved
Comment thread benchmark/runner.go Resolved
Comment thread benchmark/runner.go Resolved
Comment thread .github/workflows/release-cron.yml Resolved
Comment thread benchmark/runner.go Resolved
Comment thread .github/scripts/benchmark/teardown.sh Resolved
teardowd-infrastrcuture.sh tries to delete lambda function that might create probles during parallel runs. Deletion logic moved into run-integration-aws-iam.sh script because it manages the lifecyle(create and deletes afterwards) of lamda function. Also resiliency is improved
 for deleting.
…This time won't be included to statistics calculation

gcs bucket delete warning added incase deletion fails.

This branch was successfully deployed

1 active deployment
fork-pr-integration-test — cc09f896 Deployed Nov 16, 2026 by stephanme via Approval gate (fork PRs only) #637
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Development

Successfully merging this pull request may close these issues.

2 participants