Skip to content

test: sweep test calendars off external servers - #871

Open
tobixen wants to merge 5 commits into
masterfrom
test-calendar-sweep
Open

tobixen wants to merge 5 commits into
masterfrom
test-calendar-sweep

Conversation

@tobixen

@tobixen tobixen commented Nov 16, 2026 •

Copy link
Copy Markdown
Member

Code and text generated by Claude Opus 5.5 (Claude Code); prompts are in the commit message below.

Commit message:

The test runs kept accumulating new calendars on the server. That can be an annoyance and can also cause tests to fail; some providers allow only four or five on a free account. The async fixtures now share the sync suite's cal_ids and display names. Where calendars can be recreated, every pythoncaldav-* calendar on an external server is deleted for good when a session first uses the server and again when it ends. Where a plain DELETE leaves the id taken (Nextcloud's trashbin), the tests attempt to skip the trashbin too (that's a header-field that may be Nextcloud-specific).

Prompts:

for ecloud there are now 11 calendars coming from the caldav tests. That's too many. Every test should clean up after itself, and it's needed with some cleanup-magic also at the start of a test to remove old obsoleted test calendars. Some calendar providers only allow for 4-5 calendars on the free account.

The tests should not create calendars with UUIDs anymore (if calendaring-client is doing it, it needs to be fixed there). The ecloud server is only used for testing, so it's safe to delete calendars there.

I thought "sweep" meant "delete content of calendar, but not the calendar itself"?

I think it's needed to do some testing here. We have the empty-thrashbin-script (sic), if running that one for each test (but I don't want it into the public repo - this is a hack for testing of ecloud only), would it suffice to avoid the rate limits? If not, we need to reuse calendars. If the test run needs to create 6-8 calendars but still needs every test to be run serially, then something is wrong with the tests.

(on bare-UUID leftovers not matching the prefix) The sweep in the test suite should not touch those. That said, it's OK to delete such calendars for my ecloud account. If such calendars reappear after deleting them, it's needed to do more research.

(same finding, after the research found the server-tester creating them) [add a calendar task for the fix in caldav-server-tester]

(on deleting past the trashbin wherever the tests delete a calendar) ,,, this is NextCloud-specific, but harmless for other servers, and only in tests ... so let's do it

Take a step back. The [edited] commit message below reads better. It's not an ecloud problem, I was clearing away A LOT of junk calendars from my zimbra the other day.

Assisted-By: Claude Opus 5.5 noreply@anthropic.com

Also: test: let sync ecloud runs send a request a second - two ecloud runs at one request per two seconds got no 429 in 22 calendar creations over three hours, so the proactive pace in the ecloud hint is doubled.

Tested: unit tests (919 passed) and the Radicale integration subset on the branch rebased onto master; TestForServerRadicale::testCheckCompatibility fails on master too. Before the rebase, Radicale/Xandikos (491 passed) and the ecloud tests whose delete paths changed, which left nothing in the trashbin.

🤖 Generated with Claude Code

Also piggy-backing: feat: create_calendar_per_hour rate-limit key - an extra key on the rate-limit hint saying how many calendars an account may make per hour (10 on ecloud), so a client like calendaring's conformance suite can reuse its calendars there.

The test runs kept accumulating new calendars on the server.  That
can be an annoyance and can also cause tests to fail; some providers
allow only four or five on a free account.  The async fixtures now
share the sync suite's cal_ids and display names.  Where calendars can
be recreated, every pythoncaldav-* calendar on an external server is
deleted for good when a session first uses the server and again when
it ends.  Where a plain DELETE leaves the id taken (Nextcloud's
trashbin), the tests attempt to skip the trashbin too (that's a
header-field that may be Nextcloud-specific).

Prompts: 
> for ecloud there are now 11 calendars coming from the caldav tests.  That's
> too many.  Every test should clean up after itself, and it's needed with some 
> cleanup-magic also at the start of a test to remove old obsoleted test 
> calendars.  Some calendar providers only allow for 4-5 calendars on the 
> free account.

> The tests should not create calendars with UUIDs anymore (if 
> calendaring-client is doing it, it needs to be fixed there).  The ecloud 
> server is only used for testing, so it's safe to delete calendars there.

> I thought "sweep" meant "delete content of calendar, but not the calendar itself"?

> I think it's needed to do some testing here.  We have the 
> empty-thrashbin-script (sic), if running that one for each test (but I don't 
> want it into the public repo - this is a hack for testing of ecloud only), 
> would it suffice to avoid the rate limits?  If not, we need to reuse 
> calendars.  If the test run needs to create 6-8 calendars but still needs 
> every test to be run serially, then something is wrong with the tests.

> (on bare-UUID leftovers not matching the prefix) The sweep in the test suite 
> should not touch those.  That said, it's OK to delete such calendars for my 
> ecloud account.  If such calendars reappear after deleting them, it's needed 
> to do more research.

> (same finding, after the research found the server-tester creating them) [add 
a calendar task for the fix in caldav-server-tester]

> (on deleting past the trashbin wherever the tests delete a calendar) ,,, this 
> is NextCloud-specific, but harmless for other servers, and only in tests ... 
> so let's do it

> Take a step back.  The [edited] commit message below reads better.  It's not an 
> ecloud problem, I was clearing away A LOT of junk calendars from my zimbra 
> the other day.

Assisted-By: Claude Opus 5.5 <noreply@anthropic.com>
Reviewed-by: Tobias Brox <tobias@redpill-linpro.com>
Two ecloud test runs at one request per two seconds created 22
calendars in three hours and got no 429, while a full run took over
two hours.  The proactive pace in the ecloud hint, which only the sync
test suite applies, is doubled; the 429 handling stays as it was.

Prompts (comments made inline in the document while reviewing it):
> (on the proposal to look at the ecloud rate-limit hint, interval 2, count 1, since no 429 was observed) agreed, lower it a bit, with a comment, and we can revisit it later

Assisted-By: Claude Opus 5.5 <noreply@anthropic.com>
Reviewed-by: Tobias Brox <tobias@redpill-linpro.com>
Comment thread tests/test_fixture_helpers.py Fixed
Nextcloud lets an account make only so many calendars in an hour
(rateLimitCalendarCreation, 10 by default) and answers MKCALENDAR
beyond that with 429.  A suite that makes a calendar per test, like
calendaring's conformance suite, fails most of its tests on ecloud.
The rate-limit client-feature gets an extra key for it,
create_calendar_per_hour, set to 10 for ecloud, so such a client can
reuse its calendars instead.  It is not a server-feature: it is not
probed, and what it asks of a client depends on the client.

Prompts:
> [pick from the task list: "calendaring: conformance follow-up -
> decisions and push after 2026-11-11 session"]

> (on ecloud's calendar-creation rate limit making 174 tests fail) I
> thought we already had a fix here, reusing calendars?

> (on the options: a caldav hint, keying on the server name, or skipping
> ecloud) We go for [a caldav calendar-creation rate-limiting hint],
> probably.  I don't think we should have a probe for this.  This means
> it's not a "server-feature", but either a "client-feature" or
> "server-observation" or something like that.

> please fix it now

> (on the new key being spelled with hyphens while its neighbours use
> underscores) Consistency matters - it seems like I've (relatively?)
> consistently been choosing dash in the feature names and underscore in
> the extra keys.

Assisted-By: Claude Opus 5.5 <noreply@anthropic.com>
Reviewed-by: Tobias Brox <tobias@redpill-linpro.com>
The synology hint said DSM manages calendars only through its own UI
and API, leaving a reader with no way to clear leftover test calendars.
DSM's web API deletes them with the account's own credentials, also
those created over CalDAV, which its list method does not show.

Prompt: (on whether to mention the DSM delete in the synology hint)
  please do

Assisted-By: Claude Opus 5.5
Reviewed-by: Tobias Brox <tobias@redpill-linpro.com>
Code scanning flagged the fake ExternalTestServer in the sweep tests
for not calling the parent __init__.  The parent needs no config, so
the fake now passes its name through it instead of setting it alone.

Prompts:
> rebase, write a comment in the code why the parent `__init__` is
> skipped, and dismiss the QA-issue. [the code-scanning finding
> "Missing call to superclass __init__" on
> #871; review then showed
> the skip was not needed, so the code calls it rather than explaining]

Assisted-By: Claude Opus 5.5
Reviewed-by: Tobias Brox <tobias@redpill-linpro.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant