ADR-0002: Harvest pagination evidence

Status: Proposed

Date: 2026-09-21

Context

A listing can repeat its navigation controls above and below the vacancies. A unique element is therefore not required to identify a unique destination. Selectors can also change between pages or browser sessions.

Harvest previously treated a missing control or failed advance as the end of the listing. The harvest worker could then mark partial results as completed. A successful advance proves progress, but does not prove full coverage.

Decision

Keep model-based control identification separate from browser execution. Build selectors from observed controls. For link pagination, allow multiple matches only when all matched anchors have the same normalized, usable URL. Reject conflicting destinations. Do not select the first conflicting link.

Attempt two advances per candidate in each of two fresh browser sessions. Require new listing items and matching effects. Both sessions must produce the same number of advances. Save the advance count as verifiedAdvances with the action and effect. Existing configurations can omit this field; discovery evals require it. Keep the stop reason in the verification result. One verified advance is limited evidence and can support a two-page listing; it is not proof that the selector works on more pages.

Return explicit stopped results from the paginator. A disabled saved next control confirms completion. Missing controls, no new items, and listings without a saved advance remain unconfirmed. Navigation errors, conflicting links, and changed effects are failures. Preserve collected items and let the harvest worker store uncertain or failed harvests as incomplete.

The existing site registry is a regression set. Coverage measurements need a separate sample from the site inventory, with independently reviewed expected results. Passing the regression set does not establish coverage of that inventory.

Consequences

Verification uses six page loads when both advances succeed in both sessions. Listing-load retries can add requests. Candidates that fail early use fewer requests. Normal harvesting reuses the saved configuration and does not make new model calls.

More listing harvests can be marked incomplete, including real terminal pages that remove their controls instead of disabling them. This preserves the difference between observed completion and uncertain completion. Sitemap completion and configured page and item limits keep their existing meanings.

This decision does not add automatic configuration repair or a claim of universal website support. Those require separate evidence and operating limits.