Refresh-refcache PR fix

Follow these steps to resolve non-2XX static/refcache.json entries on the target otelbot PR. This process may involve updating or removing dead links on the site, then refreshing the refcache again until no non-2XX entries remain.

Target PR

Unless instructed otherwise, target the PR for the upstream otelbot/refcache-refresh branch. When asked to fix a spec or semconv integration branch, target the open PR whose head branch matches otelbot/spec-integration-* or otelbot/semconv-integration-*, respectively. If more than one PR matches, ask which one is intended. To list open otelbot PRs:

gh pr list --search head:otelbot/

In the steps below, TARGET_BRANCH is the head branch of the target PR.

Preparation

These steps assume you have a local clone of the repository with the upstream remote configured to point to the main repository. Run these steps locally from the repository root.

1. Determine the PR associated with upstream TARGET_BRANCH.

2. If none exists, stop.

3. If a local TARGET_BRANCH branch already exists and contains commits that are not in the upstream branch, back them up or stop before resetting anything.

4. Check out the PR branch with gh pr checkout <num>. If that fails because the local branch has diverged and you have already backed up any local-only commits, realign it with upstream:

   git fetch upstream
   git checkout TARGET_BRANCH
   git reset --hard upstream/TARGET_BRANCH
   

5. If any content modules are out of date, run npm run get:submodule.

Handling 5XX responses

Status 5XX responses are usually transient. If static/refcache.json or ./scripts/double-check-refcache-4XX.mjs (below) reports status 5XX for a URL, treat it as likely temporary (origin down, gateway errors, overload). Do not change site content or links solely to work around a 5XX; prefer re-running the double-check script (with --retry-404 if useful) or npm run fix:refcache later. Only investigate a 5XX like a real defect if it keeps failing across multiple runs over time and you have confirmed the URL is not otherwise healthy.

Resolve non-2XX entries

1. Run ./scripts/double-check-refcache-4XX.mjs --retry-404 to re-fetch URLs still cached as 4XX and fragment URLs marked INVALID FRAGMENT, then update static/refcache.json. See LinkedIn note below.

2. Scan static/refcache.json for remaining non-2XX statuses.

3. If none remain, that is, the double-check script succeeds:

   • Share the double-check summary: in your reply or PR comment (retried URLs, entries updated, final HTTP status counts, and “Processed N URLs” when shown).
   • If static/refcache.json changed, commit and push to upstream TARGET_BRANCH (as of this step).
   • For otelbot/refcache-refresh only — integration-branch PRs stay draft until their workflow finalizes them at release time:
     • Mark the PR ready for review: gh pr ready <num>.
     • Enable auto-merge, so that the PR is merged once all approvals are in and the checks pass: gh pr merge <num> --auto.
     • Remind a maintainer to approve the PR so auto-merge can complete. Provide a link to the PR: https://github.com/open-telemetry/opentelemetry.io/pull/<num>.

   Then stop unless you are also leaving notes for reviewers.

4. Otherwise (non-2XX still present after step 2), list remaining URLs and their statuses:

   jq -r 'to_entries[] | select(.value.StatusCode < 200 or .value.StatusCode >= 300) | "\(.key) \(.value.StatusCode)"' \
     static/refcache.json
   

   LinkedIn URLs

   Responses from LinkedIn.com are often unreliable (agents and bots may see 403 or 404 even when profiles exist). Do not remove or edit LinkedIn 4XX links, instead let a maintainer manually run ./scripts/double-check-refcache-4XX.mjs --retry-404 locally first.

5. Analyze and recommend. For each URL from the previous step, produce a numbered or bulleted list that includes at least:

   • The URL and HTTP status.
   • Where it originates from: provide links to files or pages.
   • A recommendation. For links into github.com, recommend a replacement link based on the last commit that contains the named resource.
   • For an integration branch: where the fix belongs — in-branch, a separate PR against main, or upstream in the spec repository.

   Pause for feedback from a reviewer.

6. Apply approved fixes. After approval, edit the suggested sources:

   • For a 404, update or remove the referring link where you identified it.
   • For other non-2XX statuses, apply the reviewed recommendation (manual inspection may still be required for ambiguous cases).
   • If any touched page is outside content/en/, follow Localization for that edit (e.g. # patched on default_lang_commit).

7. Run npm run fix:refcache to refresh static/refcache.json after those source-link changes, then repeat the steps in this section (from step 1) until no non-2XX statuses remain.