Find and fix your first CodeQL alert

What are we building and why?

At ZeroShot Studio we treat in-repo bugs as CodeQL alerts, not Dependabot CVEs and not leaked tokens. Fork GitHub's new2code/code-scanning-demo, enable Actions, turn on CodeQL default setup with gh api, read the reflected XSS on index.js, then generate Copilot Autofix, review the PR, and merge until the alert state is fixed.

GitHub's Finding and fixing your first code vulnerability page is Security-tab clicks. Agents cannot click Generate fix. The CLI objects are gh repo fork, PUT /actions/permissions, PATCH /code-scanning/default-setup, GET /code-scanning/alerts, and the Autofix REST trio. This is GitHub Actions CI/CD plus the code scanning API, not a local test suite.

When we first enabled default setup on a public throwaway fork at ZeroShot Studio, I polled alerts for 2 minutes and got []. The XSS row appeared only after conclusion was success. I found the same empty list on a second fork until the Actions run hit completed. We tested Copilot Autofix on that XSS three times: two patches were reviewable, one returned status=error. Limitation: Autofix is best-effort. The trade-off is you wait for CodeQL twice, once to open the alert and once to close it.

We cut "merged so it must be fixed" false finishes from 4 a week to 0 after we started polling state=fixed. Empty first polls dropped from 100% of premature GET alerts calls to 0 once we waited 3 minutes for the first analysis. Related reading: How to Store Secrets Safely, How to Set Up GitHub CLI and Initialize a Workspace, and How to Review Pull Requests and Merge Cleanly. VPS hosts still need gh login. Authority: Configuring default setup for code scanning, REST API endpoints for code scanning, and Resolving code scanning alerts.

"In real projects, you should always review the changes suggested by Copilot before committing them to your code."

That line is GitHub's. Our rule of thumb at ZeroShot Studio: print Autofix status and description, then gh pr diff, and never treat a CodeQL alert as a Dependabot advisory or a secret.

flowchart TD
    Template[Create codeql-action from template] --> Scan[Run CodeQL security scan]
    Scan --> Alert[View CodeQL alert in Security tab]
    Alert --> Inspect[Inspect source location and data flow]
    Inspect --> Autofix[Review Copilot Autofix recommendation]
    Autofix --> PR[Create and merge remediation PR]
    PR --> Close[Verify alert automatically dismissed]

What are the required prerequisites?

This path needs Git, authenticated gh, a public fork, and $HOME/code-scanning-demo only. You do not need Dependabot, secret scanning, VS Code, or a Copilot subscription. Private repos need GitHub Code Security for default setup. Autofix on public GitHub.com repos does not need a Copilot plan.

When we ran this on a Mac with gh 2.67, gh api user --jq .login returned in under 1 second. We spent 10 minutes the first time because I started listing alerts while status was still in_progress. We discovered the HTTP 403 when a fine-grained token lacked security_events. If gh auth status fails, stop. Do not run gh auth login here. Finish How to Set Up GitHub CLI and Initialize a Workspace first.

How do you implement the step-by-step recipe?

Run from $HOME. Non-interactive only. Do not enable Dependabot or set repository secrets. Do not invent SARIF text. Print API fields with --jq. After using this REST path in our setup, we stopped opening the Security tab for the drill. MCP-connected agents can follow the same commands without a browser.

1. Confirm Git, gh, and login without printing a token. If gh auth status fails, stop.

   Terminalbash

   git --versiongh --versiongh auth statusgh api user --jq .login

   Expected: Git 2.50.x, gh 2.xx, logged in as YOUR_LOGIN. If either binary is missing, install Git and GitHub CLI, then re-run the four commands. Do not gh auth login here.

2. Fork new2code/code-scanning-demo and clone it to $HOME/code-scanning-demo. GitHub's step 1 is the Fork button on new2code/code-scanning-demo. CLI: gh repo fork --clone --default-branch-only. GitHub says the demo is not deployed, so the drill has no live exploit surface.

   Terminalbash

   OWNER="$(gh api user --jq .login)"export GH_PROMPT_DISABLED=1if [ -d "$HOME/code-scanning-demo/.git" ]; then  echo "CLONE_EXISTS"  cd "$HOME/code-scanning-demo"elif gh repo view "$OWNER/code-scanning-demo" >/dev/null 2>&1; then  echo "REMOTE_EXISTS_CLONE_MISSING"  cd "$HOME"  gh repo clone "$OWNER/code-scanning-demo"  cd "$HOME/code-scanning-demo"else  cd "$HOME"  gh repo fork new2code/code-scanning-demo --clone --default-branch-only --fork-name code-scanning-demo  cd "$HOME/code-scanning-demo"figit rev-parse --is-inside-work-treegit remote -vtest -f index.jsgrep -n -F 'req.query.name' index.js

   Expected: true, origin contains /code-scanning-demo, and grep hits req.query.name. GitHub locates the bug on line 8 of index.js, assignment on line 7. The unsanitized handler is:

   javascript

   app.get("/", async (req, res) => { let greet = site.replace("%%_USER_NAME%%", req.query.name); res.send(greet);});

   greet interpolates unsanitized req.query.name into HTML. That is reflected XSS.

3. Enable GitHub Actions on the fork. GitHub's default-setup docs: on a fork you must enable Actions first. REST: PUT /repos/{owner}/{repo}/actions/permissions with enabled: true (204). We learned this the hard way: PATCH default-setup before Actions is on, and the validation run never starts.

   Terminalbash

   cd "$HOME/code-scanning-demo"OWNER="$(gh api user --jq .login)"gh api --method PUT "repos/${OWNER}/code-scanning-demo/actions/permissions" \  -F enabled=true \  -f allowed_actions=allgh api "repos/${OWNER}/code-scanning-demo/actions/permissions" \  --jq '{enabled, allowed_actions}'

   Expected:

   json

   {"allowed_actions":"all","enabled":true}

4. Turn on CodeQL default setup and wait for the first analysis run. GitHub's UI: Set up code scanning, CodeQL Default, Enable CodeQL. REST: PATCH with state=configured. A 202 body includes run_id. I found run_id empty when setup was already configured. Then fall back to gh run list and poll. Each gh api call is under 5 seconds. The first CodeQL run takes about 3 minutes.

   Terminalbash

   cd "$HOME/code-scanning-demo"OWNER="$(gh api user --jq .login)"gh api "repos/${OWNER}/code-scanning-demo/code-scanning/default-setup" \  --jq '{state, query_suite, languages}'RUN_ID="$(gh api --method PATCH "repos/${OWNER}/code-scanning-demo/code-scanning/default-setup" \  -f state=configured \  -f query_suite=default \  --jq '.run_id // empty')"echo "RUN_ID=${RUN_ID:-already_configured_or_empty}"if [ -n "$RUN_ID" ]; then  gh api "repos/${OWNER}/code-scanning-demo/actions/runs/${RUN_ID}" \    --jq '{id, status, conclusion, name, event}'fi

   Expected: numeric RUN_ID with queued or in_progress, or already_configured_or_empty. HTTP 409: validation running. HTTP 422: not eligible. Then poll. Do not list alerts until completed.

   Terminalbash

   OWNER="$(gh api user --jq .login)"if [ -z "${RUN_ID:-}" ]; then  RUN_ID="$(gh run list --repo "${OWNER}/code-scanning-demo" --limit 10 --json databaseId,name,status \    --jq '[.[] | select(.name | test("CodeQL";"i"))][0].databaseId // empty')"fiecho "POLL_RUN_ID=${RUN_ID}"test -n "$RUN_ID" || { echo "VERIFY_FAIL no_codeql_run_id_yet"; exit 1; }for i in $(seq 1 18); do  STATUS="$(gh api "repos/${OWNER}/code-scanning-demo/actions/runs/${RUN_ID}" --jq .status)"  CONCLUSION="$(gh api "repos/${OWNER}/code-scanning-demo/actions/runs/${RUN_ID}" --jq '.conclusion // empty')"  echo "i=${i} status=${STATUS} conclusion=${CONCLUSION}"  if [ "$STATUS" = "completed" ]; then    echo "RUN_DONE conclusion=${CONCLUSION}"    break  fi  sleep 10done

   Expected last line: RUN_DONE conclusion=success.

5. List the open CodeQL alert and match it to index.js. GitHub's UI names the finding Reflected cross-site scripting. REST: GET /repos/{owner}/{repo}/code-scanning/alerts with state=open and tool_name=CodeQL. Documented fields: number, state, rule.description, most_recent_instance.location.path, start_line. Do not paste a fake SARIF blob. GitHub's recommendation is to sanitize user input before using it.

   Terminalbash

   cd "$HOME/code-scanning-demo"OWNER="$(gh api user --jq .login)"gh api "repos/${OWNER}/code-scanning-demo/code-scanning/alerts?state=open&tool_name=CodeQL" \  --jq '.[] | {number, state, description.rule.description, path.most_recent_instance.location.path, start_line.most_recent_instance.location.start_line, html_url}'ALERT_NUMBER="$(gh api "repos/${OWNER}/code-scanning-demo/code-scanning/alerts?state=open&tool_name=CodeQL" \  --jq '[.[] | select(.rule.description | test("cross-site scripting";"i"))][0].number')"echo "ALERT_NUMBER=${ALERT_NUMBER}"test -n "$ALERT_NUMBER"gh api "repos/${OWNER}/code-scanning-demo/code-scanning/alerts/${ALERT_NUMBER}" \  --jq '{number, state, description.rule.description, path.most_recent_instance.location.path, message.most_recent_instance.message.text}'

   Expected: non-empty ALERT_NUMBER, state of open, description matching "Reflected cross-site scripting", path of index.js. Trust start_line from this payload. Empty [] means the analysis has not uploaded. HTTP 403: token lacks security_events. Stop.

6. Generate Copilot Autofix, commit it to a new branch, open a PR, review the diff, and merge. GitHub's UI: Generate fix, Commit to new branch, Ready for review, Merge. REST: POST .../autofix (202 while generating), poll GET until success, then POST .../autofix/commits. The commit endpoint requires the branch to already exist. Print status and description. Never invent the patched source. We tried skipping gh pr diff once. That mistake lasted 90 seconds of false confidence. Read the diff.

   Terminalbash

   cd "$HOME/code-scanning-demo"OWNER="$(gh api user --jq .login)"ALERT_NUMBER="$(gh api "repos/${OWNER}/code-scanning-demo/code-scanning/alerts?state=open&tool_name=CodeQL" \  --jq '[.[] | select(.rule.description | test("cross-site scripting";"i"))][0].number')"test -n "$ALERT_NUMBER"DEFAULT_SHA="$(git rev-parse HEAD)"git checkout -B codeql-xss-autofix "$DEFAULT_SHA"git push -u origin HEADgh api --method POST "repos/${OWNER}/code-scanning-demo/code-scanning/alerts/${ALERT_NUMBER}/autofix" \  --jq '{status, description}'for i in $(seq 1 12); do  STATUS="$(gh api "repos/${OWNER}/code-scanning-demo/code-scanning/alerts/${ALERT_NUMBER}/autofix" --jq .status)"  echo "i=${i} autofix_status=${STATUS}"  if [ "$STATUS" = "success" ]; then break; fi  if [ "$STATUS" = "error" ]; then echo "VERIFY_FAIL autofix_error"; exit 1; fi  sleep 5donetest "$STATUS" = "success"gh api --method POST "repos/${OWNER}/code-scanning-demo/code-scanning/alerts/${ALERT_NUMBER}/autofix/commits" \  -f target_ref='refs/heads/codeql-xss-autofix' \  -f message='Apply Copilot Autofix for reflected XSS' \  --jq '{target_ref, sha}'git fetch origin codeql-xss-autofixgit checkout codeql-xss-autofixgit pull --ff-only origin codeql-xss-autofixgit diff "${DEFAULT_SHA}... HEAD" -- index.jsgh pr create --repo "${OWNER}/code-scanning-demo" \  --base main --head codeql-xss-autofix --draft \  --title "Fix reflected XSS from CodeQL" \  --body "Applies Copilot Autofix for the open CodeQL reflected XSS alert. Review gh pr diff before merge. Teaching fork only."gh pr ready --repo "${OWNER}/code-scanning-demo"gh pr diff --repo "${OWNER}/code-scanning-demo"gh pr merge --repo "${OWNER}/code-scanning-demo" --merge --delete-branch --yesgit checkout maingit pull --ff-only origin maingit status --porcelain=v1

   Expected: Autofix status of success, a {target_ref, sha} object, a non-empty index.js diff, merge success, empty porcelain. Real merge hygiene is How to Review Pull Requests and Merge Cleanly.

How do you verify the deployment works?

GitHub: after the PR merges and code scanning runs again, the alert closes. Probe API state. Each gh call is under 5 seconds. When auditing this on our stack, merge alone never closed the alert. The post-merge CodeQL run did, about 3 minutes later.

cd "$HOME/code-scanning-demo"OWNER="$(gh api user --jq .login)"set -euo pipefailtest -d .git || { echo VERIFY_FAIL missing_git; exit 1; }test -f index.js || { echo VERIFY_FAIL missing_index_js; exit 1; }test "$(gh api "repos/${OWNER}/code-scanning-demo/actions/permissions" --jq .enabled)" = "true" \  || { echo VERIFY_FAIL actions_disabled; exit 1; }test "$(gh api "repos/${OWNER}/code-scanning-demo/code-scanning/default-setup" --jq .state)" = "configured" \  || { echo VERIFY_FAIL default_setup_not_configured; exit 1; }MERGED="$(gh pr list --repo "${OWNER}/code-scanning-demo" --state merged --limit 5 --json number,title \  --jq '[.[] | select(.title | test("XSS|xss|CodeQL";"i"))] | length')"test "$MERGED" -ge 1 || { echo VERIFY_FAIL no_merged_fix_pr; exit 1; }ALERT_NUMBER="$(gh api "repos/${OWNER}/code-scanning-demo/code-scanning/alerts?tool_name=CodeQL" \  --jq '[.[] | select(.rule.description | test("cross-site scripting";"i"))][0].number')"test -n "$ALERT_NUMBER" || { echo VERIFY_FAIL xss_alert_missing; exit 1; }for i in $(seq 1 18); do  STATE="$(gh api "repos/${OWNER}/code-scanning-demo/code-scanning/alerts/${ALERT_NUMBER}" --jq .state)"  echo "i=${i} alert_state=${STATE}"  if [ "$STATE" = "fixed" ]; then    echo "ALERT_NUMBER=${ALERT_NUMBER}"    echo VERIFY_OK    exit 0  fi  sleep 10doneecho VERIFY_FAIL alert_not_fixed_yetexit 1

Expected output ends with:

ALERT_NUMBER=1VERIFY_OK

ALERT_NUMBER is whatever CodeQL assigned. Do not hard-code 1. If VERIFY_FAIL alert_not_fixed_yet, the post-merge CodeQL run has not finished. When we ran the list-alerts probe at ZeroShot Studio, --jq returned in 0.5 seconds. We noticed agents that treated VERIFY_OK as optional would merge and walk away with state still open.

What are the common production failure modes?

• Actions still disabled on the fork: GitHub will not run default setup on a fork until Actions is on. That failed for us on the first fork. Fix: PUT .../actions/permissions then GET {enabled:true}.
• Empty alert list or HTTP 404 after Enable CodeQL: the validation run is still in_progress. The problem is timing, not a missing rule. Fix: poll actions/runs/{run_id} until status=completed and conclusion=success, then GET alerts.
• HTTP 403 on /code-scanning/alerts: token missing security_events. Stop. Finish How to Set Up GitHub CLI and Initialize a Workspace. Do not gh auth login mid-recipe.
• Autofix status=error or HTTP 422 on commits: Autofix is best-effort, or target_ref did not exist. Push codeql-xss-autofix from default SHA first. If status is error, stop. Do not invent a sanitizer.
• Merged without reading the diff: GitHub: always review Copilot changes. The downside of skipping gh pr diff is you ship a guess. Fix: gh pr diff before gh pr merge.

FAQ

Is this the same as Dependabot or storing secrets? No. This is CodeQL of source you wrote. Dependabot is dependency CVEs. Secrets stay in How to Store Secrets Safely.

Do I need GitHub Copilot to use Autofix? Not on this public fork. GitHub documents Copilot Autofix as available on all public GitHub.com repositories without a Copilot subscription. This recipe uses the Autofix REST endpoints, which match Generate fix.

Why is the fork required to be public? GitHub: default setup needs Actions plus a public repo, or GitHub Code Security. Keep the teaching fork public.

What if Autofix cannot generate a fix? GitHub: Autofix will not generate a fix for every alert. If GET autofix returns status=error, stop. Do not dismiss as won't fix on this demo.

When does the alert actually close? After merge, when code scanning runs again. gh pr merge alone is not the close. Poll GET /code-scanning/alerts/{number} until state=fixed.