Skip to main content
{ Reporting and Compliance }

GitHub Repository AI-Readiness Audit

Scores each repository in a GitHub organization on how well a coding agent can work in it, and reports the ones below the bar.

What this agent does

This read-only agent scores the source repositories in a GitHub organization from 0 to 100 on how ready each one is for a coding agent. It checks agent instructions, tests, documentation, and security guardrails. It reports each repository below the bar with the checks it fails and how to fix them.

The challenge

A coding agent works well only in a repository that tells it how to build, test, and change the code. Many repositories have no agent instructions, a thin README, or tests that CI never runs. Some also lack branch review or secret scanning, so nobody reviews an agent's mistakes before they reach the default branch. Teams cannot see which repositories are ready, so agents fail in the ones that are not.

The solution

The agent decides the plain facts from files and settings, and it judges the rest by reading the code. It records what it learns about each repository and uses those notes to keep later scores consistent. It rescans a repository only when the inputs to its score change, and it scores a bounded batch on each run. It uses a margin between the bar and the pass score, so a repository near the bar does not alternate between pass and fail on each run.

Workflow

  1. 01

    Select repositories

    List the non-archived source repositories, and pick a bounded batch that is new, changed, or due for a rescan.

  2. 02

    Check facts

    Check the files and settings that decide a check on their own, such as a README, CODEOWNERS, or branch review.

  3. 03

    Judge the code

    Read each repository to judge its tests, CI, lint setup, README, agent instructions, and documented commands.

  4. 04

    Score and report

    Score each repository, apply the hard gates and the pass margin, and report the ones below the bar.

Agent template 

# GitHub Repository AI-Readiness Audit

## Measurable outcomes

Every source repository in the organization has a current AI-readiness score. Every repository below the bar is in the report, with each failing check and its fix. Track the scanned and below-bar counts on every run.

## Procedure

For a given organization, list the non-archived source repositories, and skip the ones on a blocklist I maintain. Score a bounded batch on each run: new repositories, repositories whose score inputs changed, and repositories last scored more than 14 days ago. Score each repository out of 100 with this rubric:

```text
Agent instructions   30  AGENTS.md or CLAUDE.md present 15, substantive 10, agent tool config 5
Test mechanism       30  CI runs the tests 15, a real test suite 10, lint or format config 5
README and docs      20  README present 7, substantive 8, build and test commands documented 5
Security guardrails  20  default branch requires review 10, secret scanning and push protection 6, CODEOWNERS 4
```

Decide a check from files and settings when they settle it, such as a file that exists or a setting that is on. Judge the other checks by reading the code, never from file names alone. Open a test file to confirm it holds real assertions. Mark a check unknown only when it cannot be settled, and leave unknown checks out of the score, never count them as failures. A missing README or a default branch without required review puts a repository below the bar at any score. A repository falls below the bar under 70 and passes again only at 75 or above, so a repository near the bar does not alternate between pass and fail. Record where each repository keeps its tests, CI, build, and lint, and confirm those notes on the next run.

## Requirements

It needs GitHub API read access to the organization's repositories, their code, branch protection, and security settings, and nothing more. It never changes a repository, its settings, or its code.