The problem showing up in every maintainer’s queue

AI-assisted pull requests are arriving at open source projects at an increasing rate, and the trend is only going one direction. Some of these contributions are excellent. Others compile, pass a quick glance, and still miss the project’s design patterns entirely: the wrong base class, mutable fields where the project requires immutable ones, tests organized in a style the repo abandoned years ago, no changelog entry. The code works, but it does not belong.

Reviewing these PRs is painful in a specific way. The contributor is often acting in good faith, the diff is often large, and the problems are rarely bugs. They are convention violations, and explaining conventions one review comment at a time does not scale. I maintain hier_config, a Python library for network configuration remediation. A pattern that I have been seeing is: more AI-assisted PRs, and in some of them, code that did not adhere to the project’s design patterns and standards.

Some projects have responded by banning AI-generated contributions outright. I think that is the wrong move. Gatekeeping does not stop the flood, it just makes contributors hide how the code was written, and it turns away people who would have contributed well with a little guidance. The better response is to accept that AI-assisted code is here to stay and to build guardrails so it arrives already conforming to the project’s standards, and so reviewers can verify conformance quickly when it does not.

I recently landed on an approach in hier_config that I will walk through below and why, as a playbook other projects can adapt.

The key realization: AI agents read your repo

A human contributor might skim CONTRIBUTING.md once and never look at it again. An AI coding agent reads whatever guidance the repository puts in front of it, on every task, and follows it with much more consistency than humans do. That changes the economics of documentation. Every convention you write down in a place an agent will find becomes a convention you no longer have to enforce in review.

The inverse is also true. If your standards live in maintainers’ heads, in old PR review threads, or in documentation that has drifted into three contradictory copies, an agent cannot follow them. It will fill the gap with generic best practices, and generic best practices are exactly what a “close but not quite our style” PR looks like.

So the work splits into three parts: make the standards true and findable, state them in a form agents consume, and give contributors tooling to check their own work before a maintainer ever sees it.

Step 1: Fix the documentation before pointing agents at it

Maintaining documentation is hard and it matters most. Guidance files are only as good as the documentation they point to.

For hier_config, that meant reorganizing the docs into three audiences: a User Guide for people consuming the library, a Developer Guide for people changing it, and a Maintainer Guide for releases and CI. The Developer Guide got pages that had never existed because the knowledge lived in reviewers’ heads: testing conventions, code style, and how to extend the library with new platform drivers.

It also meant hunting down drift. The MatchRule reference existed in three copies. The supported platform table existed in three copies, and two of them were missing a platform. A unified diff walkthrough was duplicated, and one copy was orphaned from the navigation entirely. Every duplicate is a chance for an agent to read the stale copy and confidently produce outdated code. Deduplication is not cosmetic when machines are your readers.

Finally, a mkdocs build --strict job was added to CI so broken links and navigation fail the PR instead of shipping silently. Once documentation becomes the substrate your review process runs on, docs rot becomes a correctness bug, and it should be caught like one.

Step 2: One canonical guidance file, thin overlays for each tool

The AI tooling ecosystem has produced a pile of per-tool convention files: CLAUDE.md for Claude Code, .github/copilot-instructions.md for Copilot, and various others. Maintaining independent copies guarantees they drift apart, which is the same disease the docs had.

The structure I settled on:

• AGENTS.md is the canonical statement of standards for all AI tools (and a useful quick reference for humans). It is deliberately an index, not an encyclopedia: a project overview, the exact build and test commands, a brief architecture summary, the hard rules, and a task-to-documentation map.
• CLAUDE.md slims down to a three-line overlay that imports AGENTS.md. Its previous 81 lines included a platform list that had already gone stale, which proved the point about duplication.
• .github/copilot-instructions.md carries a condensed self-contained copy, because Copilot code review does not follow imports. It is short enough that keeping it in sync is tractable.

Two sections of AGENTS.md do the heaviest lifting. The first is a numbered list of hard rules, framed exactly as what they are: things CI and reviewers will block a merge over. For hier_config that includes rules like these:

1. Models: always subclass the project-local BaseModel in hier_config/models.py (it sets frozen=True, extra="forbid") — never pydantic.BaseModel directly. Model fields use immutable collections only (tuple, frozenset).
2. Typing: mypy strict + pyright strict. Full annotations everywhere, including tests. No Any, no unjustified # type: ignore or # noqa.
3. Lint: ruff select = ["ALL"] with preview, line length 88. Never loosen lint or coverage configuration to make a change pass.

Notice rule 3’s second sentence. Agents under pressure to make a build pass will reach for the config file, so the standards need to say, explicitly, that loosening the gates is itself a violation. Write your rules with an eye toward the shortcuts an agent might take.

The second load-bearing section is a task-to-documentation map: a small table pairing “Add a platform driver” with docs/dev/extending.md, “Write or fix tests” with docs/dev/testing.md, and so on. This is what lets AGENTS.md stay short. The agent reads the index, identifies its task, and pulls the deep documentation for exactly that task into context.

Step 3: Give contributors a self-review tool

Documentation makes conformance possible. Tooling makes it cheap. The centerpiece of the PR is a Claude Code skill called hier-config-review that any contributor can run before opening a PR. It performs a diff-aware review of their working branch:

1. Establish the diff and classify every changed file (library code, tests, docs, CI, changelog).
2. Run the actual gates: lint, tests with the coverage floor, and a strict docs build if docs changed.
3. Review each category against the documentation, with a pointed instruction: read the referenced doc before judging that category, because the docs are the standard, not the model’s intuition.
4. Report findings grouped by severity (blockers, should-fix, nits), each with a file and line, what is wrong, and which documented standard it violates.

The severity levels are defined in terms a contributor can act on: a blocker means CI would reject this, a should-fix means a reviewer would push back. The skill ends with a pass or fail verdict against the same checklist that appears in AGENTS.md and in the PR template.

That last part is deliberate. The PR template’s self-review checklist, the “Before Opening a PR” list in AGENTS.md, and the review skill’s final verdict are all the same list. Contributors see the standard, agents apply the standard, and reviewers enforce the standard, and it is one standard.

I validated the skill by pointing a fresh agent at a deliberately non-compliant change and confirming it caught the violations. If you build a review tool, test it against bad code, not good code.

Step 4: Encode common workflows, not just rules

Rules tell an agent what not to do. Workflow skills tell it how to do the most common tasks correctly the first time. The PR added two:

• hier-config-new-driver scaffolds support for a new network platform, walking through the enum member, the driver class, the test file, and the docs table row that a complete driver PR requires. Adding platform support is the most common category of drive-by contribution to hier_config, so it is the workflow most worth paving.
• hier-config-troubleshoot maps observable symptoms (wrong negation commands, repeated commands, inaccurate future-state predictions) to the driver rule or code layer responsible, so bug-fix PRs start in the right file instead of patching symptoms at the wrong layer.

These live in .claude/skills/, but they are plain markdown. The AGENTS.md file tells non-Claude agents that the same workflows are reachable through the docs the skills reference. Skills for one tool, documentation for every tool.

The playbook, generalized

None of this is specific to hier_config or to network automation. The general recipe:

1. Audit your docs as if a machine will follow them literally, because one will. Deduplicate, fix drift, and fill the gaps where standards exist only in reviewers’ heads. Add a strict docs build to CI.
2. Write one canonical AGENTS.md as an index: commands, architecture in brief, hard rules, and a task-to-docs map. Keep per-tool files as thin overlays or condensed copies.
3. State hard rules as merge-blockers and anticipate shortcuts. If an agent could satisfy the letter of a rule by weakening a config, say explicitly that this is forbidden.
4. Ship a self-review tool that runs your real gates and reviews the diff against your real docs, reporting findings by severity. Make its checklist identical to your PR template.
5. Pave your most common contribution paths with workflow guides so the frequent PR types arrive well-formed.

The effort is front-loaded and the payoff is compounding. Every AI-assisted PR that self-reviews before submission is a review you conduct against a conforming diff instead of a foreign one. And the humans benefit too, because “our standards are documented, current, and checkable by tooling” was always the right goal. The rise of AI-assisted contributions just made the cost of skipping it visible.

Open source has absorbed bigger shifts than this one. The projects that thrive will be the ones that treat AI-assisted contributors as contributors, hold them to the same standards as everyone else, and invest in making those standards impossible to miss.