coding-with-ai/06-verifying-and-citing/README.md

# Verifying and Citing

## Key idea

AI assistants are useful because they generate plausible output fast. They are *dangerous* for the same reason. They can write plausible-but-wrong output that looks identical to plausible-and-right output until you check. This section is about the discipline of checking, plus the two other questions that come with using AI work: *what did you give the service?* and *how do you disclose what you used?*

## Key goals

- Develop habits for verifying AI output, especially the parts that look most confident
- Understand the privacy and intellectual-property implications of what you paste
- Know how to disclose AI use in academic and professional contexts


---

## Part 1: Verifying

> **A note on terminology.** This section uses "check" and "test" with different meanings. A **unit test** is a specific software-development practice, which is a small piece of code (often written with a framework like `pytest`) that exercises a function with known inputs and confirms the output matches an expected result. Tests are automated and reusable, and they pay off when code will be edited many times by many people. A **check**, more broadly, is anything that verifies the code does what you intended: running on a known limit case, comparing to a published value, plotting and inspecting the shape, or hand-calculating a small input. Formal unit tests are one form of check, but for scientific code written for a single project they are often not the most natural form. Whenever this guide says "verification" or "check," any of these forms count; "test" appears only where an automated test is the right tool.

### Why verification matters

Hallucinations in AI-assisted coding fall into two broad categories:

1. **Loud hallucinations.** Code that fails to compile or run. Easy to catch; the tool tells you.
2. **Quiet hallucinations.** Code that runs and produces a result, but the result is wrong. These are the dangerous ones.

There is often a familiar pattern: a function that uses an API method that doesn't exist, a regex that handles all the cases you mentioned but fails silently on an edge case you didn't think to mention, a math expression that is dimensionally inconsistent but produces a number anyway. The output *looks like* an answer, so you accept it. Hours or weeks later, you discover the silent failure.

The fix is not "don't use AI." The fix is to assume every AI-generated piece of code is plausible until verified, and to make verification cheap.

### What to verify

Not everything needs the same scrutiny. A reasonable triage:

| The code is... | Verify by... |
|---|---|
| A one-line refactor in code you understand | Reading the diff |
| A new function with a clear contract | Running it on inputs whose answer you can compute or look up — a known limit case, a published value, or a hand calculation. (A formal unit test is one form; for scientific code, often not the most natural.) |
| Code that calls an unfamiliar library | Checking the library's docs for the methods and signatures the AI used |
| Code involving math, units, or numerical methods | Walking through the math on paper or with a known input |
| Multi-file changes from an agent | `git diff`, run whatever verification you normally use (tests, sanity-check scripts, known-answer comparisons), and *read* any verification scripts the agent touched to make sure they didn't get weakened |

Two specific habits are worth building:

**Check that named functions and methods actually exist.** A common hallucination is "the AI confidently uses `pd.read_excel_smart()`," which sounds real but does not exist. A quick `grep` of the library docs (or a 10-second IDE check) catches this.

**Write your own checks, especially for edges.** If the AI writes the implementation *and* the check, the check verifies the AI's interpretation of your goal, not the goal itself. Run the function on a limit case you can verify by hand (zero, infinity, a published value), compare against an analytical answer where one exists, or write a formal test if that is the right form for your code. Whatever the form, at least one check should come from you.

### Trust gradients

Trust is calibrated by domain. Some quick heuristics:

- **High trust:** Boilerplate, well-known patterns, idiomatic refactors, common-library usage at common signatures.
- **Medium trust:** Unfamiliar library usage, performance-sensitive code, security-adjacent code.
- **Low trust:** Numerical methods, statistics, anything involving units or domain-specific physics, anything safety-relevant, security-critical code, code where being subtly wrong is much worse than being obviously wrong.

Trust is not "the model is bad at X." It's "the *consequence* of being wrong about X is high, so verify proportionately."

---

## Part 2: Privacy and IP

### Baseline: similar risk profile as other cloud services

When you paste content into a chat, that content is sent to the service. Editor extensions and agents do the same with the files they read. This is the same baseline as Gmail, Google Drive, GitHub, OneDrive, Dropbox, or any other cloud service you already use. In these cases, your content lives on someone else's servers, subject to their data-handling policies. For most academic and research work, including coursework, classroom code, public datasets, open-source libraries, drafts of your own writing, the privacy risk is no different from what you accept every time you use those other services.

### Three ways AI services *are* different from Gmail

The genuine differences are narrower than the general "the cloud is watching" framing suggests, but they are real:

1. **Training-data inclusion.** Gmail and Drive do not train on your content. AI services historically have, varying by the provider and the plan. While defaults have been changing, and most paid and enterprise tiers now opt out, the precedent is real and there is no Gmail equivalent. Check the current setting for the service you use.
2. **Aggregation richness.** A chat history reveals more than an email archive. What you are working on, what you do not know, what you are puzzling over. These can accumulate in conversations in a way they do not in inboxes. Aggregated chat history is potentially a richer and more sensitive documentation of your work than aggregated email is.
3. **Routine review of flagged content.** Most AI services explicitly reserve the right to have humans review conversations flagged by their safety systems. Gmail has no equivalent "if our spam filter trips, a person may read this" policy. In practice your conversations are almost certainly not reviewed, but the legal posture is different.

A fourth, less concrete consideration is that AI services are two to three years old as widely-adopted products. We have less empirical track record of their behavior than for Gmail (20+ years) or AWS (15+).

### When this matters in a research context

For most academic work, the cloud-service baseline is the right mental model. You wouldn't upload an unpublished manuscript to a random web tool you'd never heard of, but you also do not panic about every email you send. Many researchers paste unpublished manuscripts into Overleaf without thinking about it. AI chat services from established providers are in a similar position. They are known entities with real privacy policies and enterprise tiers with stronger guarantees. The instinct that makes you cautious about a brand-new free tool you've never heard of is right, but it shouldn't make you treat Claude or ChatGPT as fundamentally different from the cloud tools your work already lives in. A few categories do warrant extra care, though:

- **Restricted research data.** Anything covered by your IRB protocol, your data-use agreement with a collaborator or industrial partner, or institutional policies around HIPAA, FERPA, export controls, or similar regimes. If a category of data is restricted on your computer, it is restricted in your chat too.
- **Unpublished work that isn't yours.** Collaborator drafts, manuscripts under review, code from a lab that hasn't been released. You don't own the right to share these regardless of how you happen to be sharing them.
- **NDA-covered or proprietary material.** From an industrial collaboration, an internship, an advisor's industry consulting work. Check the specific agreement.
- **Personally identifying information.** Participant data, survey responses, names attached to outcomes, even when "anonymized for internal use." If you need help analyzing it, paste a synthetic example with the same shape rather than the real thing.
- **Credentials, API keys, internal URLs.** Easy to leak by accident when pasting config files or logs.

For most students most of the time who are dealing with coursework, classroom exercises, your own scripts, public datasets, open-source libraries, and drafts of your own writing, the answer is "the chat is fine, same risk as email." Graduate students and undergraduate researchers working with sensitive research data are the most common case for the categories above. If that's you, take the agreements that govern your data seriously, and when in doubt, ask your advisor or your IRB.

### A practical checklist

Before pasting anything non-trivial:

1. Does this contain credentials, keys, or tokens? *(strip them)*
2. Does this contain restricted research data, sensitive participant information, or unpublished work that isn't yours? *(use a synthetic example or stop)*
3. Is this covered by an NDA or institutional agreement that restricts where it can go? *(check the agreement)*
4. Would I be comfortable with this content appearing in an email I sent through Gmail? *(if yes, the chat is fine; if no, reconsider)*

The check takes seconds and prevents most accidental disclosures.

---

## Part 3: Citing and attribution

### The landscape is in transition

AI-use disclosure norms are still settling. Most universities, journals, and employers have adopted policies in the past two years, but the policies themselves often do not capture how the tools actually work. They may treat all AI use as equivalent, conflate autocomplete with substantive drafting, or be vague about what counts as "use." In practice, compliance is uneven, too. Many people don't disclose at all, others over-disclose in ways that aren't useful, and the gap between policy and behavior is wide.

The advice in this section is therefore intended to be normative and proportional, not absolute. It is about what serves you and the integrity of your work when policies apply, not a claim that this is how the field universally behaves.

### Why disclose anyway

Two complementary reasons:

1. **Compliance, when policy applies.** Many courses, programs, and journals require disclosure for completed work. The policies vary, enforcement is uneven, but not complying is a violation even if undetected.
2. **Self-protection.** If your work is later challenged by a reviewer, an examiner, a future employer, or a collaborator, having disclosed AI use is the defensible posture. Disclosure is the conservative choice for *your* future, regardless of what others do.

### A proportional standard

The realistic bar is not "note every Copilot autocomplete." That standard is impossible to meet in practice, and treating it as required is part of why disclosure norms feel unrealistic. A more useful distinction:

- **Background assistance** that shaped *how* you worked, such as autocomplete, syntax help, name suggestions, quick lookups, debugging conversations. Usually no disclosure is needed unless your venue's policy is specific.
- **Substantive contribution** that shaped *what* you produced, such as AI drafted a section, generated significant chunks of code that you reviewed and accepted, planned the analytical approach, wrote the literature summary, debugged a critical reasoning step. These likely warrant a brief note.
- **Substituted work** where AI produced something you submitted as your own without meaningful engagement, including running an assignment through ChatGPT and turning in the output. This is the case policies are most worried about, and it sits closer to academic dishonesty than to the disclosure question.

The distinction is *substantive contribution* versus *background assistance*, and it's roughly the same one you would use for human help. Casual tutoring from a peer doesn't go in your acknowledgments, but a significant intellectual contribution does.

### Where and how to disclose

The form depends on context:

- **Course assignments.** Follow your course's stated policy. Most courses now have one; if yours doesn't, ask. When in doubt, a brief "I used [tool] for [purpose]" note at the top is the conservative choice.
- **Theses and dissertations.** Many programs now require an AI-use statement in the front matter. Check with your department.
- **Conference and journal papers.** Most major venues (ICML, NeurIPS, Nature family, ACS journals, etc.) have explicit policies as of 2024–2025. Common pattern: AI cannot be listed as an author; substantive AI use must be disclosed in methods or acknowledgements.
- **Code commits and PRs.** Practices vary here. Some teams use `Co-Authored-By: AI <noreply@...>` lines for substantial contributions. Others don't. Follow the team's convention; if there isn't one, ask. (This guide's own commits use `Co-Authored-By: Claude` lines where AI substantively contributed.)
- **Code review.** Reviewers benefit from knowing which sections of a change were AI-generated, not as a red flag, but to inform what kind of review the section needs.

A useful pattern when you do disclose is to state three things:

1. **What tool you used** (specific model and version if available, such as "Claude Opus 4.7," "ChatGPT-4o," "GitHub Copilot")
2. **What you used it for** ("debugging error messages," "drafting the introduction," "generating boilerplate code")
3. **What you did with the output** ("reviewed and edited," "used as a starting point and rewrote," "used as-is after verification")

Example:

> *I used Claude to help structure the introduction and to debug error messages I encountered while running the analysis. All AI-generated text was reviewed and edited. All AI-generated code was verified against known limit cases.*

That's enough for most academic contexts. Specific journals may require a particular form.

### What disclosure is not

Disclosure is not an admission of wrongdoing. It is an accurate description of your process. The work is yours and the AI was a tool. Be matter-of-fact about it the same way you would acknowledge a specific Python library, a colleague's helpful conversation, or a textbook reference.


## Exercises

> **Exercise 1:** Take a piece of AI-generated code you used recently. Apply the verification triage table — where on the spectrum was it, and did you verify proportionately? If not, do the verification now and note what (if anything) you find.

> **Exercise 2:** Run through the pre-paste checklist on a recent chat session. Was there anything in the paste you would do differently now?

> **Exercise 3:** Find the AI-use policy for one context that applies to you (a current course, your department, a journal you have submitted to, your employer). Read it. Note the differences from this guide's general advice — those differences are the ones that matter for you specifically.