Source Evaluation and Evidence Policy

Branch: development Status: Active IKaC policy Related: Source Packet Workflow · Repository-wide refactor governance · Main Evidence & Verification Audit

Repository-wide evaluation (effective immediately)

Intake type ≠ refactor scope. A faculty profile, grant PDF, publication DOI, or ORS annual report may contribute evidence to any knowledge layer.

Every source packet evaluation must:

Requirement	Policy
Full impact assessment	Review all layers listed in Repository-wide refactor governance
Three-way classification	Evidence-backed update · candidate knowledge · rejected claim
Candidate preservation	Insufficient evidence → preserve in section M / `extracted/candidates.yaml`; never silent discard
Promotion review (mandatory)	Every preserved candidate triggers targeted evidence discovery → re-evaluate → promote what becomes evidenced, or defer with rationale (section N). See Targeted evidence discovery and the Promotion Discovery Rule.
Layer-specific rules	Apply grant, publication, or output rules when that layer is implicated — not as a packet scope limit

Core distinction (do not collapse)

Every source packet maintains two separate concepts:

Concept	Meaning
Captured source	A file or URL snapshot saved in the packet with provenance (HTTP metadata, timestamp, SHA-256, `capture_status`).
Source used as evidence	A captured (or uploaded) source that supports one or more specific claims in `evidence_checklist.md` and proposed corpus edits.

Capture does not imply use. All captured sources are preserved for audit regardless of whether they are used as evidence. Refactor reports must list both sets explicitly.

Capture status

Intake records a capture_status for each URL snapshot:

Status	Meaning	Examples
`success`	Expected content retrieved and stored.	HTTP 200 HTML faculty profile; PDF upload saved intact.
`partial`	Something was stored, but content is incomplete, blocked, or not the intended document body.	HTTP 403 publisher landing page (error page or paywall shell saved for audit); truncated response; non-HTML body saved with note.
`failed`	No usable snapshot stored.	Network error; blocked URL; oversize response; validation failure.

HTTP 403 is always partial, never success. A saved 403 body is provenance (what the server returned at capture time), not confirmation that the scholarly record was retrieved. Titles, DOIs, and authorship for blocked publisher pages must come from higher-confidence sources in the same packet (e.g. faculty profile, Google Scholar, ORCID) unless independently verified.

High-confidence scholarly and institutional sources

These may support corpus updates when captured in a packet and cited in evidence_checklist.md:

University faculty profile pages
University laboratory websites
University department websites
University research center websites
Grant proposals
Grant award notices
Published journal articles
DOI landing pages
Google Scholar profiles
ORCID profiles
Web of Science records
Scopus records
ResearchGate profiles when clearly associated with the scholar
Professional society profiles
Official project websites
Faculty-maintained research websites

Appropriate uses

High-confidence sources may support claims about:

People (identity, affiliation, contact when public)
Publications (titles, authorship, venues, years, DOIs when listed)
Collaborations (coauthorship, joint projects when explicitly stated)
Memberships (societies, consortia, editorial boards)
Service roles (advising director, journal editor, committee service) — on resource pages, only when tied to facilities or hardware (see below)
Expertise areas and research themes
Professional activities
Organizational participation

Scholarly index and profile systems

Google Scholar, ORCID, Web of Science, Scopus, and ResearchGate are valuable evidence sources for:

Publication lists and bibliographic metadata
Coauthor networks and collaboration patterns
Research themes and expertise areas
Professional identifiers (ORCID iD)
Cross-checking titles and DOIs when publisher pages are blocked

They are insufficient by themselves for:

Facility ownership or laboratory affiliation
Equipment ownership or specific instrument assignment
Facility access levels or booking policy
Organizational authority or administrative responsibility
Operational control of shared resources

Those relationships require stronger evidence from institutional or project sources (official lab pages, department facilities pages, grant award documents, explicit facility acknowledgments in publications, access policies).

Faculty-maintained research websites sit between institutional pages and third-party indexes: strong for group research themes, news, and links; weaker alone for formal facility or access claims unless they repeat official institutional statements.

Targeted evidence discovery (promotion review)

Candidate preservation is the beginning of promotion review, not the end. When a source produces candidates (facilities, labs, centers, institutes, clinics, equipment collections, capabilities, external facilities, programs, courses, partners, people, or relationships), the refactor must actively go look for corroborating evidence rather than leaving the candidate dormant. This is the Promotion Discovery Rule.

Permitted targeted-discovery sources (all subject to the confidence rules above):

Source	Strong for	Not sufficient alone for
Institutional websites (college/department/center/facility/project pages)	facility existence, leadership, location, access	— (these are the institutional evidence)
Faculty profiles & lab pages	lab existence/leadership, people, research themes	facility ownership beyond what the page states
Grant-related pages (e.g. ORS award listings)	award existence, PI, program	facility ownership / access
ORCID, Google Scholar, ResearchGate, Web of Science, Scopus	publications, coauthor networks, expertise	facility / equipment ownership or access
DOI-linked publications	citation, authorship, funding acknowledgments	facility ownership unless explicitly named in facilities text
Official partner websites	external-partner existence, sub-facility structure, agreements	DePaul-specific relationship unless stated

Rules for promotion review:

Capture before you cite. Every newly discovered source must be saved as a packet capture with provenance (URL, date, capture_status) before it backs a corpus edit.
The evidence bar is unchanged. Promotion of facility / lab / equipment / access / ownership claims still requires institutional or project evidence; targeted discovery raises the obligation to look, not the threshold to promote.
External partners may have internal structure. A national lab or large partner can contain multiple sub-facilities reached through different agreements and contacts; record which sub-facility is implicated and the access pathway, and do not assume one agreement covers all of them.
Record outcomes in section N (sources reviewed/imported, promoted/deferred, relationships + reciprocal links added, remaining gaps). Unresolved candidates stay candidates with reason_not_promoted.

Grant and award evidence (when the source documents awards)

When a source packet includes award or grant evidence, apply Grant ingestion governance for grant-layer decisions. This does not exempt the packet from repository-wide assessment — the same source may document labs, people, publications, or equipment.

Unresolved knowledge is still knowledge. A reliable award listing may enter data/grants.yaml even when the PI is not in people.yaml or facility links are unknown.

Treat as three independent decisions:

Decision	Question
Grant existence	Did the source document this award?
PI identity	Is the named PI resolved to `people.yaml` (`resolved` / `unresolved` / `ambiguous`)?
Resource links	Is there explicit evidence for a registry facility edge (`resolved` / `unresolved` / `not_supported`)?

Do not discard evidence-backed awards solely because the PI is absent from people.yaml. Do not invent person_id, people records, or facility links. Add person_grant_links.yaml only when person_id is resolved; add grant_resource_links.yaml only when facility naming is resolved.

Bulk grant sources: see Grant ingestion filter audit for current script behavior.

Publication registry (DOI-first, when implicated)

When a source packet or refactor identifies a scholarly publication (regardless of primary intake type):

Resolve a DOI via Crossref, ORCID, Google Scholar, or publisher metadata — verify authorship and title match the discovery source (faculty profile, lab page, etc.).
Store in data/publications.yaml standard citation fields only:
title, year, venue, doi, url (https://doi.org/…)
abstract (from Crossref, OpenAlex, or publisher abstract when available)
people links to people.yaml
evidence with discovery source + evidence_url DOI
Do not save full-text PDFs or publisher HTML in the packet for registry intake — the DOI record plus verified citation metadata is sufficient for the Resource Map. Exception: the full-text facility-evidence pass may capture full text for a narrow, gated set of candidates solely to extract facility-use quotes. Captured full text is audit evidence under the source packet (fulltext/ or uploads/) with capture_status + SHA-256, is not registry content, and is not committed to the public repo when the license forbids redistribution. Publisher HTTP 403 remains partial.
Link by DOI in resource prose and YAML (evidence_url, hyperlinks to https://doi.org/… or registry PUB-### IDs).
Publisher pages returning HTTP 403 are partial captures only; DOI verification replaces publisher URL as the canonical reference.
Add person_publication_links.yaml entries for verified DePaul authorship; add publication_resource_links.yaml only when explicit facility/instrument evidence exists (see publication-intake-rules.md).

ORCID registry (people.yaml)

ORCID iDs strengthen person identity and cross-check publication authorship. They do not support facility, equipment, or access claims by themselves.

When enriching or refactoring faculty/research-eligible people:

Prefer Crossref authorship on linked publications (orcid_source: crossref, orcid_verification_status: verified).
ORCID public API (name + DePaul affiliation, or email when public) may yield probable matches (orcid_verification_status: probable) — human confirm when ambiguous.
Store on people.yaml:
orcid — normalized iD (0000-0002-0542-6495)
orcid_url — https://orcid.org/{orcid}
orcid_verification_status — verified | probable | needs_review | unresolved | not_applicable (staff-only roles)
orcid_verified_at — ISO date of last enrichment or manual confirm
orcid_source — crossref | orcid_api | manual | pending
orcid_candidates — list when multiple Crossref matches (needs_review)
Batch enrichment: python3 scripts/enrich_scholarly_identifiers.py (report) or --apply (write YAML). Review docs/reports/scholarly-identifiers-review.md.
Do not infer lab affiliation, instrument use, or resource access from ORCID alone.

Audit-driven refactors

Repository audits under docs/reports/ (e.g. hypertext-reciprocity-audit.md) guide surfacing and linking work. Full rules: Audit-driven refactor policy.

Audit findings are not evidence.
Implement only changes supported by sources already in the repository or by a source packet.
Prefer surfacing YAML/registry evidence over inventing relationships to close audit gaps.

Service and professional activity on resource pages

Do not add service, editorial, or society-membership blocks to facility/lab/infrastructure resource pages unless the role directly involves facilities or hardware (e.g. director of a named core facility, instrument manager, machine-shop oversight).

Advising directorship, journal editorship, consortium membership, and similar roles:

May appear in people.yaml notes or person-oriented records when evidenced.
Must not clutter lab resource pages (e.g. CSH-016) unless facility-relevant.

Conservative claim categories

Remain conservative — require institutional or project-level evidence — for:

Facility ownership
Equipment ownership
Facility access
Organizational authority
Administrative responsibility
Laboratory affiliation (beyond what an official profile explicitly states)
Operational control

When evidence is indirect (method lists, coauthorship alone, shared building), use supporting language, needs_review, or graph exclusion flags — do not upgrade to definitive facility or access claims.

Lower-confidence sources

Use with caution; generally not primary evidence unless corroborated:

Tier	Examples	Typical use
Lower confidence	News articles, press releases, blogs, marketing materials	Context, discovery leads, links to primary sources — corroborate before corpus edits
Very low confidence	Social media, discussion forums, anonymous websites	Do not use as primary evidence; may inform candidate links for separate intake

Refactor report requirements

Every refactor must produce refactor_output.md (or equivalent) with section A listing:

Captured sources — every file under the packet (uploads/, source_snapshot/, source_snapshot/captured/) with capture_status where applicable.
Sources used as evidence — which captures support which claims (map to evidence_checklist.md).
Sources reviewed but not used — captures opened or listed but not cited in proposed edits.
Reason not used — for each unused capture (e.g. "Google Scholar captured for audit; faculty profile used as primary evidence for publications"; "HTTP 403 publisher page — title taken from faculty profile instead").

Sections B–J, L (impact assessment), M (candidate registry), and N (promotion review summary) of the refactor output remain as defined in Source Packet Workflow and Repository-wide refactor governance. Section N is required whenever the packet produces candidates: it records the promotion-review outcome (sources reviewed/imported, candidates promoted/deferred, relationships and reciprocal links added, remaining gaps) or an explicit deferral with justification.

Curator-added provenance

Maintainer-added sources (outside the intake web form) are curator_added, not anonymous repository files. Each must record curator identity (added_by), date (added_at), and rationale (added_rationale) in packet metadata — see Source Packet Workflow § Curator-added sources.

Form-submitted sources are intake_form (intake_form_source in metadata.yaml). Captured source ≠ source used as evidence applies to both channels.

Hyperlinks extracted from URL snapshots (candidate_links.md) are not sources until a curator selects them for separate capture. After capture, each becomes a captured source subject to this policy — still distinct from "used as evidence" until the checklist and refactor cite it.

Lessons encoded (Kyle Grice packet, 2026-06-13)

A faculty profile URL intake can produce many candidate links; curators capture substantive links separately with full provenance.
Publisher HTTP 403 responses are partial captures — preserve the body and hash, do not treat as full article evidence.
Google Scholar may be captured and used for publications, collaborations, and expertise when associated with the scholar — but not as sole evidence for facility or equipment claims.
The official DePaul faculty profile remains the anchor for lab location and public affiliation; personal research sites supplement themes and contacts.
Refactors must document why a captured source was or was not used — silence is not acceptable audit practice.
Publications discovered on faculty profiles should be registered by verified DOI with citation metadata and abstract — not by publisher URL alone; do not store full text in packets.
Service roles (advising, editorship, IONiC, etc.) were removed from CSH-016 — not facility/hardware-related.