Skip to content

Publications layer plan

Preliminary schema and intake plan for a resource-evidence publications layer.

Report date: 2026-05-30
Addendum: 2026-08-20 (ORS-authorized harvest vs promotion)
Registry: data/publications.yaml
Person links: data/person_publication_links.yaml
Resource links: data/publication_resource_links.yaml
Export: scripts/export_graph_tables.py → data/graph_exports/

See also: Publication intake rules · Publication data intake template · Publications quality review · Grants layer review · Graph readiness review · Source Evaluation and Evidence Policy

Current curated registry: see publication_count in data/publications.yaml. Historical batch counts in §10 are unchanged.

Addendum (2026-08-20) — licensed harvest vs map promotion

ORS authorized a scope extension on 12 August 2026 (Daniela Stan Raicu, Lauren Miller): collaborate with the Library and its databases, and assess computing resources as available, needed, and actually used. On 19 August 2026 the Library (Kelly Hallisy, Ashley McMullin, Kirsten Yehl) authorized scraping scholars.depaul.edu and VIA theses/dissertations for the expanded map. Provenance: sources/inbox/2026-08-20-library-harvest-authorization/.

That authorization does not turn data/publications.yaml into a DePaul-wide bibliography. Two stores stay distinct:

  1. Harvest archive (sources/harvests/, plus optional indexes under data/derived/) — OAI-PMH metadata and OA HTML/PDF from Scholars and VIA under the packet above. Capture is not publication. OpenAlex (public API) and Scopus (institutional key, if issued) are separately scoped.
  2. Curated map layer (data/publications.yaml, data/outputs.yaml / theses when promoted, link registries) — only records with quoted, reviewed evidence of resource use, computing/data practice, or other map-relevant claims.

Capture does not imply use. Bulk harvest is permitted as assistive extraction into the harvest archive. Promotion into YAML or public resource pages still requires a source packet (or equivalent checklist), identity match to people.yaml without auto-creating person pages, and the evidence bar in publication-intake-rules.md. A harvest file may be cited from evidence_file only when that packet quotes it.

Theses and dissertations may be promoted as outputs or publication-like records when methods, acknowledgments, or data chapters document a map resource or a computing/data-handling practice. Authorship or a Scholars profile alone is not a facility or operational-status claim.

The Library also noted that Scholars, VIA, and related databases are incomplete and asked for help improving them (Data Services Librarian hire pending). That is a later collaboration, not a public-map completeness claim.

How ingest actually works (Library-authorized OA harvest + VPN for licensed promotion):

  • Scholars/VIA harvest still prefers the public internet (OAI, OA HTML/PDF). Bot-protection is not a reason to require VPN.
  • Licensed publisher full text for ranked promotion candidates may use campus VPN (curator 2026-08-21). Check scripts/check_campus_vpn.py. Split tunnel: campus prefixes (140.192/16, 216.220.176/20) go via VPN; doi.org often still uses the home IP — prefer Library campus hosts. If the VPN drops, stop publisher fetches and alert the curator to restart the client. Do not bulk-download publisher corpora.
  • VIA: use OAI-PMH for metadata (https://via.library.depaul.edu/do/oai/) because the Library authorized this harvest — VIA robots.txt (User-agent: *) disallows /do/ and does not disallow /cgi/viewcontent.cgi. OA thesis full text is in-scope under the packet's reading of “scraping … VIA for thesis and dissertations”; the memo does not say “PDF.” Elsevier Digital Commons OAI dataPolicy text that asks for written approval before robot full-text harvest is unchanged; this project proceeds on Library authorization in the packet, not on a conclusion that Elsevier's terms do not apply. Skip embargoed or login-walled files.
  • Scholars.depaul.edu (Pure): the public portal is OA (listing pages return 200 off-campus). The Pure API is 401 without an institutional key; the Library cannot currently provision that access. OA portal scrape is therefore the approved Scholars path, not a forbidden workaround. Honor Crawl-Delay: 5.
  • Bot-protection (Cloudflare / AWS WAF challenges on some profile and viewcontent.cgi URLs) is an engineering problem for polite harvest tooling, not a reason to require VPN. Identifiable User-Agent with contact; no UA/IP rotation or CAPTCHA-solving services; back off on 429/403; stop on operator request.

Sections 1–10 below remain the rules for the curated publications layer.

1. Why not a complete DePaul publication inventory

The Resource Map exists to document facilities, equipment, access, and collaboration paths connected to DePaul resources—not institutional research output metrics.

A full public-map publication inventory would:

  • Dilute graph signal with humanities, policy, and non-technical work unrelated to labs or instruments
  • Imply coauthorship networks that look like facility collaboration without evidence
  • Require ongoing sync with ORS, schools, and external indexes outside map maintenance scope
  • Duplicate work better done by libraries, Pure/Symplectic, or institutional CRIS systems

Publications belong in the curated YAML layer only as evidence objects when they document resource use, capability, computing or data practice, or cross-unit technical work tied to map entities. Campus-wide bibliographic completeness belongs in the harvest archive and in Library systems (Scholars, VIA), not in data/publications.yaml.

Start from the people registry in data/people.yaml (see full-refresh-report.md for current count), prioritized by hard-tech adjacency:

Priority Person categories Rationale
1 Lab/center directors and faculty_lead on CDM, CSH, MUS, TTS resources Most likely to publish methods using named facilities
2 CyberLabs / RoME / VIDA / CSH instrumentation affiliated faculty Equipment-heavy research
3 Grant PIs in data/grants.yaml with equipment or lab keywords in titles/abstracts May cite facilities in acknowledgments
4 Production/technical staff only when authorship is documented on official pages Rare; avoid staff bibliography fishing

Do not bulk-import all authors from a college or department page.

Suggested pilot cohort (~10–15 people): directors of CDM-001, CDM-004, CDM-005, CDM-011, CSH-001, CSH-007, PER-032 (Landahl), PER-064 (Schrank), PER-013 (Godage), plus grant PIs with clear hardware abstracts (e.g. Federated Learning, equipment grants).

3. Inclusion criteria

Include a publication when all apply:

  1. Hard-tech relevance — fabrication, instrumentation, sensing, imaging, HPC, media production technology, cyber-physical systems, GIS/analytics tied to map resources, or similar
  2. DePaul connection — at least one author matches a people.yaml record with evidence
  3. Resource-map utility — supports a facility link, capability claim, grant follow-on, or cross-unit technical thread
  4. Evidence in intake — title/year/venue from an allowed source (see §8); resource connection quoted when linking to resource_id

4. Exclusion criteria

Exclude publications that:

  • Have no plausible link to a map resource, grant, or technical capability
  • Are teaching materials, op-eds, or policy commentary without instrumentation
  • Mention DePaul only as author affiliation with no methods/facility detail
  • Are planned outputs (manuscript in preparation) — those belong in grants, not publications
  • Come from bulk API harvest without per-record resource evidence review
  • Duplicate entries across DOI, arXiv ID, and publisher URL (pick one canonical record)

5. Warnings: author-name ambiguity

  • Common names (e.g. “J. Smith”) may match wrong person_id
  • Name variants (Daniela Raicu / Daniela Stan Raicu) require alias notes
  • Students not in people.yaml should not auto-create people — record as person_name with person_id: null or add via separate people intake
  • Corporate authorship (“DePaul University”) is not a person link

Always record ambiguity in notes and prefer confidence: low until verified against an official DePaul profile page.

6. Warnings: old affiliations

  • Publications from before a faculty member joined DePaul may still acknowledge current DePaul labs if methods were conducted here — verify dates
  • Alumni authorship does not automatically link to current map resources
  • External postdoc or sabbatical affiliations may appear in author lists without DePaul facility use

Do not link related_resources based on author’s current map role alone.

7. Warnings: coauthorship ≠ collaboration

  • Shared authorship does not imply shared facility access or cross-lab collaboration edges
  • Large consortia papers may list DePaul as one node among many — link people, not inferred institution–institution cliques
  • Acknowledgments sections are stronger evidence for publication → resource links than author lists

Use person_publication_links for authorship; use publication_resource_links only with explicit methods/acknowledgment quotes.

8. Suggested sources for future publication searches

Use in this order (future passes only — not executed in schema pass):

Source Use when
Official DePaul faculty/lab profile pages Publications listed on depaul.edu or official lab sites already linked in people.yaml
Lab and center pages (CyberLabs, VIDA, RoME, CSH facilities) Selected papers highlighted as outputs
Google Scholar Only when URL is linked from an official DePaul page (no broad scraping)
ORCID Only when ORCID URL is on official DePaul profile
Crossref / DOI metadata Resolve title, year, authors after a paper is already identified
PubMed Biomedical / health-informatics topics only (VIDA, CCHE, biology instrumentation)
Semantic Scholar / OpenAlex Secondary enrichment for DOI disambiguation — not primary discovery

Avoid as a path into the curated YAML layer: wholesale department publication lists without resource filtering, and writing scrape output straight into data/publications.yaml. OA harvest of Scholars/VIA into sources/harvests/ (authorization packet above) is the approved bulk path (see the 2026-08-20 addendum).

  1. Pick 5 directors from the pilot cohort (§2).
  2. For each, read official DePaul profile or lab page already in depaul_profile_url — copy 0–3 papers that mention labs, instruments, or fabrication.
  3. Intake via publication-data-intake-template.md.
  4. Add person_publication_links with include_in_graph: true only for listed authorship.
  5. Add publication_resource_links only when methods/acknowledgments name a resource (e.g. “CyberLabs”, “McGowan NMR”, “Idea Realization Lab”).
  6. Run python3 scripts/export_graph_tables.py and review CSV row counts.
  7. Document pilot results in an updated section of this plan before scaling.

Target for pilot: 5–15 publications, ≥3 publication–resource links with explicit evidence.

10. Lessons from batches 1–4

Four conservative discovery passes (seed cohort, ISE/CSH/ID expansion, hard-tech faculty, directors/grants cohort) produced 34 publications and 39 person–publication links, but only 6 publication–resource links — 4 remain graph-included after quality review downgraded two profile-inference edges (PRL-PUB-003, PRL-PUB-004).

Publication–resource edges are high-evidence edges, not default edges. Batches 3 and 4 correctly added zero new publication–resource links because selected papers lacked explicit facility acknowledgment. Person–publication intake alone is the normal outcome.

Lesson Source
Authorship verified → person–publication link OK All batches
Resource linkage requires quoted facility/methods/acknowledgment text PRL-PUB-001/002 (strong) vs PRL-PUB-003/004 (downgraded)
Profile method lists ≠ resource use PRL-PUB-004 (Grice Lab)
Center director ≠ center infrastructure in paper PPL-033 (Wagner/SPARK qualitative study excluded from graph)
Grant abstract guides search, not resource edge Bystriansky wolf eel vs Poeciliid grant
Coauthorship stays on person–publication edges only No person–person collaboration from pubs
~1 weak inclusion per batch → quality review pass Cohort 4 Wagner case
Batch size 8–10 people remains appropriate Throughput ~8–9 includes per batch

Full rules: publication-intake-rules.md. QA and batch summaries: publications-quality-review.md.

Regeneration commands

python3 scripts/export_graph_tables.py
python3 -m mkdocs build --strict

Empty publication YAML files export header-only CSVs (0 data rows) without error.

Schema pass only. No public Publications tab, no web crawl, no Neo4j import.