Skip to content

RAGMap and graph analysis plan

Plan date: 2026-06-01
Status: Prototype / architecture (no Neo4j or chatbot required yet)
Current registry: full-refresh-report.md · Exports: data/graph_exports/ · Queries: graph_queries/

1. Four layers of value

Layer What it is Primary user need
Static resource map MkDocs site + YAML/Markdown catalogs Browse ownership, equipment lists, courses, and documented relationships with human-readable pages
Graph database Typed nodes and edges with properties (include_in_graph, evidence_level, confidence) Traverse multi-hop questions: “who connects unit A facilities to unit B?”, “which grants touch instrumentation?”
RAGMap (retrieval-augmented map) Graph traversal plus retrieval over evidence text (Markdown, YAML fields, reports, public URLs) Answer natural-language questions with cited snippets, not invented narratives
Decision-support tool RAGMap outputs framed for a task (proposal fit, collaborator shortlist, gap list) with explicit uncertainty ORS, chairs, and PIs choose next actions (enrich data, contact person X, cite facility Y in proposal)

The static map is the publication and stewardship surface. The graph is the relationship engine. RAG is the language interface over evidence. Decision support adds ranking, gaps, and recommended next steps without replacing human judgment.

2. What the repository already provides

Entity files (source of truth)

File Entities
data/people.yaml 115 people (PER-*), roles, affiliations, associated_resources
data/grants.yaml 33 grants (GRA-*), funders, units, optional related_resources
data/publications.yaml 34 publications (PUB-*)
data/outputs.yaml 18 outputs (OUT-*)
resources/*/*.md 78 facility/program nodes + 187 course pages (CUR-*)
data/resource_equipment.yaml, data/resource_courses.yaml Equipment lines and course catalog metadata
File Relationship
data/person_resource_links.yaml Person ↔ resource (209 links; 174 include_in_graph=true)
data/person_grant_links.yaml Person ↔ grant (PI/co-PI)
data/grant_resource_links.yaml Grant ↔ resource (facilities/equipment narrative)
data/person_publication_links.yaml Person ↔ publication
data/publication_resource_links.yaml Publication ↔ resource (high bar)
data/person_output_links.yaml Person ↔ output
data/output_resource_links.yaml Output ↔ resource
data/course_resource_links.yaml Documented course ↔ facility (2 links today)

Evidence and trust metadata

  • evidence_text / evidence_file / evidence_url on links and entities
  • confidence and verification_status on people and many entities
  • include_in_graph — export gate for graph/RAG traversal (excludes weak admin, generic affiliation, UNI-001 grant anchors, etc.)
  • include_in_public_site — separate flag for MkDocs people/contact display
  • evidence_level on CSV exports: direct, affiliation, topical, administrative, excluded, needs_review (from scripts/person_link_rules.py)

Graph export CSVs

Produced by export_graph_tables.py (via refresh_all.py) into data/graph_exports/:

  • Nodes: nodes_people.csv, nodes_resources.csv, nodes_grants.csv, nodes_publications.csv, nodes_outputs.csv
  • Edges: edges_person_resource.csv, edges_person_grant.csv, edges_grant_resource.csv, edges_person_publication.csv, edges_publication_resource.csv, edges_person_output.csv, edges_output_resource.csv

Facility resources in nodes_resources.csv are non-course resources only (CDM, CSH, MUS, etc.). Courses and equipment are not yet exported as separate node CSVs (see neo4j-import-plan.md).

Refresh validation

python3 scripts/refresh_all.py rebuilds links, regenerates resource pages, exports CSVs, and writes full-refresh-report.md with:

  • Entity and edge counts
  • Evidence-level breakdown
  • Validation errors/warnings (e.g. structured links with zero graph edges; outputs without people)
  • Hub resource list (with UNI-001 flagged)

3. What graph analysis can answer that the website cannot

The MkDocs site is excellent for single-entity lookup and curated narrative. It does not natively answer:

  • Multi-hop paths: Person A → grant → resource → publication → person B
  • Set operations: “All CSH instrumentation resources with equipment YAML but no include_in_graph person edge”
  • Hub and bridge detection: Degree centrality excluding administrative anchors (UNI-001)
  • Cross-unit overlap: People with direct edges to both CDM-* and CSH-* prefixes
  • Evidence gaps: Grants mentioning facilities in text but no grant_resource edge
  • Layer consistency: PI on grant with no publication/output linking to same resource theme
  • Weak-edge audits: All affiliation or topical edges above a degree threshold

These are analytical questions over the exported graph (Cypher, NetworkX, or a notebook), not navigation questions.

4. What RAG can answer that graph analysis alone cannot

Graph traversal returns IDs and edge properties. RAG adds unstructured evidence:

  • Wording in resources/CDM/CDM-016.md about Cinespace capabilities
  • evidence_text on why a collaborator edge was included or excluded
  • Intake rules in publication-intake-rules.md / output-intake-rules.md
  • Grant facility narratives in local PDFs (when permitted) referenced from grants.yaml
  • Full refresh warning rationales (e.g. why CDM-002 has equipment but no named operator)

RAG is needed for questions like:

  • “Summarize what we can prove about environmental sensing at DePaul.”
  • “What should we tell a program officer about Cinespace based on documented evidence?”
  • “Why was this publication excluded from the graph?”

Pure graph queries cannot generate faithful prose without retrieving source text.

Institutional decisions (collaboration, proposal claims, facility investments) require auditability. If an answer cites only “Person X is connected to Resource Y,” users will over-interpret affiliation as co-use, or grant_pi → UNI-001 as collaboration with everyone else touching ORS.

Every nontrivial RAGMap answer should include:

  1. Graph path (node IDs, relationship types, evidence_level)
  2. Evidence snippet (evidence_text, resource Markdown section, or report paragraph)
  3. Confidence / verification where available
  4. Explicit limits (“no publication–resource edge”; “affiliation only”)

Without (2), the system becomes a plausible-fiction generator—unsafe for ORS and research development use.

flowchart LR
  YAML[Repository YAML and Markdown]
  Refresh[refresh_all.py]
  CSV[graph_exports CSVs]
  Graph[Graph DB or analysis notebook]
  Retrieve[Evidence retrieval]
  QA[Question answering / planning UI]

  YAML --> Refresh
  Refresh --> CSV
  CSV --> Graph
  YAML --> Retrieve
  Graph --> QA
  Retrieve --> QA
Stage Tooling (prototype → production)
Ingest Existing repo; no new entities in prototype phase
Normalize refresh_all.py + export_graph_tables.py
Graph store Neo4j Community (optional) or Python NetworkX/pandas for notebooks
Import graph_queries/neo4j_*.cypher templates
Analysis graph_queries/analysis_queries.cypher + graph-analysis-use-cases.md
Retrieval Chunk resources/, link YAML evidence_text, reports; embed + filter by node IDs from graph
Answer Template: graph result → fetch evidence → LLM with mandatory citations (internal tool first)

Public vs internal: MkDocs shows only include_in_public_site contacts and public Markdown. Graph/RAG may use include_in_graph=true edges and internal reports; never surface local grant PDFs or ORS purchasing fields without permission (internal-data-wishlist.md).

7. Safety and trust rules

Rule Rationale
Do not treat affiliation as collaboration Generic CyberLabs affiliation text is not co-authorship or co-lab use
Do not treat ORS / UNI-001 as a collaboration hub grant_pi to UNI-001 is administrative anchoring; export enforces include_in_graph=false
Distinguish direct, weak affiliation, topical, administrative, excluded Use evidence_level and relationship type in every visualization default
Cite evidence for every nontrivial claim Tie answers to evidence_file, resource page, or report section
Keep public display separate from graph use include_in_public_site ≠ include_in_graph
Never expose private proposal text without permission Local data/*.pdf stays out of Git and out of public RAG indexes
Prefer “underdocumented” over “unused” Absence of edge ≠ proof facility is irrelevant
Flag needs_review and refresh warnings in answers OUT-014–016, CDM-002, CSH-008 are known gaps

8. Prototype milestones (no new data in this pass)

  1. Done: Planning docs + Cypher templates in graph_queries/
  2. Next: Run Neo4j import locally against current CSVs; validate hub/bridge queries match full-refresh hub table
  3. Then: Notebook or script: graph query → collect evidence_file paths → markdown retrieval
  4. Then: Internal-only CLI: one question → graph + citations (no public page, no mkdocs nav change)
  5. Later: Course and equipment node export; topic nodes from topics fields

See also: graph-analysis-use-cases.md · neo4j-import-plan.md · ragmap-question-examples.md