> For the complete documentation index, see [llms.txt](https://neurosymbolicai.gitbook.io/neuro-symbolic-ai-in-practice/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://neurosymbolicai.gitbook.io/neuro-symbolic-ai-in-practice/part-iii-core-approaches/chapter-4/4-3-hybrid-architectures/4-3-knowledge-graphs.md). # 4.3.2 Knowledge Graph Integration *** > **Background: Knowledge Representation Essentials** > > Knowledge Graphs use three foundational standards from the W3C Semantic Web stack: > > * **RDF (Resource Description Framework):** A data model for expressing facts as subject–predicate–object triples, e.g., `(Paris, capital_of, France)`. Each entity is identified by a URI, enabling globally unique identifiers across datasets. > * **SPARQL:** A query language for RDF stores. Analogous to SQL for relational databases, SPARQL supports pattern matching over triple stores, enabling complex multi-hop graph queries. > * **OWL (Web Ontology Language):** A formal language for defining ontologies — class hierarchies, property restrictions, and logical constraints over KG entities. OWL supports Description Logic inference: automated reasoning to derive implicit facts from explicit ones (e.g., if `Aspirin subClassOf NSAID` and `NSAIDs contraindicated_with RenalFailure`, then `Aspirin contraindicated_with RenalFailure`). Production-grade OWL reasoners used in practice include **Pellet**, **HermiT**, **ELK** (for the EL profile, scalable to billions of triples), and **Stardog** (commercial, with integrated SPARQL endpoint and reasoning). For most neuro-symbolic applications requiring OWL inference, ELK or Stardog are the practical starting points. > > **Open World Assumption (OWA):** KGs, unlike classical planning (Section 3.1), operate under the Open World Assumption: the absence of a triple does not mean it is false — it means it is unknown. This is both more realistic (the world is only partially known) and more demanding (reasoning must handle incompleteness explicitly). **Definition:** Multimodal architectures use neural networks to extract entities and relations from unstructured data, organize them into a structured Knowledge Graph (KG), and subsequently run graph-based symbolic reasoning to ensure accuracy, consistency, and explainability.(Pan et al., 2024)(Shen et al., 2025)(Hitzler & Sarker, 2022) Knowledge Graphs represent the world as a collection of entities and typed relations: (entity₁, relation, entity₂) triples. For example: ``` (Paris, capital_of, France) (Aspirin, treats, Headache) (BERT, is_a, Transformer) (Transformer, invented_by, Vaswani_et_al) ``` This explicit, queryable representation is ideal for downstream reasoning — graph traversal, path finding, link prediction, constraint checking. But constructing and maintaining a KG at scale requires processing vast amounts of unstructured text, images, and other data — which is where neural components excel. ### The Full Pipeline ``` Unstructured Sources (text, images, tables, PDFs) │ ▼ ┌──────────────────────────────────────────┐ │ Neural Information Extraction │ │ │ │ - Named Entity Recognition (NER) │ │ - Relation Extraction (RE) │ │ - Event Detection │ │ - Coreference Resolution │ │ - Schema-guided IE │ └──────────────────────────────────────────┘ │ (entity, relation, entity) triples ▼ ┌──────────────────────────────────────────┐ │ Knowledge Graph Construction │ │ │ │ - Entity linking (disambiguation) │ │ - Ontology alignment │ │ - Conflict resolution │ │ - Temporal fact management │ └──────────────────────────────────────────┘ │ Populated KG ▼ ┌──────────────────────────────────────────┐ │ Symbolic Reasoning over KG │ │ │ │ - SPARQL / Cypher queries │ │ - Logical rule application │ │ - Path-based reasoning │ │ - Consistency checking │ │ - OWL / Description Logic inference │ └──────────────────────────────────────────┘ │ ▼ Verified, Explainable Answers ``` ### Neural Knowledge Graph Completion Even after initial construction, KGs are inevitably incomplete — the **open world assumption** holds: absence of a triple does not mean it is false. **Knowledge Graph Completion (KGC)** is the task of predicting missing triples. Neural approaches to KGC learn **entity and relation embeddings** in a continuous space such that true triples score higher than false ones: * **TransE** (Bordes et al., 2013): Models relations as translations in embedding space: $\mathbf{h} + \mathbf{r} \approx \mathbf{t}$ for true triples $(h, r, t)$. * **RotatE** (Sun et al., 2019): Models relations as rotations in complex space, enabling representation of symmetric, antisymmetric, and transitive relations. * **ComplEx** (Trouillon et al., 2016): Uses complex-valued embeddings for asymmetric relations. * **R-GCN** (Schlichtkrull et al., 2018): Relational Graph Convolutional Networks apply message-passing GNNs to relational data. Each entity aggregates representations from its neighbors, weighted by relation type, enabling both entity classification and link prediction on KGs. R-GCN is the direct architectural predecessor to QA-GNN and DRAGON (described below), which apply the same relational message-passing principle with language model encoders.(Schlichtkrull et al., 2018) *Reference:* Schlichtkrull, Michael, et al. "Modeling Relational Data with Graph Convolutional Networks." *European Semantic Web Conference (ESWC)*, 2018. | Code: These neural embeddings can be used to score candidate triples before adding them to the KG, providing a neural "prior" for symbolic completion. ### Commonsense Knowledge Graphs The knowledge graphs discussed so far — medical KGs (SNOMED-CT), encyclopedic KGs (Wikidata), and organizational KGs — encode *factual* knowledge: verifiable statements about the world. A complementary class, **commonsense knowledge graphs**, encode the implicit everyday knowledge humans take for granted but rarely write down. **ConceptNet** (Speer et al., 2017) is a multilingual commonsense knowledge graph containing approximately **21 million edges** across millions of concepts in 83 languages at the time of the 2017 publication; the database has grown substantially since. Relations include `IsA`, `UsedFor`, `CapableOf`, `Causes`, `MotivatedByGoal`, and `HasPrerequisite`. Example: `(knife, UsedFor, cutting)`, `(cutting, HasPrerequisite, knife_is_sharp)`. **ATOMIC** (Sap et al., 2019) focuses on everyday *event* knowledge: given an event description, what are its preconditions, postconditions, and participant mental states? ATOMIC covers 300K events with 877K typed inferences across 9 relation types (xIntent, xNeed, xEffect, xReact, oEffect, etc.). **COMET** (Bosselut et al., 2019) trains a generative LM on ATOMIC to produce new commonsense inferences for arbitrary input events — effectively creating a **generative commonsense knowledge base** that can answer queries not present in the training data. **COMET-ATOMIC 2020** (Hwang et al., 2021) scales this to 1.3M everyday inference rules across 23 relation types, combining physical, social, and event-based knowledge. **Why this matters for neuro-symbolic AI:** Commonsense KGs are essential for AI agents operating in natural language domains. An agent planning a task like "prepare a meal for a guest with nut allergies" needs to know that peanut butter contains peanuts (factual KG), that nuts cause allergic reactions (ATOMIC causal knowledge), and that the guest's preference should be respected (social reasoning). No single KG class covers all of this. *References:* Speer, Robyn, Joshua Chin, and Catherine Havasi. "ConceptNet 5.5: An Open Multilingual Graph of General Knowledge." *Proceedings of AAAI*, 2017. | Data: Sap, Maarten, et al. "ATOMIC: An Atlas of Machine Commonsense for If-Then Reasoning." *Proceedings of AAAI*, 2019. | Data: Bosselut, Antoine, et al. "COMET: Commonsense Transformers for Automatic Knowledge Graph Construction." *Proceedings of ACL*, 2019. | Code: Hwang, Jena D., et al. "COMET-ATOMIC 2020: On Symbolic and Neural Commonsense Knowledge Graphs." *Proceedings of AAAI*, 2021. ### KG-Augmented Language Models (KGLM, RAG + KG) A significant recent research direction combines KGs with LLMs to reduce hallucination and improve factual accuracy: 1. **ERNIE (Tsinghua)** (Zhang et al., 2019) was the first major work to inject entity-level knowledge graph representations directly into BERT pretraining. It aligns token-level text representations with entity embeddings from Wikidata via a **Knowledge Aggregator** that fuses information from both levels at each transformer layer. ERNIE demonstrated significant improvements over vanilla BERT on knowledge-intensive tasks — particularly relation classification and entity typing — while remaining comparable on general NLP benchmarks. Specific gains depend on task and dataset.(Zhang et al., 2019) *Reference:* Zhang, Zhengyan, et al. "ERNIE: Enhanced Language Representation with Informative Entities." *Proceedings of ACL*, 2019. | Code: 2. **Retrieval-Augmented Generation (RAG) + KG:** At inference time, the LLM's query is used to retrieve relevant subgraphs from a KG. The retrieved triples are included in the LLM's context, providing factual grounding for generation. 3. **KGLM** (Logan et al., 2019): A language model conditioned on a knowledge graph. At each token generation step, the model can choose to copy entities and relations from the KG rather than generating them from scratch, dramatically reducing factual errors. 4. **GreaseLM** (X. Zhang et al., 2022): Fuses LM and KG representations at each layer of the transformer, enabling rich bidirectional interaction between language understanding and structured knowledge. 5. **GraphRAG** (Edge et al., 2024): Builds a knowledge graph from the entire document corpus at indexing time, then at query time retrieves both relevant documents *and* relevant KG subgraphs — enabling answers that require synthesizing information across many documents. Unlike standard RAG (which retrieves local document passages), GraphRAG uses the KG to perform global reasoning over the entire corpus. On summarization and question-answering benchmarks requiring multi-hop reasoning, GraphRAG significantly outperforms standard RAG, with the KG providing the cross-document bridging that flat retrieval cannot. *Reference:*\ Edge, Darren, et al. "From Local to Global: A Graph RAG Approach to Query-Focused Summarization." *arXiv preprint* arXiv:2404.16130 (2024). | Code: 6. **QA-GNN** (Yasunaga et al., 2021) introduces a **working memory** over a KG subgraph retrieved for each question. A relevance scoring module identifies which KG entities are relevant to the question and answer candidates; a Graph Neural Network propagates information within the relevant subgraph; the resulting KG representations are jointly attended with the language model outputs. QA-GNN outperforms GreaseLM on CommonsenseQA and MedQA. *Reference:* Yasunaga, Michihiro, et al. "QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering." *Proceedings of NAACL*, 2021. | Code: 7. **DRAGON** (Yasunaga et al., 2022) extends QA-GNN with **deep bidirectional language-knowledge pretraining**: the LM and KG representations are fused at every layer via cross-modal attention, creating a single unified representation that jointly encodes text context and symbolic knowledge. Pre-trained on Wikipedia and Wikidata, DRAGON achieves state of the art on CommonsenseQA, OpenBookQA, and MedQA without task-specific fine-tuning of the fusion architecture. *Reference:* Yasunaga, Michihiro, et al. "Deep Bidirectional Language-Knowledge Graph Pretraining." *Advances in Neural Information Processing Systems (NeurIPS)* 35 (2022). | Code: **Practical impact:** On medical question answering benchmarks (MedQA, BioASQ), KG-augmented models consistently outperform pure neural baselines by 5–15% absolute accuracy, with the additional benefit of providing citation trails through the KG that support explainability requirements in clinical settings.(Pan et al., 2024) ### Case Study: Clinical Decision Support A hospital's clinical decision support system integrates: 1. **Neural NLP pipeline:** Extracts diagnoses, medications, procedures, and lab results from unstructured clinical notes using fine-tuned BERT-variant models. 2. **Medical KG:** Constructed from SNOMED-CT, RxNorm, and UMLS, enriched with extracted triples. Contains 5M+ entities and 30M+ relations. 3. **Symbolic reasoning layer:** * Drug-drug interaction checking via KG path queries. * Contraindication detection using OWL ontology inference over patient conditions and prescribed medications. * Treatment pathway recommendation via graph traversal over clinical guideline KG. 4. **Explanation generation:** For each alert or recommendation, the system traverses the KG paths that triggered it and generates a human-readable explanation: "Alert: Patient has Condition X, which contraindicates Drug Y via interaction Z (see CPIC guideline \[link])." The neural component provides the breadth of perceptual coverage (processing thousands of unstructured notes); the symbolic component provides the correctness guarantees and explainability required for clinical trust.(Shen et al., 2025) *** ## References 6. Pan, Shirui, et al. "Unifying Large Language Models and Knowledge Graphs: A Roadmap." *IEEE Transactions on Knowledge and Data Engineering* (2024). 7. Shen, Yongliang, et al. "Multi-Source Knowledge Graph Reasoning for Biomedical Question Answering." *arXiv preprint* arXiv:2502.11269 (2025). 8. Hitzler, Pascal, and Md Kamruzzaman Sarker, eds. *Neuro-Symbolic Artificial Intelligence: The State of the Art*. IOS Press, 2022. 9. Bordes, Antoine, et al. "Translating Embeddings for Modeling Multi-Relational Data." *NeurIPS 2013*. 10. Sun, Zhiqing, et al. "RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space." *ICLR 2019*. | Code: 10a. Trouillon, Théo, et al. "Complex Embeddings for Simple Link Prediction." *Proceedings of the 33rd International Conference on Machine Learning (ICML)*, 2016. | Code: 10b. Schlichtkrull, Michael, et al. "Modeling Relational Data with Graph Convolutional Networks." *European Semantic Web Conference (ESWC)*, 2018. | Code: 11. Logan, Robert, et al. "Barack's Wife Hillary: Using Knowledge Graphs for Fact-Aware Language Modeling." *ACL 2019*. 12. Zhang, Xikun, et al. "GreaseLM: Graph REASoning Enhanced Language Models." *ICLR 2022*. | Code: 13. Edge, Darren, et al. "From Local to Global: A Graph RAG Approach to Query-Focused Summarization." *arXiv preprint* arXiv:2404.16130 (2024). | Code: 14. Zhang, Zhengyan, et al. "ERNIE: Enhanced Language Representation with Informative Entities." *Proceedings of ACL*, 2019. | Code: 15. Yasunaga, Michihiro, et al. "QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering." *Proceedings of NAACL*, 2021. | Code: 16. Yasunaga, Michihiro, et al. "Deep Bidirectional Language-Knowledge Graph Pretraining." *Advances in Neural Information Processing Systems (NeurIPS)* 35 (2022). | Code: 17. Speer, Robyn, Joshua Chin, and Catherine Havasi. "ConceptNet 5.5: An Open Multilingual Graph of General Knowledge." *Proceedings of AAAI*, 2017. | Data: 18. Sap, Maarten, et al. "ATOMIC: An Atlas of Machine Commonsense for If-Then Reasoning." *Proceedings of AAAI*, 2019. | Data: 19. Bosselut, Antoine, et al. "COMET: Commonsense Transformers for Automatic Knowledge Graph Construction." *Proceedings of ACL*, 2019. | Code: 20. Hwang, Jena D., et al. "COMET-ATOMIC 2020: On Symbolic and Neural Commonsense Knowledge Graphs." *Proceedings of AAAI*, 2021. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://neurosymbolicai.gitbook.io/neuro-symbolic-ai-in-practice/part-iii-core-approaches/chapter-4/4-3-hybrid-architectures/4-3-knowledge-graphs.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.