This article comes at the perfect time, truly articulating the core issue. Your consistent emphasis on ontology, semantics, and context as essential scaffolding for domain intelligence is so spot on. It clearly explains why so many "AI" projects are disconected black boxes. This clarity is invaluable.
Thanks Mark! Short answer: none of the tools do the hard part. You still have to do the upfront work of defining entities and getting those definitions right with the business. Once that’s solid, the stack and use case dictate which tools you use to operationalize it.
Semantics: Not just metadata on columns. I’m seeing teams encode domain definitions in:
dbt Semantic Layer / MetricFlow or a headless semantic layer like Cube to define entities and metrics once and reuse them everywhere (BI, apps, agents).
A catalog / glossary (e.g., Atlan) to tie business terms to concrete models, metrics and owners.
Context: Not just “dump docs into RAG.” In practice it’s mostly:
Letting models and agents call the semantic/metrics layer directly for structured answers (e.g., “comp sales by store last 13 months”).
Optionally using RAG for docs (definitions, SOPs, runbooks), but anchored to the same ontology so text points back to the same entities and metrics.
The tools help you apply and maintain the domain model; they don’t replace the initial, meticulous domain work.
Incredibly helpful! For non-enterprise, I loved how Omni is approaching the semantic layer / context by letting users add notes for each dimension: https://www.youtube.com/watch?v=BGXlAk9AIjE
I also read a snowflake paper (maybe a year ago now) about using a YAML file in the data warehouse to add the context, but also provide example SQL statments for common questions to "train" the AI.
I'm new to the technical side, so just trying to figure out non-enterprise methods to lay a strong AI foundation
This article comes at the perfect time, truly articulating the core issue. Your consistent emphasis on ontology, semantics, and context as essential scaffolding for domain intelligence is so spot on. It clearly explains why so many "AI" projects are disconected black boxes. This clarity is invaluable.
Great read! How are you seeing powerful and robust semantic and context layers actually applied?
Semantics: just metadata added or using something like dbt semantic layer product?
Context: just adding / RAGing in docs etc?
Thanks Mark! Short answer: none of the tools do the hard part. You still have to do the upfront work of defining entities and getting those definitions right with the business. Once that’s solid, the stack and use case dictate which tools you use to operationalize it.
Semantics: Not just metadata on columns. I’m seeing teams encode domain definitions in:
dbt Semantic Layer / MetricFlow or a headless semantic layer like Cube to define entities and metrics once and reuse them everywhere (BI, apps, agents).
A catalog / glossary (e.g., Atlan) to tie business terms to concrete models, metrics and owners.
Context: Not just “dump docs into RAG.” In practice it’s mostly:
Letting models and agents call the semantic/metrics layer directly for structured answers (e.g., “comp sales by store last 13 months”).
Optionally using RAG for docs (definitions, SOPs, runbooks), but anchored to the same ontology so text points back to the same entities and metrics.
The tools help you apply and maintain the domain model; they don’t replace the initial, meticulous domain work.
Incredibly helpful! For non-enterprise, I loved how Omni is approaching the semantic layer / context by letting users add notes for each dimension: https://www.youtube.com/watch?v=BGXlAk9AIjE
I also read a snowflake paper (maybe a year ago now) about using a YAML file in the data warehouse to add the context, but also provide example SQL statments for common questions to "train" the AI.
I'm new to the technical side, so just trying to figure out non-enterprise methods to lay a strong AI foundation