dbt
The dbt integration updates metadata from your dbt manifest into the data lineage graph.
Web App
Fields
Field | Value | Example |
---|---|---|
source | The name of the source, see sources | my-source |
Namespace | Namespace for the connection, see namespaces | default |
Python Library
The dbt integration can be run as a standalone python library to extract data lineage from a dbt manifest. If you want more information about working with dbt cloud see the dbt cloud integration.
The library is available via pip
pip install grai_source_dbt
More information about the API is available here.
Example
The library is split into a few distinct functions but if you only wish to extract nodes/edges from dbt you can do so from the base DbtIntegration class.
from grai_source_dbt import DbtIntegration
from grai_schemas.v1.source import SourceV1
source = SourceV1(name="my-source", type="my-type")
integration = DbtIntegration(source=source, namespace="dbt", manifest_data="my-manifest.json")
nodes, edges = integration.get_nodes_and_edges()
dbt Manifest Version Support
Manifest Version | Status |
---|---|
v1 | ✅ |
v2 | ✅ |
v3 | ✅ |
v4 | ✅ |
v5 | ✅ |
v6 | ✅ |
v7 | ✅ |
v8 | ✅ |
v9 | ✅ |
v10 | ✅ |
v11 | ✅ |