The Flat File integration is used to upload a single flat file (like csv, parquet, and feather) into the data lineage graph.
|Namespace||Namespace for the connection, see namespaces||default|
Install Flat File Grai package with pip
pip install grai-source-flatfile
Connecting & Syncing
The integration comes equipped with the client library already but we will need a python terminal or Jupyter Notebook to execute a few commands to establish a connection and begin querying the server.
Spin up your favorite python terminal then:
import os from grai_source_flatfile.base import update_server
For now we will use the default user credentials though you are free to create a new user / api keys from the server admin interface at http://localhost:8000/admin (opens in a new tab).
client = ClientV1("localhost", "8000", username="firstname.lastname@example.org", password="super_secret")
Now we can update the server with data from any postgres source. In order to do so you will need to pass credentials and namespace into the update_server function. Namespace is used to uniquely identify the nodes and when used consistently will allow you to add to the node from any source.
update_server(client, [flat_file fields])