Snowflake App Runtime has long suffered from a critical visibility problem: once a Next.js app is deployed, it vanishes from the platform's native lineage graphs. While Snowsight tracks tables and views perfectly, it stops dead at the application boundary, forcing developers to grep through source code to understand dependencies. A recent experiment by developer takimiko_gohan demonstrates how to fix this by pushing OpenLineage events directly into Snowflake's External Lineage API, effectively making App Runtime APIs and screens visible nodes in the metadata graph.

The Missing Link in Data Governance

The core issue is that Snowflake's built-in lineage capabilities are database-centric, not application-centric. When an API route queries a table, that relationship is invisible to the lineage graph. This experiment leverages Snowflake's External Lineage API, which has been generally available for some time, to ingest lineage data in the open OpenLineage format. By treating API endpoints and frontend screens as lineage nodes, the author creates a complete map from raw data to the user interface. This approach relies on Horizon Context, a feature currently in preview as of October 2026, which extends Snowflake's Horizon Catalog into a pluggable metadata layer.

Automating Extraction with SQLGlot

The pipeline works in three distinct steps: extract, analyze, and send. First, a Python script uses regex to pull SQL template literals from Next.js App Router files located in app/api/. It handles both standard constants and arrow functions, inferring HTTP methods from exported function names. Next, the extracted SQL is preprocessed to replace template variables like ${db} with actual values and normalize bind parameters. The cleaned SQL is then parsed by sqlglot, a powerful SQL parser that resolves fully qualified table names and traces column-level lineage through joins, CASE expressions, and aggregates. This step is crucial for moving beyond table-level granularity to understanding exactly which columns feed into specific API responses.

Registering Nodes in the Lineage Graph

Once parsed, the system constructs an OpenLineage COMPLETE event and sends it to Snowflake's /api/v2/lineage/external-lineage endpoint. The event defines two layers of relationships: table-to-API and API-to-screen. For the first layer, inputs are the source tables, and outputs are the API endpoints, enriched with the columnLineage facet. Snowflake currently honors only datasetType and columnLineage facets, silently ignoring others like documentation or ownership. The second layer maps frontend components to their called APIs, registering screens as downstream nodes. Write operations, such as PUT requests for master data, are correctly positioned upstream of the target tables, reflecting their data-writing role.

Practical Impact on Change Management

The result is a lineage graph where changing a single column, like CUSTOMERS.CUSTOMER_NAME, immediately reveals every affected API and screen. Using Snowflake's GET_LINEAGE function, developers can query downstream dependencies to see that both /api/customers and /api/orders rely on that column. This eliminates the guesswork involved in schema refactoring. Instead of manually auditing code, teams can rely on the graph to identify breaking changes before deployment. The visual representation in Snowsight now shows external nodes labeled as "External," bridging the gap between data engineering and application development.

Key Takeaways

  • Snowflake's External Lineage API is GA and accepts OpenLineage events, offering a robust workaround for App Runtime visibility gaps.
  • sqlglot enables precise column-level lineage extraction from Next.js TypeScript files, handling complex SQL structures like joins and CASE statements.
  • Horizon Context, currently in preview, provides the pluggable metadata layer necessary for integrating custom application nodes into the broader catalog.
  • Only datasetType and columnLineage facets are currently honored by Snowflake, limiting the richness of metadata that can be attached to external nodes.

The Bottom Line

This experiment proves that you don't need to wait for Snowflake to natively support App Runtime lineage. By automating OpenLineage ingestion, teams can achieve full-stack visibility today, turning opaque application code into actionable metadata.