ftisiot ideas about food and life
who talks suggestions soft json-pg

Put PostgreSQL at the center of your (meta)data universe!

Presented October 11, 2022 at PG Conf EU

Slides Website Venue

Extract metadata from your existing data tools and build a queryable graph interface in PostgreSQL®

Intro image

Abstract

How do we answer questions about GDPR, data lineage, impact assessment, security scans with a SQL query? If all data assets are in PostgreSQL®, well… problem solved! We can hit the metadata views and extract what’s needed. But what happens when data is scattered across several platforms using different technologies?

This talk is about the hidden treasure stored in the metadata: starting by defining use cases, we’ll then focus on how we can design a system, based on PostgreSQL and some graph theory, that collects, merges and exposes metadata coming from a variety of tools.

If you want to programmatically keep control over your ever increasing data assets using a familiar technology and SQL, this session is for you!

Useful Links

  • Graph theory
  • Aiven Metadata Parser
  • Aiven Metadata Parser Blog
  • DOT language
  • GML language
  • JSONB in PostgreSQL
  • Recursive Queries Example PostgreSQL
  • Recursive Queries Example PostgreSQL 14

Francesco Tisiot

Mastodon Francesco comes from Verona, Italy and works as a Senior Developer Advocate at Aiven. With his many years of experience as a data analyst, he has stories to tell and advice for data-wranglers everywhere. Francesco loves sharing knowledge with others as a speaker and writer, and is on a mission to defend the world from bad Italian food!

Story logo

© 2023