Event

<p>Apache DataFusion is emerging as a powerful open-source foundation for building interoperable data systems, thanks to its strongly modular design, Arrow-native execution model, and growing ecosystem of extension libraries. In this talk, we'll explore our contributions to the DataFusion ecosystem—most notably <strong><a href="https://github.com/datafusion-contrib/datafusion-federation">DataFusion Federation</a></strong> for cross-database query execution and <strong><a href="https://github.com/datafusion-contrib/datafusion-table-providers">DataFusion Table Providers</a></strong> that connect DataFusion to a wide range of backends.</p> <p>We'll show how we use these components to federate queries to databases such as <strong>TiDB</strong> and <strong>InfluxDB 2</strong>, and how this fits into a broader data fabric/API generation work we're doing at Twintag. We'll also discuss our work on Arrow-native interfaces, including an <strong><a href="https://github.com/datafusion-contrib/datafusion-flight-sql-server">Arrow Flight SQL Server implementation for DataFusion</a></strong> and a prototype Flight SQL endpoint for TiDB, which together enable a fully Arrow-based pipeline spanning query submission, execution, and federated dispatch.</p> <p>The session highlights practical patterns for building distributed data infrastructure using open libraries rather than monolithic systems, and offers a look at where Arrow and DataFusion are headed as shared interoperability layers for modern databases.</p>