Event

<p>Apache Airflow is the most popular Data Workflow Orchestrator - developed under the Apache Software Foundation umbrella. We have 120+ Python distributions in our rep, and we often release ~ 100 of them every two week. </p> <p>All those distributions are built from a single monorepo. </p> <p><code>[jarekpotiuk:~/code/airflow] find . -name 'pyproject.toml' | wc 120 120 4248</code></p> <p>This had always posed a lot of challenges and we had a lot of tooling to make it possible, however with the recent development of Python Packaging tools, multipel Packaging PEPs implemented, and with new wave of tools such as <code>uv</code> and <code>prek</code>, our setup is finally manageable and we removed 1000s of line of custom code we wrote before after we applied <code>uv</code> workspaces, switched to <code>prek</code>, started using inline script metadata.</p> <p>It's a breeze to have monorepo now. This talk explains how.</p> <p>Bonus content. If you know the differences between dynamically and statically linked libraries in C and other languages, or used NPM - you might recognise the need of being able to use different versions of the same library in the same system. It's not possible in Python. Or is it? </p> <p>We've figured out a way to eat cake and have it too - and we have "statically linked" libraries in Python. How did we do it? </p> <p>You will find out how from the talk.</p>