Polars & pandas at PyCon DE & PyData

Zero-Copy or Zero-Speed? The hidden overhead of PySpark, Arrow & SynapseML for inference

Petar Ilijevski · 2026

"Zero-copy" data transfer promises free communication between Spark's JVM and Python workers, but at 6 billion rows daily, the reality is far more complex. This session explores the low-level mechanics of distributed...

Workshop: What do we still need to learn? [no-video]

Paula Gonzalez Avalos, Sebastian Neubauer, Dr. Kristian Rother · 2026

AI agents are reshaping how we work, how we code, how we search — and quietly, how we think about ourselves as practitioners. The ground is shifting. And nobody has all the answers yet. That is exactly why we are...

Wetterdienst: Fast, Unified Access to Open Weather Data with Polars

Benjamin · 2026

Weather and environmental data power analytics, ML, and operations—but APIs differ wildly and data prep is slow. Wetterdienst is a Python library that provides a unified, Polars‑first interface to multiple weather...

To nest, or not to nest? Nested data types in Polars with big data

Daniel Finnan · 2026

Do you find yourself weighing up the pros and cons of using nested types in the Polars library - pondering whether you should encode your variables in structures using lists, arrays or opt for a flat format without...

State of In-Browser ML: WebAssembly, WebGPU, and the Modern Stack

Oleh Kostromin, Iryna Kondrashchenko · 2026

What if you could run real data/ML workflows right in your browsers - sandboxed, with no installation or sending your data anywhere? Such an approach would have tons of benefits: it is easy to distribute, safer by...

SQL is Dead, Long Live SQL: Engineering reliable analytics agent from scratch

Mehdi Ouazza, Dumky de Wilde · 2026

Is it still worth learning SQL in 2026, or can we just "chat" with our data? This hands-on tutorial explores that exact question by pushing Text-to-SQL to its absolute limits. This won't be just happy paths; we will...

Simulating the World using SimPy: A practical Example

Niklas · 2026

Modern systems are complex - and testing them in real environments is often expensive, risky, or simply not reproducible. Simulation is a practical way to explore behavior under controlled conditions: run scenarios,...

Ship Data with Confidence: Declarative Validation for PySpark & Pandas

Ryan Sequeira · 2026

Tired of data quality issues crashing your PySpark and Pandas pipelines? This talk introduces [dataframe-expectations](https://github.com/getyourguide/dataframe-expectations), a lightweight, open-source library for...

Scaling Data Processing for Training Workloads at DeepL Research with Rust

Jonas Dedden, Johanna Goergen · 2026

This talk will detail how we used Rust to solve a number of resource utilization inefficiencies while scaling data pre-processing to a petabyte scale and enable next-generation model training at DeepL. Besides other...

Roll for Architecture: DungeonPy – A D&D Companion as Server + Thin Clients

Francesco Conte · 2026

### **DungeonPy** – an interactive Dungeon&Dragons app for remote campaigns As a matter of fact, tabletop RPGs are secretly distributed systems: one canonical world state, many clients, lossy links (players), and...

Rediscovering single-node processing: When does it make sense to move from Spark to Polars?

Jonas Böer · 2026

As data engineers, we are used to spinning up a Spark Cluster every time we want to do data processing and handle the overhead that comes with using such a mighty framework. But is this really necessary? In this talk...

PyTorch and CPU-GPU Synchronizations

Tomas Ruiz · 2026

CPU–GPU synchronizations are a subtle performance killer in PyTorch: they block the host, prevent the CPU from running ahead, and create GPU idle gaps. This talk explains what host-device synchronization is, how it’s...

Open Table Formats in the Wild™ - Reloaded: Vortexing Ducks over Floating Icebergs

Franz Wöllert · 2026

Open table formats have *almost* freed us from vendor lock-in. They form a critical building block of the modern, composable data stack. The most prominent open table format is Apache Iceberg - not only because of...

Metashade: Compilerless Immediate-Mode Shader Generation in Pure Python

Pavlo Penenko · 2026

Discover how to build a GPU shader generator in pure Python, without having to write a compiler. We start by discussing how Pythonic embedded domain-specific languages (EDSLs) can help address the common challenges...

Making my Apache Spark™ talk more interesting using AI

Celeste Horgan · 2026

Writing talks is hard, but being a good conference speaker is even harder. Resultantly, this talk is recursive: I'll take a talk previously written for a London data science meetup on using Apache Spark and Apache...

How to Search Through 800 Billion Records in Real Time

Mirano Tuk, Filip Bacic · 2026

Large-scale distributed systems rarely produce clean data streams. In practice, hundreds of services continuously emit overlapping updates, retries, corrections, and partial state. Turning that constant stream of...

Heat: scaling the Python scientific stack to HPC systems

Claudia Comito, Thomas Saupe · 2026

Python’s scientific stack (NumPy/SciPy) is often confined to single-node execution. When datasets exceed local memory, researchers face a steep learning curve, typically choosing between complex manual distribution...

From Struggling to Mastery: A Practical Guide to Data Pipeline Operations

Akif Cakir · 2026

How mature are your data pipeline operations? A Roadmap to Operational Excellence. Data teams often struggle to scale their pipeline operations, trapped in a cycle of manual fixes and reactive fire-fighting. But what...

From Row-Wise to Columnar: Speeding Up PySpark UDFs with Arrow and Polars

Aimilios Tsouvelekakis · 2026

Python UDFs often become the slowest part of PySpark pipelines because they run row-by-row and pay a high cost crossing the JVM↔Python boundary. Spark’s Arrow-backed execution changes that cost model by moving data...

From Prompt to Production: How to use AI Code Assistants for Python Data Systems

Serhii Sokolenko · 2026

**Code-generating LLMs have matured** to the point where they can reliably scaffold **data pipelines and data agents**, when used in a **supervised, engineering-first workflow**. This tutorial demonstrates how to...

From Hard Problems to Proven Solutions: Solving Decision Problems with Gurobi

Silke Horn · 2026

Join us as we demonstrate how to formulate and solve hard decision problems with Gurobi. You’ll learn practical modeling techniques that integrate naturally with NumPy, SciPy.sparse, and pandas. We’ll show how...

Fight your garbage data: implementation of a pythonic data quality monitoring framework in PySpark

Rostislaw Krassow, Joshua Finger · 2026

The timeless phrase “garbage in, garbage out” is even more important today with the growing usage of non-deterministic generative neuronal networks, which amplifies the effect of bad data quality. This presentation...

Exploring Germany's Urban Geography with Census and OpenStreetMap Data

Travis Hathaway · 2026

When conducting studies of the urban form, an important resource many researchers turn to is the massive OpenStreetMap dataset. But, as extensive as this dataset is, it lacks one very important aspect about the...

Dynamic Knowledge Graphs

Jakob Leander Müller · 2026

Traditional RAG systems struggle to understand holistic connections in distributed, constantly changing knowledge sources that characterize real-world organizations. While document-based approaches using vector...

Demystifying Parallel Programming in Python: from CPU to quantum processors, including GPU and TPU

Gaël Pegliasco · 2026

This talk provides a beginner-friendly overview of Python’s parallel programming ecosystem. You’ll discover the key libraries and techniques—JIT compilation, multithreading, multiprocessing, distributed computing,...

Building Trust in Your Data Pipelines with Observability

Stefan Dienst · 2026

In the daily work of a data engineer, building new data pipelines often takes priority, while maintaining them and ensuring their correctness becomes an afterthought. This focus can quickly turn into a pitfall:...

Building reliable data pipelines with polars and dataframely

Oliver Borchert, Andreas Albert · 2026

If you have worked with real-world data before, you know that processing it can be challenging. Data often comes scattered across tables, in inconsistent encodings, with duplicated rows and is generally dirty. In...

Black Hole Stars: An Astronomical Mystery (Mostly) Solved with NumPyro and JAX

Raphael Hviding · 2026

The James Webb Space Telescope has revealed a mysterious population of "Little Red Dots": extremely distant objects that have upended our understanding of the early Universe. However, revealing the true nature of...

Beyond Kafka and S3: Python Data Pipelines with HTTP-Native Bytestreams

Johannes Dröge · 2026

Real-time bytestreams between systems in different organizations or secured environments, whether for batch dataset delivery or continuous streaming, are surprisingly hard. Traditional solutions fall short: message...

Array-Oriented Programming in Python: Libraries, Techniques, and Trade-offs

Iason Krommydas · 2026

Python has become the dominant language for scientific computing and data science, largely due to powerful array libraries that enable high-performance numerical computation. This tutorial introduces array-oriented...

Zero Code Change Acceleration: familiar interfaces and high performance ▶ Recording 📝 Transcript

Tim Head · 2025

The PyData ecosystem is home to some of the best and most popular tools for doing data-science. Every data-scientist alive today has used pandas and scikit-learn and even Large Language Models know how to use them!...

Using Causal thinking to make Media Mix Modeling ▶ Recording 📝 Transcript

Carlos Trujillo · 2025

In today's data-driven landscape, understanding causal relationships is essential for effective marketing strategies. This talk will explore the link between Bayesian causal thinking and media mix modeling, utilizing...

Topological data analysis: How to quantify "holes" in your data and why? ▶ Recording 📝 Transcript

Ondrej Draganov · 2025

Do you need to compare sets of points in a plane? Identify a potential cyclic event in high-dimensional time series data? Find the second or the third highest peak of a noisily sampled function? Topological data...

The earth is no longer flat - introducing support for spherical geometries in Spherely and GeoPandas ▶ Recording 📝 Transcript

Joris Van den Bossche · 2025

The geometries in GeoPandas, using the Shapely library, are assumed to be in projected coordinates on a flat plane. While this approximation is often just fine, for global data this runs into its limitations. This...

Streamlining the Cosmos: Pythonic Workflow Management for Astronomical Analysis ▶ Recording 📝 Transcript

Raphael Hviding · 2025

Astronomical surveys are growing rapidly in complexity and scale, necessitating accurate, efficient, and reproducible reduction and analysis pipelines. In this talk we explore Pythonic workflow managers to streamline...

Serverless Orchestration: Exploring the Future of Workflow Automation ▶ Recording 📝 Transcript

Tim Bossenmaier · 2025

Orchestration is a typical challenge in the data engineering world. Scheduling your data transformation jobs via CRON-jobs is cumbersome and error-prone. Furthermore, with an increasing number of jobs to manage it...

Scalable Python and SQL Data Engineering without Migraines ▶ Recording 📝 Transcript

Dirk Jung · 2025

This session is for data and ML engineers with a basic understanding of data engineering and Python. It shows how to easily use Python code in Snowflake Notebooks to create data pipelines. By the end, you’ll know how...

Responsible AI with fmeval - an open source library to evaluate LLMs ▶ Recording 📝 Transcript

Mia Chang · 2025

The term "Responsible AI" has seen a threefold increase in search interest compared to 2020 across the globe. As developers, the questions like "How can we build large language model-enabled applications that are...

Power up your Polars code with Polars extention ▶ Recording 📝 Transcript

Cheuk Ting Ho · 2025

While Polars is written in Rust and has the advantages of speed and multi-threaded functionalities., everything will slow down if a Python function needs to be applied to the DataFrame. To avoid that, a Polar...

Optimizing in the Python Ecosystem – Powered by Gurobi ▶ Recording 📝 Transcript

Silke Horn · 2025

Join us as we explore integrating Gurobi and prescriptive analytics into your Python ecosystem. In this session, you’ll discover model-building techniques that leverage NumPy and SciPy.sparse as well as the data...

How Narwhals is silently bringing pandas, Polars, DuckDB, PyArrow, and more together ▶ Recording 📝 Transcript

Marco Gorelli · 2025

If you were writing a data science tool in 2015, you'd have ensured it supported pandas and then called it a day. But it's not 2015 anymore, we've fast-forwarded to 2025. If you write a tool which only supports...

High-performance dataframe-agnostic GLMs with glum ▶ Recording 📝 Transcript

Martin Stancsics · 2025

Generalized linear models (GLMs) are interpretable, relatively quick to train, and specifying them helps the modeler understand the main effects in the data. This makes them a popular choice today to complement other...

Forecast of Hourly Train Counts on Rail Routes Affected by Construction Work ▶ Recording 📝 Transcript

Sebastian Folz, Dr Maren Westermann · 2025

Construction work in national railroad networks often disrupts train traffic, making it vital to estimate hourly train numbers for effective re-routing. Traditionally managed by humans, this process has been...

Extending Python with Rust, Mojo, Cuda and C and building packages ▶ Recording 📝 Transcript

Ruben Arts, Wolf Vollprecht · 2025

We all love Python - but we especially love it for its unique ability as a glue language. In this talk we will show a number of ways of extending Python: using Rust, C and Cython, C++, CUDA and Mojo! We will use the...

Distributed file-systems made easy with Python's fsspec ▶ Recording 📝 Transcript

Einat Orr, Barak Amar · 2025

The cloud native revolution has impacted all aspects of engineering, and data engineering is not exempt. One of the ongoing challenges in the data engineering world remains the local and distributed cloud native...

Dataframely — A declarative, 🐻‍❄️-native data frame validation library ▶ Recording 📝 Transcript

Daniel Elsner, Oliver Borchert · 2025

Understanding the structure and content of data frames is crucial when working with tabular data — a core requirement for the robust pipelines we build at QuantCo. Libraries such as `pandera` or `patito` already...

BayBE: A Bayesian Back End for Experimental Planning in the Low-To-No-Data Regime ▶ Recording 📝 Transcript

Martin Fitzner, Alexander Hopp, Adrian Šošić · 2025

From coffee machine settings to chemical reactions to website AB testing - iterative make-test-learn cycles are ubiquitous. The [Bayesian Back End](https://emdgroup.github.io/baybe/stable/) (BayBE) is an open-source...

Analyze data easily with duckdb - and the implications on data architectures ▶ Recording 📝 Transcript

Matthias Niehoff · 2025

duckdb is increasingly becoming a universal tool for accessing and analyzing data. In this talk I will show with slides and live demo what duckdb is capable of and will dive deeper in how it will influence modern...

A11y Need Is Love (But Accessible Docs Help Too) ▶ Recording 📝 Transcript

Smera Goel · 2025

Accessible documentation benefits everyone, from developers to end users. Using the [PyData Sphinx Theme](https://pydata-sphinx-theme.readthedocs.io/en/stable/) as a case study, this talk dives into common...

3 Ways to Speed up Your Regression Modeling in Python ▶ Recording 📝 Transcript

Alexander Fischer · 2025

Linear Regression is the workhorse of statistics and data science. Some data scientists even go as far and argue that "linear regression is all you need". In this talk, we will introduce three ways to run regression...

Whispered Secrets: Building An Open-Source Tool To Live Transcribe & Summarize Conversations ▶ Recording 📝 Transcript

John Sandall · 2024

Are you secretly a spy and/or passionate about open-source? Maybe you don't trust a cloud-hosted service with your highly classified information, or perhaps you like to build things for yourself. In this...

The Struggles We Skipped: Data Engineering for the TikTok Generation ▶ Recording 📝 Transcript

Anuun, Hiba Jamal · 2024

In a world increasingly embracing Python, plug-and-play solutions and AI-generated code, our generation growing up with these advancements may not fully grasp the challenges faced by our predecessors. Meanwhile, data...

The pragmatic Pythonic data engineer ▶ Recording 📝 Transcript

Robson Junior · 2024

Learn to make practical decisions in data engineering with Python's vast ecosystem. Avoid blindly following market guidelines and consider the reality of your situation for better performance and architecture.

The evolution of Feature Stores ▶ Recording 📝 Transcript

Olamilekan Wahab · 2024

Feature Stores have become an important component of the machine learning lifecycle. They have been particularly pivotal in bridging the gap between data engineering and machine learning workflows(experimentation,...

Prescriptive Analytics in the Python Ecosystem with Gurobi ▶ Recording 📝 Transcript

Robert Luce · 2024

Join us as we guide you through integrating Gurobi and prescriptive analytics into your greater Python ecosystem. We’ll demonstrate model-building patterns based on NumPy and SciPy.sparse data structures and explore...

Polars and Time Series: what it can do, and how to overcome any limitation ▶ Recording 📝 Transcript

Marco Gorelli · 2024

Time series analysis is ubiquitous in applied data science because of the value it delivers. In order to do effective time series analysis, you need to know your tools well. Polars has excellent built-in time series...

Pandas + Dask DataFrame 2.0 - Comparison to Spark, DuckDB and Polars ▶ Recording 📝 Transcript

Florian Jetter, Patrick Hoefler · 2024

Dask is a library for distributed computing with Python that integrates tightly with pandas. Historically, Dask was the easiest choice to use (it’s just pandas) but struggled to achieve robust performance (there were...

Mostly Harmless Fixed Effects Regression in Python with PyFixest ▶ Recording 📝 Transcript

Alexander Fischer · 2024

This session introduces PyFixest, an open source Python library inspired by the "fixest" R package. PyFixest implements fast routines for the estimation of regression models with high-dimensional fixed effects,...

Keynote - Safe Space or Trap? Creating Software like DuckDB in Academic Institutions ▶ Recording 📝 Transcript

Hannes Mühleisen · 2024

DuckDB is an in-process analytical data management system. DuckDB is free and open source and rather popular. It is one of the fastest growing data system to date, especially in the Python ecosystem. DuckDB was...

How Python helped us uncover secrets of protein motion ▶ Recording 📝 Transcript

Zoran Štefanić, Boris Gomaz · 2024

This presentation will give an overview of the scientific project that focuses on understanding how proteins move and function. Along the way a very large collection of Python tools was used, and on top of them our...

Going beyond Parquet's default settings – be surprised what you can get ▶ Recording 📝 Transcript

Uwe L. Korn · 2024

Apache Parquet has become the de facto format for storing tabular (DataFrame) data on disk. This is done through universal compression and efficient knowledge of the stored data structure. As part of this talk, we...

Data valuation for machine learning ▶ Recording 📝 Transcript

Miguel de Benito Delgado, Kristof Schröder · 2024

Data valuation techniques compute the contribution of training points to the final performance of machine learning models. They are part of so-called data-centric ML, with immediate applications in data engineering...

Boost your Data Science skills with the new Python in Excel ▶ Recording 📝 Transcript

Valerio Maggio · 2024

Python in Excel is the new integration created by Microsoft that brings Python programming directly into Excel workbooks, for advanced data analytics. With Python in Excel, it is now possible to embed Python code...

Analyzing COVID-19 Protest Movements: A Multidimensional Approach Using Geo-Social Media Data ▶ Recording 📝 Transcript

Umut Nefta Kanilmaz · 2024

The COVID-19 pandemic and associated policy measures lead to world-wide protest movements that were singled out by the spread of misinformation and conspiracy theories, predominantly on social media platforms....

A deep dive into the Arrow Columnar format with pyarrow and nanoarrow ▶ Recording 📝 Transcript

Joris Van den Bossche, Raúl Cumplido Domínguez, Alenka Frim · 2024

Apache Arrow has become a de-facto standard for efficient in-memory columnar data representation. You might have heard about Arrow or using Arrow, but do you understand the format and why it’s so useful? This...

You've got trust issues, we've got solutions: Differential Privacy

Vikram Waradpande, Sarthika Dhawan · 2023

As we are in an era of big data where large groups of information are assimilated and analyzed, for insights into human behavior, data privacy has become a hot topic. Since there is a lot of private information which...

Use Spark from anywhere: A Spark client in Python powered by Spark Connect ▶ Recording 📝 Transcript

Martin Grund · 2023

Over the past decade, developers, researchers, and the community have successfully built tens of thousands of data applications using Spark. Since then, use cases and requirements of data applications have evolved:...

Unlocking Information - Creating Synthetic Data for Open Access. ▶ Recording 📝 Transcript

Antonia Scherz · 2023

Many good project ideas fail before they even start due to the sensitive personal data required. The good news: a synthetic version of this data does not need protection. Synthetic data copies the actual data's...

The Spark of Big Data: An Introduction to Apache Spark ▶ Recording 📝 Transcript

Pasha Finkelshteyn · 2023

Get ready to level up your big data processing skills! Join us for an introductory talk on Apache Spark, the distributed computing system used by tech giants like Netflix and Amazon. We'll cover PySpark DataFrames...

The CPU in your browser: WebAssembly demystified ▶ Recording 📝 Transcript

Antonio Cuni · 2023

In the recent years we saw an explosion of usage of Python in the browser: Pyodide, CPython on WASM, PyScript, etc. All of this is possible thanks to the powerful functionalities of the underlying platform,...

The Beauty of Zarr ▶ Recording 📝 Transcript

Sanket Verma · 2023

In this talk, I’d be talking about [Zarr](https://zarr.dev/), an open-source data format for storing chunked, compressed N-dimensional arrays. This talk presents a systematic approach to understanding and...

Shrinking gigabyte sized scikit-learn models for deployment ▶ Recording 📝 Transcript

Pavel Zwerschke, Yasin Tatar · 2023

We present an open source library to shrink pickled scikit-learn and lightgbm models. We will provide insights of how pickling ML models work and how to improve the disk representation. With this approach, we can...

Raised by Pandas, striving for more: An opinionated introduction to Polars ▶ Recording 📝 Transcript

Nico Kreiling · 2023

Pandas is the de-facto standard for data manipulation in python, which I personally love for its flexible syntax and interoperability. But Pandas has well-known drawbacks such as memory in-efficiency, inconsistent...

PyLadies Panel Session. Tech Illusions and the Unbalanced Society: Finding Solutions for a Better Future ▶ Recording 📝 Transcript

· 2023

During this panel, we’ll discuss the significant role PyLadies chapters around the world have played in advocating for gender representation and leadership and combating biases and the gender pay gap.

Pragmatic ways of using Rust in your data project ▶ Recording 📝 Transcript

Christopher Prohm · 2023

Writing efficient data pipelines in Python can be tricky. The standard recommendation is to use vectorized functions implemented in Numpy, Pandas, or the like. However, what to do, when the processing task does not...

Postmodern Architecture: The Python Powered Modern Data Stack ▶ Recording 📝 Transcript

John Sandall · 2023

The Modern Data Stack has brought a lot of new buzzwords into the data engineering lexicon: "data mesh", "data observability", "reverse ETL", "data lineage", "analytics engineering". In this light-hearted talk we...

Polars - make the switch to lightning-fast dataframes ▶ Recording 📝 Transcript

Thomas Bierhance · 2023

In this talk, we will report on our experiences switching from Pandas to Polars in a real-world ML project. Polars is a new high-performance dataframe library for Python based on Apache Arrow and written in Rust. We...

Pandas 2.0 and beyond ▶ Recording 📝 Transcript

Joris Van den Bossche, Patrick Hoefler · 2023

Pandas has reached a 2.0 milestone in 2023. But what does that mean? And what is coming after 2.0? This talk will give an overview of what happened in the latest releases of pandas and highlight some topics and major...

Observability for Distributed Computing with Dask ▶ Recording 📝 Transcript

Hendrik Makait · 2023

Debugging is hard. Distributed debugging is hell. Dask is a popular library for parallel and distributed computing in Python. Dask is commonly used in data science, actual science, data engineering, and machine...

Let's contribute to pandas (3 hours) #2

Noa Tamir, Patrick Hoefler · 2023

PyData Berlin are excited to bring you this open source workshop dedicated to contributing to pandas. This tutorial is 3 hours. We will have a break and continue with the same group of people. pandas is a data...

Let's contribute to pandas (3 hours) #1

Noa Tamir, Patrick Hoefler · 2023

PyData Berlin are excited to bring you this open source workshop dedicated to contributing to pandas. This tutorial is 3 hours. We will have a break and continue with the same group of people. pandas is a data...

Geospatial Data Processing with Python: A Comprehensive Tutorial ▶ Recording 📝 Transcript

Martin Christen · 2023

In this tutorial, you will learn about the various Python modules for processing geospatial data, including GDAL, Rasterio, Pyproj, Shapely, Folium, Fiona, OSMnx, Libpysal, Geopandas, Pydeck, Whitebox, ESDA, and...

Apache Arrow: connecting and accelerating dataframe libraries across the PyData ecosystem

Joris Van den Bossche · 2023

Apache Arrow is a multi-language toolbox for accelerated data interchange and in-memory processing, and is becoming the de facto standard for tabular data. This talk will give an overview of the recent developments...

Unclear Code Hurts ▶ Recording 📝 Transcript

Dario Cannone · 2022

Code may work or not, but it will always tell a story. Computers will not complain about how you write it (except correct syntax), but human readers will. This talk is about writing clear code and caring for the...

Slack bots 101: An introduction into slack bot-based workflow automation ▶ Recording 📝 Transcript

Jordi Smit · 2022

Most developers work with Slack every day, yet very few of them know about the awesome things you can do when you build your own slack bot. For example, recently, we built a slack bot that automates our hiring...

sktime - python toolbox for time series: advanced forecasting - probabilistic, global and hierarchical

Franz Kiraly · 2022

sktime is a widely used scikit-learn compatible library for learning with time series. The forecasting module of sktime provides a unified, sklearn-compatible, and composable interface to the pydata/numfocus...

(Serious) Time for Time Series ▶ Recording 📝 Transcript

Marysia Winkels, James Hayward · 2022

From inventory to website visitors, resource planning to financial data, time-series data is all around us. Knowing what comes next is key to success in this dynamically changing world. And for that we need reliable...

Rewriting your R analysis code in Python ▶ Recording 📝 Transcript

Helena Schmidt · 2022

R and Python are two of the most powerful tools for any kind of data analysis. But both programming languages have their strengths and weaknesses. This means it can be necessary to switch from one to the other. When...

Reproducible machine learning and science with python ▶ Recording 📝 Transcript

Prabhant Singh · 2022

With machine learning being used in all domains of science, reproducibility and openness is major concern for these experiments and workflows, This tutorial will discuss various experiment tracking tools and focus on...

On Blocks, Copies and Views: updating pandas' internals ▶ Recording 📝 Transcript

Joris Van den Bossche · 2022

a.k.a. “Getting rid of the SettingWithCopyWarning” Pandas’ current behavior on whether indexing returns a view or copy is confusing, even for experienced users. But it doesn’t have to be this way. We can make this...

Introducing the Dask Active Memory Manager ▶ Recording 📝 Transcript

Guido Imperiale · 2022

The Active Memory Manager is a new experimental feature of Dask which aims to reduce the memory footprint of the cluster, prevent hard to debug out-of-memory issues, and make worker retirement more robust.

Flexible ML Experiment Tracking System for Python Coders with DVC and Streamlit ▶ Recording 📝 Transcript

Antoine Toubhans · 2022

There are tons of tools to do data science. Too often, data scientists end up using a monolithic AI platform that “does everything by clicking on a UI”. In this talk, I will walk you through a pythonic and flexible...

Easy and flexible imaging with the Core Imaging Library

Vaggelis Papoutsellis, Dr. Jakob Sauer Jørgensen · 2022

In this talk, we present the [Core Imaging Library (CIL)](https://github.com/TomographicImaging/CIL), an open-source, object-oriented Python library for solving large scale imaging inverse problems. We give a brief...

Easily build interactive plots and apps with hvPlot ▶ Recording 📝 Transcript

Philipp Rüdiger, Maxime Liquet · 2022

Do you use the `.plot()` API of pandas or xarray? Do you ever wish it was easier for you or your collaborators to try out different combinations of the parameters in your data-processing pipeline? This tutorial will...

Data Science at Scale with Dask ▶ Recording 📝 Transcript

Richard Pelgrim · 2022

A Pythonic introduction to methods for scaling your data science and machine learning work to larger datasets and larger models with Dask, all while staying within the comfort of the tools and APIs you know and love...

Data Apis: Standardization of N-dimensional arrays and dataframes ▶ Recording 📝 Transcript

Stephannie Jimenez Gacha · 2022

We would like to introduce the consortium of Data APIs, where we will be presenting our motivation, objectives and progress of the standardization process after one year of activity. We will dive into a small history...

Creating 3D Maps using Python ▶ Recording 📝 Transcript

Martin Christen · 2022

In this talk it is shown how to create 3D Maps using Open Data and Python. There are many open data sources available now for direct download, for example on AWS ( https://aws.amazon.com/opendata ). This talks shows...

conda-forge: supporting the growth of the volunteer-driven, community-based packaging project

Wolf Vollprecht, Jannis Leidel, Jaime Rodríguez-Guerra · 2022

The conda-forge project is one of the fastest growing Open Source communities out there – and most data scientists have probably heard of it. In this talk we explain the inner workings of conda-forge, its...

A data scientist's guide to code reviews ▶ Recording 📝 Transcript

Alexandra Wörner · 2022

A crucial aspect of software engineering teams' working agreements are code reviews. By applying the four-eyes principle on code, teams can reduce the number of bugs and errors, uncover misunderstandings early and...

5 Steps to Speed Up Your Data-Analysis on a Single Core ▶ Recording 📝 Transcript

Jonathan Striebel · 2022

Your data analysis pipeline works. *Nice.* Could it be faster? *Probably.* Do you need to parallelize? *Not yet.* We'll go through optimization steps that **boost the performance of your data analysis pipeline on a...

Your Name Is Invalid! ▶ Recording 📝 Transcript

Miroslav Šedivý · 2019

About people with first names, middle names, last names, one-word names, multiple names, and changing names, about names with characters beyond ASCII, and about using Python to handle them correctly, because names of...

Why you should (not) train your own BERT model for different languages or domains

Marianne Stecklina · 2019

Language models like BERT can capture general language knowledge and transfer it to new data and tasks. However, applying a pre-trained BERT to non-English text has limitations. Is training from scratch a good (and...

Why you don’t see many real-world applications of Reinforcement Learning.

Yurii Tolochko · 2019

Reinforcement learning is the closest thing to a general AI system that we have. When it works, that is. The problem is, it often doesn’t. In this talk we will discuss the difficulties of RL as well as what to look...

Where Linguistics meets Natural Language Processing ▶ Recording 📝 Transcript

Mariana Capinel · 2019

This talk explains how linguistics describes language - via phonetics-phonology, morphology, syntax, semantics and pragmatics. We will combine linguistic concepts with models through examples for NLP newbies.

Want to have a positive social impact as a data scientist? ▶ Recording 📝 Transcript

Ellen König · 2019

Discover your individual approach towards more positive social impact by conducting experiments. I'll show you how! I’ll also share my learnings from doing such experiments over the last 15 years.

vtext: text processing in Rust with Python bindings ▶ Recording 📝 Transcript

Roman Yurchak · 2019

In this we talk present how to write Python extensions in Rust, and discusse advantages and limitation of such approach. We then illustrate this approach on the vtext project, that aims to be a high-performance...

Visualizing Interactive Graph Networks in Python ▶ Recording 📝 Transcript

Jan-Benedikt Jagusch · 2019

In this talk you will learn how to visualize graph networks in Python, using `networkx`, `traitlets`, `ipywidgets` and `plotly`. The resulting plot will be fully interactive, which makes it easy to filter edges, find...

Version Control for Data Science ▶ Recording 📝 Transcript

Alessia Marcolini · 2019

Are you versioning your Machine Learning project as you would do in a traditional software project? How are you keeping track of changes in your datasets?

Using Overhead Video Capture to Analyse Grouping Behaviour of Dancers in a Silent Disco

Nelson Mooren · 2019

I built upon Python's OpenCV library to detect locations of dancers in a silent disco, using their headphone lights as a proxy and performed network analysis to investigate their grouping behaviour based on the...

Using machine learning for Level Generation in Snake (video-game) ▶ Recording 📝 Transcript

Filipe Silva · 2019

As a practical example, this tutorial uses machine learning models to predict where to best place the apple in Snake. By using datasets that contain different plays we can obtain different game experiences or models...

Using adversarial samples to break and robustify your Vision Neural Network Models

Irina Vidal Migallón · 2019

We will cover several techniques to expose weaknesses and robustify neural network models for computer vision, from basic precautions to more advanced adversarial training.

Transforming a Legacy System into a Bias-Mitigating AI Solution for Debt Repayment

Avaré Stewart · 2019

We present Phoenix, a modernization of a legacy, German rule-based system based on Tesseract and SpaCy, and uses AI Fairness 360 to build tunable, Bias-Minimizing AI solutions to offer payment incentives to debtors.

Tools that help you get your experiments under control ▶ Recording 📝 Transcript

Katharina Rasch · 2019

There is now a wealth of tools that support data science best practices (e.g. tracking experiments, versioning data). Let’s take a look at which tools are available and which ones might be right for your project.

Time series modelling with probabilistic programming

Sean Matthews, Jannes Quer · 2019

This talk focuses on a Bayesian approach of advanced time-series forecasting in the case of small data. We describe the stages of modelling from simple smoothing to advanced forecasting by applying Gaussian Processes...

Time Series Anomaly Detection for Bottling Machine Maintenance

Andrea Spichtinger · 2019

This talk discusses time series anomaly detection methods for predictive maintenance of machines in bottling plants and their implemention on AWS edge devices.

The Sound of Silence: Online Misogyny and How we Model it ▶ Recording 📝 Transcript

Teresa Ingram · 2019

Female-identifying people are being attacked and silenced online. Social media platforms act as neutral bodies and law enforcement can’t stop the abuse. When you can’t trust that your safety will be protected online,...

Take control of your hearing: Accessible methods to build a smart noise filter

Peggy Sylopp, Aislyn Rose · 2019

Have you ever wanted more control over what you do or don't hear? This talk explores the potential of Python, deep learning, and open databases to bring you towards that goal, without needing expensive licenses or software.

Tackle the problems that really matter - leverage the power of data science in the service of humanity

Eva Schreyer, Lisa Zäuner · 2019

Data Science for Social Good Berlin (DSSG) brings together data scientist volunteers and nonprofit organisations that need support in tackling various data challenges. In this talk we will tell you how data science...

skorch: A scikit-learn compatible neural network library that wraps pytorch

Benjamin Bossan · 2019

This talk is about the open source package [skorch](https://github.com/skorch-dev/skorch), a wrapper library that allows you to combine the best of sklearn and PyTorch. It covers when it makes sense to use skorch and...

Should I stay or should I go? Optimal exercise decisions using the Longstaff-Schwartz algorithm

Benedikt Rudolph · 2019

This talk presents a Python implementation of the Longstaff-Schwartz algorithm for financial exercise option valuation. The technical problems of optimal exercise decisions and exercise option valuation are...

Rethinking Open Source in the Era of Cloud & Machine Learning ▶ Recording 📝 Transcript

Peter Wang · 2019

By some measures, Open Source is a wildly successful and crucial part of many areas of modern technology. However, the ’sustainability crisis’ and the age of cloud computing have threatened its core mechanisms. Peter...

Quantum computing with Python ▶ Recording 📝 Transcript

James Wootton · 2019

Frameworks for quantum computing are a new way to use Python for cutting-edge science, and to plan for future applications of this new technology. This session will serve as an introduction to quantum computing as a...

Python-Powered OSINT! Modernising Open Source Intelligence for Investigating Disinformation

Mx Chiin-Rui Tan, Dare Imam-Lawal · 2019

OSINT, the discipline of gathering intelligence from open sources, is critical for state & citizen interest but has lacked modernisation. We present a socio-technical maturity model using Python to update legacy...

Professional Development and Career Progression for Data Scientists ▶ Recording 📝 Transcript

Noa Tamir · 2019

In this talk you will learn how to level up your skills, and develop your your career using on the job opportunities, as well as open source contributions

Production-level data pipelines that make everyone happy using Kedro

Yetunde Dada · 2019

Learn how easy it is to apply software engineering principles to your data science and data engineering code. Expect an overview of Kedro, a library that implements best practices for data pipelines with an eye...

Privacy-preserving Machine Learning for text processing ▶ Recording 📝 Transcript

Sarah Diot-Girard · 2019

Privacy is something we all care about, but when it is time to put our principles into application, it is not so trivial, especially when working with text. This talk aims at presenting a few options to handle...

Practical DevOps for the busy data scientist ▶ Recording 📝 Transcript

Dr. Tania Allard · 2019

How many times have you developed a model or a data application and tested it locally or in a staging environment just to find out that it breaks in production? This is a common issue faced by thousands of data...

Play Stupid Games, Win Stupid Prizes ▶ Recording 📝 Transcript

James Powell · 2019

This is reserved for a James Powell in-promptu talk, stay tuned! This is reserved for a James Powell in-promptu talk, stay tuned!

Panel: Turn any notebook into a deployable dashboard ▶ Recording 📝 Transcript

Philipp Rüdiger · 2019

Quickly turn your existing analyses built on the PyData stack into shareable, standalone apps and dashboards using the new open-source [Panel](http://panel.pyviz.org) library without learning the details of web development.

Managing the end-to-end machine learning lifecycle with MLFlow ▶ Recording 📝 Transcript

Tobias Sterbak · 2019

Machine learning requires experimenting with datasets, data preparation steps, and algorithms. Deploy models to a production system and retrain it on new data. MLflow is an open source platform for managing the...

Making the complex simple in data viz ▶ Recording 📝 Transcript

Tania Vasilikioti · 2019

Creating graphics that convey the desired message, are easily interpretable, but also beautiful can be a daunting task. This talk will demonstrate how to use *The Grammar of Graphics* framework to conceptualize the...

Machine learning with little data - from digital twin to predictive maintenance

Andreas Hantsch · 2019

This talk is about the coupling of a digital twin model and a machine learning predictive maintenance algorithm in order to be able to detect anomalies in the operation of a not well-known hardware system.

Loss Function Theory 101 ▶ Recording 📝 Transcript

David Wölfle · 2019

This talk covers the theoretical background behind two common loss functions, mean squared error and cross entropy, including why they are used for machine learning at all, and what limitations you should keep in mind.

Leveraging the advantages of Bayesian Methods to build a data science product using PyMC3 ▶ Recording 📝 Transcript

Korbinian Kuusisto · 2019

Bayesian models offer greater theoretical advantages compared to non-probabilistic methods, and also allow for more flexible model design. But how can one leverage these theoretical advantages to build a successful...

Lessons Learned as a Product Manager in Data Science ▶ Recording 📝 Transcript

Tereza Iofciu · 2019

The fun part about data science is that no two people really agree on the definition of the data scientist role. So how does the role of product manager in data science look like?

Law, ethics and machine learning – a curious ménage à trois

Dr. Benjamin Werthmann · 2019

The talk addresses how law and ethics can provide a framework for the “machines on the rise” that ensures fairness and societal acceptance while not interfering (too much) with the drive behind the advancement of a...

Kartothek – Table management for cloud object stores powered by Apache Arrow and Dask

Florian Jetter · 2019

Efficient data storage is an integral part of successful data applications. Cloud object stores prove to be an efficient choice but come with downsides when storing structured, tabular data. There is a way out, though.

Julia for Python ▶ Recording 📝 Transcript

Simon Danisch · 2019

Julia is a new Language, that is fast, high level, dynamic and optimized for Data Science. But due to its young age, it might not be for everyone yet. Learn about Julia's strengths and how you can integrate it in...

Interpretable Machine Learning: How to make black box models explainable

Alexander Engelhardt · 2019

Complex machine learning models make better predictions, but at the cost of turning into an unexplainable black-box model. In this talk, we'll look into a framework that allows us to explain why a model makes a...

How to write tests that need a lot of data? ▶ Recording 📝 Transcript

Sander Kooijmans · 2019

In this talk Sander explains how you can write unit and integration tests that need a lot of data. As an example Sander shows how to test code of a warehouse management system (WMS). This is not about big-data, though.

How to choose better colors for your data visualizations ▶ Recording 📝 Transcript

Daniel Ringler · 2019

Everybody is doing colorful charts with Python libraries such as matplotlib and bokeh but most people never change the basic configuration. This talk will teach you the basics of color theory to help you choose the...

How strong is my opponent? Using Bayesian methods for skill assessment

Darina Goldin · 2019

Being able to correctly estimate a competitors' skill is a crucial question in sport forecasting and matchmaking. This talk will provide an overview of the three most common algorithms for this task: Elo, Glicko2 and...

Hide Code, Minimize Dependencies, Boost Performance - The PyTorch JIT

Tilman Krokotsch · 2019

PyTorch makes developing, training and debugging deep neural networks convenient. Learn how to export your trained model using its just-in-time (JIT) compiler to hide your network architecture, minimize code...

Hidden Markov Models for Chord Recognition - Intuition and Applications ▶ Recording 📝 Transcript

Caio Miyashiro · 2019

This tutorial describes the intuition behind Hidden Markov Models, with less mathematical formulas and with an application on Music Analytics - Chord Recognition

Get to grips with pandas and scikit-learn ▶ Recording 📝 Transcript

Sandrine Pataut · 2019

This session will be an exposition of data wrangling with pandas and machine learning with scikit-learn for Python Programmers. This hands-on workshop will cover a classification project, from importing the data to...

Gaussian Progress ▶ Recording 📝 Transcript

Vincent Warmerdam · 2019

This talk is an attempt at explaining the power of the Gaussian[tm] by stepping up the ladder from Naive Bayes to Mixtures to Neural Mixtures to Gaussian Processes.

Gaussian Process for Time Series Analysis ▶ Recording 📝 Transcript

Dr. Juan Orduz · 2019

The aim of this talk is to introduce the notion of *Gaussian process* and describe how to use it to solve regressions problems and time series forecasting.

Friend or Foe: Comparison of R & Python in Data Wrangling & Visualisation

Yuta Kanzawa · 2019

R and Python are partially overlapping but different as a whole: community and language. Still, comparing them in their common fields such as data wrangling and visualisation, useRs and Pythonistas will deepen mutual...

Fighting fraud: finding duplicates at scale ▶ Recording 📝 Transcript

Alexey Grigorev · 2019

We present a duplicate detection system that we use to fight fraud in online classifieds. The system uses machine learning to analyze both text and images of 10 million ads daily and stop fraudulent listings before...

Fairness in decision-making with AI: a practical guide & hands-on tutorial using Aequitas ▶ Recording 📝 Transcript

Pedro Saleiro · 2019

In this tutorial, we are going to deep dive into algorithmic fairness, from metrics and definitions to practical case studies, including bias audits using Aequitas (http://github.com/dssg/aequitas) in real policy...

Extended Ligthning Talks CANCELLED: Crunching Numbers Like a Journalist

Marie-Louise Timcke · 2019

Get well soon Marie-Louise! As our world becomes more and more data-driven, journalists are equipping themselves with data science and programming skills to turn numbers into headlines.

Equivariance in CNNs: how generalising the weight-sharing property increases data-efficiency

Marysia Winkels · 2019

In this talk, we will explore how the weight-sharing property of the convolutional layer can be generalised to achieve equivariance towards transformations beyond just translation, how to implement this, and the...

Embrace uncertainty! Why to go beyond point estimators for valuable ML applications

Stefan Maier · 2019

Usually, uncertainties of Machine Learning predictions are just regarded as a sign of poor prediction accuracy or as a consequence of lacking input features. This talk illustrates how modeling uncertainties can...

Does hate sound the same in all languages? ▶ Recording 📝 Transcript

Andrada Pumnea · 2019

How might we make social media safer and more inclusive? Tackling hate speech online is not easy, especially if it’s in a language less circulated. This talk describes detecting hate speech in Romanian from dataset...

Detecting and Analyzing Solar Panels in Switzerland using Aerial Imagery

Martin Christen · 2019

We present a novel method for detecting solar panels and its geometry on aerial imagery. The goal is to know the exact locations, dimensions and potential of every solar installation in Switzerland.

Deep Learning for Healthcare with PyTorch

Valerio Maggio · 2019

This tutorial provides a general introduction to Deep Learning using PyTorch with specific focus on challenges and solutions for Healthcare and Computational Biology.

Decentralized and Privacy-Preserving ML via TensorFlow Federated ▶ Recording 📝 Transcript

Peter Kairouz, Amlan Chakraborty · 2019

Federated Learning is a technology to train machine learning models and run data analytics on decentralized data. This tutorial will demonstrate step-by-step how to train largescale TensorFlow models and custom...

Data Literacy for Managers ▶ Recording 📝 Transcript

Alexander CS Hendorf · 2019

Artificial Intelligence need to be better understood in enterprises. Close the communications gap between engineers and management. Making data litteracy happen in your organisation.

Dash: Interactive Data Visualization Web Apps with no Javascript ▶ Recording 📝 Transcript

Dom Weldon · 2019

Interactive web pages and visualizations with no JavaScript? What could go wrong? What you can, can't, should and probably shouldn't do with plotly/Dash.

Current affairs, updates, and the roadmap of scikit-learn and scikit-learn-contrib

Adrin Jalali · 2019

As a scikit-learn core developer, I'd give an update on recent changes, current affairs, and the roadmap of the package and the community packages included in scikit-learn-contrib. I'd also briefly talk about how new...

Creating an Interactive ML Conference Showcase ▶ Recording 📝 Transcript

Harald Bosch · 2019

Our goal is to create a simple yet interactive showcase for computer vision using a Python notebook. In a trade fair setup, we want to learn new object classes quickly using very few training examples. Thus, we rely...

CANCELLED: First steps in Julia

Felicia Burtscher · 2019

Cancelled - get well soon, Felicia! The community of Julia has been growing and the much anticipated 1.0 release of Julia is out since last summer. Targeted to Julia beginners and Python users, we will especially...

CANCELED: Create CUDA kernels from Python using Numba and CuPy.

Emergency Self-Constuct · 2019

Get well, soon, Valentin! We'll explain how to do GPU-Accelerated numerical computing from Python using the Numba Python compiler in combination with the CuPy GPU array library.

Build a Machine Learning pipeline with Jupyter and Azure

Daniel Heinze · 2019

With increasing focus on Machine Learning systems in almost every business it is important, to build a great pipeline to train, test and deploy your models. In this session we will show a way to do that with Jupyter and Azure

Birds of a feather flock together - Tracking pigeons with Python and OpenCV

Neslihan Edes · 2019

In this talk I want to demonstrate how to use OpenCV to implement basic animal movement tracking use cases. And everything without any fancy machine learning or neuronal networks ;)

Avoiding ML FOBO ▶ Recording 📝 Transcript

Rachel Berryman, Dânia Meira · 2019

Everyday there is a new package or algorithm to use- it can be hard to determine what is useful and what is only hype. The speakers offer a practical roadmap and checklist to help you cut through the hype and focus...

Automating feature engineering for supervised learning? Methods, open-source tools and prospects.

Dr. Thorben Jensen · 2019

Feature engineering is crucial for supervised learning (SL), but labor-intensive. Automating this task is less established, compared to other steps of SL. We therefore present and compare approached and open-source...

Automated Feature Engineering and Selection in Python ▶ Recording 📝 Transcript

Franziska Horn · 2019

Careful feature engineering and selection can be just as important as choosing the right ML model & hyperparameters. I will present several options for automating the feature engineering and selection process with a...

Are you sure about that?! Uncertainty Quantification in AI ▶ Recording 📝 Transcript

Florian Wilhelm · 2019

There is a strong need in many AI applications to state the certainty about their predictions. This talk elaborates on different ways to perform uncertainty quantification in deep learning and classical methods.

Applying deployment oriented mindset for building Machine Learning models

Marianna Diachuk · 2019

Developing ensemble model with hundreds of features? And getting stuck for months trying to deploy the model and fighting with data inconsistency and bugs? This talk will introduce the way to build the development...

Algo.Rules - How do we get the ethics into the code? ▶ Recording 📝 Transcript

Carla Hustedt · 2019

In the keynote I will present our Algo.Rules, 9 rules for the design of algorithmic systems and address the questions “What standards of quality should algorithms be held to?” and “How can we make sure that these...

Airflow: your ally for automating machine learning and data pipelines

Enrica Pasqua, Bahadir Uyarer · 2019

Now that you finally have your Machine Learning model trained, what’s the next step for moving to production? Orchestrating, scheduling and monitoring ML inference pipelines is a big challenge. Airflow can be your...

AI Intentions and Code Completion ▶ Recording 📝 Transcript

Vasily Korf · 2019

Datalore supports intentions – code suggestions based on what you’ve just written. They cover a wide range of situations from generating code to warnings and optimization suggestions.

Active Learning with Bayesian Nonnegative Matrix Factorization for Recommender Systems

Gönül Aycı · 2019

In most of the systems, collecting data is not always free. I will talk about an approach for a matrix completion problem that learns a distribution of data where information is incomplete or collecting it has a cost.

A Tour of JupyterLab Extensions ▶ Recording 📝 Transcript

Jeremy TULOUP, QuantStack · 2019

JupyterLab can be extended via third-party extensions written by developers from the Jupyter community. This is a tour of 20 of these extensions, in 20 minutes. Demos included!

A Bayesian Workflow with PyMC and ArviZ ▶ Recording 📝 Transcript

Corrie Bartelheimer · 2019

Bayesian Modelling has several advantages such as the handling of uncertainty. While the advantages are well known, implementing a Bayesian model can be a bit more involved and some care needs to be taken to check...

Strongly typed datasets in a weakly typed world ▶ Recording 📝 Transcript

Marco Neumann · 2018

Strongly typed Parquet Datasets / Hive Tables are often used to exchange and preserve data in a Pandas-driven environment, where types are rather unstable. This results in multiple issues and these as well as...

Scalable Scientific Computing using Dask ▶ Recording 📝 Transcript

Uwe L. Korn · 2018

Pandas and NumPy are great tools to dive through data, do analysis and train machine learning models. They provide intuitive APIs and superb performance. Sadly they are both restricted to the main memory of a single...

Pandas IO Tools: Reading and Writing DataFrames as Files and Databases

Miroslav Šedivý · 2018

Learn to use the right tools to read and write your DataFrames as pickle, CSV, JSON, Msgpack, HTML, Excel, HDF5, Parquet, and a PostgreSQL database.

Fulfilling Apache Arrow's Promises: Pandas on JVM memory without a copy ▶ Recording 📝 Transcript

Uwe L. Korn · 2018

Apache Arrow's promise was to reduce the (serialization & copy) overhead of working with columnar data between different systems. Using the latest Pandas release and Arrow's ability to share memory between the JVM...

Simple Data Engineering in python 3.5+ with Bonobo ▶ Recording

Romain Dorgueil · 2017

Simple is better than complex, and that's True for data pipelines, too. Bonobo is a python 3.5+ tool used to write and monitor data pipelines. It’s plain, simple, modern, and atomic python. This talk is a practical...

Modern ETL-ing with Python and Airflow (and Spark) ▶ Recording

Tamara Mendt · 2017

The challenge of data integration is real. The sheer amount of tools that exist to address this problem is proof that organizations struggle with it. This talk will discuss the inherent challenges of data...

How efficient is your public transport network? A data-driven approach using Geopandas and GTFS

Pieter Mulder · 2017

The presentation will explain how GeoPandas and other tools are used to analyse GTFS files to calculate the reachability of a public transport system.

Getting Scikit-Learn To Run On Top Of Pandas ▶ Recording

Ami Tavory · 2017

Scikit-Learn is built directly over numpy, Python's numerical array library. Pandas adds to numpy metadata and higher-level munging capabilities. This talk describes how to intelligently auto-wrap Scikit-Learn for...

Effective Data Analysis with Pandas Indexes ▶ Recording

Alexander Hendorf · 2017

Pandas is the Swiss-Multipurpose Knife for Data Analysis in Python. In this talk we will look deeper into how to gain productivity utilizing Pandas powerful indexing and make advanced analytics a piece of cake....

Data Plumbing 101 - ETL Pipelines for Everyday Projects ▶ Recording

Eberhard Hansis · 2017

There is no data science without ETL! This presentation is about implementing maintainable data integration for your projects. We will have a first look a ‘Ozelot’, a library based on Luigi and SQLAlchemy that helps...

Manipulating and analysing multi-dimensional data with Pandas

Sahil Dua · 2016

Introduction to Time Series Analysis with Pandas

Alexander Hendorf · 2016