Introduction to Streamlit for Data Engineering

4 min readMar 12, 2023

Data engineering is a critical aspect of any data-driven organization, where data scientists and analysts work with large amounts of data to extract insights and make data-driven decisions. In recent years, Streamlit has become a popular framework for building interactive data applications, making it easier for data engineers to create and share data-driven applications with their team members.

In this beginner’s guide, we’ll explore Streamlit, its features, and how it can be used for data engineering tasks. We’ll also provide some code samples and links to resources to help you get started with Streamlit.

What is Streamlit?

Streamlit is an open-source Python framework allowing data engineers and scientists to create and share interactive data applications. Streamlit simplifies building interactive applications by enabling developers to write code that can update in real-time as users interact.

With Streamlit, data engineers can create various data-driven applications, such as data exploration tools, dashboards, and machine learning applications. Streamlit provides a straightforward and intuitive interface that allows developers to focus on the application’s functionality without worrying about the underlying infrastructure.

Features of Streamlit

Streamlit offers several features that make it a popular choice for data engineering tasks. Some of its essential features include:

Intuitive API: Streamlit provides a simple and intuitive API that allows developers to build interactive applications without writing complex code. Developers can use Python functions to create interactive widgets that update in real-time based on user input.

Real-time updates: Streamlit allows developers to update their applications in real time as users interact with them. This feature is useful when working with frequently changing data, such as a sensor or financial data.

Customizable UI: Streamlit provides a flexible UI that can be customized to suit the application’s needs. Developers can use CSS and HTML to create custom styles and layouts that match their organization’s branding.

Support for machine learning: Streamlit provides several built-in tools that make it easy to create machine learning applications, such as interactive model training and evaluation tools.

How to Use Streamlit for Data Engineering Tasks

Now that we’ve covered what Streamlit is and its features let’s explore how it can be used for data engineering tasks. This section will cover how to build a simple data exploration tool using Streamlit.

Step 1: Install Streamlit

The first step is to install Streamlit. Streamlit can be installed using pip, the Python package manager.

pip install streamlit

Step 2: Create a Simple Data Exploration Tool

Once Streamlit is installed, we can create a simple data exploration tool. In this example, we’ll use the famous iris dataset to create a data exploration tool.

import streamlit as st
import pandas as pd
from sklearn.datasets import load_iris
# Load iris dataset
iris = load_iris()
df = pd.DataFrame(iris.data, columns=iris.feature_names)
# Create sidebar
st. sidebar.header('Filter Data')
min_value = st.sidebar.slider('Minimum Value', 0, 10, 0)
max_value = st.sidebar.slider('Maximum Value', 0, 10, 10)
# Filter data based on user input
filtered_df = df[(df['sepal length (cm)'] >= min_value) & (df['sepal length (cm)'] <= max_value)]
# Display filtered data
st.write(filtered_df)

In this example, we load the iris dataset and create a sidebar that allows users to filter the data based on the sepal length. The st.sidebar.slider function creates a slider widget that will enable users to specify the minimum and maximum values.

We then use the user input to filter and display the filtered data using the st.write function.

Step 3: Run the Streamlit Application

To run the Streamlit application, save the code above to a file, and run the following command in your terminal:

streamlit run app.py

This will start a local Streamlit server, and your application will be available in your web browser at http://localhost:8501.

When you run the application, you’ll see a sidebar on the left-hand side that allows you to filter the data based on the sepal length. As you move the sliders, the filtered data will be updated in real-time.

Conclusion

Streamlit is a powerful tool for data engineers and data scientists who need to create interactive data applications. In this beginner’s guide, we covered Streamlit, its features, and how it can be used for data engineering tasks. We also provided a code sample demonstrating how to build a simple data exploration tool using Streamlit.

If you’re new to Streamlit, we recommend checking out the Streamlit documentation and community resources, which provide additional examples, tutorials, and best practices for using Streamlit in your data engineering projects.