In a previous post, we highlighted that a significant shift is underway; one that is unlocking the vast amounts of Earth observation (EO) data gathered by the European Space Agency (ESA). We’re proud to be part of modernizing how the industry interacts with this historically unwieldy data. We know when data is more accessible, it enables better science and faster insights.
A New Model for Earth Observation Data Access
Leading this shift is ESA’s Earth Observation Processing Framework (EOPF) initiative. The initiative is converting legacy SAFE data into Zarr, an open-source, cloud-native format. With Zarr, users can work directly with data in the cloud without the need for large downloads or extensive local storage.
The EOPF Toolkit is a vital component of this initiative, focusing on identifying widely used tools, ensuring their compatibility with the new format, and supporting users as they update their existing workflows.
While enabling QGIS to natively access Zarr data, it is equally important that the R ecosystem is equipped to handle the petabytes of Zarr-formatted Sentinel data that the EOPF makes available.
Why R Integration Matters
The shift to cloud-native EO data offers exciting new possibilities and there is a large community of data scientists, researchers and academics using R for statistical analysis, research, data visualization and machine learning. Ensuring that EOPF data can be discovered and analyzed directly within R is essential for the broader adoption of the Zarr format.
To support this transition and promote early adoption, there are a number of R-specific tutorials for the EOPF Tooling Guide. These tutorials demonstrate how EOPF data can be discovered, accessed, and analyzed directly within R, using workflows that align with existing research and data science practices.
Supporting R Workflows with the EOPF Toolkit
The work builds on the existing Rarr and rstac libraries, leveraging their proven functionality for working with Zarr data and STAC catalogs in R. This integration allows R users to interactively explore, visualize, and analyze EOPF datasets using familiar tools and workflows, while benefiting from R’s strengths in reproducibility and scripting.
The EOPF Toolkit, developed in collaboration with thriveGeo, Development Seed, and Sparkgeo, is designed to support users who may be new to cloud computing by smoothing the transition through clear documentation, tested examples, and practical knowledge.
For those new to cloud-native EO, the EOPF 101 resource provides a strong foundation, introducing the EOPF, Zarr, STAC, and modern EO workflows. The EOPF Tooling Guide builds on this foundation by providing detailed tool-specific documentation, including tutorials for R users. Sparkgeo’s role focused on developing R tutorials that help transition users into Zarr and STAC-based workflows.
These tutorials were designed to be:
- Copy-pastable and easy to follow.
- Tested against the live EOPF Sample Service STAC API.
- Robust, with clear examples and proper error handling.
- Representative of real-world analytical workflows.
The EOPF ecosystem is still developing, and the tutorials found in the EOPF Tooling Guide reflect the present capabilities of the EOPF Sample Service and tool integrations. These are expected to change as the ecosystem matures.
Discovering EOPF Data with STAC and rstac
The How to for the EOPF Sample Service STAC catalog tutorial provides a foundation for accessing EOPF data using R. It introduces the EOPF Sample Service, a sandbox environment that allows users to explore sample data ahead of the full rollout.
This tutorial walks users through:
- Connecting to the EOPF STAC API endpoint.
- Handling authentication where required.
- Searching for data by spatial extent.
- Identifying and inspecting Zarr assets within STAC items.
Analyzing EOPF Zarr Data with Rarr
The Access and analyse EOPF STAC Zarr data with R tutorial moves beyond discovery to focus on practical usage in R. It shows how users can open and work with Zarr datasets directly from STAC URLs, without intermediate downloads.
The tutorial provides examples for Sentinel-1, Sentinel-2, and Sentinel-3 data and covers:
- Opening Zarr stores directly from STAC URLs.
- Understanding the structure of EOPF Zarr datasets.
- Best practices for efficient analysis, including:
- Lazy loading,
- Memory management,
- Chunking.
The Path Forward: Empowering the R Community
Supporting R users is a key part of making advanced EO analysis using ESA’s data practical and accessible. By providing clear, tested tutorials that integrate proven tools like rstac and Rarr, the EOPF Toolkit reduces friction and encourages early adoption of Zarr-based Sentinel datasets.
This work empowers current EO data users while opening the door to many new users across research, academia, and data science, marking an exciting new phase of discovery and innovation using ESA’s vast EO archives.
As the EOPF ecosystem continues to evolve, the Sample Service and Toolkit are expected to iterate and improve as well, incorporating feedback from the community and reflecting the ongoing development.
If you are interested in getting started, explore the EOPF 101 resources and the R tutorials in the EOPF Tooling Guide to begin working with Zarr-based Sentinel data today.
