September 2021

Geo-harmonizer project

Prepared by: Terrasigna and CTU in Prague

Executive summary

Geo-harmonizer geoportal (https://maps.opendatascience.eu) includes two 3D viewers created with the help of NASA WorldWind and Cesium libraries. Standardized 3D web services were deployed to provide the client applications with digital elevation model data. Technically this was achieved using GeoServer with the DDS/BIL Extension. The digital terrain data is exposed through OGC WMS in Binary Interleaved by Line (BIL) – image/bil format. The supplied GridCoverage2D undergoes appropriate subsampling, reprojection and bit-depth conversion. The output can be requested by the 3D viewers  as 16bit Int or 32bit Float. The data is also available in Direct Draw Surface (DDS) – image/dds format. Three data sources were considered for the services:

• Geo-harmonizer DEM, European coverage, based on AW3D30, MERIT DEM, GLO-30 and EU-DEM;
• Copernicus DEM (GLO-30 Public), European coverage;
• NASA DEM, global coverage.

The service is accessible at https://geoserver.deap.terrasigna.io/geoserver/opendatascience/ows?service=wms&version=1.3.0&request=GetCapabilities. It was also integrated in the metadata catalogue.

Official project outputs

A complete list of project outputs is available here. To subscribe for the newsletter and receive monthly updates, please visit this link.

Software outputs

Published libraries, plugins and web-GIS solutions:

• eumap library;
• eumap QGIS plugin;
• ST_LUCAS
• Open Data Science Europe (ODSEurope) Viewer;
• Sample WPS and Actinia services

The python library eumap has been built to enable easier access to several spatial layers prepared for Continental Europe (Landsat and Sentinel mosaics, DTM and climate datasets, land cover and environment quality maps), as well the processing classes and functions used to produce them. The library implements efficient raster access through rasterio, multiple gapfilling approaches, spatial and spacetime overlay, training samples preparation (LUCAS points), and Ensemble Machine Learning applied to spatial predictions (fully compatible with scikit-learn). The eumap QGIS plugin enables access to the data catalog through QGIS.

ST_LUCAS was developed as a complete system for harmonization, aggregation and distribution of LUCAS in-situ observations, consisting of a server providing access to the data through an OGC interface, as well as clients for consuming the data from QGIS and Python.

Sample services were developed as examples for setting up web processing capabilities through OGC WPS and Actinia.

A QGIS WPS Plugin was built (in cooperation with OpenGeoLabs s.r.o) to extend the capabilities of QGIS with accessing OGC WPS.All Geo-harmonized project software can be followed via the project GitLab repository.

Datasets

Datasets are usually released using Zenodo.org, in the case they exceed the limits of zenodo, they are published only via our Wasabi cloud service. Metadata for all released layers: https://data.opendatascience.eu and https://stac.opendatascience.eu. Production steps: https://gitlab.com/geoharmonizer_inea/spatial-layers. Geo-harmonized data via the official portal for European data: https://data.europa.eu/data/datasets?catalog=opendatascienceeurope

Datasets released so far:

• Heisig, Johannes, & Hengl, Tomislav. (2020). Harmonized Tree Species Occurrence Points for Europe (0.2) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.5524611 
• Hengl, T., & Parente, L. (2021). Continental Europe Digital Terrain Model geomorphometry derivatives at 30 m, 100 m and 250 m (v0.2) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.4495449 
• Hengl, T., & Parente, L. (2021). MODIS LST monthly daytime and nighttime low (0.05), median (0.50) and high (0.95) temperatures for year 20** at 1-km (v1.1) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.4501970 
• Hengl, T. (2021). Continental Europe surface lithology based on EGDI / OneGeology map at 1:1M scale (v1.0) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.4787632 
• Hengl, Tomislav, Leal Parente, Leandro, Krizan, Josip, & Bonannella, Carmelo. (2020). Continental Europe Digital Terrain Model at 30 m resolution based on GEDI, ICESat-2, AW3D, GLO-30, EUDEM, MERIT DEM and background layers (v0.3) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.4724549 
• Križan, Josip, & Antonić, Luka. (2021). Seamless 30 meter Sentinel-2 L2A Pan-European seasonal cloudless mosaics from winter 2018 to spring 2020 (1.0.1) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.5155680; 
• Leandro Parente, Tomislav Hengl, Josip Krizan, Martin Landa, Lukas Brodsky, & Martijn Witjes. (2020). Input dataset for gap filling and land-cover mapping using eumap Library – 2000 to 2020 (v0.3) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.4311598 
• Martin Landa, Lukas Brodsky, Leandro Parente, Martijn Witjes, & Tomislav Hengl. (2021). Multi-year harmonized land cover samples based on LUCAS and CORINE datasets (v0.1) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.4740691
• Ibrahim Saleem, Landa Martin,  Halounová Lena,  Pešek Ondřej,  Pavelka Karel. (2021). Geo-Harmonized Atmospheric Dataset for Aerosol optical depth at 550 nm (v1.0) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.5675427
• Bonannella, Carmelo, Hengl, Tomislav, Heisig, Johannes, Leal Parente, Leandro, Wright, Marvin, Herold, Martin, & de Bruin, Sytze. (2022). Presence-Absence Points for Tree Species Distribution Modelling for Europe (v0.3) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.6516590
• Bonannella, Carmelo, Hengl, Tomislav, Heisig, Johannes, Leal Parente, Leandro, Wright, Marvin, Herold, Martin, & de Bruin, Sytze. (2022). Potential and realized distribution at 30m for Silver fir (Abies alba) in Europe for 2000 – 2020 [Data set]. Zenodo. https://doi.org/10.5281/zenodo.5873413
• Bonannella, Carmelo, Hengl, Tomislav, Heisig, Johannes, Leal Parente, Leandro, Wright, Marvin, Herold, Martin, & de Bruin, Sytze. (2022). Potential and realized distribution at 30m for Sweet chestnut (Castanea sativa) in Europe for 2000 – 2020 [Data set]. Zenodo. https://doi.org/10.5281/zenodo.5873918
• Bonannella, Carmelo, Hengl, Tomislav, Heisig, Johannes, Leal Parente, Leandro, Wright, Marvin, Herold, Martin, & de Bruin, Sytze. (2022). Potential and realized distribution at 30m for Common hazel (Corylus avellana) in Europe for 2000 – 2020 [Data set]. Zenodo. https://doi.org/10.5281/zenodo.5874797
• Bonannella, Carmelo, Hengl, Tomislav, Heisig, Johannes, Leal Parente, Leandro, Wright, Marvin, Herold, Martin, & de Bruin, Sytze. (2022). Potential and realized distribution at 30m for the European beech (Fagus sylvatica) in Europe for 2000 – 2020 [Data set]. Zenodo. https://doi.org/10.5281/zenodo.5877787
• Bonannella, Carmelo, Hengl, Tomislav, Heisig, Johannes, Leal Parente, Leandro, Wright, Marvin, Herold, Martin, & de Bruin, Sytze. (2022). Potential and realized distribution at 30m for Olive tree (Olea europaea) in Europe for 2000 – 2020 [Data set]. Zenodo. https://doi.org/10.5281/zenodo.5879372
• Bonannella, Carmelo, Hengl, Tomislav, Heisig, Johannes, Leal Parente, Leandro, Wright, Marvin, Herold, Martin, & de Bruin, Sytze. (2022). Potential and realized distribution at 30m for Norway spruce (Picea abies) in Europe for 2000 – 2020 [Data set]. Zenodo. https://doi.org/10.5281/zenodo.5881700
• Bonannella, Carmelo, Hengl, Tomislav, Heisig, Johannes, Leal Parente, Leandro, Wright, Marvin, Herold, Martin, & de Bruin, Sytze. (2022). Potential and realized distribution at 30m for Aleppo pine (Pinus halepensis) in Europe for 2000 – 2020 [Data set]. Zenodo. https://doi.org/10.5281/zenodo.5882764
• Bonannella, Carmelo, Hengl, Tomislav, Heisig, Johannes, Leal Parente, Leandro, Wright, Marvin, Herold, Martin, & de Bruin, Sytze. (2022). Potential and realized distribution at 30m for Austrian pine (Pinus nigra) in Europe for 2000 – 2020 [Data set]. Zenodo. https://doi.org/10.5281/zenodo.5883181
• Bonannella, Carmelo, Hengl, Tomislav, Heisig, Johannes, Leal Parente, Leandro, Wright, Marvin, Herold, Martin, & de Bruin, Sytze. (2022). Potential and realized distribution at 30m for Stone pine (Pinus pinea) in Europe for 2000 – 2020 [Data set]. Zenodo. https://doi.org/10.5281/zenodo.5883782
• Bonannella, Carmelo, Hengl, Tomislav, Heisig, Johannes, Leal Parente, Leandro, Wright, Marvin, Herold, Martin, & de Bruin, Sytze. (2022). Potential and realized distribution at 30m for Scots pine (Pinus sylvestris) in Europe for 2000 – 2020 [Data set]. Zenodo. https://doi.org/10.5281/zenodo.5884262
• Bonannella, Carmelo, Hengl, Tomislav, Heisig, Johannes, Leal Parente, Leandro, Wright, Marvin, Herold, Martin, & de Bruin, Sytze. (2022). Potential and realized distribution at 30m for Sweet cherry (Prunus avium) in Europe for 2000 – 2020 [Data set]. Zenodo. https://doi.org/10.5281/zenodo.5885197
• Bonannella, Carmelo, Hengl, Tomislav, Heisig, Johannes, Leal Parente, Leandro, Wright, Marvin, Herold, Martin, & de Bruin, Sytze. (2022). Potential and realized distribution at 30m for Turkey oak (Quercus cerris) in Europe for 2000 – 2020 [Data set]. Zenodo. https://doi.org/10.5281/zenodo.5886709
• Bonannella, Carmelo, Hengl, Tomislav, Heisig, Johannes, Leal Parente, Leandro, Wright, Marvin, Herold, Martin, & de Bruin, Sytze. (2022). Potential and realized distribution at 30m for Holm oak (Quercus ilex) in Europe for 2000 – 2020 [Data set]. Zenodo. https://doi.org/10.5281/zenodo.5887008
• Bonannella, Carmelo, Hengl, Tomislav, Heisig, Johannes, Leal Parente, Leandro, Wright, Marvin, Herold, Martin, & de Bruin, Sytze. (2022). Potential and realized distribution at 30m for pedunculate oak (Quercus robur) in Europe for 2000 – 2020 [Data set]. Zenodo. https://doi.org/10.5281/zenodo.5887416
• Bonannella, Carmelo, Hengl, Tomislav, Heisig, Johannes, Leal Parente, Leandro, Wright, Marvin, Herold, Martin, & de Bruin, Sytze. (2022). Potential and realized distribution at 30m for Cork oak (Quercus suber) in Europe for 2000 – 2020 [Data set]. Zenodo. https://doi.org/10.5281/zenodo.5896621
• Bonannella, Carmelo, Hengl, Tomislav, Heisig, Johannes, Leal Parente, Leandro, Wright, Marvin, Herold, Martin, & de Bruin, Sytze. (2022). Potential and realized distribution at 30m for Goat willow (Salix caprea) in Europe for 2000 – 2020 [Data set]. Zenodo. https://doi.org/10.5281/zenodo.5896666
• Ibrahim Saleem; Landa Martin; Pešek Ondřej; Brodský Lukáš; Halounová Lena (2022). Geo-Harmonized PM2.5 maps over Europe for the years 2018-2020 (v1.0) [Data set]. Zenodo. https://zenodo.org/record/6798975

Training materials

Python tutorials:

• https://eumap.readthedocs.io/en/latest/tutorials.html

R tutorials:

• Hengl, T., Parente, L. & Bonannella, C. (2022). Spatial and spatiotemporal interpolation using Ensemble Machine Learning: https://opengeohub.github.io/spatial-prediction-eml/index.html
• Hengl, T., Parente, L., & Bonannella, C. (2021, September 17). Predictive mapping using spatiotemporal Ensemble ML (R tutorial). Zenodo. https://doi.org/10.5281/zenodo.5513827

Video tutorials:

• Antonić L.: “Introduction to spatial and spatiotemporal data in Python”
• Brodský L., Landa M., Bouček T.: “Working with harmonized LUCAS datasets”
• Hengl, T., Bonannella C.: “Spatiotemporal Ensemble ML in R”
• Hengl T.: “Computing with Cloud-Optimized GeoTIFFs”
• Hengl T.: “Introduction to spatial and spatiotemporal data in R”
• Hengl T.: “Modeling with spatial and spatiotemporal data in R: spatial interpolation”
• Landa M.: “Using OGC Web Services in Python”
• Parente L.: “Introduction to ODSE datasets in Python”
• Parente L., Antonić L.: “Working with Cloud-Optimized GeoTIFFs in Python”
• Parente L., van Diemen C.: “Spatiotemporal machine learning in Python”
• Parente L. “High performance computing in Python”

A copy of Videos with DOI is available via: Open Data Science Workshop series (see also complete list of videos).

Publications

To subscribe for Geo-harmonizer publications please use the project code “2018-EU-IA-0095”. Currently submitted / published publications:

• Ibrahim, S.; Landa, M.; Pešek, O.; Brodský, L.; Halounová, L. Machine Learning-Based Approach Using Open Data to Estimate PM2.5 over Europe. Remote Sens. 2022, 14, 3392. https://doi.org/10.3390/rs14143392
• Bonannella, C., Hengl, T., Heisig, J., Parente, L., Wright, M.N., Herold, M., de Bruin, S. (2022). Forest tree species distribution for Europe 2000–2020: mapping potential and realized distributions using spatiotemporal Machine Learning. PeerJ, accepted, https://doi.org/10.21203/rs.3.rs-1252972/v3
• Ibrahim, S., Landa, M., Pešek, O., Pavelka, K., & Halounova, L. (2021). Space-Time Machine Learning Models to Analyze COVID-19 Pandemic Lockdown Effects on Aerosol Optical Depth over Europe. Remote Sensing, 13(15), 3027. https://doi.org/10.3390/rs13153027
• Landa M, Brodský L, Halounová L, Bouček T, Pešek O. (2022). Open Geospatial System for LUCAS In Situ Data Harmonization and Distribution. ISPRS International Journal of Geo-Information, 11(7):361. https://doi.org/10.3390/ijgi11070361
• Witjes, M., Parente, L., van Diemen, C. J., Hengl, T., Landa, M., Brodsky, L., … & Glusica, L. (2022). A spatiotemporal ensemble machine learning framework for generating land use/land cover time-series maps for Europe (2000–2019) based on LUCAS, CORINE and GLAD Landsat. PeerJ, accepted, https://doi.org/10.21203/rs.3.rs-561383/v4
• Hengl, T., Parente, L., & Bonannella, C. (2022). Spatial and Spatiotemporal Interpolation / Prediction using Ensemble Machine Learning (v0.1) [Computer software]. Zenodo. https://doi.org/10.5281/zenodo.5894924 
• Sheykhmousa, M., Mahdianpari, M., Ghanbari, H., Mohammadimanesh, F., Ghamisi, P., & Homayouni, S. (2020). Support vector machine vs. random forest for remote sensing image classification: A meta-analysis and systematic review. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. https://doi.org/10.1109/JSTARS.2020.3026724 
• Ibrahim, S., & Halounova, L. (2019). Statistical study of MODIS algorithms in estimating aerosol optical depth over the Czech Republic. Civil Engineering Journal, (4). https://dx.doi.org/10.14311/CEJ.2019.04.0043

Events organized

Complete list of events is available here.

International workshops:

• Open Data Science Europe Workshop 2021, 6-10 Sept, 2021, Wageningen International Conference Center;
• Open Data Science Europe Workshop 2022, 13-16 Jun, 2022, CVUT Prague, Czech Republic;

Geo-harmonizer project

Prepared by: OpenGeoHub and CTU in Prague

Executive summary

The python library eumap has been built to enable easier access to several spatial layers prepared for Continental Europe (Landsat and Sentinel mosaics, DTM and climate datasets, land cover and environment quality maps), as well the processing classes and functions used to produce them. The library implements efficient raster access through rasterio, multiple gapfiling approaches, spatial and spacetime overlay, training samples preparation (LUCAS points), and Ensemble Machine Learning applied to spatial predictions (fully compatible with scikit-learn). To learn more about eumap and Open Data Science Europe, please access the official documentation and watch some of the demos from the Open Data Science Workshop 2021:

Spatiotemporal overlay and machine learning using the mlr, terra and rgdal packages in R is also explained in detail in the following videos and the R markdown tutorial:

To learn more about how was these libraries were used to produce spatiotemporal predictions of land cover classes and forest tree species for the continental EU, please see:

• Hengl, T., Parente, L., & Bonannella, C. (2021, September 17). Predictive mapping using spatiotemporal Ensemble ML (R tutorial). Zenodo. https://doi.org/10.5281/zenodo.5513827
• Witjes, M., Parente, L., van Diemen, C. J., Hengl, T., Landa, M., Brodsky, L., … & Glusica, L. (2021??). A spatiotemporal ensemble machine learning framework for generating land use/land cover time-series maps for Europe (2000–2019) based on LUCAS, CORINE and GLAD Landsat. PeerJ, in review, https://doi.org/10.21203/rs.3.rs-561383/v4 
• Eumap library documentation: https://eumap.readthedocs.io
• Gitlab repository with all datasets and code: https://gitlab.com/geoharmonizer_inea/odse-workshop-2021/ & https://gitlab.com/geoharmonizer_inea/odse-workshop-2022/