Robin's Blog

Learning resources for GIS in Python with cloud-native geospatial, PostGIS and more

I recently gave a careers talk to students at Solent University, and through that I got to know a MSc student there who had previous GIS experience and was now doing a Data Analytics and AI MSc course. Her GIS experience was mostly in the ESRI stack (ArcGIS and related tools) and she was keen to learn other tools and how to combine her new Python and data knowledge with her previous GIS knowledge. I wrote her a long email with links to loads of resources and, with her permission, I’m publishing it here as it may be useful to others. The general focus is on the tools I use, which are mostly Python-focused, but also on becoming familiar with a range of tools rather than using tools from just one ecosystem (like ESRI). I hope it is useful to you.

Tools to investigate:

  • GDAL
    GDAL is a library that consists of two parts GDAL and OGR. It provides ways to read and write geospatial data formats like shapefile, geopackage, GeoJSON, GeoTIFF etc – both raster (GDAL) and vector (OGR). It has a load of command-line tools like gdal_translate, ogr2ogr, gdalwarp and so on. These are extremely useful for converting between formats, importing data to databases, doing basic processing etc. It will be very useful for you to become familiar with the GDAL command-line tools. It comes with a Python interface which is a bit of a pain to use, and there are nicer libraries that are easier for using GDAL functionality from Python. A good tutorial for the command-line tools is at https://courses.spatialthoughts.com/gdal-tools.html

  • Command line tools in general
    Getting familiar with running things from the command-line (Command Prompt on Windows) is very useful. On Windows I suggest installing ‘Windows Terminal’ (https://apps.microsoft.com/detail/9n0dx20hk701?hl=en-GB&gl=GB) and using that – but make sure you select standard command prompt not Powershell when you open a terminal using it.

  • Git
    There’s a good tutorial at https://swcarpentry.github.io/git-novice/

  • qgis2web
    This is a plugin for QGIS that will export a QGIS project to a standalone web map, with the styling/visualisation as close to the original QGIS project as they can manage. It’s very useful for exporting maps to share easily. There’s a tutorial at https://www.qgistutorials.com/en/docs/web_mapping_with_qgis2web.html and the main page is at https://plugins.qgis.org/plugins/qgis2web/

Python libraries to investigate:

  • GeoPandas – like pandas but including geometry columns for geospatial information. Try the geopandas explore() method, which will do an automatic webmap of a GeoPandas GeoDataFrame (like you did manually with Folium, but automatically)
  • rasterio – a nice wrapper around GDAL functionality that lets you easily load/manipulate/save raster geospatial data
  • fiona – a similar wrapper for vector data in GDAL/OGR
  • shapely – a library for representing vector geometries in Python – used by fiona, GeoPandas etc
  • rasterstats – a library for doing ‘zonal statistics’ – ie. getting raster values within a polygon, at a point etc

Cloud Native Geospatial
There’s a good ‘zine’ that explains the basics behind cloud-native geospatial – see https://zines.developmentseed.org/zines/cloud-native/. Understanding the basics of the topics in there would be good. There are loads of good tutorials online for using STAC catalogues, COG files and so on. See https://planetarycomputer.microsoft.com/docs/quickstarts/reading-stac/ and https://planetarycomputer.microsoft.com/docs/tutorials/cloudless-mosaic-sentinel2/ and https://github.com/microsoft/PlanetaryComputerExamples/blob/main/tutorials/surface_analysis.ipynb

My Blog
You can subscribe via email on the left-hand side at the bottom of the sidebar
Relevant posts:

Conference talks
These can be a really good way to get a brief intro to a topic, to know where to delve in deeper later. I often put them on and half-listen while I’m doing something else, and then switch to focusing on them fully if they get particularly interesting. There are loads of links here, don’t feel like you have to look at them all!

PostGIS Day conference: https://www.crunchydata.com/blog/postgis-day-2024-summary
Particularly relevant talks:

FOSS4G UK conference last year in Bristol: https://uk.osgeo.org/foss4guk2024/bristol.html
Most relevant talks for you are the following (just the slides):

FOSS4G conference YouTube videos: https://www.youtube.com/@FOSS4G/videos – they have a load of ones from 2022 at the top for some reason, but if you scroll down a long way you can find 2023 and 2024 stuff. Actually, better is to use this playlist of talks from the 2023 global conference: https://www.youtube.com/playlist?list=PLqa06jy1NEM2Kna9Gt_LDKZHv1dl4xUoZ
Here’s a few talks that might be particularly interesting/relevant to you, in no particular order

Suggestions for learning projects/tasks
(These are quite closely related to the MSc project that this student might be doing, but are probably useful for people generally)
I know when you’re starting off it is hard to work out what sort of things to do to develop your skills. One thing that is really useful is to become a bit of a ‘tool polyglot’, so you can do the same task in various tools depending on what makes sense in the circumstances.

I’ve listed a couple of tasks below. I’d suggest trying to complete them in a few ways:

  1. Using QGIS and clicking around in the GUI
  2. Using Python libraries like geopandas, rasterio and so on
  3. Using PostGIS
  4. (Possibly – not essential) Using the QGIS command-line, or model builder or similar

Task 1 – Flood risk

  1. Download the ‘Flood Zone 2’ flood risk data from https://environment.data.gov.uk/explore/86ec354f-d465-11e4-b09e-f0def148f590?download=true for a particular area (maybe the whole of Southampton?)
  2. Download OS data on buildings from this page – https://automaticknowledge.org/gb/ – you can download it for a specific local authority area
  3. Find all buildings at risk of flooding, and provide a count of buildings at risk and a map of buildings at risk (static map or web map)
  4. Extension task: also provide a total ground area of buildings at risk

Task 2 – Elevation data
(Don’t do this with PostGIS as its raster functionality isn’t great, but you could probably do all of this with GDAL command-line tools if you wanted)

  1. Download Digital Terrain Model data from https://environment.data.gov.uk/survey – download multiple tiles
  2. Mosaic the tiles together into one large image file
  3. Do some basic processing on the DEM data. For example, try:
    a) Subtracting the minimum value, so the lowest elevation comes out as a value of zero
    b) Running a smoothing algorithm across the DEM to remove noise
  4. Produce a map – either static or web map

If you found this post useful, please consider buying me a coffee.
This post originally appeared on Robin's Blog.

Tagged with:

Categorised as: Academic, GIS, Programming, Python, Remote Sensing

One Comment

  1. Scott says:
    March 28, 2025 at 9:45 pm

    It’s good to see GDAL right at the top. Every other geospatial software uses it via libgdal.

    More often than not, using the GDAL utilities directly negate the use of any other software, including python.

Leave a Reply

Your email address will not be published. Required fields are marked *