Robin's Blog

Interactive cloud frequency web map, with Google Earth Engine

Summary: I’ve developed an interactive cloud frequency map, available here. It may be particularly useful for satellite imaging researchers working out where they can acquire imagery easily.

Screen Shot 2015-07-25 at 22.32.11

One of the major issues with optical satellite imaging is that you can’t see through clouds: so normally when its cloudy, you can’t get anything useful from your images. This actually has a big effect on where you can use satellite imaging effectively: for example, a lot of people have used satellite data to monitor changes in the Amazon rainforest, but it’s quite challenging to find cloud-free images due to the climate in that region of the world.

Similarly, I remember a friend of mine struggling throughout his PhD with cloud cover. He was trying to observe vegetation in India, and needed to look at images taken around the monsoon because the vegetation was growing most vigorously at that time of year. The problem, of course, is that its very cloudy during the monsoon season – so there were barely any images he could use, and he ended up spending half of his PhD developing a new method to classify cloud from his images, so that he could extract the small fraction of the data that was actually usable. I’ve run into similar problems too – for example, some research in Hydrabad ran into problems caused by the limited availability of data due to cloud cover.

I’ve often found myself wanting to look at cloud frequency in different areas so that when I have a number of options for where to use as a case study for something, I can easily pick the area that is likely to have the most cloud-free data available. I’ve been a ‘Trusted Tester’ of Google Earth Engine for a long time, and had written a short script in Earth Engine to produce a map of cloud frequency.

Screen Shot 2015-07-25 at 22.49.22

I found myself using this frequently with colleagues, but I wasn’t able to share the data or interface with anyone easily. So, a few weekends ago I sat down and altered one of the EarthEngine demonstration applications (the ‘trendy-lights’ demo) to work with my cloud frequency code. After a bit of trial and error I got it working: the webapp is available here, and the code is on Github. I’m fairly new to Javascript development, but I think it all works fairly well: you should be able to search to find a location on the map, and you can click anywhere on the map to produce a pop-up box with the cloud cover percentage. The higher this percentage, the more days in a year are considered to be cloudy by the MODIS satellite (more details are available from the Info link in the top right).

So, I hope you find this useful (and even if you’re not using it as a satellite imaging researcher, you may find the cloud cover patterns across the world to be fascinating anyway…)


Categorised as: Academic, Programming, Remote Sensing


3 Comments

  1. Ali says:

    Hi Robin. This is quite interesting. Thanks for sharing your work.

    I tried to run your app but I got an error. I cloned your app from your repository and updated the application name in app.yaml, EE_ACCOUNT in config.py and the privatekey.pem file. I didn’t do any other change.

    I am getting the error shown below. And if I go to localhost:8080 the page is blank.

    I am running osgeo Lubuntu on VirtualBox running on Windows 8.1. (VirtualBox-5.0.0-101573-Win,
    osgeo-live-8.5, Python 2.7.6, Google App Engine 1.9.24)

    Do you have any idea what this error refers to?

    Any help is appreciated. Thanks,

    Ali

    [email protected]:~/shared/git/CloudFrequencyApp$ /home/user/google_appengine/dev_appserver.py ./
    INFO 2015-08-13 19:56:50,126 sdk_update_checker.py:229] Checking for updates to the SDK.
    INFO 2015-08-13 19:56:51,499 sdk_update_checker.py:257] The SDK is up to date.
    INFO 2015-08-13 19:56:51,574 api_server.py:205] Starting API server at: http://localhost:50930
    INFO 2015-08-13 19:56:51,579 dispatcher.py:197] Starting module “default” running at: http://localhost:8080
    INFO 2015-08-13 19:56:51,580 admin_server.py:118] Starting admin server at: http://localhost:8000
    INFO 2015-08-13 19:57:06,533 client.py:515] Attempting refresh to obtain initial access_token
    INFO 2015-08-13 19:57:06,578 client.py:755] Refreshing access_token
    ERROR 2015-08-13 19:57:09,146 wsgi.py:263]
    Traceback (most recent call last):
    File “/home/user/google_appengine/google/appengine/runtime/wsgi.py”, line 240, in Handle
    handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
    File “/home/user/google_appengine/google/appengine/runtime/wsgi.py”, line 299, in _LoadHandler
    handler, path, err = LoadObject(self._handler)
    File “/home/user/google_appengine/google/appengine/runtime/wsgi.py”, line 85, in LoadObject
    obj = __import__(path[0])
    File “/home/user/shared/git/CloudFrequencyApp/server.py”, line 254, in
    ee.Initialize(EE_CREDENTIALS)
    File “/home/user/shared/git/CloudFrequencyApp/ee/__init__.py”, line 83, in Initialize
    ApiFunction.initialize()
    File “/home/user/shared/git/CloudFrequencyApp/ee/apifunction.py”, line 145, in initialize
    signatures = data.getAlgorithms()
    File “/home/user/shared/git/CloudFrequencyApp/ee/data.py”, line 343, in getAlgorithms
    return send_(‘/algorithms’, {}, ‘GET’)
    File “/home/user/shared/git/CloudFrequencyApp/ee/data.py”, line 596, in send_
    response.status)
    EEException: Server returned HTTP code: 404
    INFO 2015-08-13 19:57:09,154 module.py:809] default: “GET / HTTP/1.1” 500 –

  2. Robin Wilson says:

    Hmm, I’m not really sure what’s going in there. You will need to make sure that you have a ‘Service Account’ set up for Earth Engine (as in the docs), but beyond that I’m not really sure. Maybe try posting on the EE Google Group for help?

  3. Ali says:

    Thanks for your answer. I posted my question to the GEE forum. My problem was solved by the GEE administrators. Basically my account had to be whitelisted. Now all is working as expected.

Leave a Reply

Your email address will not be published. Required fields are marked *