Summary: Fascinating book covering the whole breadth of high performance Python. It starts with detailed discussion of various profiling methods, continues with chapters on performance in standard Python, then focuses on high performance using arrays, compiling to C and various approaches to parallel programming. I learnt a lot from the book, and have already started improving […]
The last few months have seen a flurry of activity in Py6S – probably caused by procrastinating from working on my PhD thesis! Anyway, I thought it was about time that I summarised the various updates and new features which have been released, and gave a few more details on how to use them. These […]
As part of my PhD I’ve developed a number of algorithms which are implemented as a class in Python code. An example would be something like this: class Algorithm: def __init__(self, input_filename, output_basename, thresh, n_iter=10): self.input_filename = input_filename self.output_basename = output_basename self.thresh = thresh self.n_iter = n_iter def run(self): self.preprocess() self.do_iterations() self.postprocess() def preprocess(self): # […]
I do a lot of my academic programming in Python, and – even though I often write about the importance of reproducible research – I don’t always document my code very well. This sometimes leads to problems where I have some code running fine, but I don’t know which modules it requires. These could be […]
10. Can you re-generate any intermediate data set from the original raw data by running a series of scripts? It depends which of my projects you’re talking about. For some of my nicely self-contained projects then this is very easy – everything is encapsulated in a script or a series of scripts, and you can […]
Recently I ran into a situation where I needed to select Landsat scenes by various criteria – for example, to find images over a certain location, within a certain date range, with other requirements on cloudiness and so on. Normally I’d do this sort of filtering using a tool like EarthExplorer, but I needed to […]
7. Do you use version control for your scripts? Yes, almost always. I’ve found this a lot easier since I started using Git – to start using version control with Git simply requires running "git init" – whereas with SVN you had to configure a new repository and do all sorts of admin work before […]
I recently read the article Don’t be a technical masochist on John D. Cook’s blog, and it struck a chord with me about the way that I see people choosing software and programming tools in my field. John states "Sometimes tech choices are that easy: if something is too hard, stop doing it. A great […]
This is the second in my series of posts examining how well I fulfil each of the items on the Philip Test. The first part, with an explanation of exactly what this is, is available here, this time we’re moving on to the next three items in the list: 4. Are your scripts, data sets […]
Philip Guo, who writes a wonderful blog on his views and experiences of academia – including a lot of interesting programming stuff – came up with a research programming version of The Joel Test last summer, and since then I’ve been thinking of writing a series commenting on how well I fulfil each of the items on […]