Sometimes you may find that your Mac OS X installation has an old URL for the software update service in its preferences, which may mean that you can’t successfully run the Software Update tool under the Apple menu. In my case, I got my Mac from my university and the update server was set to some internal server in their organisation. This server didn’t seem to have the latest updates, and half of the time I couldn’t even connect to it.
From various searches around the internet I found the following process to reset the update server URL to the default:
Open the /Library/Preferences folder on the main hard drive.
Find the com.apple.SoftwareUpdate.plist file and move this to the desktop (the easiest way is just to drag and drop the file to the Desktop). If you find a similar file with a .apple at the end (com.apple.SoftwareUpdate.plist.apple) then move that too. Moving them to the desktop means that you can restore them if this goes wrong.
Run the Software Update tool (Software Update on the Apple menu on the top left of the main menu bar)
It should all work fine! If not, then you can always restore the files that you put on the desktop (just move them back to /Library/Preferences)
I was thinking recently about a post I made a while back about my academic values, and happened to come across the Reproducible Research website. I can’t emphasise how much I agree with the idea of reproducible research – it would remove so many problems that I have been having with my research, and will help my research to fulfil one of the key requirements of science, that is to be fully reproducible.
When trying to implement image segmentation algorithms for my RTWOBIA project (which is still progressing, albeit very slowly) I had great problems with (a) understanding and (b) implementing the algorithms in the literature. Some of these papers deliberately left out details of the algorithms for commercial reasons (for example Baatz and Schape (2000) who describe the algorithm used for multi-resolution segmentation in eCognition, but leave out some key details), but others just didn’t seem to include the details that would be required to implement the algorithm.
This was incredibly frustrating when trying to implement (and possibly extend) these algorithms. Obviously, if I had been able to implement them (and I am still slowly working on the implementation) then this would have benefited both me and the authors of the paper, as I would have been able to continue with my research, and the authors would have received citations from me and greater exposure of their work.
The concept of reproducible research aims to change this, by ensuring that all published research is accompanied by a website containing all of the details of the algorithm, the code used by the authors to produce the results shown in the paper, and the data used in the study. There is a very simple how-to document available which lists everything that should be provided to enable the research to be fully reproduced, and a very readable paper by Vandewalle and Kovacevic (2009) which explains the idea and the benefits. These benefits are for both the readers and the authors: the readers will have all of the information needed to reproduce the research, and therefore all of the information needed to use the research in their work, or to extend it. The authors are likely to get more citations due to this ease of reproducibility, and, even more important, will find it easier to reproduce their research themselves. I’ve lost track of how many times I’ve struggled to reproduce research outputs that I’ve produced, and ensuring that anyone can reproduce them should help the original authors to reproduce them too!
Unfortunately, I can see a number of issues with fully implementing the concepts of reproducible research in the fields of remote sensing and GIS. However, I will start by working on reproducible research pages for my current publications, and post a follow-up post detailing the issues (and hopefully the solutions) for doing reproducible research in my field.
References:
Baatz, Martin, and Arno Schape. 2000. Multiresolution Segmentation: an optimization approach for high quality multi-scale image segmentation, Journal of Photogrammetry and Remote Sensing 58, no. 3-4: 12-23.
Vandewalle, Patrick and Kovacevic, Jelena and Vetterli, Martin (2009) Reproducible Research in Signal Processing – What, why, and how. IEEE Signal Processing Magazine, 26 (3). pp. 37-47. PDF
Summary: If you can get past the strange underlying story, then this gives a good introduction to R to someone with no programming experience. However, if you have any experience with other programming languages then another book is likely to be more suitable.
Reference: Quick, J. M., Statistical Analysis in R: Beginners Guide, Packt Publishing, Birmingham, UK, 300 pages. Sample Chapter
As a new science PhD student, I wanted to get to grips with the most powerful statistical analysis software around, and that meant trying to understand the intricacies of programming in the R Project for Statistical Computing. John M. Quick has posted extensively on the internet about programming in R, so I had high hopes for his new book, Statistical Analysis with R in the Beginner’s Guide series recently published by Packt Publishing. Unfortunately, my high hopes were not entirely fulfilled. The book provides useful and correct information about programming in R, but it is underlain by a strange story about Chinese wars and has a number of niggling problems that prevent me from fully recommending it.
Before starting, I should explain my previous experience. I have used a number of pieces of statistical software in the past (such as SPSS) and have a small amount of experience with R from a PhD statistics skills class. I do, however, have significant programming experience in a number of languages. I think this is the part of my experience that distinguishes me from the intended audience for this book, as it is designed for real beginners. Those who have any experience in a modern programming language or linux shell will find the first few chapters very easy, and therefore somewhat frustrating. However, these do give a good basic introduction to running commands in the R shell, and working out which lines are commands and which are outputs. It also explains the [1] that appears at the beginning of R output lines, which is not mentioned in many introduction to R tutorials that I have read. Once we’re past the rather contrived example of solving a magic square by using R as a calculator the interesting bit starts…
However, before describing the data analysis section of the book, I should explain the underlying story used throughout the book. The introductory chapter gives a bit of ancient Chinese history, and states that you, the reader, have been chosen to succeed the famous military leader Zhuge Liang and need to learn how to use R to analyse his data and plan the future of the military campaign. The rest of the book takes on this theme, both in the data analysis (comparing the Shu and Wei armies, and predicting battle outcomes using regression) and the general phrasing (headings like "Have a go hero!" and emphasis that if you fail the Chinese kingdom will collapse). I’ll be honest: this story doesn’t work for me at all. In fact, it drives me nuts having it constantly throughout the book. As for why it annoys me, I’m not entirely sure: probably partly because it seems like the examples have had to be twisted rather to make them fit the story, and partly because I have no interest in ancient Chinese kingdoms, or using R to plan military campaigns.
I understand that not all readers will agree with me here, and that putting a story like this behind the scary process of learning new statistics software may help people get to grips with it. From my point of view I would have preferred to see a range of datasets used from the examples provided with R (all of the datasets listed here are built in to R), as this would (a) mean that the datasets are always available from within R and (b) provide interest for a wide range of readers.
So, apart from my personal views of the underlying story, the actual content of the book is quite good. The chapters cover the whole process of statistical analysis from data import (Chapter 4), through summary statistics (Chapter 5) and modelling (Chapters 5-7) to graphical output (Chapter 10). The final chapter of the book gives good pointers for more help on R, from the inbuilt help through to recommended blogs and websites. It also covers installing packages, although more emphasis could have been made of how useful packages can be when performing analysis in R. The book assumes some statistical knowledge, but briefly explains concepts the reader may not have experienced before (such as correlation coefficients and AIC). Each chapter takes the form of some instructions (‘Time for action’), followed by an explanation (‘What just happened?’), a few questions (‘Pop quiz’) and a suggestion for the reader to try and do (‘Have a go hero’), and this approach seems to suit the material quite well. Although it can get frustrating at times (I tend to automatically skip over quizzes in books), I think the structure would help less confident readers.
The range of content that the book covers is impressive, as it goes from installing R to comparing models using AIC and customising graphs, although at times the explanations seem a bit verbose. More worryingly, the code examples, although completely correct, are written in a programming style that I suspect no real-world R programmer uses. The arguments for each command are stored as variables before the command is run (not that unusual for complex arguments, but a bit strange to do for every argument) and these variables have incredibly long names. The code snippet below (from page 172) is a good example:
&> #create a box plot that compares the number of soldiers required across the battle methods
&> #get the data formula to be used in the plot
&> boxplotAllMethodsShuSoldiersData <- battleHistory$ShuSoldiers ~ battleHistory$Method
&> #customize the plot
&> boxPlotAllMethodsShuSoldiersLabelMain <- "Number of Soldiers Required by Battle Method"
&> boxPlotAllMethodsShuSoldiersLabelX <- "Battle Method"
&> boxPlotAllMethodsShuSoldiersLabelY <- "Number of Soldiers"
&> #use boxplot(...) to create and display the box plot
&> boxplot(formula = boxplotAllMethodsShuSoldiersData, main =
boxPlotAllMethodsShuSoldiersLabelMain, xlab = boxPlotAllMethodsShuSoldiersLabelX,
ylab = boxPlotAllMethodsShuSoldiersLabelY)
John’s code samples on the internet (for example here) do not have this verboseness, so I assume he put it in to try and make the code samples more easily readable. The problem is that, particularly when there is no colour syntax highlighting in the book, it actually makes it far more difficult to read and understand the code.
In conclusion, this book provides a good overview of using R and is correctly pitched for an audience of beginners. The underlying story frustrates me, but this is likely to be a matter of personal taste (try looking at the sample chapter linked at the top of this post to see how you feel about the story). Apart from the verbosity of the code examples the information is accurate and up-to-date. I would recommend this book for someone who has absolutely no experience with R or other programming languages and is somewhat scared of trying to learn R, as the underlying story and structure will provide a safe and comfortable environment for learning, but for those who feel they are more confident another book may be more suitable.
(Disclaimer: I was given a free review copy of this book by PacktPub)
(Yes the title is meant to remind you of snakes on a plane!)
Goodness! At this time of year there really are a lot of people coughing on trains! I travelled from Southampton to Tunbridge Wells today, and my journey was accompanied by a large range of coughs and sneezes, from the high-pitched cough of the lady opposite me, to the spluttering of an old gentleman in the corner. Leaving aside the public health aspects of coughing (particularly as a number of my fellow travellers did not cover their mouths when they coughed), I wondered if this could be used somehow to analyse the severity of cough/cold illnesses in the UK.
Proposal: All we need is a few willing volunteers who commute on trains. We can give them tally-counter to carry (they could count on a piece of paper, or in their head, but I’d get terribly confused doing that), and instruct them to count every cough they hear in their part of the carriage. This data can then be plotted on a graph over time, and (if we have enough people), over geographic area, and may provide a useful public health dataset. On the other hand, it may be rubbish! But who will know unless we try…
Unfortunately (well, fortunately for me!), I don’t commute on a train, but I’ll analyse the data if you’ll collect it!
Of course all we need now is to find a way to get commuters to count coughs while still reading the Metro!
Well, as mentioned before, I’ve recently been given (by my PhD funding body) a MacBook Pro laptop. I’m really getting to like it, but there are still some things that frustrate me. So, this will be one of two posts: my favourite things, followed by my least-favourite things. So, the good bits first:
Instant Sleep/Wakeup
This feature has actually replaced my desire for an iPad! When I close the lid of my MacBook it goes to sleep in about ten seconds and I can put it back in my bag. When I want to use it again, I simply open it up, tap a key, and I’m immediately back exactly where I was. This has really changed how I use my laptop. I’d only get out my Windows laptop when I was sure I’d have a significant amount of time to use it, as it took so long to wake up from sleep/hibernation mode. Now, if I have five minutes to spare I just whip out my laptop and manage to get around four minutes work done! Also, just as an aside, the sleep indicator on the MacBook Pro flashes in a very soothing way…hardly surprising as they deliberately designed it to mimic the human breathing rate during sleep (see here).
POSIX base
I’ve always been a fan of linux (I installed my first Linux distro at the age of around 12), but I’ve never really managed to persuade myself to use it on my main every-day machine. Servers yes, but laptops no – it didn’t have the right software, it was too finicky to do stuff with when I need my laptop to work every day. But now I have a solution – OS X is a very polished desktop operating system, with a linux-y (technically, POSIX-compatible) base. If I want to use GCC on my Mac, it’s there. If I want to install some linux software it’ll often just work. If I want to use a linux-based GUI then an X server is built in to the operating system. And, probably most importantly, there is a decent bash terminal built-in to the operating system. Best of both worlds? Definitely!
Touchpad
I didn’t realise how great this was until I moved back to my Windows laptop for a while. The touchpad on my MacBook is just so smooth and easy to use. Two-fingered scrolling is so easy, as is pinch-to-zoom. I’ve always hated scrolling on touchpads before (they often have a horrible ‘scroll zone’ on the right-hand side) but on my MacBook I use the touchpad to scroll even if I’m plugged into an external mouse!
So, those are my three favourite things. It’s interesting to note that these three things aren’t particularly difficult or complicated things, but they just make my MacBook far nicer to use on a day-to-day basis.
Next time: what frustrates me about my MacBook Pro.
This is the first in a series of posts on the lessons I’ve learnt from various episodes in my life. First up: my dissertation.
In case you’re not familiar with dissertations: they are the large written projects which are often given to students in the final year of their degree. The details of mine are available on my academic website. Here I want to try and distil my experience into some generically applicable lessons that can be applied to other dissertations, PhD theses, other projects…and maybe even life generally!
1. Start early
Apparently the most common piece of feedback from students after they’ve finished their dissertations is "I wish I’d started it earlier". I took this advice and started very early, doing a lot of work on my dissertation during the Easter holiday of my second year, but I still had a bit of a rush at the end. I also put an extra week into my schedule to allow for ‘unforeseen problems’ and was really very glad I had done this, as I suddenly found a major problem with my work a week before the deadline!
2. Keep things organised
I’m a big fan of PhD Comics and one of their comics perfectly describes the bad habits I tend to fall into: A story in filenames. As I’m working in remote sensing I have a large number of image files, both original images from satellites, and a large number of images I’ve created through processing the original files. It gets terribly difficult to find sensible names for the files, particularly when you’re a rushing a bit, and so I often slip into calling them test.bsq or test_again_more.bsq or even the particularly bad test22222_robin.bsq. Through doing this I’ve spent far more time than I need trying to find particular files, and have often ended up with file duplicates taking up extra hard drive space.
3. Spend a lot of time just thinking
I find it very easy to slip into a very mechanical way of doing my academic work: click this button, save this file, produce this graph, write this paragraph – so much so, that I sometimes forget to actually properly think about my work. I got into this phase a number of times during my dissertation, and in fact the major problem I found at the end was caused by this (I’d blindly continued down my planned path for the discussion section without realising that I wasn’t actually discussing the questions I was meant to be answering). I was reminded of this today when I started writing a very simple, mechanical discussion section for a project – along the lines of "This value is big, this means X. This other value is small, this means Y. Overall this worked ok-ish" (although obviously in more academic language!). At this point I noticed, forced myself to stop, and then took my notes away from my computer and went to have a think. I actually went and sat on the sofa near the School of Geography reception and sat and thought deeply about the problem. Fifteen minutes later I came away with a lot more insight into my project, what I’d found, and what further work could be done.
I should probably first state that I’m writing this a few days after I fixed this problem, so I may have some of the details wrong. Apologies if that is the case. Now, on to the problem:
Problem
You have EndNote and Microsoft Word 2010 installed. When loading Microsoft Word the EndNote tab that normally appears has been replaced with a tab labelled EndNote Web. When clicking on this tab you get asked to login to EndNote Web, and cannot access any references you have stored in your local EndNote library.
Solution
The solution to your problem depends exactly how badly things have gone wrong. Try the following steps in order:
Change EndNote Cite-While-You-Write (Cwyw) Settings: Go to the EndNote Web tab in Word and click the Preferences button. Go to the Application tab and look for the Application dropdown. Use this to select EndNote rather than EndNote Web. Once you’ve changed this, restart Word and it should work. However, you may have found that the Application dropdown is greyed out and can’t be changed. If so, go to step 2.
Remove EndNote Cwyw add-in and reinstall: Go to the File menu on the left of the tab bar and select Options and then the Add-Ins option on the left. You should see a list of Active Application Add-ins, in which there will be a number of EndNote related items. To remove these, select COM Add-ins in the dropdown box at the bottom, and then click Go. Select each of the EndNote-related items in the list and click Remove. Repeat this for all other items in the dropdown box so everything related to EndNote has been removed. Now the Cwyw functionality should have been completely removed from Word, so you can now re-enable it. The EndNote X4 Configuration Wizard (accessed through Configure EndNote in the Start Menu) has an option to enable EndNote in Word 2007, but not for Word 2010. Therefore, the only way I have found to re-enable EndNote in Word is to reinstall EndNote. Do that now (but make sure Word is closed first!), and then restart Word and it should work. If not, go to Step 3.
Remove every trace of EndNote from all of Word’s folders and then reinstall: I have found that EndNote seems to hide things away everywhere it possibly can, which makes it very difficult to be sure you’ve removed everything. To make this easier, I would download a copy of the Everything utility and search for "cwyw". You will get a number of results, including folders, .dll files and .dot and .dotm files. Delete all of the .dot and .dotm files. Then reinstall EndNote and, hopefully, everything will work!
Summary: Very unusual approach, but also provides an interesting new view of geography.
Reference: De Landa, M. 1997 A Thousand Years of Nonlinear History, Zone Books, New York. 333 pages. [amazon_link id="0942299329" target="_blank" ]Amazon Link[/amazon_link]
When dipping into a chapter entitled Geological History 1000-1700 AD one would expect to find information on rock types, the development of landforms and possibly the history of the development of geological thought. In Manuel De Landa’s book A Thousand Years of Nonlinear History however, this is not the case – what will actually be found is discussion of Christaller’s Central Place Theory, the development of urban areas in both Europe and the Far East and different philosophical perspectives on these. This aspect of surprise continues throughout the book – De Landa’s approach to all the topics covered is novel, and the insights gained from these approaches are huge.
Although the book is entitled A Thousand Years of Nonlinear History, it is by no means a standard history book – it focuses on the application of historical processes, and generally the passage of time, to many areas within human geography. The most important word in the title is probably ‘nonlinear’ as this is the way in which De Landa approaches all the areas covered in his book. It is very difficult to define what is meant by nonlinear – the author takes many pages for his explanation – but simply put it is considering history as a tree with many branches rather than one pure and straight linear course. This idea of nonlinearity is extended throughout the book to cover different types of nonlinear development (such as hierarchies and meshworks) and is used as part of the explanation for many areas of geographical development.
The book is divided into three parts (Lavas and Magmas, Flesh and Genes and Memes and Norms) each of which contains chapters which look at the specified topic from 1000-1700 AD and then from 1700-2000 AD. Sandwiched in the middle of each part is a section elaborating on some of the ideas introduced in the part – for example the Sandstone and Granite chapter within Lavas and Magmas elaborates on the ideas of hierarchies and meshworks, their definitions (within a variety of fields from biology to economics) and their effect on the development of urban geography. As mentioned in the first paragraph of this review, the names of the parts are metaphors for the content within them. For example, the first part is entitled Lavas and Magmas, and this metaphor is explained towards the end of the part by an analogy between lava and the physical constructs of cities. Some of these analogies are rather tenuous, but they all serve to give interesting new perspectives on familiar aspects of human geography.
Although De Landa’s book is very interesting, and in many ways unique, it is also a difficult read. This is really par for the course when one is explaining the sort of complex ideas which are used in this book, and some may find this book completely inaccessible because of the complexity of the ideas discussed. The majority of topics are explained very well, but some topics come across as rather confusing. Also, some of the language is rather pretentious, and one can’t help feeling that some of the ideas are not quite as complex as De Landa makes them out to be.
The presentation of A Thousand Years of Nonlinear History is, like the rest of the book, rather unusual. The striking front cover design makes the book stand out on a bookshelf, although the complexity of this cover design hinders the reading of the blurb on the back – one of the first places a prospective reader will look for information about the book. The choice of font size throughout the book is also interesting. De Landa has chosen to use larger font sizes at the beginning of each chapter, gradually reducing to a rather small font for the majority of each chapter and then increasing again towards the end. I assume this was chosen to accentuate the introduction and conclusion of each chapter, and in some ways that is a good aim. However, this has not helped my reading of the book, or my identification of the important parts of the chapter. It also has the side-effect of making the body of the chapter look very small, and this has made it quite difficult and tiring to read.
Overall, De Landa’s book is a very interesting read. It takes a new approach to almost every topic covered and provides much food for thought. Although A Thousand Years of Nonlinear History should not be used as the main text for any of the topics covered it provides much useful background reading. Some parts of the book are difficult to read and understand, but perseverance will result in appreciation of the new perspectives raised by this unusual book.
Summary: More mathematical than I thought it would be, but very thorough and well explained. I thought that there would be more instruction in how to program in IDL and use the ENVI API, but much of this can be learnt by examining the code given in the book. The book is very comprehensive and most major algorithms are covered in enough detail to allow the reader to implement them.
Reference: Canty, M. J., 2010, Image Analysis, Classification and Change Detection in Remote Sensing: With algorithms for ENVI/IDL, 2nd edition, 441 pages, ISBN: 978-1-4200-8713-0
I purchased this book towards the end of my undergraduate BSc Geography degree, as I was writing a number of image processing algorithms for my undergraduate dissertation, and thought that the book would be useful for the PhD in Remote Sensing that I would be starting soon. I’m happy to say that it has been very useful – read on to find out why.
I did not realise quite how mathematical the book was when I purchased it (I bought it through Amazon), and was slightly scared when I opened it to find headings like Theorem 4.2 and lots of mathematical symbols I’d never come across before! This certainly reminded me that one of key skills I need to improve in the next few years is my ability to read and understand mathematics. Once I got over the shock of seeing so much maths I found the book very useful. To start with I read it without really reading the maths, but have gradually managed to get in to the maths and understand the important bits.
I would recommend thoroughly reading and understanding Chapter 1 for those readers who don’t have much mathematical experience. I jumped in to later sections, but found that I could understand them far better once I’d read the basic material at the start. However, for someone with a higher mathematical understanding the later sections are likely to be able to be understood on their own.
So, what do these later sections contain? Well, the book can be divided into three main sections: Image Analysis, Classification and Change Detection (as you might expect from the title). Image analysis includes everything from simple image statistics and transformations (such as MNF and PCA) to filters, feature extraction and topographic modelling. Classification is split, as you would expect, into Unsupervised and Supervised, with sections focussed on Support Vector Machine classification and post-classification analysis. Finally, the Change Detection section focuses mainly on Canty’s own work on Multivariate Alteration Detection, but also covers some other methods such as simple differencing and PCA-based change detection.
I was impressed that Canty explains each algorithm from the ‘bottom up’ (which is where all of the mathematics comes in), in enough detail to allow the reader to write their own implementation. Where appropriate he mentions the ENVI built-in functions and often provides an explanation of the differences between ENVI’s implementation and the mathematics he has described. This approach has proved very useful for me when attempting to implement simple algorithms in languages such as Python.
The Appendices provide more mathematical information, efficient algorithms for neural-network training and, most importantly, a list of all of the ENVI code which is available from the author’s website. This code is well documented and of high quality, with good error-checking (a nice change to a lot of ENVI/IDL code that I have seen).
Overall, this book is a very useful addition to my Remote Sensing library. Simple answers about algorithms can be gained more easily from other books, but this provides the in-depth approach when that is required. I would recommend this book to postgraduate students, as most undergraduate students would not have the time or mathematical ability to make full use of it. It will be most useful to researchers who are implementing existing image analysis algorithms or those who are developing their own algorithms and want to see what has been done already so they can extend them. The focus on MAD in the Change Detection section is particularly useful as an example of a modern quantitative remote sensing algorithm.
Quite through chance I just happened to stumble upon the USGS New EarthExplorer, and I’m really quite impressed. Remote sensing is a field that is almost entirely based around new technology: satellite sensors, computer processing techniques and so on, but for ages actually acquiring the data has been an exercise in frustration.
Much as the interface of various alternative tools such as GloVIS look alright at first glance, but then have loads of little annoyances when you try to actually use them. From my (admittedly limited) experience so far, the New EarthExplorer seems to solve many of these issues.
When you browse to the website you enter a location (in pretty much any form) and it shows you a Google Map of the area. You can then select a date range for images, then various different imagery archives (Landsat 4/5, Landsat 7 SLC-on and Landsat 7 SLC-off). A list of images is then shown, and – here comes the key bit – you can then overlay the images on the Google Map. This really helps when you’re trying to find a number of images that cover the same area, particularly if your area of interest is near the edge of an image.
Once you’ve chosen your image a quick click of the download button will start the download of a zip file containing the image.
Simple, easy and quick – and, in my opinion, far better than the previous offerings.
