peawee dot net

No, this post isn’t Star Trek related. It’s about SCIENCE!

I realize that the real world makes a lot of things difficult for those who collect data. Consider this, then, a gripe list of things I may or may not expect to get fixed.

Moving on, I work on computer models to better understand physical processes. These models are built and parameterized off of real world data of some way or another. This means I consume a lot of data in my job. However, how I as a modeler would like data is not often how I get the data. Here’s some of the bigger issues:

• Oddball formats. On a given day, I have to deal with files in Excel format, varieties of plain text, oddball Fortran binary files (undocumented, of course), NetCDF, HDF, ESRI Shapefiles, GeoTIFF, and a whole bunch of other raster formats. And this doesn’t particularly phase me, either. What phases me? Incomplete oddball formats. Whenever a new and interesting bit of geodata ends up on my desk in an oddball format, I can wind up spending over a day poking at it. When I get gridded data, I need to know a few details about each datapoint in the grid:
  1. The lat/lon coordinates
  2. Units being used
  3. The digital representation of the data

  Most data I get tends to follow this: well-put-together NetCDF & HDF files, as well as anything produced by or with a GIS (such as GeoTIFF, Shapefiles, etc.), tend to be alright. I can work with this. However, if I have to figure things out, it slows me down. Sometimes a lot if I misinterpret it- if it looks reasonable to my eye but actually is not, this Can Cause Baddness. This is largely the case with Fortran binary files, but I get a lot of plain text that also shares these problems- I have an unclear idea of where or what is being represented. Examples:

  1. Points are addressed from 0 to 360 degrees east instead of -180 to 180 (or vice versa)
  2. There may be a README file with the line “value is thousands / 20″ or something similarly vague.
• Oddball Measurements. Let’s start with something basic. If a value is between zero and one, then why do your measurements include numbers both less than zero and greater than one? Saying “sometimes in real life, things work out like that” doesn’t satisfy me in building a numerical model where the textbook definition of a function expects a value to be between zero and one. Tell me why this is, with perhaps a textbook reference going more in depth. I don’t mind that your measurements falls outside the traditionally accepted bounds of reality . I mind that there’s no systematic explanation why this is.
• Oddball time-series. In modeling, we want to see a lot of things over time. The more fine-grained our models become, the more fine-grained we want our measurement data. If I’m simulating the growth of a crop, I would like to see the growth of the crop as well as after it’s done growing. It’s hard to determine a trend if one only has one to two datapoints to work from.

I suppose the grand summary of this griping is that as a modeler, I have my hands full with a lot of things. Spending weeks trying to work with bad data is frustrating; human hands entered the data into a computer, so why can’t those same hands explain the data?

Welcome to the fun and glamorous side of computers!
- my Dad, whenever I was frustrated with progress bars as a child

ESRI, makers of ArcMap: Why, oh why, can’t you at least multi-thread ArcMap. Or make it 64-bit. Or *something*. I’m severely not a fan of ArcMap taking 5,000 years to complete a spatial join. Yes, I know I’m doing this on a 1.83 GHz Mac Mini. But it has two cores, and it’s 64-bit. And it runs MATLAB very nicely, thankyouverymuch, so it’s not impotent by any means.

I’m going to grow old and die before I have the 2000-2005 temperate forest cover change mapped into a 0.5° grid. Gaah.

I’ve been using Windows 7 a lot lately.  In fact, I’ve taken to bringing my dual boot MacBook from home to work every day as I find myself hunched over the small thing most of the time doing work.  It has a smaller keyboard, wireless network (as opposed to 1 Gbps ethernet), and a 13” screen as opposed to my desktop’s huge 24” Dell.

I’m a proud Mac fanboy.  When it works, it works well.  Things have a very high value of JustWork, and more or less I’ve been very pleased with it.

My problem comes into being with trying to interact a lot with more of the outside world.  Office 2008 for the Mac manages to snatch Defeat from the Jaws of Victory.  It’s slow and buggy.  The PDFs that Apple products create (such as from Pages) tend to be incorrect when viewed from Adobe products.  For better or worse, ESRI’s ArcGIS consumes enough resources that it cannot be run in a virtual machine on my mac.

All of these issues were still “put-up-able” enough while the choice was XP or Vista versus MacOS.  I like how Apple has set up the user interface.  That plus a couple of free programs (like Blacktree’s excellent Quicksilver launcher application) mean that I can really focus on doing things without having to think about how I’m doing it.

Now it seems that Microsoft has managed to figure out most of what’s wrong with using Windows with Windows 7.  The interface for most everything I’m using is regular and normal. Its search functionality is as good as Apple’s Spotlight (which admittedly was a Vista feature), and all in all it has a very high value of JustWork.  It moves the major annoyances of computer use from the OS (to which MacOS was the cure), to the applications I’m using.

I’m using Windows 7 right now.  And I like it.