Since I’m doing PhD in geospatial science, I need to keep my finger on the pulse of anything location-based. And the Silicon Valley buzz last year was location-based social networks. Foursquare, Gowalla, Facebook Places.
The problem is, none of those services seem to fill a need. Why would you want to be “mayor” of a cafe? And if you want your friends to know where you are, why not use Twitter? It has geotagging.
One idea I came across for an LBS application went like this – you sign up for a service that continually tracks your movements and stores them in a database. It compares these movements with every other user in the database, and if it find that you cross paths with someone on a regular basis it recommends that you become friends. How sweet. No more Craiglist-style “missed connections”.
Now, putting on my scientist killjoy hat, there are some fundamental technical and practical flaws with that model – GPS power consumption, GPS not working indoors, dealing with multi-storey buildings, gaining a critical mass of users – but it got me thinking. Where are the location-based dating applications? In the online world, dating is a killer app. Why not the mobile world?
One thing I learned in my MBA is that advertising is more effective the closer you get to the point of sale. Ideally, you want to put ads in the supermarket. I’m sure the same applies to dating. A dating application is most effective when you’re all dressed up, drunk, and looking to pick up – in other words when you’re in a bar on a typical Friday or Saturday night.
Which is why I’m surprised that there are so few apps. The Economist mentions Skout and Flirtomatic (although the latter doesn’t seem to be location-based), and I’ve seen an iPhone app called Urban Signals in action. Nothing for Android 1.6, unfortunately, so I can’t comment on usability. But why haven’t the big players got involved? RSVP, eHarmony, and OkCupid certainly have the resources to pull it off.
I guess one of the problems is privacy. You probably don’t want to broadcast your exact location to all and sundry. Perhaps a system that releases more accurate information to people that meet your target profile. This needs some thought.
The release of augmented reality applications such as Wikitude and Layar has generated quite a lot of buzz, so I thought I’d try writing my own. It was also an opportunity to get familiar with the Android accelerometer, compass, and real-time camera APIs.
The application has in two modes. When the handset is held horizontally (landscape mode) is displays an augmented reality overlay, in this case a compass heading. When held normally (portrait mode) it displays a Google map, centred on your current location, that rotates to line up with the real world.
I was planning to use the application to display the radar rainfall data provided by the Bureau of Meteorology, centred on your current position. But unfortunately, the BoM doesn’t provide a central index of all their radar images along with location, range, and timestamp attributes. In theory I could scrape the site, but radars go down for maintenance and backup radars are switched on, so it wouldn’t be reliable. So I stuck with a simple compass heading overlay.
From a technical perspective, the application works fine. But from a usability perspective, there are some serious flaws.
- The magnetic compass is not reliable indoors. Too many electric fields.
- Holding up the phone in landscape mode is tiring, and it’s hard to find the information you need. The ergonomics are wrong.
- Rotating Google maps aren’t much use because the labels rotate as well, making them almost impossible to read.
Conclusion: when it comes to displaying spatial data on a smart phone, overlays on a static map are more useful than augmented reality or rotating maps, especially if supplemented with a discreet compess needle. Augmented reality would be pretty good if displayed using glasses, and rotating maps might be usable if the labels rotated (a la OpenStreetMap), but as things stand augmented reality is just a fun toy, not really useful.
I used every optimization trick I could think of, but couldn’t get the FFT routines fast enough to do real-time 3D sound. A 1.125 second clip, 49612 samples at 44.1kHz, takes 3.9 seconds with integer FFT, and 5.5 seconds with floating point.
That’s surprising. My HTC Tattoo has a 528 MHz CPU, so the integer conversion is taking about 2 billion clock cycles. The HRTF converts 128 samples at a time, and calculates the left and right channels separately, so that’s about 2.7 million clock cycles per call.
Each HRTF call does an FFT-256 and inverse FFT-256, plus some overhead. FFT runs in O(NlogN), so the inner loop must be taking about 650 clock cycles. Honestly, that’s a long time for code that consists of four array reads, four array writes, eight integer adds, four integer multiplies, and a comparison. Around 30 clock cycles per command? Crazy.
Potentially the code could be sped up by optimizing for FFT-256 (instead of generic FFT) and unrolling some loops. But that wouldn’t get the necessary four-fold speedup.
The other option is to write the FFT in native code. That may be fast enough, but it wouldn’t be portable across handsets. Seriously, why isn’t there an FFT class in the standard Java distribution?
But all is not lost. It turns out that 3D sound isn’t that effective at distinguishing front/back and up/down anyway. Maybe I’ve got strangely-shaped ears or crappy headphones, but I can only guess the direction slightly better than chance. But left/right is easy.
Your head is about 17cm across, and sound covers that distance in about half a millisecond. If you delay the sound coming into your left ear by that much, then the perceived direction is clearly from the right. So much so that you can’t actually hear the sound in your left ear, even though it’s playing at the same volume in both ears.
So left/right sound works really well, and it doesn’t require a fourier transform. You can’t distinguish between front/back, but you could overcome that with an interactive system that updates the left/right phase as you turn your head. That gets you 90 percent of the benefit, and the phone’s CPU would barely notice.
Back in July 2007 I attended the Mobile Media 2007 conference in Sydney, mostly because I was offered a free ticket by one of the sponsors. Generally the talks were too focused on social and cultural theory for my liking, with a lack of hard facts and working technology.
But one of them got my attention, “These Foolish Things: On Intimacy and Mobile Media” by Kate Crawford. It was the first time I’d seen Twitter in action, a rather impressive demo using its geotagging capability mashed up with Google maps.
The gist of the talk was that communications are becoming shorter and more frequent. Decades ago we spent hours writing a letter, and sent them monthly. Several years ago we spent a few minutes writing an e-mail, and sent them daily. Now we knock up a tweet or SMS in a few seconds and send them hourly.
So I wondered, can I get in front of this trend? Can I develop a form of communication that’s even faster and more superficial than SMS? Well, inspired by idea of scribbling on a Post-It note, I developed Scribble, an Android application that lets you draw a message on the touch screen and send the image as a JPEG via MMS.
There were a few technical problems.
- The customized toggle buttons used to select pen colour and size don’t appear correctly on all handsets.
- The action that’s supposed to launch the built-in Android MMS application, doesn’t. Instead you’re offered a choice of uploading to Picasa or sending as an e-mail attachment.
- The camera API captures images at a lower resolution than the screen, resulting in poor quality background images.
But those issues are probably not insurmountable. What really caused me to pull the plug on Scribble was the fact that writing on a touch screen with your fingertip is really hard. Witness my effort above. Entering text via T9 or a virtual keyboard is much faster.
So while there may be demand for a faster, more superficial, form of communication, Scribble isn’t it.
I’m currently working on three dimensional sound.
Have you ever noticed that we can figure out where sound comes from, even though we only have two ears? Sorting out left vs right is pretty easy, but how do we determine front/back and up/down? It turns out that the sound entering our ear canals follows multiple paths, bouncing off the structure of our ear and causing interference patterns. Sounds coming from different directions have different patterns and our brain picks up on that.
In 1994 a team from MIT set out to record these interference patterns, using a dummy head with dummy ears and two microphones. They placed the head in a recording studio and played audio samples from 710 different positions. The resulting recordings, when combined with some DSP magic, can be used to simulate sound coming from anywhere. This technique is known as Head-Related Transfer Function, or HRTF.
I’m trying to get this working on my Android through stereo headphones, but performance is a problem. The MIT data set was sampled at 44.1kHz, so it can only be used to transform audio sampled at that rate. Unfortunately my HTC Tattoo can’t do that in real-time. The current version of the software takes 5.9 seconds to transform a 1.1 second clip.
The bottleneck seems to be my Fast Fourier Transform code. After digging deeper I found that the Tattoo doesn’t have hardware floating point! Sheesh, it’s like living in 1995. Anyway, I’ll see if I can get it working with integer arithmetic and whether it’ll speed things up significantly.
Not exactly rocket science, but I built a simple spectrum analyzer for my Android phone. The main reason was to get familiar with the Android microphone API, but it was also a chance to test the Java port of some fourier transform code.
As a bonus the application turned out to be a useful testbed for trying out different colour schemes to represent intensities, something that may come in handy in the data visualization part of my thesis.
This project got its own blog so the press release would have something to refer to. If you want to know more, check out http://geosms.wordpress.com
Welcome to my blog.
A while back I decided to spend 20 per cent of my time on speculative research, not directly related to the PhD. Rather than let all that research effort go to waste, I though I’d make it available through a blog. There’s a bit of a backlog, but I’ll try to get it written up over the next few weeks.
Nearly all the research involves the development of Android smart phone applications. There are a few reasons for that …
- Smart phones are the fastest-moving technology at the moment, so there are lots of things still to invent.
- Smart phones are the fastest-moving technology at the moment, so it’s the area where your skills become obsolete fastest.
- I needed to write some Android apps for the thesis, and programming for Android is something that you learn by doing.
But above all, I’m really inspired by the potential of smart phones. I started writing software for a living back in the early 90’s, and for the first several years it was great. Learning new techniques and making a computer bend to your will. But then it started getting repetitive.
After all, there’s only so much you can do with a stand-alone personal computer, and after nearly a decade I’d exhausted most of the possibilities. Luckily, along came the internet, and a whole new world of possibilities opened up. You could write applications that communicated with other machines, and it became trivial to build cross-platform user interfaces thanks to HTML and browsers. But toward the end of the naughties even that started to wear thin.
Smart phones make things interesting again. They’re mobile and aware of their environment in real-time. They have cameras, microphones, touch screens, accelerometers, magnetic compasses, and GPS, as well as all the computing power and network connectivity of a regular computer. In short, if you can think of an application, you can probably get it working on a smart phone.