Since I’m doing PhD in geospatial science, I need to keep my finger on the pulse of anything location-based. And the Silicon Valley buzz last year was location-based social networks. Foursquare, Gowalla, Facebook Places.
The problem is, none of those services seem to fill a need. Why would you want to be “mayor” of a cafe? And if you want your friends to know where you are, why not use Twitter? It has geotagging.
One idea I came across for an LBS application went like this – you sign up for a service that continually tracks your movements and stores them in a database. It compares these movements with every other user in the database, and if it find that you cross paths with someone on a regular basis it recommends that you become friends. How sweet. No more Craiglist-style “missed connections”.
Now, putting on my scientist killjoy hat, there are some fundamental technical and practical flaws with that model – GPS power consumption, GPS not working indoors, dealing with multi-storey buildings, gaining a critical mass of users – but it got me thinking. Where are the location-based dating applications? In the online world, dating is a killer app. Why not the mobile world?
One thing I learned in my MBA is that advertising is more effective the closer you get to the point of sale. Ideally, you want to put ads in the supermarket. I’m sure the same applies to dating. A dating application is most effective when you’re all dressed up, drunk, and looking to pick up – in other words when you’re in a bar on a typical Friday or Saturday night.
Which is why I’m surprised that there are so few apps. The Economist mentions Skout and Flirtomatic (although the latter doesn’t seem to be location-based), and I’ve seen an iPhone app called Urban Signals in action. Nothing for Android 1.6, unfortunately, so I can’t comment on usability. But why haven’t the big players got involved? RSVP, eHarmony, and OkCupid certainly have the resources to pull it off.
I guess one of the problems is privacy. You probably don’t want to broadcast your exact location to all and sundry. Perhaps a system that releases more accurate information to people that meet your target profile. This needs some thought.
The release of augmented reality applications such as Wikitude and Layar has generated quite a lot of buzz, so I thought I’d try writing my own. It was also an opportunity to get familiar with the Android accelerometer, compass, and real-time camera APIs.
The application has in two modes. When the handset is held horizontally (landscape mode) is displays an augmented reality overlay, in this case a compass heading. When held normally (portrait mode) it displays a Google map, centred on your current location, that rotates to line up with the real world.
I was planning to use the application to display the radar rainfall data provided by the Bureau of Meteorology, centred on your current position. But unfortunately, the BoM doesn’t provide a central index of all their radar images along with location, range, and timestamp attributes. In theory I could scrape the site, but radars go down for maintenance and backup radars are switched on, so it wouldn’t be reliable. So I stuck with a simple compass heading overlay.
From a technical perspective, the application works fine. But from a usability perspective, there are some serious flaws.
- The magnetic compass is not reliable indoors. Too many electric fields.
- Holding up the phone in landscape mode is tiring, and it’s hard to find the information you need. The ergonomics are wrong.
- Rotating Google maps aren’t much use because the labels rotate as well, making them almost impossible to read.
Conclusion: when it comes to displaying spatial data on a smart phone, overlays on a static map are more useful than augmented reality or rotating maps, especially if supplemented with a discreet compess needle. Augmented reality would be pretty good if displayed using glasses, and rotating maps might be usable if the labels rotated (a la OpenStreetMap), but as things stand augmented reality is just a fun toy, not really useful.
I used every optimization trick I could think of, but couldn’t get the FFT routines fast enough to do real-time 3D sound. A 1.125 second clip, 49612 samples at 44.1kHz, takes 3.9 seconds with integer FFT, and 5.5 seconds with floating point.
That’s surprising. My HTC Tattoo has a 528 MHz CPU, so the integer conversion is taking about 2 billion clock cycles. The HRTF converts 128 samples at a time, and calculates the left and right channels separately, so that’s about 2.7 million clock cycles per call.
Each HRTF call does an FFT-256 and inverse FFT-256, plus some overhead. FFT runs in O(NlogN), so the inner loop must be taking about 650 clock cycles. Honestly, that’s a long time for code that consists of four array reads, four array writes, eight integer adds, four integer multiplies, and a comparison. Around 30 clock cycles per command? Crazy.
Potentially the code could be sped up by optimizing for FFT-256 (instead of generic FFT) and unrolling some loops. But that wouldn’t get the necessary four-fold speedup.
The other option is to write the FFT in native code. That may be fast enough, but it wouldn’t be portable across handsets. Seriously, why isn’t there an FFT class in the standard Java distribution?
But all is not lost. It turns out that 3D sound isn’t that effective at distinguishing front/back and up/down anyway. Maybe I’ve got strangely-shaped ears or crappy headphones, but I can only guess the direction slightly better than chance. But left/right is easy.
Your head is about 17cm across, and sound covers that distance in about half a millisecond. If you delay the sound coming into your left ear by that much, then the perceived direction is clearly from the right. So much so that you can’t actually hear the sound in your left ear, even though it’s playing at the same volume in both ears.
So left/right sound works really well, and it doesn’t require a fourier transform. You can’t distinguish between front/back, but you could overcome that with an interactive system that updates the left/right phase as you turn your head. That gets you 90 percent of the benefit, and the phone’s CPU would barely notice.
Back in July 2007 I attended the Mobile Media 2007 conference in Sydney, mostly because I was offered a free ticket by one of the sponsors. Generally the talks were too focused on social and cultural theory for my liking, with a lack of hard facts and working technology.
But one of them got my attention, “These Foolish Things: On Intimacy and Mobile Media” by Kate Crawford. It was the first time I’d seen Twitter in action, a rather impressive demo using its geotagging capability mashed up with Google maps.
The gist of the talk was that communications are becoming shorter and more frequent. Decades ago we spent hours writing a letter, and sent them monthly. Several years ago we spent a few minutes writing an e-mail, and sent them daily. Now we knock up a tweet or SMS in a few seconds and send them hourly.
So I wondered, can I get in front of this trend? Can I develop a form of communication that’s even faster and more superficial than SMS? Well, inspired by idea of scribbling on a Post-It note, I developed Scribble, an Android application that lets you draw a message on the touch screen and send the image as a JPEG via MMS.
There were a few technical problems.
- The customized toggle buttons used to select pen colour and size don’t appear correctly on all handsets.
- The action that’s supposed to launch the built-in Android MMS application, doesn’t. Instead you’re offered a choice of uploading to Picasa or sending as an e-mail attachment.
- The camera API captures images at a lower resolution than the screen, resulting in poor quality background images.
But those issues are probably not insurmountable. What really caused me to pull the plug on Scribble was the fact that writing on a touch screen with your fingertip is really hard. Witness my effort above. Entering text via T9 or a virtual keyboard is much faster.
So while there may be demand for a faster, more superficial, form of communication, Scribble isn’t it.
I’m currently working on three dimensional sound.
Have you ever noticed that we can figure out where sound comes from, even though we only have two ears? Sorting out left vs right is pretty easy, but how do we determine front/back and up/down? It turns out that the sound entering our ear canals follows multiple paths, bouncing off the structure of our ear and causing interference patterns. Sounds coming from different directions have different patterns and our brain picks up on that.
In 1994 a team from MIT set out to record these interference patterns, using a dummy head with dummy ears and two microphones. They placed the head in a recording studio and played audio samples from 710 different positions. The resulting recordings, when combined with some DSP magic, can be used to simulate sound coming from anywhere. This technique is known as Head-Related Transfer Function, or HRTF.
I’m trying to get this working on my Android through stereo headphones, but performance is a problem. The MIT data set was sampled at 44.1kHz, so it can only be used to transform audio sampled at that rate. Unfortunately my HTC Tattoo can’t do that in real-time. The current version of the software takes 5.9 seconds to transform a 1.1 second clip.
The bottleneck seems to be my Fast Fourier Transform code. After digging deeper I found that the Tattoo doesn’t have hardware floating point! Sheesh, it’s like living in 1995. Anyway, I’ll see if I can get it working with integer arithmetic and whether it’ll speed things up significantly.
Not exactly rocket science, but I built a simple spectrum analyzer for my Android phone. The main reason was to get familiar with the Android microphone API, but it was also a chance to test the Java port of some fourier transform code.
As a bonus the application turned out to be a useful testbed for trying out different colour schemes to represent intensities, something that may come in handy in the data visualization part of my thesis.