Do you know why it’s so hard to swat a fly? Chiefly, it’s because they see ten times faster than we do. When we think we’re aiming a quick smack at a pest, what it sees is a threat creeping towards it in slow-mo. We’re an embarrassment, and now we know it thanks to an extraordinary five-year effort to develop a virtual reality world for flies.
For the scientific team behind the VR world for aviating insects –Prof. Shannon Olsson and doctoral student Pavan Kumar Kaushik at the Tata Institute of Fundamental Research in Bengaluru, India, with Marian Renz at Universität Bielefeld, Germany – the paper published Monday in the Proceedings of the National Academy of Sciences, “Characterizing long-range search behavior in Diptera using complex 3D virtual environments,” is a groundbreaking investigation about fly search behavior in the environment bombarding them with stimuli, and how they navigate in fruitful directions. For us it’s all about how flies navigate away from us.
Apple of their compound eye
The flies in question were apple flies, a type of specialist fruit fly. The Mediterranean fruit fly that graces our homes in the Middle East will eat anything, happily landing on, consuming, mating on and laying their little eggs in decomposing fruit, vegetables, onions, bread, even cat food and grimy kitchen rags in a pinch, it isn’t fussy. This is a sensible evolutionary strategy because it will always find something to eat and infest.
The work was done in India, which has flies. But the team chose to work with the American apple fly as the test subject for the virtual reality world, which was based on a vision Prof. Olsson has been working on for years, with the VR world being built by Mr. Kaushik. “It only likes apples on apple trees; it mates on the apples; lays eggs on the apples; eats the apples; the apple is the apple of its eye,” Olsson explained to Haaretz.
Darwin would be appalled, but their selection is a convenience if one is building a virtual reality world for flying insects. Not only because the apple fly is beautiful as flies go, with black and white zebra stripes and go-hither iridescent compound eyes, but the main advantage is that it has only one objective in mind: apples.
Building a VR world for a regular fruit fly in order to investigate its search and navigation mechanisms would be impossible: it could go for anything, Olsson observes. “When you want to find out if they would recognize objects [in the VR world] you want to choose an animal for which you know the object it likes – it will go to the apple tree. If it doesn’t, we have done something wrong.”
Why did they build a virtual reality world featuring apple trees? Because flies have shown navigational skills in flying towards food and mates and it would be useful for pest controllers to figure out how they accomplish this “computationally intensive feat during flight, especially over large spatial scales,” as the team puts it.
It took five years to design a VR world for the apple fly, Shannon confides, because first one has to nail down how the insect views the world. There’s little point in testing its skills if we put it in a virtual movie made for humans. Not only do they have compound eyes: but they also see much faster than we do.
Movies for humans are projected at 24 frames per second. “For insects to see a movie they need 200 to 300 frames per second,” the professor explains. “Under 200 frames per second they would see the individual images flickering, just like we would at 10 frames per second.”
Which explains why we flail our fly-swatters in vain. “They see you coming a lot faster than you see them,” she sums up.
Leashing a fly
Virtual reality for humans can have side effects, such as nausea, disorientation and headaches (after the headset is removed). We have no information if the apple fly felt queasy after the experiments but in contrast to paying tourists at malls, the subject fly was tethered so it would watch the movie. Learning that, one can’t help but have a disquieting flashback to “Clockwork Orange.”
“We used tethered insects for a specific reason – when the insects are ‘free-range’ we have no way currently to know exactly what they are seeing, feeling, or smelling at any one particular time, and therefore why they moved in the directions they did,” Olsson tells Haaretz. “By tethering the insects (which is more like a harness – we do not puncture or otherwise harm the insect so they survive after the experiments), we can move the world and its sights and smells around the insect and better understand how they navigate at any point in time,” she explains.
Asked for more details, she says, “I’ve done a Tedx talk about this research and so many people asked how can you torture flies. They’re actually held in place. Perhaps it’s not fun for them but they were fine afterwards. We fed them while tethered,” she adds reassuringly for the Fruit Fly Rights advocates among us, elaborating further: “It wasn’t impaled.”
So what was it? “You can imagine the tether like an immobile leash or harness,” Olsson elaborates. “We attached a tiny needle to the upper part of their thoracic exoskeleton (about where their shoulder blades would be if they had them) by using a small drop of glue under a microscope. This held them in place, but allowed them to move their legs and wings freely. We fed them frequently during experiments with a little sugar-soaked ball they could hold with their legs and lick like a lollipop. Afterwards, they could be returned to our rearing room.”
And they were fine with that? Olsson agreed that it is “hard to know how difficult this experience was for them. But we observed that the flies would fly in the VR world sometimes for hours, and they always had the option to stop, at which point we would end the experiment. Our love of insects and their importance for the planet is what inspired us to perform this research, so we always took care to treat them with respect for helping us get into their amazing, tiny minds.”
The team discovered that the apple flies employ multiple sensory inputs to locate and navigate toward the virtual apple trees in a complex 3D landscape emulating their natural environment, featuring “dynamic 3D visual scenery, windscapes, and odor flux over large spatial and temporal scales.”
Here is another film of what the fly was seeing.
Bottom line: the flies virtually headed for flowers or trees. Crucially the apple flies were more likely to approach close trees than distant trees. The implication is that the flies can judge distance, using depth cues such as perspective and motion parallax. They were also clearly attracted by scent. “Taken together, the findings demonstrate that flying insects integrate multiple types of sensory cues to locate and navigate toward virtual objects in a complex 3D landscape,” the team sums up.
And this is useful, how? “The findings could be used to optimize strategies for pest control, crop pollination, and disease vector management.” As for us, sure, go ahead and try to deduce which way the fly will go and aim your swat there.