Copyright Ian Pearson, BT Futurologist
Click here for contact details, other articles and personal details
Virtual Air
A
novel way to consider & exploit location-based services with augmented
reality.
March
03
Robin
Mannings, Ian Pearson
Virtual Air is an analogous way of
understanding positioning systems and the information and communication
services that they support. Achievement of optimal position knowledge, in
particular when indoors, employs a heterogeneous set of technologies which will
be discussed. Virtual Air (VA) however, is a useful way of conceptualising the
benefits in a business context whilst avoiding the more academic descriptions
involving positioning field theory. The paper starts by explaining the concepts
and basics of positioning and then discusses how these support the development
of mobile augmented reality applications and services.
The idea of virtual and usually digital
overlays on the real world has been around for well over a decade and is known
as augmented reality (AR). Several companies have developed ranges of
applications linking position to networked functionality, triggering
appropriate functions when a certain person or object is at a particular place.
Projects such as Hewlett Packard's Cooltown illustrate what can already be done
with existing systems to digitally enrich our environment, given a little effort
to put in information beacons, some use of positioning and pervasive computing,
and a bit of imagination.
There is growing consumer and business use
of the U.S. militarys Global Positioning System (GPS), which relies on
transmissions from a number of satellites to allow a receiver to triangulate
its position (using atomic standard time), relative to the surface of the
earth. The recent decision by the EU Government (at the taxpayer's expense) to
deploy Galileo, a non-military and improved satellite positioning system is
further evidence of the relevance of positioning systems to business.
Positioning has even been dubbed the fourth utility. Positioning knowledge
leads to navigation and to a wide variety of location-based services, most of
which use both fixed and mobile communications services. The main problem with
all classic radio based positioning systems is accuracy and outage. The
situation can, however, be greatly improved by the use of supplementary
technologies such as map matching, inertial systems and local beacons. The
resulting positional information is thus a probability distribution that varies
in time and space overlaid on the real world and which thus affects the
dependant data and information services. In other words we could visualise it
as a cloud or Virtual Air (VA).
BTs own Brightstar venture, Rocking Frog
makes good use of location to deliver position and context dependent services
to customers using BT patented techniques based on intelligent agent software.
Software systems and supporting platforms such as these will be needed to
create VA whilst personal electronic equipment will interact with VA. We can
stimulate creativity of many other services if we imagine that air or space
itself is effectively being digitised, by bathing it in a positioning field.
The air won't actually contain digital devices (at least not for a few years
yet), but we can emulate the same effects via computers, smart databases and
positioning systems. Computers can deal with moving points just as easily as
they can static ones. Air can move around in space. So too can information!
Air is often still, and in this analog,
virtual air maps simply onto the static digital environment that we inhabit
today, with fixed positions and fixed information sources. But in the real
world, we are also very familiar with wind, which is simply moving air. Take an
analogy in the digital world and we have bit streams, data flows, broadcasts,
and multicasts. These are all linear flows and we are very familiar with them.
But not everything is linear. Air exhibits turbulence, pressure gradients, and
many non-linear chaotic weather behaviours. It can be clean or contaminated,
hot or cold, with different gaseous and particulate composition. We get rain,
hail, snow and sleet occasionally. It carries many life forms such as bacteria
and viruses, pollen, seeds, insects and birds. It is used as a signalling
medium for many different species, simultaneously, and each is mostly unaware
of the signals from other species (unless they use audio). The air may be clear
or foggy, and we get condensation. We have clouds and jet streams. It forms
layers as we move to different heights, some of which we can bounce signals
off. Any, perhaps all, of these properties may be useful analogs to stimulate
new service ideas. A molecule or pollen grain could be a piece of information,
a message or an advert. Information chemistry can occur in analogy to pollution
cycles in the air. We could have information entities that move around driven
by VA currents. Now instead of a static information overlay, we have a
thoroughly dynamic one, with information from different locations interacting
continuously with other information in a multitude of different ways.
The essence of VA is that it utilises a
combination of many simultaneous multidimensional positions in different
frames. There is not just one positioning system, but many. Some of these move
relative to others, so there can be much interaction between objects and the
environment depending on the relative position in each of many systems at a
given point in time. Some of the systems are highly localised to a building or
an object (which may be mobile), or a person, while some are regional or global
and static. Some may change with time. It is this wide range of flexibility
that is the key advantage but which also makes analysis and integration
difficult.
There are two classes of methods to determine
spatial position. Firstly, using the temporal properties of propagating energy
fields and secondly, using sensing combined with a static location data base.
Examples of the first methodology include GPS, various terrestrial maritime
radio systems such as Loran and Decca and short-range ultrasonic systems. The
second method could be as simple as the map and compass, or at the other
extreme the Tercom inertial navigation used by cruise missiles, or indeed the
biological systems used by bats, homing pigeons and many other species which
perform incredible feats of navigation. Yet with all this knowledge, there is
no reliable, cost effective or standardised method available to businesses to
be able to gain positional information concerning objects of value or people.
Positioning using radiating fields works
best when there is an unobstructed path between the transmitter and receiver,
which must not only be line of sight but must also take account of obstructions
that could cause significant diffraction. Poor quality signals which are
normally described as noisy lead to positioning errors since the erroneous
energy fluctuations are translated into phase or time errors. Additional
temporal errors arise from reflected waves or waves that are obstructed. Whilst
digital wireless systems using channel equalisation can make good use of
reflected (or multipath) signals, in positioning systems what is needed is an
unambiguous arrival of the most direct signal. Clearly, the geometry of
building interiors make this sort of positioning very difficult but it is still
a very cost effective way to position due to the sparse distribution of the
transmitters. A minimum of three stations is generally needed to provide a
position (which is located at the intersection of the loci of either the
hyperbolae associated with equal signal phase or the circles associated with
equal signal time).
There is now much interest in the combined
use or wireless communications systems that provide positioning as a
by-product. This is the first concrete example of VA. Indeed, although it may
be somewhat perverse, 3G revenue may come more from value added services based
on VA than the traditional networking business!
Map and sensor systems are only as good as
the quality of the maps or the accuracy of the sensors. Autonomous map matching
is widely used for vehicular systems and works particularly well when direction
changes frequently and the system self-corrects its accumulated errors. Driving
along long straight roads is the worst case, particularly when there is a close
parallel road which might then be confused with the correct one. To correct for
these errors and to provide an occasional fixed reference a sensor readings are
needed. In the limit, if no map is used then there must be a dense network of
sensors with various strategies of inter-sensor interpolation. Like the field based positioning
systems, these fixed references may be communications hot spots such as
wireless LAN access points. The most basic way to position is to detect the
presence of absence of signal about a threshold. More advanced approaches could
use radar techniques (sensing and processing signal direction, using timing and
Kalman filtering techniques).
Other sensor systems include use of ultrasonic, infrared and magnetic or
electrostatic fields.
The optimum approach to positioning would
be to use a combination of the long range field systems and the short range
sensor systems together with as much map matching as is possible (assuming the
availability and quality of the maps).
To implement VA we thus have a
heterogeneous set of technologies providing positional information to both
nomadic and static objects and people. Behind the scenes are the computer
platforms integrating the information. What is missing is the methodology to
combine the positional and related information in a standardised way. Work in
BTexact on location based mark up languages may be one technique to fill this
gap.
There are essentially three information
spaces which are involved. Firstly, the physical world of the real objects and
people, supported by a second distributed pervasive computational world of
software objects delivering information services in third space of the Virtual
Air.
VA will be supported on a variety of
sensor, storage, communications and computing platforms, some of which fall in
the field of pervasive computing. Many companies are working busily on
pervasive computing including BTexact, and many of these activities will feed
into or interact with VA. We, the authors, are working on a new project which
considers novel pervasive computing platforms to support VA, looking at
distributed sensor networks, novel computing and communications architectures
and high precision positioning techniques, even the fringes of biotechnology
and information technology integration. These go much further than the scope of
this article, which can only outline a few of the basic VA concepts and some of
the basic positioning systems that may be used. Many of the other platform
technologies, including some novel positioning systems, will be discussed in
future articles.
Air can get trapped in our clothing, and in
much the same way, information in a location could cling to us as we walk
around, gradually dispersing, interacting with information resources elsewhere
as we move around. For example, a marketeer could upload tokens to us in one
place that might link to particular discounts or marketing messages in another
place. Or as it disperses, it conveys marketing information to detectors at
locations we pass through. Or it could be used by clubs to alert people to
others they pass in the street with similar interests
A token could be physically uploaded into a
portable device that we are carrying, or it could simply be a conditional event
logged in a database, triggered by detection of our presence. This duality is
fundamental to location based services in general. The token that is physical
and portable has a counterpart that is virtual and on a server. Synchronisation
software updates the two data sets when appropriate. The physical token could
be thought of as a sort of internet browser cookie which is trapped or cached
information that enables the continuation of application sessions.
The most widespread implementation of the
trapped air concept today is perhaps manifest in the storage facility of the
latest RFID tags.
We can have a whole bubble of VA around us
- a highly localised relative positioning field, independent of other global,
urban or building systems. The bubble could have a sharp cut-off, so the
positioning system would have very clear limits on its domain. A shared
position may overlay this, which allows interaction with external devices. This
could of course be implemented by using encryption technology to permit access
to the position only to the person and devices within the bubble, even though
radio signals cannot be so easily confined. Devices on our person may use this
system to optimise communications amongst themselves and with devices on other
people, when bubbles interact. Our personal positioning system could allow us
to track the movement of our fingers or limbs so that we can push virtual
buttons or operate virtual controls. We could implement air-mice and
air-typing. Our digital bubble would make all of our data available to all of
the devices on our person, and act as an interface to other bubbles and the
outside world. It could have filters to protect us from electronic attack, and
share information and capability with other people and machines according to
our wishes. These systems are already emerging in virtual reality gaming
applications.
Bubble interactions may facilitate interworking
between groups of people, allowing route optimisation and helping delimit
boundaries. The concept of personal space varies according to context and
culture, but our bubbles might also interact in context-dependent ways. Social
support services such as tribal communication may make extensive use of
bubbles. Bubbles are a also productive model for thinking through digital
filters, information exchange and information force fields.
Bubbles may also be used to allow complex
interactions with software. An individual may be unable to access some
functions that are reserved for groups, and very specific roles may exist. Only
by having both bubbles present does the functionality become accessible. In
much the same way as fantasy heroes might keep part of a key to unlock the
treasure, a digital encryption key may be distributed amongst several digital
bubbles. They may thus form a part of a security system such as those used to
prevent the accidental use of atomic weapons, or access to wireless LANs.
Yet another bubble application is in
symbiotic networking. Many of us will soon have a personal area network that
extends just a short distance from our bodies, linking our wearable and
portable devices. We may have other networks that extend tens or hundreds of
metres away, accessing wireless LANs or other communication devices, and 'last
resort' access to paid-for public networks such as GSM, 3G or commercial
wireless LANs. These bubbles may be simply concentric, or they may be highly
directional lobes that intersect at the person. As bubbles from different
people meet, they may adjust their various electromagnetic activities to make
mutually beneficial (symbiotic) networks. As many bubbles interact in the
densely populated area of a town, we will see some bubbles co-operate and join
to form long chains, chambers and tunnels that allow messages and information
entities to travel right across town free of charge. These tunnels will then
form part of the infrastructure that will support many other VA applications
and entities.
Clouds are a novel way of approaching
mobile marketing. Natural clouds form over certain areas in certain conditions,
and may blow around. Sometimes we get rain or other forms of precipitation.
Digital clouds in VA may analogously form and drift over a target area as
myriads of small marketing 'droplets' gather together, occasionally
distributing marketing output to people underneath (unless they carry a digital
umbrella!). This pseudo-naturally randomised distribution of marketing might be
more acceptable than continuous broadcasts from thousands of marketing sources.
Taking another approach, clouds are very visible, and in augmented reality,
visible virtual clouds would be an ideal platform for adverts or other
messages. In fact, the whole virtual sky would be such a platform. Clouds could
be used as personal navigation tools, or for group maintenance, even used as
emotion indicators (a dark cloud hovering over someone's head in augmented
reality would be fairly self-explanatory). Each friend may have a digital cloud
far above them so that a quick glance at the AR sky would reveal whether any of
your friends are close enough to get together at a coffee shop.
Implementation of mobile augmented reality
is still a few years away from commercial deployment, so this won't all happen
just yet. Although niche applications in the military and in gaming exist,
convenient, cheap and portable personal head-up displays supported by
sufficient processing capability is not yet economic for everyday use. It is
highly likely that this will only become feasible in the mid term future.
Digital pheromones may be given off by
people as they walk around a town, advertising their presence, characteristics
and availability to people (or machines) with appropriate receptor types. They
would be concentrated near the person emitting them, but would also drift and
linger for a while, allowing people to follow trails. Similarly, digital aromas
may be give off alongside chemical ones by the local chip shop or the hot dog
trolley, alerting people, who may be physically upwind, that a tasty snack is
just around the corner. Gangs of youths may police mark their territories with
digital pheromones, as well as digital booby-traps to ward off enemy gang members.
Other town inhabitants may be totally unaware of their presence. Other groups
may use this technique to lay treasure trails, to mark out special routes or
leave reminders for future visits. Linking this idea to image generation, and
particularly avatars, it would be possible to realise a ghost trail in a town,
where ghosts would pop up from time to time with artificial intelligence based
personalities and behaviours.
Although we have computer viruses already
none yet appear to be location dependent, still less those that wander around
the spatial environment until they find a suitable target. Today's viruses and
worms inhabit a simple one dimensional world of fixed communication links and
occasionally the 3 dimensional world of wireless LANs. They don't sit and wait
in augmented reality ambushes. But they will soon! And they will begin to
emulate behaviours from the organic world, such as working in teams,
co-operating in sophisticated attacks, stalking stealthily, and timing their
attacks to perfection. Location or path-specific viruses could add a deeper
level of sophistication to an old problem.
We can also expect a digital food chain to
arise naturally in a VA environment that would never develop properly in a
conceptual mesh network, even though the basic physical architecture is the
same. The mere creation of a shared conceptual space is enough (a few virtual
zoos have been produced on limited spaces already), but overlaying this on real
space will accelerate its development, because people can identify more easily
and closely with it. A hierarchy of digital creatures with their own virtual
life cycles could emerge, populating our augmented reality with virtual fauna
and flora, digital gardens and jungles. A peer to peer computing infrastructure
using evolution engines could keep it well populated and provide continuous
interest.
Scenery would be important to most users of
an AR world, and since virtual physics is completely customisable, we could have
some very elaborate architectures, some of which are static and stable, others
which pop in and out of existence, move around, or change appearance
continuously. This virtual space-time environment could attract the worst
excesses of today's web sites, with pop-ups, flashing banners, and invade our
portable devices with software. Offensive marketing creatures could follow us
around town, annoying us all day once they have found us, trying to interact
with everything that we do. It will be imperative to police the AR world to
remove such annoyances, or it will destroy its value for legitimate and
desirable activities. A walk around town could be like a bad trip on LSD! Smart
filters will be very important components in our VA bubbles.
We may go even further with this process,
linking real environments to digital photography. Since the camera will soon
know where a picture was taken, the computer could overlay virtual objects on
the photograph each time the use opens it. A building or person might have one
appearance in real life, and a totally different appearance in the photo.
Again, there are so many potential abuses of this capability that we would need
to manage it carefully. On the good side, it would allow pictures to be taken
virtually at any time period, and in any weather conditions, not just the time
that the photographer happened to be there. Photographs could become very much
more versatile.
This uncoupling of actual and virtual
appearance could transform the physical appearance of our buildings too. If we
don't have to have real life shop windows, shop signs, or much other street
furniture, we may see very different architectures for these. The coolest shop
in town might have a very plain front in real life, but appear wonderful in VA
to its target clientele. Buildings that are only 'visible' to their users group
could be expected to have an interesting effect on city culture. The
information functionality can be provided in cyberspace, so the building itself
can concentrate on physical form and function in the non-augmented world,
independent of its purpose in the electronically enhanced world. Each passer-by
might see a different faade, according to their own profiles and filters.
If we take this deeper, we might conclude
that owners of buildings might actually have very little ultimate control over
their appearance. Groups of people might customise the virtual appearance of
buildings with or without the owner's permission of knowledge. Today, wireless
LAN hunters make chalk marks to indicate the existence and properties of a
nearby wireless LAN. The chalk marks could be replaced soon by VA indicators.
For some purposes it might be useful to
make information and information flows visible. If we could see which direction
data is flowing, we might be able to see where an antenna is (most antennas are
physically small, they aren't all cellphone masts), which may help us get
better reception. Seeing who is communicating with whom, or what sorts of
things they are doing electronically would be useful or interesting too
sometimes, and privacy issues can easily be addressed. Many air analogies could
be appropriate -plumes of smoke, clouds of coloured gases, flying virtual
creatures, virtual leaves moving in a virtual breeze, even visual packets or
characters floating or zooming around. Certainly, being able to see marketing
information without having to read it would make it less invasive. A shop could
leave aesthetically wrapped virtual brochures or vouchers in a virtual box, to
be picked up only by those that want them. They might only be visible to the
target group.
Information and VA services may stream from
various beacons placed around town by tourist authorities, business, groups and
individuals. While many of these may be authorised by building owners or local
authorities, many more may be informal and placed by people for a wide variety
of purposes. They may have a short transmission lifetime, based on a simple low
power transmitter and small battery. Such devices could be fabricated quite
cheaply, so might appeal to many people who want to add colour or information
to the VA landscape. Some might provide comments on the quality of service in a
shop or restaurant - the transmitter might be stuck on a wall or hidden in a
nearby hedge or plastic waste bin. Others could be culture specific or tribal.
While some of the functionality might be implemented using conventional VA
techniques, the use of actual radio devices would have some significant
advantages. They might be much easier to deploy than arranging for entries in a
town database, especially if such entries are censored. Such beacons could also
be mobile, based on cars or in people's clothes.
This paper maps out a new paradigm for
mobile services which are location dependant and focussed on the delivery of
mobile augmented reality. It has covered a time line from the present when
there is very limited potential to deliver the concept of virtual air, to a
future where positioning systems are much more sophisticated and allow a
virtual world to truly overlay the real world. By using the analogy of air, we
can allow our imaginations to conceive novel mobile services which will be both
useful and highly entertaining and which have enormous scope for network centric
value added services. The principal of virtual air assumes a very rich
environment of both fixed and ubiquitous mobile communications both with
conventional architectures and a more ad hoc approach. It certainly highlights
many ideas that aren't immediately obvious when thinking in conventional ways
based on fixed positioning systems. By integrating the positional informational
aspects of the heterogeneous communications and sensor networks it will then
enable the ability to deliver a virtual reality experience on the move.
The main lesson to draw from this paper for
today, is not to ignore the positioning and locational aspects of new networks.
Instead we should seek methods to standardise and facilitate the integration of
a generic approach to location definition, location information distribution
and its use within augmented reality systems, software and services. Many
potentially valuable services could result. It would be a shame to miss them.
Robin Mannings
is BTs most experienced and well-known expert in the field of Mobility and
Disruptive Technologies. He works extensively with academia, major corporate
customers, manufacturers, and teams within BT on future mobile, wireless and
disruptive technologies. Robin's career includes mobile radio system
development (with Philips), university research into mobile radio data and
signal processing (at the
University of Bath), and the application of new technology to business (with BT
at Martlesham). His international reputation was principally established
through his research into Mobile Telematics, Intelligent Transportation Systems
(ITS), positioning and tracking technology.
Robin has written numerous technical papers
and book chapters, has filed many patents concerning mobile inventions and he
frequently appears at conferences, specialist gatherings and in the media. He is a Chartered Engineer, a Member of
the Institution of Electrical Engineers and is a Founder Member of the
Association of Professional Futurists.
Ian Pearson graduated in Applied Maths and
Theoretical Physics in 1981 from Queens University, Belfast. He worked for four
years in missile design and then moved to BT Laboratories where he has worked
ever since. He has been involved in most areas of information technology, from
computer systems analysis to high speed telecomms. For the last twelve years he
has been a futurologist, anticipating trends in technology and working out the
likely consequences for business and society. He has written three books and
many papers on the subject, several of which have received awards. He spends
about half of his time thinking about the future and the other half explaining
it to other people, both inside and outside of BT. He is a Fellow of the RSA
and a Founder Member of the Association of Professional Futurists.