• E-mail
• Print
• Comment
• Font Size
• Digg
• del.icio.us
• Discuss article

Control Freak

Posted on: Thursday, 13 October 2005, 09:00 CDT

By Snook, Steve

New user interfaces, including handwriting recognition, configurable display systems, even facial and hand gestures, are among the ways being considered to give drivers more control in the cockpit. Steve Snook reports

Daimler Chrysler's ergonomics test bench is described as a unique facility as the interior dimensions of future models can be varied at will at the touch of a button. Around 90 electric motors move the structure of aluminium sections and panels.

Daimler Chrysler's ergonomics test bench is described as a unique facility as the interior dimensions of future models can be varied at will at the touch of a button. Around 90 electric motors move the structure of aluminium sections and panels.

OEMs and their suppliers are caught between a rock and a hard place when it comes to introducing new infotainment services into their vehicles. The iPod generation wants total connectivity while the regulatory bodies want to ensure that new systems do not undermine safety.

"Customer requests for unrestricted use of consumer equipment in the vehicle represents a major task area," says Werner Hamberger, operating concepts project manager at Audi. "Such equipment includes mobile phones, PDAs, iPods and many others. The heart of the problem is that such devices are not developed for use and operation in a moving motor vehicle. They can consequently neither be readily operated in the sense of exhibiting classic ergonomic principles, for example large keys on a mobile phone, nor use ergonomics that are cognitive, in other words easily understood.

"One example is the fact that a mobile phone reverts directly to the start menu after a certain time has elapsed. This is not conducive to the safe handling of a vehicle, as the user will be encouraged to conclude inputs that have already been started in order not to lose them, instead of devoting sufficient attention to the traffic. A solution to this conflict between customer wishes and ease of operation can be achieved by means of improved networking or compatibility between the vehicle and additional devices. Bluetooth currently offers the best prospects for development. Devices thus integrated into the vehicle can then be operated using the vehicle's own input and output devices in the same way, and consequently operated using a uniform logic."

"Distraction occurs when the driver shifts attention away from the primary driving to a thought, event, activity or person," says Joseph Kanianthra, associate administrator for vehicle safely research at the US National Highway Traffic Safety Administration (NHTSA). "Distracting sources have always been present, but new sources are increasing in numbers and complexity. These new sources can affect driver behaviour and performance in different ways than conventional distractions. We do not know what will be a source of driver distraction in the future as technology is advancing at a rapid rate with, for example, more voice interfaces.

"Data on the effects of distractions have been limited by driver dishonesty, driver and witness misperceptions and the low priority given to these factors in police investigation. Problems from new 'electronic distracters' may not show up in crash data for years. However, today's data is clear that existing electronic distracters contribute to crashes."

A naturalistic driving study sponsored by the NHTSA found that the three leading causes of roadway crashes are driver inattention, distraction and fatigue, with driver inattention leading at nearly 80 per cent of all crashes. The study, completed recently by the Virginia Tech Transportation Institute (VTTI), collected pre-crash naturalistic driving data using 100 cars fitted with instrumentation that gathered information through five channels of digital compressed video and sensors that recorded vehicle state and kinetic data. More than 42,000 hours of data were collected over 3.2 million kilometres of driving by 241 participants.

The mobile phone problem

Last month, the Insurance Institute for Highway Safety (IIHS) in the USA released research concluding that drivers using mobile phones are four times more likely to get into crashes serious enough to injure themselves. The fourfold increase in crash likelihood was consistent among drivers, with male and female drivers experiencing similar levels of risk, as well as senior drivers and drivers under 30. The study also found that weather played a nearly non-existent role, and that there was no distinction between hands-free and hand- held mobile phones.

"This isn't intuitive," says Anne McCartt, IIHS vice president for research. "You would think using a hands-free phone would be less distracting, so it wouldn't increase crash risk as much as using a handheld phone. But we found that either phone type increased the risk."

The study was conducted in the Western Australian city of Perth. The institute first tried to conduct the research in the USA, but phone companies were unwilling to make customers' billing records available, even with the drivers' permission. Phone records could be obtained in Australia, and the researchers got a high rate of cooperation among drivers who had been in crashes.

Another reason for conducting the study in Australia was to estimate crash risk in a jurisdiction where hand-held phone use is banned. It has been illegal while driving in Western Australia since July 2001. Still one-third of the drivers said their calls had been placed on hand-held phones.

While safety is obviously compromised whenever a driver takes his hands off the wheel or eyes off the road, the underlying risk factor is that the driver's mind is taken off driving.

"The brain can't simultaneously give full attention to both the visual task of driving and the auditory task of listening," says a Steven Yantis, a professor in the Department of Psychological & Brain Sciences at Johns Hopkins University. "Directing attention to listening effectively 'turns down the volume' on input to the visual parts of the brain. The evidence strongly suggests that attention is strictly limited; when attention is deployed to one modality it necessarily extracts a cost on another."

Renault makes extensive use of high-tech systems to analyse driver inputs

Arne Nbo, Saab's chief ergonomist, says: "There's no doubt increasing traffic densities and the growth of in-car infotainment systems are putting an increasing workload on the driver. Car manufacturers have so far focused on managing information inputs for the driver in the safest possible way. We now think it is time to take a rather less passive approach.

"In everyday driving, we know people actually do take their eyes off the road quite a lot and we are now developing a means of helping drivers to help themselves. This system will help prevent a dangerous habit we call 'cognitive tunnelling'. For example, the driver may have a map book open on the passenger seat and keep glancing between the map and the road. The driver can eventually get too absorbed in trying to overcome the challenge of plotting directions on the map and simply gets distracted for too long. That's what we mean by tunnelling."

Saab is developing a safety system with optical technology specialist SmartEye AB of Gothenburg that monitors eye and head movements, issuing a warning signal if attention strays too long.

Keeping an eye on the eyes

Two miniature cameras with infra-red lenses are installed, one at the base of the driver's A-pillar and the other in the centre of the front fascia, to monitor eye and head movement. As soon as the driver's gaze moves away from what Saab calls the "primary attention zone" a timer starts counting. If the driver's eyes and head do not return to the 'straight ahead' position within about two seconds, a buzzer will sound. And if there is still no response, a brake pulse will be delivered through the car's ESP system.

Measurement and processing of the infra-red image includes the relationship between eye gaze direction and head movement. It is sufficiently accurate to detect when the driver retains some peripheral vision of the road ahead - such as while looking in the rear-view mirror or turning a corner and will consequently allow a slightly longer time to elapse before activating the warning buzzer. The software is speed-sensitive and can distinguish between a busy, city driving environment and more open highway driving at faster speeds.

Apart from its primary driving safety function, there are further potential applications for this technology, says Nbo. "All infra- red facial images are unique and it would be quite simple for the 'face-print' of all individuals using a particular car to be memorised. Adjustments could then be made automatically to all seat, steering wheel and mirror settings once the driver is in position.

"The system is functioning extremely well in our testing. And there is no problem with the hardware, which is reliable and relatively inexpensive. We are now concentrating on fine-tuning the timing of the buzzer and the nature of the final warning alarm. A brake pulse, which does not engage the rear stop lights, is currently the preferred option."

Delphi Delco has been working with the NHTSA for some time on a project (SAVE-IT) to measure driver workload and adapt user interfaces to improve safety. "We are exploring the benefits that might be achieved if the driver state and driver intentions can be detected in real time to enhance the effectiveness \of collision warning systems as well as to help minimise unsafe multitasking," says Kanianthra. "Sensors outside the vehicle record driving task workload levels. Sensors inside record driver distraction. Algorithms determine in real time any mismatches between the primary task demands and the driver's level of attention.

"The goal is to explore ways to monitor distraction in real time and automatically adapt the man-machine interface (MMI) depending on the driving task demands and the level of driver distraction away from those tasks. Adaptive interface possibilities may include locking out information or changing the threshold for triggering crash warnings. The programme is developing a concept vehicle to evaluate the potential of this countermeasure concept and is scheduled to be completed in 2006."

Despite the mixed response to BMW's first i-Drive systems, this and similar control devices are gaining in popularity. Both Audi's MMI and the Mercedes COMAND systems have been subjected to further customer testing and subsequent enhancement.

DaimlerChrysler has redesigned the Cockpit Management and Data System (COMAND) for the latest S-Class vehicles. In addition to the familiar telematics and audio units, various vehicle functions that previously required individual switches have now been integrated. The central operating unit is a new COMAND controller on the transmission tunnel. It is used to select the main and submenus with which the required functions are activated.

Rapid access to well used functions

The redesign is based on findings from extensive ergonomie and perceptive psychology studies, and also takes into account the results of acceptance tests with Mercedes customers around the world.

A major design target for the intelligent operating concept in the S-Class was rapid access to functions used particularly frequently. The system is therefore specifically designed for redundancy - depending on his preferred habits, the driver is able to control functions such as the radio, TV tuner, CD/DVD changer, telephone and navigation system either using conventional switches, via buttons on the steering wheel or with the help of the COMAND system.

Direct control switches in the centre console are ergonomically positioned so that the driver is able to operate them without looking down. It is also possible to operate the standard automatic climate control system in two different ways: via COMAND or using an array of switches beneath the air outlets.

The steering wheel and instrument cluster integrate all the important information and functions required by the driver. The centrepiece is a colour screen with the latest display technology. The lower edge of the cockpit display has a plain-text array with which up to seven main menus for individual settings, display modes and operating functions can be controlled. For rapid selection of these functions Mercedes-Benz has developed the controls on the standard multifunction steering wheel further, using circular, illuminated five-way buttons that the driver operates by light thumb pressure.

Audi has continued to develop its MMI operating concept, introduced in the 2002 A8. "The complex range of functions within a single system necessitates a simple, transparent and intuitive operating concept for the user," says Werner Hamberger. "All the derivative versions based on the Audi MMI, from various suppliers and using various sizes and resolutions of display, benefit considerably from its consistent design and uniform menu principles.

"The Audi MMI integrates a large number of functions in a single system. The level of functional integration achieved reduces the driver's workload, as they do not need to operate the various different units in different ways. The Audi MMI will be available in our new Q7 with new functionalities that are available for the first time in an Audi. Special features in infotainment include a Bluetooth car phone, the entire operating procedure for the external phone book, a higher-level address book with automated import function and DAB data services.

"Our aim was to make it possible to operate these new functions intuitively, and therefore to make the vehicle as a whole more customer-friendly. It should be made clear, however, that further enhancing user friendliness and ease of operation does not mean all functions are now to be operated solely via the MMI.

"For example, when inputting destinations in the navigation system a touchpad with handwriting recognition offers immense advantages for entering free texts in terms of the number of operating stages and times the driver needs to look at it. This advantage is particularly apparent when one considers the special features of the Chinese market: whereas it has until now only been possible to enter Chinese characters into the vehicle system by converting them into a phonetic notation, Pinyin, using the Latin alphabet, the introduction of a touchpad with handwriting recognition offers scope for making the entire input process faster, more convenient and safer.

Button selection without looking

"Sensor buttons offer similar advantages for the user, as it is not necessary to establish eye contact with the controls in order to select the desired buttons. Sensor buttons are capable of registering and signalling contact between the user's hand and the button. If this signalling of the hand's position is provided by means of a display in the user's primary field of view, it is possible for the driver to operate it without taking their eyes off the traffic."

"You have to make sure that drivers don't become distracted when operating any system," says Dr Nhu NguyenThien, head of the SiemensVDO Automotive team developing the EasyCo multi-modal operating concept. Among other things, EasyCo makes it possible for an infotainment platform to receive commands based on hand movements using a sensitive medium such as a touchpad in combination with a high-performance recognition system.

"In such a scenario, names and telephone numbers could be written directly with a finger on the touchpad,"says NguyenThien."You could also draw symbols. For example, if you draw a heart, the system would automatically dial the number of your wife or husband. If you draw a musical note, the radio would play classical music. In each case, you would decide for yourself what the various symbols should stand for. Tests confirm that users find it easy to operate while driving, and they feel less distracted by it than they do by conventional systems."

Alpine Electronics launched its FoilseTouch technology last year. The system provides tactile feedback through vibration, pressure and sound to simulate the sensation of using real buttons on a touch screen. This year, its in-car infotainment control systems have been enhanced by the addition of GlideTouch, a small horizontal strip on the lower front of the head unit that replaces the traditional presets.

Johnson Controls is working on thinner, lighter consoles that ease driver workload

GlideTouch, developed in Japan by Alpine's parent company ALPS, uses film substrate printing technology to create a pressure- sensitive sensor on the surface of the film substrate with a controlling element made of silicon. Resembling the silicon rubber touch pad on a laptop computer, GlideTouch recognises the pressure and speed of the user's fingertip to provide an easy interface with infotainment systems.

"PulseTouch broke new ground in human interface design and we can see GlideTouch contributing in two ways," says Alpine's Dave Sheen. "First by making navigating the multimedia system safer on the move and second in offering the vehicle manufacturers greater opportunities to differentiate their systems interfaces. We are working closely with leading manufacturers across Europe and expect to see the system in showrooms in the next three years."

Touch sensor systems from TRW Automotive Electronics & Components will appear in 2007 vehicles from two German OEMs. The technology can replace traditional mechanical control functions, from simple on and off switches to larger input devices such as HVAC. The company has also developed a multifunction control module to interface with driver information and communication systems. A key feature is the capability for handwriting recognition, which enables fast and intuitive operation of mobile phones and navigation systems. TRW is developing both the simple and multilevel control and handwriting concepts for the 2007 launch.

"Touch sensor technology has significant advantages for the driver," says TRW business development manager Ernst Hafner. "We can group many functions together, and our research has demonstrated that it offers significant ergonomic and packaging improvements. Handwriting recognition offers reduced driver distraction and improved control and character input speed compared with the driver interfacing directly with a mobile telephone or navigation system. The user 'writes' characters with a single finger on the touch sensor area which is recognised by software and can be acknowledged by audible feedback, enabling complex input without requiring the driver's eyes to leave the road ahead."

BMW is working on driver input using gestures

Man/machine communication options now being examined by BMW are voice, gestures and mimics. This also includes the measurement of eye movements conducted by engineers in driving simulators and on test drives.

"Consistent analysis of these movements serves to place instruments and controls at the right point, ensuring that the driver is hardly required to take his eyes off the traffic around him," says Dr Frank Althoff, a specialist at BMW in man/machine interaction. "Hence knowledge of this kind is going into the development of future MMI concepts, with greater emphasis on voice control, a high position of the display screen near to the traffic around the car and the use of the head-up \display. Recognition of gestures such as emotions and further refinement of voice entry systems could open up entirely new levels of dialogue and communication.

Fiat's MultiController, shown in a concept car, uses Johnson Controls' ElekTex pressure-sensitive fabric to transmit impulses from a touch

"Even the best voice entry system will reach its limits under unfavourable conditions, for example when disturbed by loud noise in the vicinity. We are examining whether other, additional types of human communication might be used to interact with the car. One option is to focus on gestures: a simple camera may serve to monitor movements of the driver's head and hands, checking out whether the car should respond accordingly. If, for example, the driver answers a question asked by the voice system with a 'no', but speaks too quietly, or if the background noise is too loud, the system today will repeat the question. But if the system perceives not just the language signal but also the driver shaking his head, it may correctly understand this sign to be a 'no'.

"Movements of the driver's hands may also be used in this way, if necessary even providing independent control options. Waving his hand from left to right, for example, the driver might 'tell' the system to 'tune to the next station on the radio'. Moving his hand down, in turn, he might symbolise the telephone being placed back on the hook, meaning he wants to end a call."

Devining intentions

A particularly important requirement is to determine whether a certain gesture is indeed an intentional movement by the user serving to control the system. Initial test experience at BMW has shown that various gesture scenarios have to be carefully considered: "Where some people would like to respond by moving their head, others would prefer to press a button or speak out a loud answer," says Althoff. "The customer will only accept the system as useful if the vehicle allows a wide range of different options, since this really ensures a significant improvement of user comfort."

BMW is also attempting to recognise the driver's 'emotions' as a further tool to optimise voice control. "The two types of expressing emotions important in this context are mimics and the driver's tone of voice, audio evaluation of the latter having made the greatest progress so far," says Althoff. "Focusing on different signal characteristics such as the basic frequency, the intensity and duration of loud commands, a voice monitoring system can already distinguish today whether the individual concerned is, say, angry and aroused.To begin with, we only want to distinguish between positive, neutral and negative expressions, restricting the scope of this for the time being."

An example quoted by Althoff is the onboard computer asking several questions in response when the driver's voice entry is unclear, since drivers respond differently under such circumstances. Some regard even the second question asked by the system as annoying, others find the waiting times between repeated questions too short.

"Once we are able to recognise emotions, we will also be able to adjust the dialogue better to the driver, without requiring him to intervene himself. This capturing of 'multi-modal' information will very important in adjusting vehicle functions to the user, with the system relating emotions detected to other factors such as the driver's control history and current ambient conditions."

Voicing approval

Notwithstanding the latest safety concerns regarding hands-free mobile phone use in vehicles, voice control will be the dominant feature of systems being introduced over the next five years. But speaking to a machine consumes the attention of the driver and a top priority is ensuring that the driver remains focused while operating the vehicle. For this reason, spontaneity and naturalness in the spoken interaction is a must, so that drivers can keep most of their attention on driving.

Volkswagen's Electronic Research Laboratory (ERL) in Palo Alto, California is working on a Natural Language Dialog Systems project - a collaborative, US government sponsored research endeavour between the ERL, Robert Bosch Corporation, SRI International and Stanford University. The goal is to advance speech systems by enabling natural conversation in the car for multiple devices beyond typical 'command and control'.

In a speech system with these conversational capabilities, careful attention is paid to dialogue design - especially what the system should say to the user when it does not understand a sentence. Misinformed responses from the dialogue system can lead to a circle of frustration and ultimately abandonment by the customer. However, useful dialogue has the potential to enable speech as a viable input device for large-scale information management while reducing driver distraction.

"Voice entry is one of the key technologies for even more efficient use of the car," says Dr Frank Althoff, BMW's specialist for man/machine interaction. "The potential in this area is huge. As convenient as it already is today to control the navigation system or telephone by voice messages, the driver still has to make do with the limited vocabulary memorised within the onboard system.

"Once the driver, in the navigation process, is able to loudly speak out the names of cities and streets, we will have completed the next step in the evaluation progress. But simply enlarging the system's vocabulary is not enough, since a larger number of words recognised by the system means a larger statistical risk of misunderstanding. Distinguishing between similar sounding names such as New York and Newark, for example, requires very precise processing of signals and their meanings. So in any case of doubt, the user will still be able to choose his destination from a list of preselected options on the screen.

"A second great challenge is the complexity of human statements. We need to move beyond simple instructions and commands. This starts with the use of complete sentences onboard systems are not yet able to understand. 'I want to drive to London' is a normal statement in our everyday language but the machine is not yet able to understand a statement of that kind. And things become even more complicated when the driver interrupts a sentence, uses additional words not actually required or corrects himself.

"The long-term vision of a perfect voice detection system lies in the correct recognition of indirect statements made by the driver such as: 'My feet are cold'. The system should 'understand' the wish expressed by the driver in this way and reset the air conditioning accordingly."

According to Werner Hamberger, Audi has identified specific design principles for voice control systems: "For example, 'Speak what you see' means every menu command visually displayed can also be spoken. Whole-word input of navigation destinations, radio stations and address book entries increases scope for natural- speech inputs and avoids the need for tiresome spelling. 'Barge-in' enables the user to react directly to a spoken output by speaking further commands without needing to wait until the system has completed its speech output."

Other Audi design principles include: a variety of user modes to reflect varying needs for feedback depending on the user's experience in using the system; 'situation-based feedback', modified according to the driving situation; and use of text-to-speech to increase the number of spoken system outputs, as all written texts can also be spoken without these first needing to be recorded in all system languages by actual speakers.

Copyright Findlay Publications Limited Sep 2005

Source: European Automotive Design

More News in this Category