Video: Paving the way to an autonomous future

Lynn Gayowski

CES 2016 is now underway, and our kickoff to the year wouldn’t be complete without a behind-the-scenes look at the making of our new technology concept vehicle and updated reference vehicle.

The video below follows the journey of building our vehicles for CES 2016 and highlights the technologies we’re using to speed progress towards automated driving — and the list of tech that QNX covers is impressive! It includes advanced driver assistance systems (ADAS), V2X, and augmented reality, not to mention digital instrument clusters, in-car communication, and infotainment:



QNX Software Systems continues to innovate in automotive, with a vision for the evolution of automated driving and a trusted foundation for building reliable, adaptable systems. At risk of giving away the big finale, I think John Wall, head of QNX, sums up perfectly what QNX is on target for in the automotive industry: “We will dominate the cockpit of the car.” It’s a bold statement but we’re already amassing some imposing stats that back this up:

The simpler, the better: a first look at the new QNX technology concept vehicle

Bringing the KISS principle to the dashboard.

Paul Leroux

“From sensors to smartphones, the car is experiencing a massive influx of new technologies, and automakers must blend these in a way that is simple, helpful, and non-distracting.” That statement comes from a press release we issued a year ago, but it’s as true today as it was then — if not more so. The fact is, the car is undergoing a massive transformation as it becomes more connnected and more automated. And with that transformation comes higher volumes of data and greater system complexity.

But here’s the thing. From the driver’s perspective, this complexity doesn’t matter, nor should it matter. In fact, it can’t matter. Because the driver needs to stay focused on the most important thing: driving. (At least until fully automated driving becomes reality, at which point a nap might be in order!) Consequently, it’s the job of automakers and their suppliers to harness all these technologies in a simple, intuitive way that makes driving easier, safer, and more enjoyable. Specifically, they need to provide the driver with relevant, contextually sensitive information that is easy to consume, without causing distraction.

That is the challenge that the new QNX technology concept vehicle, based on a Toyota Highlander, sets out to explore.

So what are we waiting for? Let’s take a look! (And remember, you can click on any image to magnify it.)

The oh-so-glossy exterior
As with any QNX technology concept vehicle, it’s what’s inside that counts. But to signal that this is no ordinary Highlander, we gave the exterior a luxurious, brushed-metal finish that just screams to have its picture taken. So we obliged:



The integrated display that keeps you focused
When modifying the Highlander, simplicity was the watchword. So instead of equipping the vehicle with both a digital instrument cluster and a head unit, we created a “glass cockpit” that combines the functions of both systems, along with ADAS safety alerts, into one seamless display. Everything is presented directly in front of the driver, where it is easiest to see.

For instance, in the following scenario, the cockpit allows the driver to see several pieces of important information at a glance: a forward-collision warning, an alert that the car is exceeding the local speed limit by 12 mph, and turn-by-turn navigation:



Mind you, the cockpit can display much more information than you see here, including a tachometer, album art, incoming phone calls, and the current radio station. But to keep distraction to a minimum, it displays only the information that the driver currently requires, and no more. Because simplicity.

To further minimize distraction, the cockpit uses voice as the primary way to control the user interface, including control of media, navigation, and phone connectivity. As a result, drivers can access infotainment content while keeping their hands on the wheel and eyes on the road.

Thoughtful touches abound. For instance, the HERE Auto navigation software running in the cockpit interfaces with a HERE Auto Companion App running on a BlackBerry PRIV smartphone. So when the driver steps into the vehicle, navigation route information from the smartphone is transferred automatically to the vehicle, providing a continuous user experience. How cool is that?

Here’s a slightly different view of the cockpit, showing how it can display a photo of your destination — just the thing when you are driving to a location for the first time and would like visual confirmation of what it looks like:



Before I forget, here are some additional tech specs: the cockpit is built on the QNX CAR Platform for Infotainment, uses an interface based on Qt 5.5, integrates iHeartRadio, and runs on a Renesas R-Car H2 system-on-chip.

The acoustics feature that keeps you from shouting
The glass cockpit does a great job of keeping your eyes focused straight ahead. But what’s the use of that if, as a driver, you have to turn your head every time you want to speak to someone in the back seat? If you’ve ever struggled to hold a conversation in a car at highway speeds, especially in a larger vehicle, you know what I’m talking about.

QNX acoustics to the rescue! Earlier today, QNX Software Systems announced the QNX Acoustics Management Platform, a new solution that replaces the traditional piecemeal approach to in-car acoustics with a holistic model that enables faster-time-to-production and lower system costs. The platform comes with several innovative features, including QNX In-Car Communication (ICC) technology, which enhances the voice of the driver and relays it to infotainment loudspeakers in the rear of the car.

Long story short: instead of shouting or having to turn around to be heard, the driver can talk normally while keeping his or her eyes on the road. QNX ICC dynamically adapts to noise conditions and adds enhancement only when needed. Better yet, it allows automakers to leverage their existing handsfree telephony microphones and infotainment loudspeakers.



The reference vehicle that keeps evolving
Before you go, I also want to share some updates to the QNX reference vehicle, which is based on a Jeep Wrangler. Like the Highlander, the Jeep got a slick new exterior for CES 2016:



Since 2012, the Jeep has been our go-to vehicle for showcasing the latest capabilities of the QNX CAR Platform for Infotainment. But for over a year now, it has done double-duty as a concept vehicle, showing how QNX technology can help developers build next-generation instrument clusters and ADAS solutions.

Take, for example, the Jeep’s new instrument cluster, which makes its debut this week at CES. In addition to providing all the information that you’d expect, such as speed and RPM, it displays crosswalk notifications, forward collision warnings, speed limit warnings, and turn-by-turn navigation:



The QNX reference vehicle also includes a full-featured head unit that demonstrates the latest out-of-the-box capabilities of the QNX CAR Platform for Infotainment. For example, in this image, the head unit is displaying HERE Auto navigation:



Other features of the platform include:

• A voice interface that uses natural language processing, making it easy to launch applications, play music, select radio stations, control volume, use the navigation system, and perform a variety of other tasks.
• A new, easy-to-navigate UI based on Qt 5.5 that supports a variety of touch gestures, including tap, swipe, pinch, and zoom.
• QNX acoustics technology that enables clear, easy-to-understand hands-free calls through advanced echo cancellation and noise reduction.
• Cellular connectivity provided by the QNX Wireless Framework, which simplifies system design by managing the complexities of modem control on behalf of applications.
• Flexible support for a variety of smartphone integration protocols.

Additional tech specs: The Jeep’s cluster runs on a Qualcomm Snapdragon 602A processor and its user interface was designed by our partner Rightware, using the Rightware Kanzi tool. The head unit, meanwhile, runs on an Intel Atom E3827 processor.

ADAS, augmented reality, V2X, IoT, and more
I have only scratched the surface of what BlackBerry and QNX Software Systems are demonstrating this week at CES 2016. There’s much more to see and experience, including a very cool V2X demonstration, IoT solutions for the automotive and transportation industries, as well as ADAS and augmented reality systems that integrate with the digital clusters described in this post. To learn more, read the press release that QNX issued today and stay tuned to this channel.

QNX announces new platforms for automated driving systems and in-car acoustics

Paul Leroux

Every year, at CES, QNX Software Systems showcases its immense range of solutions for infotainment systems, digital instrument clusters, telematics systems, advanced driving assistance systems (ADAS), and in-car acoustics. This year is no different. Well, actually… let me take that back. Because this year, we are also announcing two new and very important software platforms: one that can speed the development of automated driving systems, and one that can transform how acoustics applications are implemented in the car.

QNX Platform for ADAS
The automotive industry is at an inflection point, with autonomous and semiautonomous vehicles moving from theory to reality. The new QNX Platform for ADAS is designed to help drive this industry transformation. Based on our deep automotive experience and 30-year history in safety-critical systems, the platform can help automotive companies reduce the time and effort of building a full range of ADAS and automated driving applications:

• from informational ADAS systems that provide a multi-camera, 360° surround view of the vehicle…
• to sensor fusion systems that combine data from multiple sources such as cameras and radar…
• to advanced high-performance systems that make control decisions in fully autonomous vehicles

Highlights of the platform include:

• The QNX OS for Safety, a highly reliable OS pre-certified at all of the automotive safety integrity levels needed for automated driving systems.
• An OS architecture that can simplify the integration of new sensor technologies and purpose-built ADAS processors.
• Frameworks and reference implementations to speed the development of multi-camera vision systems and V2X applications (vehicle-to-vehicle and vehicle-to-infrastructure communications).
• Pre-integrated partner technologies, including systems-on-chip (SoCs), vision algorithms, and V2X modules, to enable faster time-to-market for customers.

This week, at CES 2016, QNX will present several ADAS and V2X demonstrations, including:

• Demos that show how QNX-based ADAS systems can perform realtime analysis of complex traffic scenarios to enhance driver awareness or enable various levels of automated driving.
• QNX-based V2X technology that allows cars to “talk” to each other and to traffic infrastructure (e.g. traffic lights) to prevent collisions and improve traffic flow.

To learn more, check out the ADAS platform press release, as well as the press release that provides a full overview of our many CES demos — including, of course, the latest QNX technology concept vehicle!

QNX Acoustics Management Platform
It’s a lesser-known fact, but QNX is a leader in automotive acoustics — its software for handsfree voice communications has shipped in over 40 million automotive systems worldwide. This week, QNX is demonstrating once again why it is a leader in this space, with a new, holistic approach to managing acoustics in the car, the QNX Acoustics Management Platform (AMP):

• Enables automakers to enhance the audio and acoustic experience for drivers and passengers, while reducing system costs and complexity.
• Replaces the traditional piecemeal approach to in-car acoustics with a unified model: automakers can now manage all aspects of in-car acoustics efficiently and holistically, for easier integration and tuning, and for faster time-to-production.
• Reduces hardware costs with a new, low-latency audio architecture that eliminates the need for dedicated digital signal processors or specialized external hardware.
• Integrates a full suite of acoustics modules, including QNX Acoustics for Voice (for handsfree systems), QNX Acoustics for Engine Sound Enhancement, and the brand new QNX In-Car Communication (ICC).

For anyone who has struggled to hold a conversation in a car at highway speeds, QNX ICC enhances the voice of the driver and relays it to loudspeakers in the back of the vehicle. Instead of shouting or having to turn around to be heard, the driver can talk normally while keeping his or her eyes on the road. QNX will demonstrate ICC this week at CES, in its latest technology concept car, based on a Toyota Highlander.

Read the press release to learn more about QNX AMP.

Ford ports SmartDeviceLink to QNX CAR Platform

QNX joins Ford, Toyota, and other industry leaders to help drive new standard for app integration.

Paul Leroux

For as long as I can remember, QNX Software Systems has been at the forefront of integrating cars and smartphones. Through our flexible OS architecture and large automotive ecosystem, we provide automakers and Tier 1 suppliers with the ultimate choice in connectivity options for smartphones and other smart devices. And now, QNX customers will have even greater choice, with the availability of Ford’s SmartDeviceLink (SDL) technology for the QNX CAR Platform for Infotainment.

If you’ve never heard of SDL, it’s the open source version of Ford AppLink, the software that allows Ford SYNC users to access smartphone apps through voice commands and dashboard controls. Ford donated AppLink to the open source community to create a standard way for consumers to interact with smartphone apps, regardless of which phone they use or vehicle they drive.

SDL is quickly gaining industry advocates, including Toyota, UI Evolution, and, of course QNX. What’s more, companies like PSA, Honda, Subaru, Mazda are evaluating it for use in next-generation vehicles.

Why the interest in SDL? Because it’s a flexible, vendor-neutral standard that can benefit drivers, automakers, and developers alike. With SDL:

• Drivers can interact with apps by using voice commands, steering-wheel buttons, and other in-car controls, so they can keep their eyes on the road and hands on the wheel.
• Automakers can deliver a consistent app experience across vehicles, while retaining the flexibility to customize that experience for each vehicle brand or model.
• Developers can create apps that can work across multiple smart devices and multiple automotive brands — which means they have greater incentive to create automotive apps.

SDL for QNX builds on a history of successful collaborations between Ford and QNX, including the QNX-powered Ford SYNC 3 infotainment system. According to Paul Elsila, CEO of Livio, the Ford subsidiary that maintains the SDL open source project, “With its large market share, QNX can play a key role in driving the adoption of auto industry standards, and we are excited to work with them in building vendor-neutral technology that can simplify the integration of smartphone apps in any brand or type of vehicle.”

SDL works with multiple smartphone platforms. Moreover, it is highly flexible: it can work across a full range of vehicles, from entry-level to premium, and across a wide range of displays. It can even be used in systems without displays — for instance, in systems that use a voice interface.

To learn more about SDL, check out the announcements that Ford, Toyota, and QNX issued this morning.

The demo is in the details

A new video of the 2015 QNX technology concept car reveals some thoughtful touches.

Paul Leroux

QNX technology concept cars serve a variety of purposes. They demonstrate, for example, how the flexibility of QNX technology can help automakers deliver unique user experiences. They also serve as vehicles — pun fully intended — for showcasing our vision of connected driving. And they explore how thoughtful integration of new technologies can make driving easier and more enjoyable.

It is this thoughtfulness that impresses me most about the cars. It is also the hardest aspect to convey in words and pictures — nothing beats sitting inside one of the cars and experiencing the nuances first hand.

The minute you get behind the wheel, you realize that our concept team is exploring answers to a multitude of questions. For instance, how do you bring more content into a car, without distracting the driver? How do you take types of information previously distributed across two or more screens and integrate them on a single display? How do you combine information about local speed limits with speedometer readouts to promote better driving? How do you make familiar activities, such as using the car radio, simpler and more intuitive? And how much should a car’s UX rely on the touch gestures that have become commonplace on smartphones and tablets?

Okay, enough from me. To see how our 2015 technology concept car, based on a Maserati Quattroporte, addresses these and other questions, check out this new video with my esteemed colleague Justin Moon. Justin does a great job of highlighting many of the nuances I just alluded to:



In just over a month, QNX will unveil a brand new technology concept vehicle. What kinds of questions will it explore? What kinds of answers will it propose? We can’t say too much yet, but stay tuned to this channel and to our CES 2016 microsite.

Bringing a bird’s eye view to a car near you

QNX and TI team up to enable surround-view systems in mass-volume vehicles

Paul Leroux

Uh-oh. You are 10 minutes late for your appointment and can’t find a place to park. At long last, a space opens up, but sure enough, it’s the parking spot from hell: cramped, hard to access, with almost no room to maneuver.

Fortunately, you’ve got this covered. You push a button on your steering wheel, and out pops a camera drone from the car’s trunk. The drone rises a few feet and begins to transmit a bird’s eye view of your car to the dashboard display — you can now see at a glance whether you are about to bump into curbs, cars, concrete barriers, or anything else standing between you and parking nirvana. Seconds later, you have backed perfectly into the spot and are off to your meeting.

Okay, that’s the fantasy. In reality, cars with dedicated camera drones will be a long time coming. In the meantime, we have something just as good and a lot more practicable — an ADAS application called surround view.

Getting aligned

Approaching an old problem from a
new perspective. Credit: TI

Surround-view systems typically use four to six fisheye cameras installed at the front, back, and sides of the vehicle. Together, these cameras capture a complete view of the area around your car, but there’s a catch: the video frames they generate are highly distorted. So, to start, the surround-view system performs geometric alignment of every frame. Which is to say, it irons all the curves out.

Next, the system stitches the corrected video frames into a single bird’s eye view. Mind you, this step isn’t simply a matter of aligning pixels from several overlapping frames. Because each camera points in a different direction, each will generate video with unique color balance and brightness levels. Consequently, the system must perform photometric alignment of the image. In other words, it corrects these mismatches to make the resulting output look as if it were taken by a single camera hovering over the vehicle.

Moving down-market
If you think that all this work takes serious compute power, you’re right. The real trick, though, is to make the system affordable so that luxury car owners aren’t the only ones who can benefit from surround view.

Which brings me to QNX Software Systems’ support for TI’s new TDA2Eco system-on-chip (SoC), which is optimized for 3D surround view and park-assist applications. The TDA2Eco integrates a variety of automotive peripherals, including CAN and Gigabit Ethernet AVB, and supports up to eight cameras through parallel, serial and CSI-2 interfaces. To enable 3D viewing, the TDA2Eco includes an image processing accelerator for decoding multiple camera streams, along with graphics accelerators for rendering virtual views.

Naturally, surround view also needs software, which is where the QNX OS for Safety comes in. The OS can play several roles in surround-view systems, such as handling camera input, hosting device drivers for camera panning and control, and rendering the processed video onto the display screen, using QNX Software Systems’ high-performance Screen windowing system. The QNX OS for Safety complies with the ISO 26262 automotive functional safety standard and has a proven history in safety-critical systems, making it ideally suited for collision warning, surround view, and a variety of other ADAS applications.

Okay, enough from me. Let’s look at a video, hosted by TI’s Gaurav Agarwal, to see how the TDAx product line can support surround-view applications:



For more information on the TDAx product line, visit the TI website; for more on the QNX OS for Safety, visit the QNX website.

An ADAS glossary for the acronym challenged

If you’ve got ACD, you’ve come to the right place.

Paul Leroux

Someday, in the not-so-distant future, your mechanic will tell you that your CTA sensor has gone MIA. Or that your EDA needs an OTA update. Or that the camera system for your PLD has OSD. And when that day happens, you’ll be glad you stumbled across this post. Because I am about to point you to a useful little glossary that takes the mystery out of ADAS acronyms. (The irony being, of course, that ADAS is itself an acronym.)

Kidding aside, acronyms can stand in the way of clear communication — but only when used at the wrong time and place. Otherwise, they serve as useful shorthand, especially among industry insiders who have better things to do than say “advanced driver assistance system” 100 times a day when they can simply say ADAS instead.

In any case, you can find the glossary here. And when you look at it, you’ll appreciate my ulterior motive for sharing the link — to demonstrate that the ADAS industry is moving apace. The glossary makes it abundantly clear that the industry is working on, or has already developed, a large variety of ADAS systems. The number will only increase, thanks to government calls for vehicle safety standards, technology advances that make ADAS solutions more cost-effective, and growing consumer interest in cars that can avoid crashes. In fact, Visiongain has estimated that the global ADAS market will experience double-digit growth between 2014 and 2024, from a baseline estimate of $18.2 billion.

And in case you’re wondering, ACD stands for acronym challenged disorder. ;-)