Sure, HoloLens is cute, but Ford was making VR work before it was cool
Elizabeth Baron found genuine use for virtual reality in the design process
Interview In an era defined by Microsoft, Facebook, Google, Apple and Samsung, it's understandable, easy even, to become complacent and look only to Silicon Valley for tech innovation and leadership. Smartphones, touchscreen, mega search, the app economy... if all these things and more didn't come from a handful of US tech firms, they were certainly popularised and monetised by them.
With virtual and augmented reality, again it's the usual suspects. Microsoft has HoloLens, Apple the iPhone X. Then there's Samsung Gear and Facebook's Oculus.
Except someone else was there first – and not even from a sector you'd normally associate with the words "technology" or "innovation". It was Detroit car maker Ford.
Twenty years ago, automobile design and manufacture used many of the same, slow, analogue processes developed at the beginning of the industrial era – bespoke models, hard to create, hard to change, and impossible to simulate. Gran Turismo wasn't just leading video gaming – it was way ahead of the automakers.
It took a generation for the auto industry to rethink, retool, and redevelop its design and engineering processes to take advantage of the same simulation techniques pioneered in video gaming. But for the indefatigable efforts of one woman, it might never have happened at all.
Virtual Reality and Advanced Visualisation Technical Specialist Elizabeth Baron's leadership produced the Ford Immersive Vehicle Environment (FIVE) lab for vehicle prototyping in virtual reality and Immersive Cinematic Engineering 3D computer rendering. The system today employs an 80-inch, 4K monitor connected to a computer, and the viewer wears a VR headset to explore the vehicle with Autodesk VRED 3D-visualisation software. FIVE has since attracted the interest of NASA and Boeing.
Now most automakers tout their "virtual" design capabilities, but Ford was there a decade ago. With integration across design, engineering, manufacturing, and even marketing, a global company fabricating a complex product finds itself connected, collaborating and "glued" together by simulation.
For her work, Baron received Ford's Dr Haren Gandhi Research and Innovation Award, the car maker's highest technical accolade. As the inventor of modern automotive emissions control, Gandhi's research helped keep air around a breathable standard in busy cities.
The Gandhi award had always been pinned to one of the boffins in Ford's research division for fundamental advances in the field. But Baron had never worked in the research division, always the outsider solving design and engineering challenges the research folks never saw.
Baron getting the gong was unprecedented – and a very public recognition of the value Ford saw in her work (that Baron was the first woman to receive the award made it a touch sweeter).
That award recognised a generation-long effort by Baron to bring simulation into the heart of the automotive industry, a transition Ford's competitors have yet to fully make.
Twenty-nine years ago, however, Baron was a humble programmer analyst at Ford. She worked on Ford's custom-built CAD system – a bit of big iron that helped the manufacturer develop the "digital" vehicle, collating all the data associated with each component. Although useful to the business as a whole, that kind of data collection offered little support or guidance to designers, engineers or manufacturers.
Baron sensed that by placing this wealth of data within realistic simulations, Ford could radically transform their entire manufacturing process, and began a career migration away from CAD towards real-time simulation. By the mid-1990s, virtual reality had gone "mainstream" as a concept, though few organisations (Boeing famously at the forefront) made any real use of it. Promising, sure – but table stakes for VR demanded a graphics supercomputer that cost several hundred thousand dollars.
Having spent the last few years structuring the dataset for these digital vehicles, Baron reckoned the next logical step would be to visualise that data – creating a representation of a car from the underlying data. Although a straightforward concept, the road to its realisation took the better part of 20 years.
Are they heavy? Then they're expensive
A Ford employee demonstrates a new virtual reality technology at the 2012 Chicago Auto Show Media Preview on February 9, 2012 in Chicago, Illinois. Pic: Darren Brode / Shutterstock
Step one: create a virtual car. That wasn't as easy as it sounds. Twenty years ago, all the optical tracking we take for granted today (even our smartphones can do it) lay on the other side of ten iterations of Moore's Law, and a thousandfold improvement in computation.
Heavy head-mounted displays, generally pricey and mil-spec, required integration with baroque motion-tracking systems that employed sensors moving through magnetic fields.
That sounds simple enough until you remember cars are generally made of steel, a material that has a tendency to block or warp magnetic fields. Baron talked some of Ford's more artisanal craftsmen into building her a fully adjustable vehicle prop – out of wood. What appeared a step backward in materials meant a great leap forward in technology, because now Baron could invite an automotive designer into her "studio" – tucked away in an overlooked corner of a Ford facility – and give them an experience of their proposed designs from the perspective of both a 4ft 7in woman and a 6ft 4in man, simply by modifying the dimensions of that magically modifiable prop – complete with steering wheel and cupholder.
That was the first time anyone at Ford had a real sense of the value of simulation as a design aid – a conclusion Baron had already come to, but one she had struggled to demonstrate. But demonstrate it she would, again and again and again over the following years, finding design questions that required her particular expertise and tools.
"The majority of folks at Ford weren't convinced of the value of simulation," Baron told me in a recent interview. "They'd tell me: 'If it could do X, that'd make my job easier.' So I'd work with them to solve that specific problem."
"I'd look for the problems that were hard to solve any other way. Create a solution. Document it as a use case. Then move on to the next problem. Eventually, I had a big spreadsheet with solutions."
A classic example in automotive design – where do you put the radio knobs? "I'd create a simulation, so the designer could have a look around, and show them the three options they'd need to decide between," Baron said. She tracked their hands in the simulation. "They could reach out – and feel whether a particular placement worked."
Baron had arrived at a genuinely new way to think about automobile design – as one element within a broader a visualisation process – and gave designers the capacity to quickly confirm or trash their design decisions. At the time, an automobile would take five years from conception to manufacturing, and the millions of design decisions involved could never be fully inspected or tested until it was too late to make meaningful changes. Difficult dashboards, door latches and instrumentation weren't intentions – they simply couldn't be corrected in time.
Even with this huge win for designers, Ford was cautious about building up its reservoir of virtual technique. "There was a sense from Ford leadership that this tech was coming – someday," Baron recounted. "They invested just enough to stay ahead of the curve, and mostly that was just me, working on this until I got another person – full time – in 2003."
If that seems unnecessarily parsimonious, it should be remembered that automobile manufacturers run lean – particularly in the wake of the global financial crisis, when Ford's US competitors both went bust.
Another Ford employee told me about her brand new position and 40-year-old office furniture. These are not the anything-goes playgrounds of Google and Facebook. Tech is deployed sparingly, and only when it solves a specific business problem.
Which problems best fit Baron's simulations? "I'd look for the problems that were hard to solve any other way. Create a solution. Document it as a use case. Then move on to the next problem. Eventually, I had a big spreadsheet with solutions," she told The Reg.
By 2007, Ford had begun to shift its design strategy toward virtual-physical simulation, lead by an exploding tech sector delivering high-performance GPUs and high-resolution displays, and an innovative if not somewhat low-tech adjustable prop (complete with a Hollywood-style prop department) adding controls and displays to the car or truck being simulated – just the right constellation of advances to move the needle on simulation from "way too hard" to "can it help?"
Oh snap, this totally helps
"There were three points when it changed," Baron noted. "First, when we wanted to streamline the design process. How can we visualise things to take time off product development? That's when we built our first CAVE (a room-scale virtual environment), so designers could walk around a simulated vehicle, get inside, and get a feel for it. That's when virtual assessments of design became an integrated part of the design process – for the first time.
"Second, when simulation extended from design into manufacturing. We had all the data for our vehicles – that was a product of work at Ford many years before – now we could take that data and simulate it along with the design. That made it possible to represent the build conditions in the immersive environment. Not just how the car looks, but how it comes together on the assembly line, during the manufacturing process.
"Finally, we got to the point where simulation allowed us to have a global collaboration and design review. We had folks in Australia talking to executives in America, everyone looking at the same model, everyone with the same capability to point at something, ask questions about it, make changes to it – changes that everyone would see immediately. That's something we'd never been able to do before, and as soon as we got to that point, that made the case at Ford for going all in on immersive simulation and visualisation.
"The efficiencies are phenomenal. Because it's relevant to everyone in the company – through their own lens. What an artist understands, what an engineer understands, what someone on the manufacturing line understands – they're all different, but they're all here in the simulation. Just change the view to fit their lens."
Now Baron is back to using supercomputers to handle the demands of a company-wide platform for simulation. "We deliver real-time ray tracing of the immersed perspective in 4K resolution – that takes a lot of CPU. But it means a designer can specify the physical qualities of the materials they're using. Everything looks and acts real."
That feedback between designer, engineer and manufacturer means the enormous capital and material flows required to make a car – one of the most complicated mass-produced items – can now be simulated all the way from idea to customer.
In another 20 years, the integration of simulation with manufacturing will have moved from exotic to ordinary, with AI systems supporting design decisions. Ford recently launched a new AI research team to tackle the issues of autonomy, mobility, sensors and drones.
All of these product pipelines will be knitted together with next-generation mixed-reality systems, providing a lens onto any process an employee needs to focus on at the moment. While that may sound a bit far-out, it's closer to reality than Baron's vision, 25 years ago, for a fully simulated design and manufacturing process.
As simulation integrates into our industrial processes, those processes change, and our capacity to be creative with them grows accordingly. The stage is set for an explosion of forms, features and futures for transport. Simulation is touching the real. ®