Accelerating disruption in the automotive industry with Unity – Unite LA | AutoTech Summit

Accelerating disruption in the automotive industry with Unity – Unite LA | AutoTech Summit


good afternoon everyone can you all hear me okay good now your response I presume you you just had lunch you’re going to take a small nap feel free you know I know how tiring it can be to listen to these presentations so yeah thank you very much for taking the time being of being here we’re visionaries triple7 we we are based out of Hong Kong we started the business five years ago will briefly introduce each other I come from a business background and architectural media architecture interaction design with lighting and I’ve lived in Hong Kong for 14 years and my business partners friends yes so my name is friends and basically I used to work at Lego before 10 years ago bridging the physical and digital with air and then moved to Hong Kong created visionary super 7 with my other business partner Nikolas which is not here today because he’s having a baby congratulations to him and we’re going to show you a different thing so how we accelerated the disruption in the automotive industry with a couple of projects showcase that we did invest so our slogan has been anticipating the future because this is what Nico and friends were doing inside Lego there were really pioneers in the whole augmented reality topic they started the first patent was done in 2009 9 yeah so a long long time ago with Lego we requite fortunately when we started the business we pitched the idea to Lego and they you know there were first customer as well as we sorted and as new friends were saying we’ve kept Lego over the years we still do R&D for them anything to do with a our VR and mr and so we basically have an agreement with them where we cannot work with any other toy company in the world which we don’t need to because Lego is probably the most amazing company in the planets but any any kind of orangey that we do we can then use for other type of industry and we’ll show you some of these examples so what we’ve been building over the years is what we call an ecosystem our target is to influence the customer lifecycle and product sorry customer journey and product lifecycle we are very keen on figuring out what is the place of AR V R and M are within this ecosystem so if you can think of it this way how can people who design a vehicle can use this kind of tools the guys who are manufacturing the vehicle in factories how can they use these tools you know to do maintenance in Maschine the people who are then using the vehicle images for 3d assets who create all the marketing material once a person purchased the vehicle can we use these 3d assets and technologies to have people do maintenance themselves through augmented reality multiple things after sales so that’s what we’ve been doing over the years and we’ll show you a few examples on this topic we’ve been quite fortunate even though we’re based out of Hong Kong we have had the pleasure and and the fortunate lack of working with global leaders such as the ones that you can see here so pretty much all of our customers are outsides Hong Kong except for infinity that belongs to the NYSE and Renault Alliance and their headquarters in Hong Kong so we really only work with the headquarters of all these companies not only the company the car manufactures but also the tier 1 and tier 2 suppliers the automotive industry so this was our first customer three years ago if you guys are a fan of you know watching films this vision was in the fastens Fast and Furious 7 7 which I’ve never seen so there was the like and from W motors it’s a 3.5 million US dollar vehicle three years ago you know we couldn’t really approach any other big brands so we just went for this hyper luxury brand in Dubai they really liked what we were doing and three years ago they granted us our wish of allowing us to do the first configurator for them and maybe friends can explain what we did in detail yeah so on as you can see like cows are still sold with traditional marketing materials and especially that 13.5 million u.s. dollar car sold with basically paper brochures showing you the different customization possible this guy is actually highly customizable you can change the can choose the color of the body paints the calipers the reins the carbon fiber the stitching and so on and what we did here here is to create an interactive 3d real-time configurator that runs on mobile tablets so typically an iPad iPad pro and we really put a lot of effort to optimize the 3d asset so that it can run smoothly on such devices and also the river a really realistic quality that was back in 2016 and yeah so we had to basically come up with a benchmark of what could be the weight of this video set with here the car is about half a million triangles we have baked a met occlusion maps we have saleable textures we created to you to UV sets so that we can have terrible textures and unwrapped AO maps and as you can see we replicated all the options you can find in a brochure so you can basically customize every single aspect of your car and also explore it from inside I’ll open the trunk to do the hood the doors etc and change sceneries so after that wo chose did a joint venture with a company called iconic from China they do luxury multi-purpose vehicles they’re the first ones electric for MVP MPV in China so these are basically these cars are used with just a driver at the back here very comfortable seats very popular in China and so our job was to blend both brands through a virtual showroom we launched in China we basically grabbed the newest vehicle from wmo chose the veneer and they’re all vehicle to liken and there are two new iconic vehicles and the VR experience may be friends again here until so yeah so here as you can see we have forecasts available we have the like and anchors for the Feeny Super Sport and the two iconic 7 VAP in premium and all this running in HTC vibes we had to also come up with a level of detail that is acceptable and running smoothly with forecast on the screen where he can go inside and outside and can again navigate intuitively around the cars and customize all the different aspects here it was really relevant because actually on the show no one could get in the cars those cars were really premium and only VIP customers could actually visit the interior of the car so that was the way for anyone to actually experience it from inside out and so of course one of the great things and the reason why we love working with unity and it’s the the main software we use all the time is because of the flexibility of unity that we can have the 3d assets then used for multiple purposes with this with a an app on an iPhone or an iPad VR experience or as friends is going to show you next what we did when we released the air kid function so basically a week after the air kid was launched we already released this with W Moltres we’re quite fortunate that these brands really believe in what we do and so they give us budget to do Aaron D R&D and a lot of experimentation as we’re quite fortunate we are always looking for the newest technologies when it comes to hardware and as soon as it’s ready we deploy it for customers regardless of what industry they’re in so as you can see here we can use a kit and park the car in the street to one-to-one scale we also experimented a bit with capturing the ambient lighting in reflections of the surrounding to give more realistic lighting of the car and then again yeah it was our first attempt with our kids to integrate it and once again unity was really great to work with because in like a few days playing with their kid we could easily integrate it in our existing app that was on the App Store and we had this basically at the launch of where kids and the latest development we’ve done in showcase is also in partly thanks to Vinnie and Ileana from PTC who were here with before yeah super emporia of course as an amazing software they’re also pioneers in this kind of technologies and this is something that we proposed to the team from like and from W motors you know could we come up with a concept where people could visualize all the configuration the possible configurations in terms of color without having to go through just a digital version and yeah I think France is always very good at explaining exactly how we did it so so yeah so here we were part of the before yeah early access programs we had access to the latest model target tracking and here as you can see we can detect the car with a simple mobile phone smart phone and we can alter the color of the car dynamically and even paste and vinyls around the car so this we created a special shader that blends with the actual cars we are not completely masking the product we are really blending a new paint with the actual existing paint of the car and as you can see the illusion is quite good when you move around the car all the vinyls stay in place and have the shadows the rear shadows and reflection of the car so it really gives a realistic look and you can really see the potential of having that in the future it’s quite stable you can walk around the car and all 360 angles and yeah you can just have fun and customize it in all sorts of way yeah we we included this slide as soon as we knew that our BMW customers were going to come in to the conference Dylan and pizza back there he wants to stop them later so we this is a project and just kidding very good project but it was a very interesting project because with BMW they wanted to figure out what was the future of digital especially in showrooms so we had been working on a project called the vision gates and the vision gate was the concept was principle was in principle can we bring the products to life before they are in the showrooms and the project is quite extends it’s been launched in multiple countries around the world the idea is that five or six months before the carrot in the showroom we can start showcasing what this car looks like not only physically but also why is it technology inside the vehicle and many of these dealers of course they are not very keen on spending huge amounts of money in terms of the hardware resources and so we’re always trying to figure out you know is it an iPad and a Samsung we are heavier that could be the entry point that they could spend to have an interactive experience so in this case we did use the Samsung VR headgear we try to make it as realistic as possible but that meant that anyone could just buy this downstairs and they had access to a VR experience yeah maybe friends yes yes so here we had we were really constrained by the capabilities of Samsung gear VR so we had to again come up with a way of visualizing the interior of the car in the most realistic way we could do so don’t if you can notice but actually when the car moves and turns like we have dynamic lighting and reflections so this is not actually a video playing we only have a 4k 360 video for the city environment but the car is actually a combination of different channels of 360 images which gives us access to to dynamically change the reflection on the reflective paths the lighting as well we can even go to a night mode to see the interior ambient lighting of B&W so you will see in seconds afterwards so you have different hotspots around the car that you can different features of the interior features yeah let’s just move on to them later now we’ll send you the videos if you guys want to look at them so ultra ultra is a tier 1 supplier or tier 2 supplier to the automotive industry they make these little things semiconductor components are used for liner technology infrared they’re very tiny technologies that are very important for autonomous driving vehicles hand gesture and multiple things so this is the traditional marketing that they’ve been using so when they would approach company like BMW companies like Tesla this is what they would be using and so when we approached them we asked them you know if you guys have a technology company maybe it’s worth speaking a different language when you approach your customers and so our solution was to sit down with the engineers responsible from every for each of these components and create a sales tool that would explain the application of all of the components basically what the component did instead of just showing boring technical spec sheets and the solution was basically an app where we for every component once again we created an animation during the meetings so let’s say Tesla is now meeting with also means meeting Tesla they could during the meeting send a technical specification details just via via email and they could create an access account so the engineer could also download the app and maybe friends can explain to you what we did technically so here as you can see we developed an iPad applications where you can see the car with all the different applications around the car so here if I select headlights I can dive into the scenario of the headlights and learn about adaptive front lighting so what is that observe front lighting basically it adapts to different Road environment so here we are in the city and basically it’s a step by step scenario that explains you all the benefits so when you go to a crossroad it’s going to activate cornering lighting and at any point of time you can drive it into the headlamp block and open it and dive into the component level of oran so these components are like few millimeters really hard to visualize and understand and here we can display all the technical specifications but more importantly the applications on the road of this vehicle and and and so bear in mind that this kind of product is only used by maybe 27 salespeople within the organization so this is on a public app there’s an app that we create we develop we host we do the maintenance for this app but these 27 or 28 people they bring around maybe 800 700 million euro revenue into the organization so it’s a very important tool for them and that’s what we think that we are trying to we’re trying to push the limits in the automotive industry not just with the guys who are very visible like the BMWs of the world with Daimler’s but more about the other companies that are in the ecosystem like tier 1 and tier 2 supplies which are equally important in the in the whole industry and here we also were technical said we basically created asset bundles that we hosted on the cloud so that we can easily update each component and specifications if we need to rather than doing a full update of the application so infinity is a great example they are a much smaller brand from the parent brands the Nissan renault alliance which also allows us to be more more experimental with the things that they have been doing and so the first job that we did was the most important launch of the history’s brands was a qx50 here in LA there were multiple projects that we did for this which friends will go through them but the most important thing is that we managed for the first time to modularize the software so that it could be used for multiple purposes at different stages and you will see how one project became multiple kind of meet micro projects being used worldwide by different stakeholders in the industry the brand themselves the dealers and the distributors and and somebody else who which I forgot right now anyway let’s go through so here as you can see we created the first exhibit was a screen on a sliding rail on 80 120 degree and the screen is actually displaying the real image of the car so there’s a camera behind that films the car and we overlay the digital content blended with the car to create an x-ray illusion so here we can show in place the the boot of the car with the stroller that he can fit inside we can show the steel structure with the different robustness of steel at different parts we can show the engine the new Biturbo engine through the hood of the car and yes sir each basically angle of around the car is showing a different parts different aspects of this new visible platform part of the new qx50 then the second exhibit we here explained the pro pilot technologies so propeller technologies from Nissan that are inside the infiniti cars and could be a little bit difficult or difficult to understand for non engineer non-technical customers so here intelligent all-wheel drive what does it mean on the road when when the conditions are bad what is predictive forward collision warning all these things and here we bring basically to life the clay model we again we film the car and then we merge digital to create an augmented with the illusion and we contextualize two different scenarios that can intuitively understand into each pieces of technologies and the third one was focused on the design aspect of the car so here it was a highly realistic configurator running on the 4k screen and users were interacting it through a remote iPad that you can see on the table so you can use it as a touchpad to open the doors etc but moreover you can interact with the physical samples so when you take the red car paints and you place it on the iPad basically it’s going to tell me dynamically change the paint on the screen so there is a nice relationship of a physical and digital because customers they like to feel things you know the leather is very important to feel the texture so you cannot get away with this when it’s digital and same for the paint you want to see how it looks like in real light and then you can see the result in in the screen so this was also very playful and interactive for people and yeah and with the approach that we’ve always had with our customers is to really think of scalability and easy deploy ability so when we developed the the main project we developed it in such as in such a way that we could then replicate it as you can see it was widely used very much they didn’t act they didn’t a technical support because it was very straightforward the the installation in many cases and and this is definitely one once again the strength of unity that it can be as complex or as simple as you want to make it but it definitely serves its purpose really well and so we approached infinity once again and we said you know you clearly have a need for these kind of technologies but your dealers and the people who are doing the sales from the ground worldwide will not be able to afford such physical installations what we could do is dilute a bit the complexity in terms of hardware without removing the software experience and so we we had a few different experiences so for example here this is one the one that Franz just showed you where you have the 4k screen but in what in another case we could just connect the iPads in another case it could just be the whole configuration just and iPads without having to have the whole computer attached to it and here are a few examples of how we simplified everything to make sure that it worked on an iPad as well yeah so exactly so the idea here is like also sometimes you know like the the OEMs they don’t really have a technical knowledge of how complex it is you’ve done it for 4k screen with an alienware computer and I can you copy/paste the software to an iPad the answer is not that simple so there is a lot of optimizations that needs to be done which is fortunately quite easy to do with unity so here you can see we are very close to the quality that we had on the 4k screen but running on an iPad pro so we did like a few optimizations here and there so that to make sure that those traders are optimized for mobile we got rid of unwrapped ambient occlusion textures but we did baked vertex sale instead so we can save a lot of texture space and we end up with only one in research instead of having to so this makes this really asset creation much easier the cars also modular because we have different trims different models of the car so sometimes they have the sunroof sometimes they don’t we don’t have one cup or configuration that would take too much space we have warning FBX for things so instead we have like basically an excel sheet with all the different parts that are activated or not depending of the configuration and yes we can see we have highlights we have like traveling short story shots aroma car and so on and the next example we approached before yes and we got access very early to the model target and this was a great example of how we could use that kind of technology so what we did was it’s not showing it so what we did was we had the product in the showrooms this small car model in the showrooms before the physical car was there and what we did was we reused the content that we had had the 3d assets that we had used on digital solutions previously to do the model tracking so we could display all the technology and that is sometimes invisible within the within the car model yes you as you can see here we can detect the car fairly if we don’t fall over fairly accurately and the lighting is really not that great to be honest there’s a lot of shadows there’s a lot of reflections like the the car is like pretty lit up at some areas like there’s like huge shadow here but as you can still sit quite robust to it detects the car quite well and you can really go far away from it and yes so basically we transposed again the sliding screen experience there was a challenge but thanks to the capabilities of the model target from unity from before then it was fairly easy to to replicate the same experience so yeah and then at any point of time you can take out the different things and get a closer look of it yeah so these kind of tools is very important and as you as you can see here we can basically it works on a small model but in this video in this video basically it also works on the one-to-one scale car so this scale doesn’t really matter you just need to prepare the model targets accordingly with the correct scale but it’s great because you can have a small experience transportable but you can also have it on the real car so here the dealers they end up with a tool to accompany the customer when they visit the shroom and it’s this is much more powerful and attractive than coming to the customer with a paper brochure here you have something much cooler it’s ice breaking it’s it’s new it’s it’s astonishing and you can learn a lot about all the different features of the car in the dealership and this is just the beginning many of the things that we cannot show unfortunately but they will be used widely in the industry is how are these tools going to be used for example to do maintenance on a machine what is that industry side going to look like so right now we’re looking very much at the marketing side but this will definitely revolutionize the way that we approach many of the day-to-day tasks that we do for example in a factory floor and this is very exciting for us and yeah eventually we will be able to show you guys right now is still very very handy driven yeah the next example is all your friends yes so basically as you know unity ERP came out quite recently and we as soon as it was available we get our hands on it and we were basically trying to evolve our existing car configurator that you have seen before with this new render pipeline and with a little bit of help from unity Japan a Hilux hook’s who gave us like some tips and tricks which are like we we managed to with a fairly short amount of time get something that is really from our point of view looking quite good and highly realistic and it’s a huge comparison with what we had before it’s a huge improvement and it’s really important for the customer because they’re always trying to like they compare with what they have seen on the market with videos they can see on YouTube made with other softwares and now with a unity a GRP we can really bring a level of realism which is really astonishing so here you can see the quality of the leather interior it’s like it’s really starting to compete with the pre-rendered CGI images but the good thing is that it interactable so if I want now to move the seat back and forward that’s possible because it’s real-time 3d so yeah so this is like still like a prototype experiment but we are planning to roll this in the new dealers of infinity with this new update yeah let’s just play the video a bit longer so we have you have any music on the video there’s no music no trace I’ll sing for you guys yeah of course right now it definitely takes high-tech you know high specification computer to run this kind of graphics but as we all know power is weak computing power is becoming more and more small so eventually we will manage to get this kind of quality on our iPhones or Android devices within a couple of its use yeah so there’s another example that we were allowed to show a company like infinity they spent a lot of money trying to figure out what is the best way to design a vehicle before they go into production traditionally the way this the way this is done is they build a physical mock-up of vehicle they seat people inside read the demographic and but you know of course there’s restrictions with this you have someone sitting down in front of a mock-up vehicle and you know let’s say we’re sitting here they ask you you know imagine this dashboard now blue now red now whatever your brown which one would you like more so you don’t know if people are telling the truth you know you’re very restricted because shipping this physical product is quite difficult worldwide so we came up with a VR experience that can be used in multiple countries multiple languages we partner up with a company doing neural marketing where they monitor your brainwaves so they claim I’m just kidding they’re amazing and then we compare results what we’re doing here actually you’re better at explaining the technical side yeah so this prototype where basically we combined the AI Tracking’s we are using a glass that is plugged in the HTC vive and you can see this red cube moving and the steering wheel and now the speedometer etc so this represents the eye tracking we are tracking the eye position of the user over session of time we are also measuring at the same time the brainwaves emotional values and so from the interior the exterior so we record everything over timer and then afterwards we cross combine all the data together so here basically right now right now the 3d model we use it’s not the final 3d model of course the quality as you saw previously can be done much better right now it’s still in the experimental science but you will see more and more of this happening in multiple industries the automotive is one of the obvious ones because wherever you can save costs with these kind of tools then you know you can really justify it to the CFO or the financial person and why they should invest in these kind of tools and more importantly the metrics that were collecting behind would just show it to you here yeah so here we can see those two analytics so this would this is what after the person has finished the experience we have their name we know their gender whatnot and that’s what they see every time that they spend seconds and milliseconds tracking certain part of the vehicle we track that and the more time they spend looking at one area we can then generate a heat map based on the the you know visual patterns during the whole experience and then the idea of the world is to cross this heat map of the eye tracking with the brainwave data and have more accurate values of what actually the users are interested in and what emotional emotions it generates at the same time yeah and so this is a partnership with it with a company called Johanna cooks in Munich they are amazing at doing cutaway physical engines so companies like Audi they would give them the e-tron is it yeah it’s one vehicle engine and they basically put the whole engine apart they cut it and they help you visualize how the mechanics of thinking of these things work on the inside and we’re basically once again using before his model targets to then create tools that can help people visualize things that sometimes are invisible to the naked eye and of course this is not only used for marketing but they’re already using it for training inside their workshop so that they can train people on how to better repair or do maintenance on these engines without having you know to have someone you know chop their fingers off or in the best case scenario so yeah here you can see this is like a demonstration of different possibilities we can do so so here we can do highlights of different paths we can take the engine apart like exploded to have an exploded view we can visualize also the vehicle so what kind of engineer is actually this is a Kawasaki engine and it fits here in this vehicle so you could imagine the applications here to visualize mechanical complex parts where does it sits in the vehicle and we have x-ray function etc and this is all possible again with before you model target tracking so you can see even some of the parts of the physical engine are moving all those parts are highly reflective so this this is a quite complex with tracking it works really quite well and if you don’t have the engine we also have an air kit way to visualize it so here I can basically place the engine over there and have similar experience where I can have the annotations around I can highlight the different parts I can see an x-ray of all the moving Pistons inside I can explode it I can see the very crude as well here go from here yeah exactly so I can see the engine with the motorbike so once again cross combining different technologies together yeah so every single presentation would been to people go over time we want to make sure we left it really short so you guys could go and enjoy some coffee or tea or if you had any questions so yeah that’s pretty much it guys I hope thank you [Music] feel free if you have any questions have you have a question meet up time feel free to shout them out Frannie single in case anyone is interested yeah yes thank you yeah so it’s a V formula that’s the name of the SDK and it belongs to PTC and those two guys are here so you can ask more details about it and they haven’t they have a stand-down service in video two motive sides they’re very kind people they’re very cool should really speak to them very friendly yeah thank you that’s it well anyway you’ve got our emails any questions thank you very much hope to see you soon and don’t hesitate to visit our websites most of what you saw is actually available online you


Leave a Reply

Your email address will not be published. Required fields are marked *