Active sensing technologies for automotive applications

Active sensing technologies for automotive applications


[Music] thank you very much okay that’s all okay thank you it’s very nice to be here and I probably will relate very much to the previous speaker here before we start I just want to say that we will have a personal quirks and might happen to be that I speak very fast so please raise your hand if I get carried away and for those non-native speakers so don’t understand me raise your hand okay so this is the talk and this is the what I’m gonna be covering here very briefly about what panasonic does in eiders some motivations why we need active sensors and then in turn I will talk it briefly about sonar and such diffusion example a Raider and the Raider is going to be a phased array radar and a Miami Raider and then finally we’re even talking about imaging radar slider there’s some confusion of concepts there’s some people they’re tough cameras they’re sliders and they’re all tough essentially so tradition reasons 12 cameras the lightest I didn’t separate you so I’ll just briefly discuss both and then in conclusions and outlook so this is what panasonic does a lot of things it’s not known for being a desk company and it’s it’s fair to say that we’re quite recent reasonably enter the ADA’s business so we did a lot of entertainment stuff we work at the home and we also work now with the chorus we do both software and hardware we’re looking all tier levels which make life sometimes a bit complicated that also took quite instinct so yeah most you know that we do a lot of work in cameras so cameras are fantastic sensors for classification for tracking the distraction I’m gonna argue this talk that we need both active sensors as well as passive sensing so the many things you can’t do I think I claim in a way with the passive sensors that you can do with active sensors we are now we’re working with Altos on I personally do a lot of work in outer song we’re moving into the areas of lighter and radar so why do we need radar for example well this is something I stole from internet shamefully this is the average number of rainy days in Munich and as you can see there is potentially if we use same cameras they can see through rain this is just one example yeah you could go to Sweden where I come from you know it’s even worse if you can if you go to Newfoundland you you know yes I think you’re having a vision cameras on you varmint I think this can be quite a challenge so we need 19a operations we also need sensor fusion and we need to attract range and angle we need situation awareness woody navigation we need all these things and I think the bottom of the line I think is they also will need resilience to graceful degradation but something goes wrong the core should just stop we have to have a sense of their complement each other and I think for these reason only I think we need a combination of active and passive sensors working together so just to make things clear here there are a lot of you can divide this order the sensor space into different rooms you have active passive sensors this is what I’m going to talk about but you also have cooperative and non cooperative sensors which is it right slightly different and so I just kind of I’m just pointing out of there we’ll talk about this this this red corner here so we now have a currently working with the duration three so Nurse Corps basic but they do the job we’re moving into more advanced owners and for valet parking here for for auto braking I’m gonna show you some example so how we use this together with an automatic parking system so yeah for example this you can use the standard tricks I think that most people know it’s called trial iteration which means that you try to figure out a point in space using ranges there’s a very interesting by the way group a from única do the same thing but a bit more advanced so we can do it as spatially with the rear sensors so we can do with it temporally would you park so there’s an example who uses this to the left we see an occupancy grid map together with this supposed to illustrate the way your sonís use the sonís to figure out where you have no information that is it’s clear they have clear map but you also use sonís to paint out the rest of the course we use sonís to add things and remove things from the map so using and the white stuff you see here it’s mostly from structure for motion points so this owners and start of emotion points work in tandem to to get this you get the octopus a good map and to the right to see how we use is to find a Prague spaces so and I go to ship show a film here so the black stuff is free free space and the red stuff is destructive emotion points using a monocular camera and the Sonos you see and a purple and the green stuff is actually a circular motion points on the ground and you see now the we shot off a circular motion point so you old house so now now when the car is turning and now we switch on the SFM points again here and yeah this is how is how is used and I think what is interesting is that you can do this even when they sort of medium-range sooner so yeah and now the next step at the next sensor gonna present is the highest power is risen later and I think like a previous PQ we also go into the area seventy seven and seven to nine gigahertz at the many reason for this for example in cluttered areas and cities you want to be able to separate pedestrians and this you can do if you have a high not only high resolution spatial resolution but also if you have high frame rate you need both it’s also use very useful for rock rails detection I will show some example of this also and we also we are also moving away from a standard FNC v-ray of course believe that’s theirs they have a lot of inherent problems they have some advantages to certainly but it also has some inherent problems so we go into a coded pulse modulation technique they have they have their own problems to example the correlator has to be designed with this in care so it’s not trivial but it’s possible to do so and you can clear it with the pulse modulation you get much clearer detections this is the main the main advantages here and also it’s an actual just steer array which has been used in defense for very long time I can tell you I can tell you too much about the respects sir but I can tell you it’s a 20 Hertz right there and so to achieve gigahertz bandwidth so it’s about hundred twenty degrees and with so there is a forward crossing traffic alert situation so on the top you have the top view camera we overlay the radar tracks so it’s not tracking animation it’s tracking a radar and to the left you see the bottom right button bottom left you see the actual road I will actually side of the car you see this technique so yeah it’s quite accurate and here is a 40 I would say I was not able to bring a video here but it’s a it’s it’s one of the use endcap person obvious you have the pedestrian coming out of the two cars and I think what strikes me in a way we see this is that you have very low arrow covariances so the error is very low we’re talking about I think the previous peak is at this table I think is something like 7 centimeters it’s a very high rec receipt and what I say also you can see only the first car not the second car and this is occlusion problem of course so you can’t see through cars so and the second application now is the inter car distance were to be able to measure distances between cars quite accurately and the here I think the ground truth is 1 meter and the way that says 1.1 major I don’t think for this application I think it’s quite you know I think it’s quite a good result can we show you this video again already is which first it’s okay so and we also do some work and this sensor by the way I was saying I said advanced prototype this sensor is more like a research date but we also work with my my raters and I have seen example of a pedestrian walking across a bridge and you can see the entire person track four limbs so this is a potentially very advanced Raider but it’s still in research phase I think as this previous speaker said also I think Raiders in general people there is some talk in the community the buzz about pros and cons of sensors I’m not a pro lighter or pro grader or I’m a profusion person so I definitely need a few sensors this is the main message of my talk actually okay so the sensor I think we should also think is of interest ears of flashlights I mean this is a crude representation of what is a flash hider what we’re talking about here is basically Thomas add sensors and flash ideas I think have a bright future really it comes on just I think which has been used in defense for a long time and the principle is very very simple you simply have a timer on each pixel and you just count the when you want you to you trigger trigger to the policy and just simply count the time when it comes back it’s very simple principle of course in practice is much more difficult what it gets you is it gets you a depth map and you get the accuracy determined by the pulse length this is quite important so at long distances you get some you can measure up to maybe 100 hundred fifty 200 meters depends on the rider you can get some something like a 1 and a 1/2 nature of be 2 meters accuracy which is very good I think in that range however for close range I would say it’s not so good so is this differently I think you very good sense of a four range but not so much with close range and also they arranged by the way the range is pronounced limited to the output power now the tough camera has been used in I think in gaming for quite a long time and I think the most traditional of modulation principle is continuous wave which are things were very successful I think in gaming but not so much for outer applications so principle it’s what simple you send out a pulse and you have two gates Commission the gate one gate you open exactly when you saw at the Powis another one you you open just when the power cells closed so we have like two gates your measured simple the the ratio of the of the electrons essentially then of course this is not enough what you need to do also is to measure the ambient light in the background so there couple of ways to do this you can send a third pulse or a fourth pulse and not measure anything for the pulse but simply the background noise and this is what we actually do this what everybody does but this is the basic principle and the good thing here is that you get for free an image you get a grayscale image as well as a depth map so you can principle fuse information from the depth map and the image at the same time so there’s no need to sort of you know impose or have they simultaneously mounted video camera and put a lighter image of that you can do this it’s complicated you can do this but this you don’t have to do this in this case here and the thing that’s also the accuracy if you do the numbers you can see the accuracy it depends it’s a lot more luckylucky like a parabola so close range is very good we’re talking about couple of centimeters for range up to 20 meters maybe we’re talking about maybe 15 20 centimeters for curve detection this is quite sufficient reporting I think you can easily detect curve sir I’m gonna show you a video of this so this is the traditional left traditional scan you lighter to the left to the left and here we have the top camera this pen develop in Japan to the right and you see the boom very there to the right but you can’t see it on the left is more less invisible there unless you now we can see a few lines so I think this is quite drastic a demonstration of you know you know the the power of this stuff cameras the tough promise or the way is full VGA resolution it’s 30 Hertz it’s a typical I would say close range media rate sensor and pretty much you can do to see pedestrians poles chains curb stones and the second video is free space detection in urban areas same thing here you see the free space in the middle so let us see the car there and yeah so I think I think the accuracy depends very much on vacation for parking I think is essential to have very good accuracy at the curb detection so here I think it’s enough to see the curve there’s a curve there that’s enough but I think of the parking you actually to measure the height of the curb also and the we also do some work on developing flash lighters so we would do a some work on on Geiger mode a PD devices by the way this is a lot of jargon here huh how many people have not heard about APD devices I will ask photodiodes raise your hand that’s good no one person no it’s it’s a it’s like an old-fashioned photo multiplicative tube so we simply send in a photo mentor device and you have an exploding cascade of electrons essentially you can device the sensors in two different ways either linear mode or like counting photos so we said we’ll do some very preliminary this is a like to research work so we also has some some work and I’m trying to adapt the modulation so you can get away from this problem that these sensors have sometimes that you dark objects are high problem sometimes with the sensors so we were aiming at they’re also a high resolution waving at the 1200 by 400 resolution for pedestrian detection at nighttime where it’s up to 200 meters this depends very much of course on the clothing some of these specs you specify at 90 percent reflectivity or the worst case we you know you have to go down to 10 percent I think they were there they’re now coming out specs that actually they specify this quite quite well so nah this is a server interesting slide this you can have a lot of arguments about I deliberately did not put a a B here because there will be heated to date about what their vision is impossible or possible to use for breaking my main message here is you have to use sensor fusion without fusion you you can’t build an autonomous course is my person belief business sense are fantastic you can do a lot of things machine learning of detection but things like tracking for example and also tracking at high frame rates is it’s very difficult for vision you can’t do a lot of things with the ocean but this is something that you well you will have difficult to do now you can all get with this should some of these we yellow somewhere red I’m not sure I think bad weather example you know cameras are pretty bad right now sir very good you know ACC for example I think cameras are terrible so knows I’m not very good Raiders are really good – or not sure yeah – are actually indicated and the green here so that’s this is my awesome my belief and for automatic Parking miserable tough cameras they are able to measure accurately a quite close range so put them in greater where’s lighter I put in red this might be a bit unfair but what I say yeah we’re not saying here is that lighter is a bad sensor in general I’m saying is only for this specific application it’s doesn’t do the job I think for yeah when you see this image by the way you have any spontaneous reactions if I might sort of do the Q&A bit before you have any violent it takes it out – you know – to eat or ingest okay no erections that’s good we all agree that’s great so to conclude I think you need active sensors definitely even though it’s been debated about oh you know we could you miss can drive cars you know we’re know active sensors but also has previously been denoted I mean we have not only this and we have hearing we have tactile sensors we have all this sensor that we integrate in our brain so we implicitly actually not only use vision but will you misuse other sensors as well and I think for short mini range I think so is in caramel great I don’t think so Nizar dead at all contrary to what people say I think stoners have a bright future also it for some applications I think we’re meeting range range me to rate medium range census you have Raiders who have tough the lighters and cameras for long range should have Raiders and flash hiders so flashlights this also has a really bright future I think so and sensor fusion again I think sensor fusion is essential as I said before I mean with a tough camera we can actually do Boris for free depth map and fusion with the grayscale you can also do I mean standard stuff and anybody’s been working with the range smashing sensing with poor poor angular resolution you have that trial iteration and you have occupies a grid mapping of course that we do and we have track to track fusion which is what many people do so what’s probably missing I think is probably standards for for fusion you can fuse data on so many different levels and a lot of people experimenting out with different solutions so yeah there’s no one way to do this and once again I think grace of degradation this is something you have to think about when a design systems I know in aviation for example some modern aircraft Lisa especially middle tier aircraft because I have a background defense they now use separate internal networks for tactical information and critical information this is something that definitely I think people need to be you know you have to have in courses as well yeah that’s it [Applause] [Music]


Leave a Reply

Your email address will not be published. Required fields are marked *