While having high-level technological accumulation of SenseTime Group as a base, SenseTime Japan also places weight on its original techniques and know-how it develops and progresses in Japan.

The Company holds up the following points as its major technologies; 1) Face detection / authentication using original face recognition deep-learning algorithm, face image database and deep learning platform, 2) Detection / tracking of vehicles or pedestrians in the scenes where kinds of vehicles and people are mixed up in road environments, 3) Detection of traffic lanes, stop lines, traffic signs, and traffic signals in road environments through image recognition, 4) Real-time detection of self-position or posture / construction of 3D models of environment by the visual-inertial navigation SLAM (Simultaneous Localization and Mapping) System developed in-house, 5) Detection of human body posture, 6) Liveness check to recognize if the shot object is a human face, a photograph, or a deceptive mask, and 7) “SenseParrots”, which is an AI learning framework originally developed to assist large-scale deep learning. Solutions that utilize these technologies includes the following; 1) The automobile field such as autonomous driving, Advanced Driver Assistance System (ADAS), and development of vehicle electronic technologies; 2) Support for security measures, traffic detection / tracking in smart cities, and smart retail, and 3) Fields of FA and robotics.

Among them, technology development of basic algorithms of AI and its platforms, for example, are performed in the Chinese side (SenseTime Group). In marketing them in the Japanese market, they are customized by SenseTime Japan as needed to guarantee the required quality and provided as solutions, according to Ms. Chikayo Nishioka, PR Leader, Corporate Planning Div., Strategic Planning Dept. of the Company.

Technologies of SenseTime	UC-win/Road DS utilized in Demonstration of SenseDrive DMS

MHI Sensing Dept., Automotive Business Unit we interviewed aims at developing and productization of a variety of on-board applications using AI algorithm, while another group in the same Unit is specifically addressing ADAS.

One of those that The Unit currently places emphasis on most is development of DMS. This is for checking and estimating the driver’s physical conditions in order to secure safety and comfort of the driver. In particular, at the level 3 of automated driving, it is so stipulated that “the driver can leave driving operation to the system, e.g. by letting go of the steering wheel” under a given condition. There, it is supposed to be required to have HMI with a “function to monitor the driver’s condition capable of taking over driving operation from the system, and to give warning” when needed. In other words, the function similar to DMS is required.

Concerning this, the Company utilized face recognition / authentication technology they have fostered through technology and product development within SenseTime Group. At present, they are promoting development of DMS capable of not only detecting the driver’s dangerous behavior or conditions that interfere with safe driving in the on-board environment, but also realizing the following functions with high accuracy and high speed; 1) personal authentication for security, 2) gesture recognition for HMI between the vehicle and the driver, and 3) attribute recognition and line of sight recognition of the driver. At the same time, they are also making efforts to create opportunity to appeal their technologies through exhibiting them at multiple events.

	Action Interface System
Bus passenger management system that applies driver monitoring system	Face Entry System

The Unit’s start of full-scale development of DMS dates back to about 3 years ago, when Mr. Sato, Manager was invited to the Company.
“(At that time, in the Unit) we were performing sensing while doing image processing in variable ways, using algorithm of AI for studies related with automated driving.”

On the other hand, the Company owns excellent materials such as of image recognition technologies that SenseTime Group has originally developed and accumulated. Since these technologies have already been commercialized in the Chinese market, they tried to find a way to spread their use also in Japan. As they have attracted attention from their potential customers through exhibition at events etc., he came upon the idea of expanding objects of developing applications including DMS, using face authentication technology that the Company was good at.

So, they consulted with the concerned people, saying “Some tool for simulation will be necessary to develop DMS. What kind of things shall we use for what purpose?” When he was working at his previous post in the automobile industry,　he had been using UC-win/Road for simulation for sensing related with ADAS-related diverse devices for many years. He explained his experiences of reproducing real-vehicle environment, repeating evaluation experiments, leading them to development. Then it was decided to adopt a similar approach.

Following this, the Company introduced UC-win/Road in the late 2018. They started to build up a mechanism to detect phenomena such as sleepiness or looking aside by DMS while the test subject plays traveling simulation by operating DS that reflects VR of traveling environment close to the actual situation created using by UC-win/Road.

At that time, since they were also preparing for exhibition of “Automotive World” (Tokyo) to be held in Jan. in the following year of 2019, they met the demand of “being in time for the exhibition in any case” (Ms. Nishioka). It was aimed to systemize it not only to assist development but also to use for exhibition. Since then, this type of DS has been actively used for multiple events the Company participated. In addition to those for permanent exhibition in the showroom to be established in the Tokyo Office, 2 units of DS were prepared and exhibited in Automotive World in January 2020.

Exhibition in ”Automotive Engineering Exposition 2019 Nagoya”

If there had been no UC-win/Road DS, we would have to perform travel experiment using an actual vehicle. Therefore, many problems were assumed in consideration of time, manpower, and cost etc. to spend there. In this sense, the DS is not only easy to use but also easy to rewrite data and switch day and night or weather. It is also capable of reproducing a variety of experiment environments set to see the driver’s condition. Mr. Sato mentions advantages of using UC-win/Road as a simulator for development, in particular, in that UC-win/Road can be used not only as a single item but also in connection with other diverse kinds of things.

Paying attention to future function extension of UC-win/Road inconsideration of new HMI-related research and development, he has been mostly checking the sensible part of the drivers so far with DMS. He expects that in the future, using the technology of UC-win/Road, not only vehicles and roads but also the complicated interior of vehicles and even facial expressions of drivers will be represented, enabling to reflect the change in driving environments more realistically.

“(Partly because customers’ needs are greatly evolving,) it will be very favorable if it will become possible to represent diverse elements from the vehicle’s exterior to interior seamlessly.”