A Future where AI and Robots Coexist in Harmony with Humans

[Fujitsu Forum 2016 Exhibition Report]

Fujitsu Forum 2016 was held on May 19 and 20 at the Tokyo International Forum. In the exhibition hall, we introduced artificial intelligence (AI) and robots that utilize state-of-the-art technology—which have gained much attention recently—along with case studies, allowing visitors to gain hands-on experience.

Fujitsu's mediator robot Robopin showed great hospitality to each visitor. Robopin can express a variety of emotions by combining movements using its uniquely structured six joints, such as a banzai pose where both hands are raised in the air, with changes in the colors of the LED lights (blue, green, red, pink, etc.) surrounding a monocular camera.

Robots Welcome Visitors with Hospitality—Creating Fun Learning Experiences

Robots connected with a cloud-based AI can recommend services tailored to individual needs by connecting the individual and ICT in a natural way—this new way of providing services is what Fujitsu calls the ROBOT FUTURE VISION, in which robots coexist in harmony with humans.

For example, the mediator robot Robopin served visitors at the exhibition hall of Fujitsu Forum 2016. Beacons were distributed to the visitors at the entrance, allowing Robopin to extract data from them in order to identify the visitors. Robopin then selected and recommended exhibits and demonstrations tailored to the visitors' individual needs based on the profile information that they registered in the cloud in advance, such as their names, jobs, and topics of interest, as well as the data on their past activity transmitted from the beacons. In this way, Robopin was able to provide guidance to visitors with its charming voice and movements from various corners of the hall.

Exhibitions Using AI and Zinrai—Fujitsu's AI Technology Cultivated for Over 30 Years

One of the important technologies that support ROBOT FUTURE VISION is Fujitsu's Human Centric AI Zinrai, announced in November 2015. The term Zinrai is derived from the Japanese proverb shippu-jinrai, which means "as fast as lightning." True to its name, Zinrai is as fast as lightning when it comes to supporting the decisions and actions of its human operators. Zinrai, which is the culmination of AI technologies developed over 30 years by Fujitsu Laboratories and other FUJITSU organizations, has been incorporated into a range of Fujitsu products and services.

Based on the AI technology of Zinrai, Fujitsu is providing consulting services that utilize AI, making it possible for AI researchers and curators to work with customers in creating the optimal AI application usage.

In Fujitsu Forum, we showcased many examples of AI application in a variety of fields that Fujitsu has worked on together with our customers.

Zinrai features four core technologies: sensing recognition, knowledge, processing decision and support, and the advanced learning system that supports these three technologies.

Time-series Data Deep Learning: New Technology that Enables the Analysis of IoT and Sensor Data

Time-series Data Deep Learning technology analyzes data with a high degree of accuracy

Deep learning technology (*1), a core part of Human Centric AI Zinrai, is an advanced machine learning method. It has so far been difficult to analyze data other than images and sounds. In particular, it has been very difficult to accurately classify highly volatile data (time-series data) taken from gyroscopes (angular velocity sensors) installed in wearable devices and smartphones.

To resolve this issue, Fujitsu Laboratories has developed a deep learning technology that can analyze time-series data using the chaos theory and topological data analysis. (*2)
The complex waveform data captured from gyroscopic sensors has about 25% higher accuracy over existing techniques, making it possible to identify subtle differences, such as whether a person is running or walking at a fast pace. This technology can be used to detect equipment anomalies or forecast breakdowns from the vibration of equipment.

This technology also classifies complex human brainwaves with a high degree of accuracy, allowing us to determine whether a person is sleeping or awake. When combined with a mechanism for detecting drivers' sleepiness, this technology can also be used to prevent drowsy driving and accidents by long-distance bus drivers.

(*1) A type of machine learning using a multi-layered neural-network model.
(*2) A data analysis technique that treats data as a set of points arrayed within a certain space, from which geometric information can be extracted.

AI Helping Users Make Travel Arrangements and Recommending Sightseeing Spots with Natural Dialogues

AI that understands the user's speech correctly and can converse naturally as if it was an actual person

Fujitsu also showcased a technology that can carry natural conversations using AI.
In the exhibition hall, a demonstration of a customer service at a travel agency was conducted. When the user entered information into a tablet, the computer automatically started a conversation. For example, the computer will ask "Where would you like to visit?" If the user enters in response, "I wanted to go to Rome, but I have changed my mind and decided to go to Naples," then the computer will reply, "Naples is famous for Stadio San Paolo," or "Who are you traveling with?" and continue the conversation naturally as if it was a human.

This conversation is easy for humans, but it is quite difficult for a computer to interpret the meaning of a sentence that contains both 'desire' (I wanted to go to Rome) and 'intention' (decided to go to Naples) at the same time.

To address this problem, Fujitsu Laboratories developed a new technology that can structurally extract the relationship between the meanings of words, using the analysis technology from its machine translation engine that contains a large dictionary with millions of words. This made it possible for the AI to have a highly accurate understanding of the user's speech and carry a normal conversation. Unlike conventional systems, where the AI would have conversations based on typical scenarios that were registered in advance, this technology creates friendly conversations and makes users feel like they are chatting with a real person. Zinrai reads the conversation history data and learns from it, resulting in more natural-sounding conversations.

In the field test that Fujitsu Laboratories conducted for Tokio Marine & Nichido Fire Insurance Co., Ltd., we confirmed that the technology was able to give the correct responses and have a natural conversation. Fujitsu Laboratories aims to incorporate this technology into Fujitsu Limited's solutions within fiscal year 2016.

Expectations on the Application of AI Technology in a Variety of Fields

Fujitsu also demonstrated various use cases where Zinrai is used in combination with affective media technology, such as bank ATMs, mirrors in changing rooms, and touch screens for taking orders at burger restaurants. For example, our gaze tracking technology determines what a user is looking at on the ATM display. If the user continues to look at the same thing for a period of time, it will conclude that the user is having difficulty reading it and provide assistance by enlarging that section. It is also able to detect the customer's stress level from their telephone conversation with a bank clerk. Moreover, it aims to prevent from suffering phone fraud damage where the user is tricked into transferring money by detecting certain keywords that indicate a possible phone fraud and notifying the caution of fraud to related people for support.

Use cases of affective media technology

AI and ATMs (assisting ATM operations by analyzing user's gaze; detecting and preventing phone fraud using the voice data of telephone conversations)
AI and ticket vending machines (displaying recommended menu items on the machine by analyzing the attribute information and user's gaze, the climate, etc.)

AI and mirrors (displaying recommended outfit coordination on the mirror by analyzing the customer's purchase history and user's gaze, as well as inventory information)

Fujitsu also presented various applications of AI technology, including the Intelligent Dashboard that supports factory management while interacting with Anna (an AI agent), the Mobility Security Services that incorporates AI learning for automated and safe driving, and the application of AI technology for advanced cyber-attacks.

AI and Intelligent Dashboard (detecting problems in factories and offering solutions using natural conversations by analyzing past information)
AI and mobility (sensing and analyzing the surrounding environment of a vehicle and people and warning the driver of any dangers to ensure a safe, secure and comfortable journey)

Fujitsu will promote the research, development, and commercial use of robots and AI, aiming to further enrich people's lives.