Analyzing Disaster Big Data to Support Disaster Prevention with Timely and Accurate Forecasts

As a result of rapid climate change, natural disasters have been increasing in intensity in recent years. There is an increase in the annual number of rainfall events with precipitation exceeding 80 mm per hour.(*1) Against this background, more advanced technologies are needed to cope with natural disasters that may lead to serious damage, such as floods and landslides caused by heavy rain and local torrential downpours.

Government agencies in charge of disaster prevention strive to issue evacuation recommendations in a timely manner based on accurate disaster information. However, they often face a lack of information required to make decisions, especially during the early stages of a disaster. In order to resolve dilemmas like this, Fujitsu is researching and developing disaster prevention solutions using ICT.

Fujitsu's Technology for Analyzing Disaster Big Data

To prevent damage from floods and landslides, it is essential to gather information from many water level sensors over wide areas. However, there are limits to the extent of areas that can be covered by sensors. Furthermore, due to rapid housing development, houses are now being built in areas without sensors, making it all the more difficult to gather relevant information.

The technology for analyzing disaster big data developed by Fujitsu enables one-dimensional sensor information to be expanded into two-dimensional information through simulations using big data on past rainfall during floods along with topographical information. The technology predicts and estimates conditions in areas where sensors are not installed, thereby providing an advanced, next-generation disaster prevention solution.

1. Technology for Estimating the Occurrence of Disasters from Social Media Data

The large number of comments (tweets) on social media sites includes comments about disasters. Such comments are frequently used?especially in recent years?in news reports as on-site information during disasters. However, comments on social network sites include rumors and hearsay. As a result, it has been difficult for personnel in charge of disaster prevention to sort out reliable information and use it to make decisions.

To overcome such difficulties, Fujitsu has developed a technology for estimating the occurrence of disasters from social network data by applying the technology for analyzing big data used in marketing and other areas. The new technology not only sorts out direct sighting information from miscellaneous social network data on disasters, but also estimates where comments are made and predicts the occurrence of disasters by identifying changes in the number of relevant comments.

More specifically, the technology uses a natural language processing technique to gather comments that include key words related to disasters, such as "flooding" and "inundation" (Figure 1-1: Disaster Information Collection). We repeatedly reviewed key words to create a system that can gather disaster data efficiently by using a limited number of key words. By using a hearsay elimination technique based on a probability model and machine learning, it is eliminated from the comments collected by categorizing them into information based on sightings and observation, direct hearsay information, and indirect hearsay information (Figure 1-2: Hearsay Information Elimination). Since there are very few comments that include GPS information (less than 0.5%, according to our research), we analyze comments about train stations, crossroads, landmarks and other elements in order to estimate the specific location of disaster occurrence (Figure 1-3: Location Estimation). Finally, based on information on increases in the number of comments within a limited time or space, we detect abnormal events to estimate the occurrence of disasters (Figure 1-4: Disaster Occurrence Estimation) and display our findings graphically on a map.

Figure 1: System for Estimating the Occurrence of Disasters

Figure 2: System for Disaster Estimation from Social Media Data (Screen Image)

Fujitsu's technology visualizes what disasters are occurring when and in which municipalities. It enables judgments to be made as to whether they are occurring in your own community now or in a neighboring community and whether it is necessary to take immediate action, thereby supporting disaster management.

This model was tested by applying it to real social media data during a flood that occurred in the Kansai Region on August 2012. The test results show that it was possible to detect the occurrence of disasters with a probability of 80% (Approx.).

2. Mathematical Optimization for Simulating Floods

Flood forecasting simulation technology developed by the Public Works Research Institute (PWRI) predicts changes in the amount of river flow during rainfall. In flood forecasting simulation, we divide an area into cross-sectional 500-by-500 square meters, for example, and use a distributed runoff model developed to study phenomena in which rainwater infiltrates soil and discharges into a river. However, it requires a high level of professional skills and expertise in river engineering and hydrology as well as enormous amounts of work to set each of the simulation parameters correctly. In addition, appropriate tuning of the parameters is necessary to provide accurate forecasts. For these reasons, using this technology is very burdensome.

Fujitsu carried out joint research with the PWRI and developed a technology that automatically adjusts and optimizes parameters to minimize errors between simulated discharges and measured discharges by applying mathematical optimization algorithms to a flood forecasting simulator. This technology can be used even by inexperienced engineers without having to make adjustments that require expert knowledge.

Mathematical optimization is a method for obtaining the best combination of parameters from among many parameters with a small number of calculations. This method has a wide range of applications and is used in designing semiconductors as well as improving engine combustion efficiency.

However, to find the best combination of parameters with a small number of calculations, it is essential to use an optimization algorithm that best fits the simulation model. To this end, we assessed a total of 75 optimization algorithms, including genetic optimization, particle swarm optimization, differential evolution optimization and simulated annealing, thereby selecting 13 optimization algorithms suitable for the distributed runoff model used in flood forecasting simulation. At the same time, we are also developing a mathematical optimization platform that automatically selects parameters.

Figure 3: Mathematical Optimization

Graph 1: Rainfall and Comparison between Actual and Simulated Discharge

This newly developed technology makes it possible to manage water flows in entire basins of first-class rivers managed by the Ministry of Land, Infrastructure, Transport and Tourism Japan, by using existing sensors. Used in combination with compact sensors, it also helps local governments manage rivers sufficiently and improve disaster management.

Using ICT to Realize a Safe and Secure Society

Fujitsu has been engaged in developing a number of disaster prevention solutions through the use of ICT.

For example, the most effective way to detect sewer overflows in early stages is to install sensors in manholes. However, it is difficult to install sensors over a wide area because of the costs involved. To overcome this challenge, Fujitsu developed technology that analyzes data on water flow time to determine the location and number of manholes to install sensors in. This technology makes it possible to analyze and predict the water flow in an entire sewer system with only one-fifth of the number of sensors previously thought to be used. In addition, the technology optimally controls measurement parameters in accordance with changes in the water level, thereby reducing power consumption by a maximum of 70% and contributing to minimizing operational costs.(*2)

The Great East Japan Earthquake highlighted the need to predict not only the height of tsunami waves but also the areas of inundation. To meet this need, Fujitsu developed a high-resolution tsunami model, along with technology for predicting tsunami inundation patterns in real time by using the K Computer's high-efficiency parallel computing capability. During the Great East Japan Earthquake, tsunami waves began inundating Sendai one hour after the earthquake occurred. This technology makes it possible to estimate the area of inundation in approximately 10 minutes.(*3)

The use of technology for analyzing disaster big data enables administrators in charge of disaster prevention to obtain previously unavailable disaster information about when, where and how disaster damage spreads and to take protective measures against the damage quickly in the initial stages. In the future, this technology may also be used to develop services for providing disaster prevention information that best suits the needs of each individual.

Fujitsu will continue developing ICT solutions for disaster prevention in order to realize a society where people can live safe and secure lives.