Scoring Gymnastics Competitions More Accurately with 3D Sensing Technology--What Is the Future of Judging?

Increasingly Sophisticated Gymnastics Techniques Make Judging More Difficult Every Year

Japan captured the men's Olympic team and individual all-around gymnastic gold medals. The nation once again showcased its prowess to the world, and our excitement has yet to subside.

It is said that gymnastics was first introduced to Japan in 1830 by feudal domains (called "Han") to train recruits, after which it was adopted into school education. Since the All Japan Gymnastics Association (the present Japan Gymnastics Association, a public interest incorporated foundation) was established in 1930, gymnastics athletes have continued to participate and win medals in individual events and individual all-round competitions.

In gymnastics, a sport that awards points through a judging system, considerable efforts and reforms have been made over many years to ensure consistently fair and accurate scoring. However, with the recent remarkable advances in gymnastic techniques, there are times when it is difficult to accurately judge athletes with the naked eye. As a result, judges face an escalating burden to make accurate split-second scoring decisions.

Motion capture technology* is used to analyze human movements. In the technology, there is a well-known method that uses multiple cameras and markers; however, it is virtually impossible to employ in competitions and everyday practice because it requires markers to be placed on athletes' bodies and a complicated camera configuration.

* Technology used in the fields of sports and movie production for recording human body movements as digital data.

Bringing New Innovation to Sports through "Sports and ICT"

On May 17, 2016, Fujitsu, Fujitsu Laboratories, and the Japan Gymnastics Association announced the start of joint research on a scoring support technology for gymnastics competitions.

This research will combine the Japan Gymnastics Association's expertise in recognizing proper gymnastics techniques, in scoring, and in competitions with Fujitsu Laboratories' 3D laser sensors and 3D data processing technology in order to develop a technology that supports fairer, more accurate, real-time scoring of gymnastics competitions.

Measuring movement

Specifying Techniques in Real Time with High Accuracy by 3D Sensing

The details of the technology are as follows.

Point 1: Accurately measure human movements with 3D laser sensors
Distance is measured by emitting approximately 2.3 million laser beams per second toward an object and measuring the time until the beams return. A solid object that is moving can be accurately identified by emitting lasers while finely changing the angle. By expanding or narrowing down the range of laser emission depending on the distance to the object, the distance can be measured with a constant resolution even if the distance changes. This measuring technology employing lasers, which does not require athletes to wear sensors, can be used in everyday practice as well as actual competitions.

Point 2: Specify techniques based on 3D laser sensor information
Once body movements have been accurately measured, the person's joint positions and their curving conditions are derived from the data to specify the techniques displayed. This technology combines two technologies: the model method (high speed, low accuracy), in which the positions of the hands and feet are estimated by searching for similar data from a data dictionary of human movements, and the fitting method (low speed, high accuracy), in which positions are fitted to actual body movements. The technology accurately identifies the temporal movements of joints by capturing them at high speed with high accuracy and then specifies the techniques displayed by judging the positions of the hands and feet as well as the number of twists of the body. This enables real-time, high-accuracy scoring.

Developer's voice
Akihiko Yabuki, Life Innovation Laboratories, Applied Innovation Research Center, Fujitsu Laboratories Ltd.

I'm working on a technology for recognizing "techniques" based on joint position information. We combined a 3D laser sensor technology that was originally developed for application to vehicle peripheral monitoring devices with skeleton recognition technology that was developed for rehabilitation purposes in order to develop this human movement sensing technology for application to sports. In particular, we focus on gymnastics, which is a challenging target because it involves many basic factors of body movements.

We are now developing a unique algorithm to accurately recognize a gymnast's quick, complicated, and smooth movements from changes in joint positions and to judge whether the gymnast performed techniques successfully. For example, techniques such as "flop" and "combine" of the pommel horse are in fact composed of multiple techniques. Therefore, the completion requirements for each technique must be accurately captured. To this end, with the cooperation of the Japan Gymnastics Association, while listening to lectures from judges about scoring and reading the scoring rulebook carefully, we have repeatedly discussed as a team how to utilize ICT to do scoring. We think that we must offer evidence of the reliability of this technology for it to be employed to support scoring in gymnastics and other competitions. In this respect, we are highly motivated and feel great responsibility.

We think that real-time measurement technology refined in this specific sports environment can also be applied to various other fields. We aim to use this technology not only in sports but elsewhere in the not-so-distant future, such as in traditional performing arts, rehabilitation, and evaluating the movements of skilled workers.