AI to Raise the Bar for Competitive Sports―Elevating the Skills of Athletes, Coaches,and Judges

ICT usage in sports has entered a new stage. Until now, focus has been placed on "digitalizing sports" by using sensors and video processing; obtained data was analyzed manually. In a new trend, AI can perform data analysis for people. This article explores the cutting edge of "ICT x Sports" ("ICT meets sports").

Digitalization of Athletes' Movements Fundamental to "ICT x Sports"

With or without AI, the basis for ICT usage in sports remains the same: digitalizing athletes' movements and/or overall team play. The spread of digitalization has enabled us to accumulate a massive amount of diverse types of data, and this data is analyzed from various perspectives to improve play accuracy and game plans.

Before looking at AI case examples, let's examine some concrete examples of digitalization using ICT.

Improving Player Skills with ICT

One familiar example is to use ICT to help players improve their skills. Digitalizing player movements with sensors and image processing enables more accurate, intuitive grasping of movements, making it easier to find what needs to be corrected. A typical example is checking the form of a golf swing. Although you can objectively spot bad habits by videotaping your golf swing, you can achieve a more accurate analysis of body movements by attaching sensors and viewing processed video, which helps coaches and more experienced players point out problems that need to be corrected. Moreover, after practicing one's swing to correct the problems, the digitized data can be used again to recheck the form.

Sensing-based Sports Form Digitalization (Japanese)
Sensing Technology for Analyzing Golf Swings

In addition to improving one's form, some athletes have started to digitize the entire competition to acquire winning skills. For example, in summer 2017, the Japanese Windsurfing Association, together with Fujitsu and Lapis Semiconductor, conducted IoT-based field trials to improve windsurfers' sailing skills. In these field trials, a device developed by Lapis Semiconductor that can simultaneously record GPS and sensor data was attached to windsurfing sails. The collected data was analyzed using Fujitsu's cloud service in order to visualize sail movement through 3D models and graphs. Using such data to visualize sail handling, surfers can compare their data with that of higher ranked competitors and see the differences in 3D models or numeric values in order to find ways to improve their own sailing skills.

Windsurfing Lab: IoT-based Training System for Improving Sailing Skills (Japanese)
IoT-based Training System for Improving Sailing Skills

Analyzing the Opponent's Game Plan

In professional sports in which an athlete(s) plays directly against an opponent or opposing team, digitized game data can be used as basic data to analyze said opponent's game plan. In sports such as tennis, volleyball, and soccer, identifying the opponent's game plan and creating one's own plan accordingly is as important to winning the game as improving one's play. Analyzing the opponent's game data before a match and preparing plays or a game plan to neutralize the opponent's moves is common even in amateur sports.

Here is a more advanced use case. A team collects real-time playing data during a match, and coaches analyze the match using this data, come up with a more effective game plan, and tell the plan to players during the match. For example, the Women's Tennis Association, a sports association for female pro tennis players, has introduced an on-court coaching rule, which allows a coach to enter the court between games or sets during a match to direct the player. A coach can explain to a player why she is losing while showing her game data on a portable device and modify her game plan. Since both players now flexibly change their game plans, it is said that the number of close matches has increased, making tennis a more interesting sporting event.

Managing Players' Conditions

In professional team sports, more and more teams are using digitized data to manage players' physical conditions. Since the performance of highly paid players greatly affects team performance, protecting players from injury and shortening their out-of-action time is the top priority.

For example, the players of Leicester City Football Club, an English professional soccer team that competes in the Premier League, put on GPS wearable devices with embedded sensors such as GPS and an accelerometer in order to accurately measure the loads on players during matches. The wearable records the total distance run, the distance run at top speed, acceleration, deceleration, and so forth for each player during the match. By checking the data against the actual injuries that occur, the team can understand the load on each player and the correlations between different types of movement and injuries in detail. Since this analysis reveals injury-prone situations for individual players, the team can minimize injuries by closely managing each player's physical condition and bench him if he becomes fatigued. In fact, in Leicester City FC's 2015-2016 championship season, the team recorded the fewest number of injuries among all Premier League teams.

AI Judging for Humans

In which areas do we anticipate "AI x Sports" ("AI meets sports")?

One area is in tagging collected data. Using ICT to digitize data enables users to accurately record and replay both player and ball movements. However, game plan analysis requires preprocessing of data that identifies the purpose of each play and how formations change, and such data must be divided into groups accordingly; this task is called tagging. Currently, tagging is done one-by-one by human analysts well-versed in the sport while they watch the match. If AI can take over tagging work, analysts can spend more time conducting analysis. Since image processing is AI's forte, AI can be expected to perform tagging with a certain level of accuracy if users prepare many classification examples for AI to learn from.

Team Sports: Basketball

One practical example is Fujitsu's women's basketball team RedWave. AI is used to improve the accuracy of motion tracking technology, which automatically tracks the players and ball by image processing and digitizes their plays. RedWave has installed eight cameras in its home court in Kawasaki City. The team introduced a system to recognize and record player movements, formations, and shots during games.
Fujitsu is improving play recognition accuracy through machine learning based on massive numbers of player and ball images so that even if players overlap during key offensive or defensive plays in the area near the basket, the system can correctly recognize the players. Development is underway to realize automatic tagging of play images and events such as shots made or missed, players' moves, and passing the ball.
The motion tracking data thus obtained will enable scientific, data-based coaching and facilitate replaying video on the court in real time, rather than traditional coaching based on the results of analysis that coaches have conducted after games or practice. The system helps players improve their skills. For example, it can be used for tactical analysis, such as identifying offensive patterns with a high success rate, how to stop opponents from scoring, and checking tracking data for each player's position.
Moreover, recorded video can also be used to entertain fans, such as zooming in on and following their favorite players. After we commercialize a system for basketball, we plan to apply this motion tracking technology to a wider range of sports, such as volleyball and handball.

Fujitsu's Motion Tracking Technology Assists Basketball Players Making Better Plays (Japanese)
The system identifies players, their jersey numbers, and the ball based on the difference between the background and moving objects from video shot with eight cameras. Every step of the process employs AI.

Individual Sports: Gymnastics

To use ICT to maximize judges' capabilities, we are developing a system to support judges to determine score when evaluating individual performances in competition. Jointly developed with the Japan Gymnastics Association, this system numerically expresses "a technique's degree of perfection (execution)."

Currently, judges visually confirms the composition of successive elements during the performance and fill out a score sheet by hand at the same time. The scores are then tallied afterwards. However, since gymnastics is evolving rapidly and gymnasts execute very difficult movement very quickly, even seasoned judges have a hard time making evaluations instantly.

Combining the judges' evaluation techniques, provided by the Japan Gymnastics Association, with Fujitsu's proprietary 3D sensing technology, we are developing a system to support judges to achieve fairer, and more objective scoring. More specifically, Fujitsu's original 3D sensing technology converts gymnasts' performances into numerically identifiable data in real time. This data is then matched against a "dictionary of executed elements (in performance)" (movement database), which the Japan Gymnastics Association and Fujitsu are co-developing, in order to instantly confirm the elements of competitive performance. At present, we are using AI (machine learning) to develop this dictionary. Digitalizing the complex movements of gymnastics requires a massive amount of computing resources. Therefore, we use the Fujitsu Cloud Service K5 cloud platform to run parallel and distributed processing (Hadoop) to secure the necessary development speed.
In future, judges can deliver more convincing and impartial evaluations with the objective and numerical data, supported by Fujitsu's system. Furthermore, offering spectators and TV viewers new information (content) based on such athletes' movement data will help the novice audience better understand gymnasts' performances and how great they truly are.

Fujitsu's 3D Sensing Opens a New World of Sports (Japanese)
3D Sensing Technology Supports Gymnastics Scoring

The New Possibilities AI Brings to Sports

Exploration of "AI x Sports" has only just begun. We expect to see various new systems arise in the future. AI's strength is its capability to learn a massive amount of knowledge and experience from various experts. By letting AI learn great plays and calls made by legendary players and coaches, we may be able to have a "personal master coach" to advise us whenever necessary. Moreover, AI may be able to solve the often-discussed issue of depending on humans in artistic merit scores. If AI learns large quantities of data about "performances that impressed many people," it may be able to give scores that satisfy athletes, coaches, and spectators.

In the future, playing sports may become even more enjoyable via an application provided over the Internet that gives you advice from your favorite players or coaches.