Pose estimation: utilising AI to improve tennis technique

Pose estimation is a computer vision task that infers the pose of a person or an object in an image or a video. We can also think of pose estimation as the problem of determining the position and orientation of a camera relative to a given person or object.

This is typically done by identifying, locating and tracking a number of key points on a given object or person. For objects, this could be corners or other significant features. For humans, these key points will normally represent major joints like an elbow or a knee. This is referred to as human pose estimation.

Humans fall into a particular category of objects that are flexible. By bending our arms or legs, key points will be in different positions relative to others. Most inanimate objects are rigid. For instance, the corners of a brick are always the same distance apart regardless of the brick’s orientation. Predicting the position of these objects is known as rigid pose estimation.

By analysing the movement and position of these key points, pose estimation can be used to understand and improve a person’s posture, balance and overall movement patterns. It has a wide range of applications, from sports analysis to medical diagnosis and rehabilitation.

Section1: How does pose estimation work?

There are two overarching approaches to pose estimation: a bottom-up approach and a top-down one.

With a bottom-up approach, the model detects every instance of a particular keypoint (e.g. all left hands) in a given image and then attempts to assemble groups of keypoints into skeletons for distinct objects. This can be time consuming and process intensive.

A top-down approach is the inverse – the network first uses an object detector to draw a box around each instance of an object and then estimates the key points within each cropped region.

In this blog post, we will break down how pose estimation can be used in tennis today and what it could look like in the future!

Section 2: How tennis players can use pose estimation

In tennis, pose estimation can be a valuable tool for both players and coaches. By using this technology, players can analyse their technique and identify areas where they need to improve.

For instance, when it comes to analysing a player’s serve technique, pose estimation can provide valuable insights into his/her body positioning and movement patterns. By tracking key points on the player’s body, like their elbow and shoulder joints, pose estimation can reveal any inconsistencies or flaws in the serve technique. For example, if a player’s toss is consistently off-centre, pose estimation can help highlight the underlying issue, such as a lack of proper shoulder positioning or elbow alignment.

Pose estimation can also be used to track a player’s movement on the court, providing a detailed and quantitative analysis of their footwork and positioning. By analysing a player’s footwork, coaches and players can identify potential weaknesses such as their inability to move laterally or quickly change direction. This can help players improve their overall movement patterns on the court, allowing them to cover more ground and hit shots with more core stability, hence increasing power and accuracy. 

Another way that pose estimation can be used in tennis is when analysing a player’s swing mechanics. By tracking key points on the player’s body during their swing, pose estimation can provide a detailed analysis of their swing mechanics, revealing any flaws in their technique. This can help players make significant adjustments, which can ultimately lead to improved shot accuracy and power.

Section 3: Pose estimation used by tennis pros

It’s all well and good being told the benefits of pose estimation but do any of the professional players use it? 

Well, in 2020, the Novak Tennis Center in Serbia began using pose estimation technology to analyse Djokovic’s serve. By tracking key points on Djokovic’s body during his serve, the technology was able to provide a detailed analysis of his technique, revealing areas where he needed to make improvements. According to reports, Djokovic was able to make improvements to his serve as a result of the analysis.

Patrick Mouratoglou, a former coach of Serena Williams, has been a vocal advocate for the use of technology in tennis training. In 2018, he launched a new training programme that uses pose estimation technology to analyse players’ movements on the court. 

Another example is the work done by Infosys, a technology services company that partnered with the ATP Tour to develop a system that captures and analyses player data. This technology, which includes pose estimation, has been used to track the performance of players such as Novak Djokovic and Roger Federer, providing valuable insights into their technique and helping them improve their game.

Section 4: Pose estimation mobile apps

Pose estimation is not just available for the professional players, it is now available from different mobile applications, making it readily available to anyone who wants to use such technology to their benefit.

There are multiple mobile applications which provide joint tracking and human pose estimation amongst their main features. A few examples are: 

• Yogger: This app provides easy to use automatic motion analysis with a mobile device or webcam. No sensors, no motion capture systems, no setup. Just click, record, and then you can start to analyse. It is available for $50 a year.

• Team Sportz: This app is more for a coach or a team which is looking to have everything in one system. It offers features such as events calendars, game attendance tracking and a video library, as well as Training Video Analysis where pose estimation can be implemented. The app is available in 3 variants, the first for free, the second for £34.80 / year and the third for £46.80  / year.

• Binary Sports: This is an application which has partnered with Premier League football clubs such as Chelsea and Everton as well as the Italian SuperTennis TV channel. The app has a number of features including: highlighting, annotating, multiple video layouts, measurement and pose estimation, to name a few. This app is a high level AI analysis app and has 3 options: Free, just under $360 / year or a one time fee of $999 for lifetime access.

• Deeptimize: This is an application that claims to extract data from your videos in the most simple, fast and automatic way possible. It is particularly useful for analysis of posture when doing things like weightlifting, ensuring you are not in a position where the chances of injury are high. It is available as a free demo on their website where you can send in your videos and get a reply within 24 hours with your pose estimation analysis completed.

• PoseAI: This is an application which is more used for entertainment purposes over sports, however, the technology used is very similar. It is the type of technology which gets used in game development as movements can be tracked and replicated in real time on a digital display. An example of this can be found on their website. It is available for free on both the App Store and Google Play.

• SportReTiNA: This is a new web app that allows users to upload their own tennis videos or use videos from other web sources and run pose estimation on them. Users can select key point angles they want to monitor and display in the output video. This can be useful for instance when analysing tennis player actions such as a serve motion. Videos can also be stored on the cloud storage space provided for later retrieval. The app is in beta and currently available for free.

Section 5: Future applications of pose estimation in tennis and other sports

Going forward, what improvements do we expect to see in the world of pose estimation and sports technology?

Something which is already being produced and developed is the ability to get real time analytics data from AI. This could be a game changer for the world of sport. For instance, as a tennis player, the technology opens the doors to the possibility of training more often and without always having to have a coach on the court with you.

This doesn’t mean that tennis coaches will become obsolete. On the contrary, their role will become more refined and professional because players will still need their help and guidance in interpreting and evaluating the data in order to make the sought progress with their game and performance.

Still taking the example of tennis, a player could have a tracker on their dominant arm in the form of a bracelet or a sweatband that tracks and uses sensor measurements in real time together with pose estimation to analyse their performance on the court. The player could have an earpiece which is connected via Bluetooth that provides audio feedback during practice telling the player what they need to focus on and what areas can be improved. 
This technology is yet to emerge in tennis. However, a brand called Carv offers a similar solution for skiing with a wearable device that clips onto your ski boots to track your sensor data in real time. It provides feedback as you are skiing and then,  at the end of your run, it tells you what you did well and how you can improve on your next run, providing actionable feedback.

Conclusion

In conclusion, pose estimation technology has the potential to revolutionise the way tennis players and coaches analyse and improve their game. 

With the ability to track key points on a player’s body, pose estimation can provide valuable insights into a player’s technique, movement patterns and swing mechanics. Furthermore, with the help of several mobile applications that have integrated pose estimation technology, it is becoming more accessible for everyone. 

In the future, real-time analysis with the help of AI could be a game-changer for sports and we may see the introduction of wearable devices that track and provide audio feedback to players during practice or a match. Overall, the possibilities of pose estimation technology and AI in sports are vast and it will be exciting to see how it will continue to develop and change the practice of tennis and many other sports.