The HAWKEYE is one of the most commonly used technologies in the game of cricket today. It has been put to a variety of uses, such as providing a way to collect interesting statistics, generate very suggestive visual representations of the game play and even helping viewers to better understand the umpiring decisions, especially in the case of LBWs. While the system provides for things which we see every day on television, there is very impressive technology going into it, which many of us are oblivious to.
The game of cricket has attained great commercial importance and Popularity over the past few years. As result, there has been felt a need to make the game more interesting for the spectators and also to try and make it affair as possible. The component of human error in making judgments of crucial decisions often turns out to be decisive. It is not uncommon to see matches turning from being interesting to being one sided due to a couple of bad umpiring decisions. There is thus a need to bring in technology to try and minimize the chances of human error in such decision making.Teams across the world are becoming more and more professional with the way they play the game. Teams now have official strategists and technical support staff which help players to study their past games and improve. Devising strategies against opponent teams or specific players is also Very common in modern day cricket. All this has become possible due to the advent of technology. Technological Developments have been harnessed to collect various data very precisely and use it for various purposes.
The HAWKEYE is one such technology which is considered to be really top notch in cricket. The basic idea into monitor the trajectory of the cricket ball during the entire duration of play. This data is then processed to produce life Like visualizations showing the paths which the ball took. Such data has been used for various purposes, popular uses including the LBW decision making software and colorful wagon wheels showing various statistics. This paper attempts to explain the intricate details of the technology which goes behind the HAWKEYE. We first start off with a general overview of the system and an outline of the challenges that we might face, then move on to the details of the technology and end with various applications where one sees this technology being put to use.
Cricket is a ball game played within a predetermined area. A system comprising of video cameras mounted at specific angles can be used to take pictures. These pictures are then used to locate the position of the ball. The images are then put together and superimposed on a predetermined model to form a complete visualization of the trajectory of the ball. The model includes, in this case, the pitch, the field, the batsmen and fielders etc. For this to be possible, we need to sample images at a very high rate and thus need efficient algorithms which can process data in real time. Such technologies are widely used today in various sports such as Tennis, Billiards which also fall in the category of ball game splayed within a restricted area. Our discussion will mostly contain applications which specific to the game of cricket, however in some cases, we will mention how similar techniques are applied in other games. There are various issues which crop up when one tries to design and implement such a system. In the game of cricket, the general issues are:
- The distance at which the cameras see the pitch and the ball are dependent on the dimensions of each ground and can vary greatly.
- Just the individual images don‟t help too much; for the system to be of practical use, one must ensure that it can track the 3D trajectory of the ball with high precision. In order to get this accuracy, the field of view of each camera should be restricted to a small region – this means one needs more cameras to get the coverage of the entire field.
- Fielders and spectators might obstruct the camera‟s view of the ball and the ball might get „lost‟ in its flight in one or more of the cameras. The system should be robust enough to handle this, possibly by providing some Redundancy.
- The ball might get confused with other similar objects for instance, with flying birds or the shadow of the ball itself. The image processing techniques used need to take care of these issues. Luckily, there are techniques which are easy to implement and are well known to the Image Processing community on the whole, to take care of these.
- To help in judging LBW calls, the system needs to beamed aware of the style of the batsman – whether he is right or left handed. This is because the rules of Blare dependent on the position of the stumps and are not symmetrical about the middle stump. Thus, the system needs to detect whether a particular ball has pitched outside the leg stump of a batsman or not.
- To determine the points at which the ball makes contact with the pitch, the batsmen or other objects is very hard. This is because we don‟t really know these spots beforehand and the model and the real pictures taken by cameras need to be merged to give such a view. We will see how the HAWKEYE technology successfully treats each of these issues and provides robust system to be used in practice. The top-level schematic picture of the system and its various parts is as shown below (each color represents a block of steps which are related):
The figure above shows precisely the steps that are involved in the computation. The process is started with some calibration of the cameras. This is required to deal with the problem raised in 1 above, about the non-uniform distance of the cameras from the playing area. After this basic calibration is done and the system is up and running, we can start processing the video input which we get from the cameras. In each of the images obtained, the first aim is to find the ball in it. Once this is done, a geometric algorithm is used to look at multiple images (which are 2D) and then combine them cleverly to get the co-ordinates of the ball in 3D space. This process is now repeated for multiple times every second (typically at the rate of 100 times per second). Thus, we have the position of the ball in 3D space at many moments in every second. The final step is to process these multiple
Positions and find a suitable fitting curve which best describes the flight of the ball. As we have sampled the positions of the ball at very short time intervals, the flight of the ball can be very accurately determined.
A description of the exact algorithms involved in the entire process will be skipped here. We instead try to give an intuitive description of each step in great detail, so as to give the reader a feel of what goes into the system, without plunging into the gory details.
Typically, for a cricket field, 6 cameras are used. As one can see, the 6 cameras in use are positioned at roughly 60 degree from each other. They are placed high in the stands, so that there is lesser chance of their view being blocked by the fielders. There are two cameras, one each looking at the wickets directly in sideways fashion. These 6 cameras are calibrated according to the distance they are at from the pitch. In order to get good accuracy, one needs to restrict the view of each camera to a smaller region. This means each camera image would show a more prominent picture of the ball and hence the ball will be located more accurately. However, we also need to keep in mind that the whole field of play has to be covered by just the 6 cameras which are available. This puts some limitation on how restricted the view of a camera can be. Nevertheless, the accuracy obtained by using 6 cameras is acceptable to the standards prevalent today.
Some further setting up is essential for the system to work correctly. The cameras need to be fixed to some frame of reference, which is defined very conveniently in terms of the wickets on the pitch, and the line joining them. This is useful when we want to use an automated program to merge images from different cameras to form one 3D image.
Also, to avoid unnecessary computation and make the system more efficient, the cameras can be operated in active or passive mode. In the passive mode, no imaging is done and hence the system is more or less completely inactive. The cameras can be triggered into active mode either by detecting some motion in the vicinity of the pitch, or manually by some external trigger. In either case, all the cameras are synchronized and go into active mode simultaneously. The cameras are then designed to stay in the active mode for a fixed time before going off into passive mode. This action of going into passive mode can be manually overridden in exceptional cases. The different modes for the cameras are especially effective for a game like cricket as the game involves significant pauses between phases of actual play.
As described in 5 in the list of issues, the system needs to know if the batsman is right or left handed. The front view cameras are used to do this. This information, as previously said is useful in making LBW decisions and formulating other statistics. For instance, we commonly see the analysis of a bowler‟s pitching areas done separately for a left and aright handed batsman. While this is not a very difficult task to do manually every time the batsman on strike changes, the system does provide some way of automating it.
Once this setting is done, the cameras are ready to take pictures in their field of view and have them sent to a computer which processes them.
Preparation before starting to process:
Additional features might be loaded into the system to enable it to process the data in a more reliable and useful manner. These might include a statistical generator, which is used to produce statistics based on the data collected. These are the statistics which we see on television during and after the match for analysis. Such statistics can also be used by teams and players to study their game and devise strategies against their opponents. Indeed, the raw data about the paths of the ball might be too much for any human to digest and such statistics turn out to be easier to hand lead understand. The statistics generator might also aid in storing data such as the average velocity of the ball. This data is crucial as it can help the ball detection algorithm to predict the rough location of the ball in an image given the position in the previous image. Such considerations are useful to reduce the computations involved in the processing of the data collected from the video cameras. Once such additional machinery is setup correctly, we are all set to start collecting data and start processing it to churn out tangible statistics and visualizations. It might be noted at this stage that there is some more information which might be required to process the data correctly. We will point out such things at later points in the paper, where it fits in more appropriately.
Core Image Processing Job:
This part of the system can be further divided into 3 major parts:
- Identifying pixels representing the ball in each image.
- Applying some geometric algorithm on the set of
Images at each instant.
- Coming up with the 3D position of the ball in space.
Putting frames at various times together:
Now we have the exact position of the ball in 3D space at given instant of time. Next, what needs to be done is putting together this data, collected at various time instants into single picture which shows us the trajectory of the ball. Wean split this part of the process into two parts. Again, the reader should understand that these parts are very much related and we split them here in our explanation just to make it easily understandable. The two parts to this computation are:
- Tracking the ball at various instants.
- Predicting the flight or trajectory of the ball.
APPLICATION OF HAWKEYE
HAWKEYE has had far-reaching consequences in many sports. Primarily in cricket, HAWKEYE is a process that makes the current judgmental call on a LBW decision, very predictive. While no technology is flawless and HAWKEYE has its own share of these, it is up to 99.9% accurate. This has made the LBW decision, a predictive one. More importantly, such technology can be used to evaluate the skills of the umpire as well. The England Cricket Board (ECB) has already set-up the HAWKEYE system not only at about 10cricket venues around the country but also in the training academy to aid umpires, as well.
While the Hawk-Eye has made its mark and derives its appeal from the ability to predict the flight of the delivery, it‟s a very useful tool for collecting statistics. The information associated with each delivery bowled is routinely processed, even when the outcome of the delivery is not doubted. As result, the strategy used by a bowler as a function of bowling spells, delivery no. in the over, batsman facing the delivery and so on can be gauged. Similarly, the scoring patterns of batsman around the ground using wagon-wheels are routine in match day telecasts. These are so cleverly generated that they give a real-life feel to it. Commentators also are able to move them about to make a finer point, about a batsman. However appealing and nice that it may seem, a keen cricketing eye will notice that the wagon wheel is less accurate than the other data. This is because the wagon-wheelies generated from data collected from outside the predetermined pitch area. The location, depth, trajectory of the ball in-flight at an arbitrary point on the ground is more difficult to determine, than when it is on the pitch. As a result, some errors manifest. These difficulties aren‟t faced in tennis – where HAWKEYE is used to decide whether the ball, was within the court limits or not. In the case of Tennis, lines calls made by HAWKEYE are completely accurate. Tennis has been quick to adapt to this technology and HAWKEYE arbitrations are legal since the NASDAQ-100 tennis tournament. Players can challenge line calls, following which HAWKEYE determines whether the ball was pitched in or out! The recently concluded US Open QF match between Feeder and Madelyn had a match-point being decided after a line-call challenge. We now briefly look at the various applications of HAWKEYE which the cricket broadcasters regularly use these days.
As mentioned previously, the HAWKEYE can accurately capture the trajectory of the ball and also predict the future direction of the ball using mathematical calculations. This is put to use in deciding whether a batsman was OUT LBW on a particular ball. Thus, the system determines the exact point at which the ball struck the batsman. Using the trajectory of the ball up to that point, the system predicts the path the ball would have taken had the Batsman not been present in the way. Thus one can know the lateral position of the ball with respect to the stumps as wells the height of the ball at the point when it reaches the line of the stumps. The figure below gives an example of the trajectory of the ball being predicted. Note that in this picture, the system has got rid of the batsman from the picture so as to give us a complete view of the path of the ball since it left the bowler‟s hand. This is exactly what one needs to decide if the ball would have hit the stumps and if that is the case, the batsman has a chance of being given OUT LBW.
The system is well equipped to handle the various complex clauses which the LBW rule has. For instance, it can check if the ball had pitched outside the leg stump of the batsman. If this is the case, the batsman is NOT OUT even if the ball is going on to the stumps. Recall that the front view Cameras are used to determine whether a batsman is right or left handed. That information is useful here. Another clause states that the batsman should not be given OUT if he is hit outside the line of off-stump and is attempting to play a shot. Now, the part of whether the batsman is playing a shot has to be decided manually and the system is not capable of doing it. However, the point of impact is accurately known and one can see exactly where the batsman was hit.
The kind of accuracy which HAWKEYE offers is difficult to get for any human umpire. The system also includes a way to do probabilistic analysis and hence bring in the factor of “benefit of doubt” which goes to batsman currently. The main idea behind this is to have a region which the human umpire would believe the ball would have been in this region is just taken to be a circle centered at the accurate position of the ball and radius .The value of this radius is calculated taking into consideration the distance between the Point of impact of the ball with the batsman and the stumps –that is, the distance which the ball is yet to cover. This models quite accurately the uncertainty which the umpires feel while making the decision manually. Thus, if the batsman is playing forward, the radius will have a higher value, than when the batsman is struck, playing back. To keep the “benefit of doubt” still with the batsman, the decision goes in favor of the bowler only if a significant portion of the probable region (circle or radius described above) lies in line with the stumps. The system thus is very robust and seems to be better than human umpires as it stands – and it can only improve. Hence there is heated discussion these days on whether one should completely rely on the HAWKEYE for LBW decisions. We choose not to go into those discussions here, in this paper.
The trajectories which the ball has taken after being hit by the batsman are recorded in the system. This is used to generate a graphic showing 1s, 2s, 3s, 4s, and 6s all indifferent colors for a batsman. These details allow the commentators, spectators and players to analyze the scoring areas of the batsman and also judge if he has played more shots along the turf or in the air. Such information is vital for fielding captain, who might alter his field placement in subsequent matches to adapt to the hitting pattern of a Particular batsman.
As shown above, the Pitch Map graphic uses information about the position where the ball bounced on the pitch. The image above clearly shows the pitch being divided into various “zones” which the experts consider in their
Analysis. It can be very easily seen where the bowler has been pitching the ball primarily. Based on such pitch maps, one can easily see general characteristics of bowlers-for instance, on a particular day a bowler might be taken for a lot of runs. HAWKEYE can show the areas in which the bowler Landed the balls and he might be able to find out he was too short on most occasions and hence was being taken for runs. Batsmen also use such graphics to study the general tendency of the bowler and can plan to play him in subsequent games
The DE Spin graphics help us in understanding how the ball has deviated after pitching. The graphic produced shows the predicted path of the ball, had it held its line even after pitching. This is particularly interesting to look at, in the case of spinners, where one can see both the flight being given byte bowler and the spin that he manages to extract from the pitch. Looking at the action and the DE Spin graphics for particular bowler is useful for batsman to notice any changes in action when the spinner is bowling a “trick” ball –which might be a goodly or flipper in the case of a leg spinner, or a “doors” in the case of an off spinner.
The Railcar graphics show a sideways view of the ballast it left the bowler‟s hand. This is useful to compare the speeds of various deliveries bowled and the bounce the bowler was able to extract from the pitch. As a simulation against time, the slower balls can clearly be seen to reach the line of the stumps much later than the faster balls.
This graphic shows the position of various balls in the plane of the batsman. So, irrespective of whether the batsman played a shot or not, the system places a mark on the plane showing us the point at which the ball passed/would have passed the batsman. At some times, this might be part of the actual trajectory, while in other cases, it might be an extrapolated path. To add to the usefulness, the system canals show the balls on which the batsman scored, in one colour and the ones which he defended in another. This helps to get a very good idea of the strengths and weaknesses of batsman and his scoring zones. The bowler can easily make out if he needs to be bowling away from the body or into the body of the batsman, whether he should be bouncing it hard into the deck or pitching it up and invite the drive etc.
We have looked at various aspects of the HAWKEYE technology. Initially, We outlined the main problems which one could encounter while trying to implement such a system for a sport like cricket. Then, We looked into the details of each step of the process which finally gives us the wonderful looking graphics that we see on TV during cricket analysis shows. We got a fair understanding of the algorithms and mathematics which goes into the system. With the help of examples, We looked at the applications which the technology finds in modern day sport, with cricket being our main focus. We got an understanding of how the graphics can be produced, using the setup, which also was described in detail. We have thus seen that the HAWKEYE is a great innovation, which puts technology to good use in the field of sports. The technology is used widely these days, in sports such as Tennis and Cricket. The accuracy which can be achieved with the use of the system is making the authorities think seriously about reducing the human error component involved in important decisions. As the system runs in real-time, there is no extra time required to see the visualizations and graphics. The system is also a great tool which can bemused by players, statisticians, tacticians, coaches to analyze previous games and come up with strategies for subsequent ones.