Improved Global Wine Medal Rating (GWMR) or “Faster, Higher, Stronger!”
Actual results (Faster!)
More than a year and a half has passed since the idea of the Global Medal Wine Rating was born. During all this rather long time, the project did not stand still at all and, on the contrary, stubbornly moved towards its goal, which at first seemed incredibly distant: a single end-to-end rating of all the world’s wines that received awards at significant professional wine competitions.
It was an exhausting marathon race, where the main rival was the persistent wave of more and more medals awarded at hundreds of professional competitions. At the same time, the route of the race ran through very rough terrain. The team had to find narrow paths of optimization in the prickly jungle of technical problems, random errors and organizational turmoil.
More than 350 events have been studied, giving out more than 250,000 medals over the past two and a half years. The subsequent processing of this information led to the creation of an extensive database of more than 140,000 wines of more than 27,000 producers from 76 different countries of the world. And even quite extensive at the start of the directory of grape varieties quickly grew to more than 1,000 entries.
But now the finish line is in sight. The processing of information on medals for 2020–2021 is coming to an end, but before summing up the intermediate results, we decided to return to mathematics and re-evaluate the existing rating calculation algorithms with a fresh look.
Improvement of algorithms (Higher!)
After accumulating a sufficient amount of data on wines and their medals, we finally had the opportunity to evaluate its correlation with other wine ratings within a very large and representative sampling. The results obtained prompted us to several possible solutions to improve the quality of the mathematical component and bring the calculation algorithms to a new, really high level.
If the calculation of the significance of the competition (coefficient of notoriety — kNoT) did not cause any particular complaints, and turned out to be quite adequate and fair, then the calculation of the rating of a particular vintage was subject to detailed study and consistent improvement.
Let’s have a look at the distribution of medals and ratings at one of the most famous wine competitions — Decanter World Wine Awards (DWWA). It is one of the few where the organizers publish exact scores for each wine.
The range of awards given at DWWA is as follows:
Bronze — 86–89.99
Silver — 90–94.99
Gold — 95–100
We specifically do not yet take into account the awards “Platinum” and “Best in Show”, as these medals are awarded based on the results of the initial score points.
As can be seen from the diagram, the distribution of points and awards for all 3 years is basically similar. Let’s try to determine this pattern.
The graph has a fairly articulated structure and clearly defined transition boundaries. It can be hypothesized that this type of distribution is typical for competitions where score points are given and prizes are awarded. For example, sports competitions where the criteria do not have strict boundaries, such as figure skating, diving, sports dancing, etc. Most likely, this type of distribution may be also typical for assessments of various tests or scorings, where there are border zones like “not accepted / accepted / awarded”. Let’s call this Award Points Distribution. It can also be assumed that this type of distribution is not continuous and has break points in the boundaries between the categories of awards. This is a theme for further research.
We can see that the the distribution graph for segment corresponding to bronze medals has a unique look, while all the rest graphs are similar.
First, let’s look at the “silver” section as the subsequent sections have the same structure.
This section has a pronounced exponential component of the form:
Empirically, we have obtained the value λ=π/N , where N is the number of segments in the given reward range. In our case, N = 94–90 = 4. For this type of distribution, the mean is 1/λ . The mean score in the silver award range is 90+ N/π , which is 91.273 and is significantly different from the simple arithmetic average of 92. This principle is applicable to all ranges except the first (“bronze”).
Consider now the distribution on the first segment, “bronze”.
With an increase in the number of segments, i.e. range of scores, the “tail” of the graph is significantly lengthened, the “cliff” on the right is actually kept at the N-1 level.
In general, this distribution looks like this:
The average value of this distribution should be added to the minimum value of the “bronze” range.
Thus, at the first stage, for competitions where only the type of award and the range of points are indicated, we calculate the average value and assign this value to this wine in this competition.
Where Xmax is the upper limit of the score range of the corresponding award, Xmin is the lower limit of the score range, ti is the corresponding element of the distribution.
Thus, we have replaced the rating range with the average value in the rating range for each award. It should be noted that since the range of scores may differ for different competitions, the average value is calculated in each interval separately. For example, for the “silver” intervals 88–92 and 85–95, the average values will be 89.27 and 88.18, respectively.
If there are awards in only one category, it is enough to get the average value. However, with awards of various denominations, it is necessary to adjust each value, according to those estimates where the range of estimates was, taking into account the number of awards in other categories. It is clear that in the presence of awards of greater value, it is necessary to shift the initial assessment up, and for awards of lesser value — down. This methodology is applicable only for estimates calculated instead of a range; for estimates with an initially fixed estimate, this does not apply.
To shift estimates, we apply the following algorithm:
Where Xavg is the arithmetic average of all scores for the respective award, |X| — the number of awards in this category.
After calculating the new values, they replace the previously calculated averages simultaneously for all categories.
At the next stage, we calculate the final rating of the wine, taking into account the coefficients of the competition.
where M is the total number of wine medals, ri are points for a particular medal, kNoTj is the coefficient of significance of the competition.
The applied methodology allows to calculate a very deliberated wine rating, as it takes into account many factors — the marks given not by a single expert, but by a competent jury, the statistical distribution of marks for various types of awards, the significance coefficients of competitions. It is safe to say that today this is one of the most “honest” ways to determine the rating of wine.
As a result of the work done, we improved the algorithm in such a way that, according to preliminary estimates, increased the relevance of the GWMR rating by 30%. The Gustos team has already assembled an impressive database of over 190,000 different medals from 205 international competitions. To assess the quality of the algorithm, a reconciliation was carried out with the prestigious wine-searcher.com portal for more than 1000 wines. The average deviation was only 0.51 points, despite the fact that wine-searcher publishes the rating in integers, and the GWMR rating is calculated with an accuracy of two decimalы. It should be noted that an accurate comparison across the entire database is difficult due to the different spellings of the same wines. In the next article, we plan to present data from such a comparison with a detailed description of the benefits of the GWMR rating.
Products and solutions (Stronger!)
More than a year ago we talked about the practical application of the results of calculating the GWMR rating for most of the world’s medal-winning wines. And now ready-to-use solutions are already available for many participants in the international market.
First of all, we implemented the promised widget, but in a more interesting format than originally planned. In addition to displaying directly the GWMR rating and the latest medals received, the widget optionally displays the Vivino rating.
However, the connection between the wines in our database and their counterparts in the Vivino system is quite a laborious process and will continue for some time. In the future, we are planning the same integration with other rating systems, such as Wine Searcher.
Placing a widget on any external pages is absolutely free and easy to implement. For technical issues of implementation, please contact: ar@gmail.com
The second, but much more significant result of the work done was the creation of the Global Wine Medals & Competitions Report, which, in more than 2000 pages, describes the complete picture of the global market for wine competitions, medals and medaled wines!
This report will be available for download on October 14th but is already open for pre-order with a significant 50% discount.
In addition to the usual report option, it is also possible to order a Personal report for the Winery, where in addition to the full version of the main report there will be a separate personalized file that contains:
- Your indicators as a manufacturer and your place within the World, Continent and Country for a number of parameters.
- The success of your top wines and their place within the World, Continent and Country for each individual wine category.
- Comparative analysis of your performance with the average values within the country and wine region.
There is also an option to order a Special Report for Distributors, which, in addition to the full version of the main report, will contain several large tables:
- Table of manufacturers by country with their model performance by category of awards, as well as in dynamics over several years.
- A table of the best wines by country with their model performance by award category, as well as in dynamics over several years
