Four languages, four sets of sites

We asked where to eat in Costa Adeje in 4 languages and collected 102 different sites. Of these, 17 are cited in all four languages, which is one in six, while 57, more than half, come out in one language only.

There were 5 questions, always the same ones, translated faithfully and changed in nothing else, with 15 readings per language: 60 in all, none dropped. The sites cited are 52 in Italian, 56 in English, 61 in German and 62 in Spanish.

Between two languages the overlap runs from 0.32 to 0.45: of every hundred sites cited in either of two languages, between thirty two and forty five come out in both. The low value is between Italian and English, the high one between Italian and German.

It is not the country suffix

The first suspicion was a dull one. If one assistant cites tripadvisor.it and another tripadvisor.de, we count two sites, and then the difference between languages could be something our own counting invented.

So we redid the count with the country suffix stripped, so that tripadvisor.it and tripadvisor.de counted as one. The numbers barely move: 0.32 to 0.46 instead of 0.32 to 0.45. The four languages are not drawing on local editions of the same site, they are drawing on different sites.

The 17 shared sites are the international review and booking portals, plus the venues' own sites. The other 57 are blogs, newspapers and guides that come out in one language and nowhere else: 21 in Spanish only, 16 in English only, 10 in Italian only and as many in German only.

The number meant to decide had no room to move

Before spending we had written down what we would look at, that is the first venues cited in each language and how much those groups resemble each other. We read 0.75, on average three quarters in common, which in the band we had declared beforehand means not conclusive.

Then we went to see why. Over the whole run 7 venues from our list came out, and the first five in each language were drawing five of them, so the groups we were comparing were nearly identical by construction and not by result. That number had no room to move. We worked it out after paying for the readings.

What we learned was in the column next to it, the one about sources, which decided nothing.

What this does not say

0.32 is the overlap of this run, not a rate. We translated the English, German and Spanish questions ourselves from the 5 Italian ones, and they are faithful, but a German speaker will not necessarily phrase the question that way. German, for that matter, is not even a language our product serves: we measured it because the language lives entirely in the text of the question. And we count the sources each assistant declares, so one that does not show them does not have zero sources. It has an empty list, which is not the same thing.