Tag Archives: machine translation

How to translate ‘Cette phrase est en français’ ? (This sentence is in French) – updated

Let us consider the following French sentence: Le comté de Kronoberg est un comté suédois dont le nom signifie en français ‘Couronne de montagne’. It translates into Corsican: A cuntea di Kronoberg hè una cuntea svedese chì u so nome significheghja in francese ‘Curona di muntagna’. (The County of Kronoberg is a Swedish county whose name means in French ‘Mountain crown’.) But it should be translated more accurately as: A cuntea di Kronoberg hè una cuntea svedese situata in u sudu di u paese, è chì u so nome significheghja in corsu ‘Curona di muntagna’ since the words significheghja in francese (means in French) are utterly false.

Now a semantic difficulty is lurking whose core can be related to self-reference: How should we translate ‘Cette phrase est en français’ ? Self-reference stems here from ‘cette phrase’ (this sentence). Litterally, it translates into: This sentence is in French). But a sense-preserving translation would be: This sentence is in English).

A much complicated instance of self-reference within translation is as follows: ‘Cette phrase ne comprend que sept mots’ (This sentence contains only seven words). It translates into Corsican: ‘Ss’infrasata ùn cumprendi ch’è setti paroli. It is also true of the Corsican translation, but false of the English one, which includes only six words. Arguably, a better English translation, which is sense-preserving is then:
This sentence contains only six words. Such translation ability is currently beyond the scope of present MT. We can tag it as an ability that would be required from superintelligent MT. It would then include: identifying sef-referent parts of discourse, such as: this sentence, these words, this proposition, this paragraph, this text, … But not all self-referring discourse is concerned here. For example, the Liar paradox (this sentence is false) is irrelevant here, since we only place ourselves from the standpoint of MT. Interestingly, such superintelligent ability also requires some meta-knowledge, i.e. the language of the source text and of the target text. For a shift from the source language to the target language is needed here.

What is required from Artificial General Intelligence with regard to Machine Translation?

Illustration from www.pixabay.com

We will be interested in a series of posts to try to define what is required of an AGI (Artificial General Intelligence) in order to reach the level of superintelligence in MT (machine translation). (All this is highly speculative, but we shall give it a try.)
One of the difficulties that arise in machine translation relates to the translation of expressions. This leads us to mention one of the required skills of a superintelligence. It is the ability to identify an expression within a text in a given language and then to translate it into another language. Let us mention that expressions are of different types: verbal, nominal, adjectival, adverbial, … To fix the ideas we can focus here on verbal expressions. For example, the French expression ‘couper les cheveux en quatre’ (litterally, cut the hairs in four, i.e. to split hairs), which translates into Corsican language into either castrà i falchetti (litterally, to chastise the hawks) or castrà i cucchi (litterally, to chastise the cuckoos). In order to properly translate such an expression, a superintelligence must be able to:

  • identify ‘couper les cheveux en quatre’ as a verbal expression in a French corpus
  • identify castrà i falchetti as a verbal expression within a Corsican corpus
  • associate the two expressions as the proper translation of each other

It appears here that such an aptitude falls under the scope of AGI (Artificial general intelligence).

Superintelligent machine translation (updated)

Illustration from pixabay.com

Let us consider superintelligence with regard to machine translation. To fix ideas, we can propose a rough definition: it consists of a machine with the ability to translate with 99% (or above) accuracy from one of the 8000 languages to another. It seems relevant here to mention the present 8000 human languages, including some 4000 or 5000 languages which are at risk of extinction before the end of the XXIth century. It could also include relevantly some extinct languages which are somewhat well-described and meet the conditions for building rule-based translation. But arguably, this definition needs some additional criteria. What appears to be the most important is the ability to self-improve its performance. In practise, this could be done by reading or hearing texts. The superintelligent translation machine should be able to acquire new vocabulary from its readings or hearings: not only words and vocabulary, but also locutions (noun locutions, adjective locutions, adverbial locutions, verbal locutions, etc.). It should also be able to acquire new sentence structures from its readings and enrich its database of grammatical sentence structures. It should also be able to make grow its database of word meanings for ambiguous words and instantly build the associate disambiguation rules. In addition, it should be capable of detecting and implementing specific grammatical structures.
It seems superintelligence will be reached when the superintelligent translation machine will be able to perform all that without any human help.

Also relevant in this discussion is the fact, previously argued, that rule-based translation is better suited to endangered langages translation than statistic-based translation. Why? Because high-scale corpora do not exist for endangered languages. From the above definition of SMT, it follows that rule-based translation is also best suited to SMT, since it massively includes endangered languages (but arguably, statistic-based MT could still be used for translating main languages one into another).

Let us speculate now on how this path to superintelligent translation will be achieved. We can mention here:

  • a quantitative scenario: (i) acquire, fist, an ability to translate very accurately, say, 100 languages. (ii) develop, second, the ability to self-improve (iii) extend, third, the translation ability to whole set of 8000 human languages.
  • alternatively, there could be a qualitative scenario: (i) acquire, first, an ability to translate somewhat accurately the 8000 languages (the accuracy could vary from language to language, especially with rare endangered languages). (ii) suggest improvements to vocabulary, locutions, sentence structures, disambiguation rules, etc. that are verified and validated by human (iii) acquire, third, the ability to self-improve by reading texts or hearing conversations.
  • it is worth mentioning a third alternative that would consist of  an hybrid scenario, i.e. a mix of quantitative and qualitative improvements. It will be our preferred scenario.

But we should provide more details on how these steps could be achieved. To fix ideas, let us focus on the word self-improvement module: it allows the superintelligent machine translation to extend its vocabulary in any language. This could be accomplished by reading or hearing new texts in any language. When facing a new word, the superintelligent machine translation (SMT, for short) should be able to translate it instantly into the 8000 other languages and add it to its vocabulary database.

To give another example, another module would be locution self-improvement module: it allows the superintelligent machine translation to extend its locution knowledge in any language.

Also relevant to this topic is the following question: could SMT be achieved without AGI ( general AI)? We shall address this question later.

 

Is rule-based MT more ethical than statistical MT?

In the ongoing debate on safe IA, it is a relevant open question of whether rule-based MT is more ethical than statistical MT. Here are some arguments in favor of rule-based MT in this context (without blaming statistical MT which has its own strengths):

  • it emulates human reasoning: it translates a text just as a human would do
  • there is much control on rule-based MT since the resulting translated text can be traced back: a detailed step-by-step translation process can be provided if required
  • rule-based MT can be consistently part of and integrate itself into a whole project of brain emulation, which emulates general human reasoning

Rough typology of remaining errors (updated march 2018)

French to Corsican: performing on French wikipedia sample test currently amounts to 94% on average. Below is a rough typology of remaining errors (presumably an average scoring of 95% on the open test should be attainable on the basis of correction of ‘easy’ tagged errors):

  • unknown vocabulary: 40% (easy)
  • basic disambiguation: 25%  (easy or medium difficulty)
  • false positives: 5% (medium difficulty or hard). This type of error  is mostly related to proper nouns, i.e. English termes that should remain un translated. For example: ‘North American Aviation’ translates erroneously into ‘North American Aviazione’. In this case, ‘Aviation’ should remain untranslated.
  • inadequate locution: 10% (medium difficulty or hard)
  • anaphora resolution related to complex sentence’s structure: 5% (hard)
  • semantic disambiguation: 5% (hard). For example, disambiguating French ‘échecs’ = fiaschi/scacchi (failures/chess)
  • erroneous accord related to gender mismatch from French to Corsican, i.e. (i) words that are masculine in French and feminine in Corsican language; and (ii) ) words that are feminine in French and masculine in Corsican language: 1% (medium difficulty).
  • erroneous accord related to number mismatch from French to Corsican, i.e. (i) words that are singular in French and plural in Corsican language; and (ii) ) words that are plural in French and singular in Corsican language (for example French ‘la canicule’ translates into ‘i sulleoni’ in Corsican language: 1% (medium difficulty).
  • specific grammatical case: 2% (hard)
  • anaphora resolution associated with gender or number mismatch: 1% (hard)
  • unknown, unclassified: 6% (hard)

Evaluation of machine translation: why not self-evaluation?

Evaluation of machine translation is usually done via external tools (to cite some instances: ARPA, BLEU, METEOR, LEPOR, …). But let us investigate the idea of self-evaluation. For it seems that the software itself is capable of having an accurate idea of its possible errors.

In the above example, human evaluation yields a score of 1 – 5/88 = 94.31%. Contrast with self-evaluation which sums its possible errors: unknown words and disambiguation errors, thus entailing a self-evaluation of 92,05%, due to 7 hypothesized errors. In this case, self-evaluation computes the maximum error rate. But even here, there are some false positives: ‘apellation’ is left untranslated, being unrecognized. In effect, the correct spelling is ‘appellation’. To sum up: the software identifies an unknown word (and lefts it untranslated) and counts it as a possible error.

Let us sketch what could be the pros and cons of MT self-evaluation. To begin with, the pros:

  • it could provide a detailed taxonomy of possible errors: unknown words, unresolved grammatical disambiguation, unresolved semantical disambiguation, …
  • it could identify precisely the suspected errors
  • evaluation would be very fast and uncostly
  • self-evaluation would work with whatever text or corpus
  • self-evaluation could pave the way to further self-improvement and self-correction of errors
  • its reliability could be good

And the cons:

  • MT may be unaware of some types of errors, i.e. errors related to expressions and locutions
  • MT may be unaware of some types of errors, i.e. errors related to expressions and locutions
  • MT self-evaluation could be especially blind to grammatical errors
  • it would sometimes count as unknown words some foreign words that should remain untranslated
  • MT would be unaware of erroneous disambiguations

Semantic disambiguation of French ‘femme’: in the mud, gold is still shining

In Corsican language, French word ‘femme’ can be translated, depending on the context

  • either into donna (woman)
  • or into moglia (wife)

The above sample still contains a lot of vocabulary and grammatical disambiguation errors (easy/medium difficulty), but it handles successfully the semantic disambiguation (hard) of ‘femme’, two instances of which are properly translated into moglia (wife). As the Corsican proverb says, in a cianga l’oru luci sempri (in the mud, gold is still shining).

French samples are from the French corpora of the University of Leipzig.

A Special Case of Anaphora Resolution

After improper anaphora resolution

Anaphora resolution usually refers to pronouns. But we face here a special case of anaphora resolution that relates to an adjective. The following sentence: ‘un vase de Chine authentique’ (an authentic vase of China) is translated erroneously as un vasu di China autentica, due to erroneous anaphora resolution. In this sample, the adjective ‘authentique’ refers to ‘vase’ (English: vase) and not to ‘Chine’ (China).

The same goes for ‘une chanson du Portugal mythique’, where ‘mythique’ refers to ‘chanson’ and not to ‘Portugal’.

After appropriate anaphora resolution

Four consecutive ambiguous words


Translating the following sentence: ‘ce fait est unique’ is not as easy as it could seem at first glance. In effect, it is made up of four consecutive ambiguous words:

  • ‘ce’: ‘ssu (demonstrative pronoun, this) or ciò (it, relative pronoun)
  • ‘fait’: fattu (masculine singular noun, fact), fattu (past participe, done) or faci (does, third person singular of the verb to do at the present tense)
  • ‘est’: estu (masculine singular noun, east) or (is, third person singular of the verb to be at the present tense)
  • ‘unique’: unicu (masculine singular adjective, unique in English) or unica (feminine singular adjective, unique in English)

What are the conditions for a given endangered language to be a candidate for rule-based machine translation?

What are the conditions for a given endangered language to be a candidate for rule-based machine translation? For a given endangered language to be a candidate for rule-based machine translation, some requirements are in order. There is notably need for:

  • a dictionary: some specialized lexicons are useful too
  • a list of locutions and their translation: to be more accurate what is needed are noun locutions, adjective locutions, adverbial locutions, verbal locutions and their translations in other language.
  • a detailed grammar (in any language): ideally, the grammar should be very detailed, mentioning notably irregular verbs, noun plurals, etc. Subjonctive, conditional tenses must also be accurately described.
  • in addition, elision rules, euphony rules, should also be described.
  • most importantly: a description of the main variants of the language and their differences. This is needed to handle what we can call the ‘variant problem’ (we shall say a bit more about this later): as an effect of diversity, endangered languages are often polynomic and come with variants. But translation must be coherent and a mix of several variants is not acceptable as a translation.

Let us mention that endangered languages are commonly associated with another language, being in a diglossia relationship one with another. To take an example, Corsican language is associated with French. So we consider the French-Corsican pair, and what is relevant is a French-Corsican. If we consider the sardinian gallurese language (‘gaddhuresu’), the relevant pair is Italian-Gallurese. Other relevant pairs are:

  • Italian-Sassarese
  • Italian-Sicilian
  • Italian-Venetian

Solving fivefold ambiguity: translation for French ‘poste’

French word ‘poste’ has (at least) fivefold ambiguity. For it can designate:

  • ‘poste’ (masculine singular noun) : postu, masculine singular noun (set, i.e. television set)
  • ‘poste’ (masculine singular noun): posta, feminine singular noun (position): erroneously translated as postu in the present case ; it should read a so posta
  • ‘poste’ (feminine singular noun) : posta, feminine singular noun (post office)
  • ‘poste’: impostu (from the verb impustà (‘poster’, to station o.s.) at singular first person)
  • ‘poste’: imposta (from the verb impustà (‘poster’, to station o.s.) at singular third person)

(However, it is more complex than that, since there is another sense of the verb ‘poster’ (to post/to mail).

Another case of firstname ambiguity: ‘Noël’

Translation of the French word ‘Noël’ yields another case of ambiguity. For ‘Noël’ can translate:

  • either into Natali (Christmas, Christmas Day): the annual festival commemorating Jesus Christ’s birth
  • or into, identically, Natali (‘Noel‘): the firstname

Now it seems there is no case of disambiguation, since in either case, ‘Noël’ in French translates into Natali (Natali in sartinese and taravese variants; Natale in cismuntincu variant). But ambiguity lurks when one considers some sentences including ‘Noël’. Let us consider then the following sentence: ‘Je l’ai donné à Noël.’ Now it can be translated:

  • either into: L’aghju datu in Natali. (I gave it at Christmas.)
  • or into: L’aghju datu à Natali (I gave it to Noel.)

since French preposition ‘à’ translates differently in both cases. A phenomenon of the same nature occurs when one considers translation from French to English.

Interestingly, when the two ambiguous consecutive words are repeated, ambiguity vanishes. Since ‘Je l’ai donné à Noël à Noël.’ translates unambiguously into L’aghju datu à Natali in Natali (I gave it to Noel at Christmas.). For we can ignore the order: L’aghju datu in Natali à Natali (I gave it at Christmas to Noel.) amounts to the same. In this last case, the  translation is meaning-preserving.

Interesting case of first name disambiguation

Here is an interesting case of first name disambiguation for machine translation. Consider the following first name ‘Camille’. It can apply to both genders. In Corsican (taravese or sartinese variants) it translates either into Cameddu (masculine) or Camedda (feminine). In some cases, the corresponding disambiguation relies on mere grammatical grounds. For example, ‘Camille était beau’ translates into Cameddu era beddu (Camille was beautiful), on grammatical grounds alone. The same goes for ‘Camille était belle’, that translates straightforwardly into Camedda era bedda (Camille was beautiful), according to the adjective gender.

Now the related disambiguation can result in a hard case, relying only on semantic context. Hence, ‘Camille était pacifique” can translate either into Cameddu era pacificu or into Camedda era pacifica, depending on the context (which can be text or even an image…). In effect, it cannot be translated merely on grammatical grounds, since ‘pacifique’ (pacific) is gender-ambiguous: it can translate either into pacificu of pacifica.

Now the same goes for French first name ‘Dominique’ (Dominic), which translates either into ‘Dumenicu (masculine) or ‘Dumenica‘ (feminine). Hence, ‘Dominique était pacifique’ (Dominic was pacific) can translate either into ‘Dumenicu era pacificu‘ or into ‘Dumenica era pacifica‘, depending on the context.

Writing differences between Corsican and Gallurese

Here are some writing differences between Corsican and Sardinian gallurese, that result from historical writing habits. These writing differences prevail, even when the words are the same:

  • ghj is replaced by gghj: acciaghju (corsu), acciagghju (gallurese) , steel
  • chj is replaced by cchj: finochju (corsu), finocchju (gallurese), fennel
  • tonic accent is marked systematically in gallurese whereas it is not compulsory in Corsican: apostulu (Corsican), apòstulu (gallurese), apostle
  • cc is prefered in Gallurese language instead of cq in Corsican: acquistu (corsu), accuistu (gallurese), purchase
  • dd in Corsican taravese or sartinese is replaced with ddh in Gallurese: beddu bedda beddi (corsu), beddhu beddha beddhi (gallurese), beautiful
  • final è in Corsican is replaced with é in Gallurese: sapè (corsu), sapé (gallurese), know

Quandu da la forza à la raghjoni cuntrasta Tandu vinci la forza è la raghjoni ùn basta

Quandu da la forza à la raghjoni cuntrasta
Tandu vinci la forza è la raghjoni ùn basta.

This is a rare Corsican proverb. In French, litterally: “Lorsque la force et la raison s’opposent, alors la force gagne car la raison ne suffit pas” (When strength and reason are opposed, then strength gains because reason is not enough).

But it seems better translated in French, as: “La raison du plus fort est toujours la meilleure.” (Jean de la Fontaine). Litterally: the reason of the strongest is always the best. This is semantically equivalent to: might makes right. This is the first verse of the fable of Jean de La Fontaine, The Wolf and the Lamb. To be compared with Aesop, from which this fable originates ; the conclusion of the story, in Aesop’s terms, was: This fable shows that with people decided to do the most righteous evil, defense remains without effect.

This rare Corsican proverb is a small wonder (heard from people of Laretu di Tallà). The proverb is in poetry, with the main rhyme cuntrasta/basta, but there is also a secondary rhyme at the beginning of the verses: Quandu/Tandu.

 

More generally, this raises the problem of the equivalence of meaning and the translation of the proverbs from one language to another. If one considers that Quandu da la forza à la raghjoni cuntrasta Tandu vinci la forza è la raghjoni ùn basta is semantically equivalent to “La raison du plus fort est toujours la meilleure” in French, it is somewhat surprising since there are several differences between the two versions:

  • The Corsican proverb is poetry while the French version is in prose
  • The phrase is longer in Corsica, and contains more words than the French version (the English version being even more concise but sense-preserving)
 

How rule-based and statistical machine translation can help each other

Here are a few suggestions on how rule-based and statistical machine translation  can help each other:

To begin with, rule-based and statistical machine translation are often contrasted and compared: it would be oversimplifying to conclude that one is better than the other. From a more objective standpoint, let us consider that each method has its strengths and weaknesses. Let us investigate on how one could make them collaborate in order to add up their respective strengths

in the case of an endangered language, the lack of good quality corpora has been pointed out. But one way for rule-based and statistical machine translation to collaborate would be to use rule-based translation for building a better quality corpus for statistical machine translation

suppose we begin with a statistical machine translation software that performs 50% on average with regard to French to Corsican translation

let us sketch the process of creating these better corpora: let us take the example of the French-Corsican diglossic pair (the Corsican language being considered by Unesco as a definitely endangered language). Now presently we lack a quality French-Corsican corpus or to say it more accurately, the corpus at our disposal is a low-quality one. The idea would be to use rule-based machine translation to create a much better corpus to use with statistical machine translation.

let us sketch now the different steps of this collaborative process: (i) create a French-Corsican corpus with the help of rule-based machine translation: if the software has some average 90% performance, then the corpus would be on average 90% reliable. With appropriate training, statistical MT should now perform some, say, 80% on average (to be compared with the previous 50% performance)
(ii) from this French-Corsican corpus, other corpora pairs can be created, such as Italian-Corsican, English-Corsican, etc. since French-Italian, English-Italian, etc. corpora of excellent quality already exist. The performance gain should then extend to other language pairs such as Italian-Corsican, English-Corsican, etc.

with the help of this process, we re finally in a position to combine and add up the strengths of the two complementary approaches to MT: on the one hand, rule-based MT is able to translate with good accuracy even in the lack of corpora; on the other hand, statistical machine translation is able to handle successfully and fastly a great many language pairs. To sum up, as the Corsican proverb says: una mani lava l’altra (One hand washes the other).

Why rule-based translation is (presently) best suited to endangered languages

Here are some arguments in favor of the choice of rule-based translation concerning machine translation of endangered languages (it relates to the philosophy of language policy):

  • there does not exist at present time a reliable corpus between the given endangered language and other languages
  • endangered languages are often polynomic, i.e. there exist some main variants of the language that coexist: it is important to preserve them since (i) it is a feature of diversity and (ii) it is an inherent feature of the given endangered language, and to distinguish between these variants. In addition, any translation should not contain a mix up of these variants. This also complicates the process of building a proper corpus, since the scarce existing corpus is made up of different variants of the language.
  • in the lack of an adequate corpus, statistical machine translation is not able to provide quality translation of the given endangered language (while on the other hand it succeeds with common languages where excellent corpora are available): arguably, providing low quality translation (although the attempt is meritable) could harm these endangered languages that are by definition vulnerable, since people could use and diffuse the resulting low quality translation. On those grounds, given this vulnerability, it could be argued that a minimum 80% quality translation is needed for a given pair involving an endangered language.
  • in addition, it should be pointed out that endangered languages are usually in a ‘diglossic’ relationship with another language: what is needed as a matter of priority is to provide translation between the two languages of this pair

(to be continued)

Word-sense disambiguation: first test of new engine

Now testing the new engine with the semantically ambiguous French ‘échecs’ = fiaschi/scacchi (failures/chess).

What is interesting here is that semantic disambiguation transfers successfully into English (although the French/English engine is still in its infancy as there are still a lot of grammatical errors):

Now further tests are needed with some other semantically ambiguous words:

  • ‘défense’: defense/tusk; Corsican: difesa/sanna
  • ‘fils’: sons/wires; Corsican: figlioli/fili
  • ‘comprendre’:
    understand/comprise; Corsican: capisce/cumprende
  • ‘vol’: flight/theft; Corsican: bulu/arrubecciu
  • ‘voler’: fly/steal; Corsican: bulà/arrubà
  • ‘échecs’: chess/failures; Corsican: scacchi/fiaschi
  • ‘palais’: palace/palaces/palate/palates; Corsican: palazzu/palazzi/palate/palates

In the background, the unresolved threefold ambiguity of French ‘partie’ = parti/partita/partita (part/game/gone) is lurking…

French ‘fin’ followed by a year number: fixed

Tagger improvement: fixed this issue. French ‘l’Empire allemand’ now translates properly into l’Imperu alimanu (the German Empire). French word ‘fin’ is now identified as a preposition when followed by a year number.

The above excerpt is translated into the ‘sartinesu’ variant of Corsican language.

This issue relates to the more general problem of the grammatical status of numbers, a problem to which we shall return later.