mod_translation

This module provides support for dealing with multiple languages.

How content and static strings are translated is explained in full in Translation.

Language as part of the URL

By default, mod_translation prefixes each URL (using URL rewriting) in your website with the code of the current language. The idea behind this is that each language version of a resource gets its own URL, and is as such indexable for Google.

This behaviour is enabled by default, but can be switched off in the admin, by going to Structure, Translation. There is a checkbox labelled “Show the language in the URL”.

Alternatively you can set the config key mod_translation.rewrite_url to false.

Default language fallback

By default, the site default language is added as the first fallback after the selected languages. This makes untranslated content use the site default language before other fallback languages are tried. Set i18n.default_language_is_fallback to false to keep the fallback language order based only on the selected request language and the previous context language fallbacks.

This can be changed in the admin by going to Structure, Translation. There is a checkbox labelled “Use the default language for untranslated texts”.

Language recognition

When a resource is saved without a language, Zotonic can infer one from its text. It extracts plain text from title, chapeau, subtitle, summary, and body, then compares its n-gram statistics with trained language profiles. Short input (at most 10 bytes after stripping HTML and trimming) or an unsuccessful detection falls back to the current context language. Detection is a heuristic; review the selected language when importing short or multilingual content.

translation_detect:detect(Text, Context) restricts matches to the site's editable languages; translation_detect:detect(Text) considers all installed profiles. Both return {ok, LanguageCode} or {error, nomatch}. Recognition identifies the language of existing text; it does not call an automatic translation service.

The bundled profiles in priv/data/profiles cover:

Training texts come from Wikipedia, typically articles about a country or language. Country articles also include names characteristic of the language.

Adding a recognition profile

1. Add a UTF-8 training file named <language-code>.txt to priv/data/texts. The code must be recognized by z_language.
2. In the Erlang shell, run translation_detect:generate_profile_data(). This generates profiles from the training files and loads the profile data.
3. Check the training text and generated priv/data/profiles/<language-code>.dat into version control. Review other generated files before committing them.
4. To load newly installed profiles without regenerating them, run translation_detect:load_profile_data(). Use translation_detect:profiles() to list the installed profile codes.

Programmatically switching languages

In a template, you can use mod_translation’s postback hook to switch between languages:

{% button text="Dutch" postback={set_language code="nl"} delegate=`mod_translation` %}

Creates a button which switches to Dutch. And another one for english:

{% button text="English" postback={set_language code="en"} delegate=`mod_translation` %}

Supporting right-to-left languages

For basic use you don’t need to do anything. Zotonic base site adds a lang attribute to the html tag, and when a right-to-left language is selected (for instance Arabic), the browser will interpret lang="ar" and automatically adapt the content to right-to-left.

Custom right-to-left content

If you write your own templates, you can add the lang tag in the html or body tag, for instance:

<body {% include "_language_attrs.tpl" id=id %} >

This will generate the following, when Zotonic selected Arabic for the page with id id:

<body xml:lang="ar" lang="ar" dir="rtl" class="rtl">

When you want to add an extra class added to the rtl or ltr class you can use:

<body {% include "_language_attrs.tpl" id=id class="my-body-class" %} >

To create individual right-to-left elements, you can use the same principle:

<div {% include "_language_attrs.tpl" %}></div>

And when you want to force a specific language:

<div {% include "_language_attrs.tpl" language=`en` %} >This is English content</div>

Translating a single resource

Use the translation dialog in the admin or frontend editor to manage a page's languages. Select an existing source language, a destination language, and one of these methods:

• Automatic translation sends the source texts to the configured translation service. This option is available when a module provides a translation service.
• Copy texts fills the destination fields with the source texts for manual translation.
• Leave texts empty enables the destination language so its texts can be entered manually.

Copying and automatic translation fill blank destination fields. Select Overwrite existing texts with new translations to replace filled fields as well. These actions update the edit form: review the texts and save the page to persist them. If automatic translation is incomplete, the editor uses the source text for missing translations and asks the user to review the result.

The dialog also lists the page's current translations. Deleting a translation requires confirmation and disables that language on the edit form; its translated texts are removed from the database when the page is saved.

Translating a saved resource from Erlang

Use m_translation:add_translation/5 to translate and save a resource directly. For example, to translate resource Id from English to Dutch while preserving existing Dutch texts:

ok = m_translation:add_translation(Id, en, nl, false, Context).

The resource must be editable in Context, both languages must be editable languages for the site, and the source language must be present on the resource. The call returns ok or {error, Reason} and waits for the translation service and resource update to finish. Successful translations add the destination language and mark it for review in translation_status. Pass true as the fourth argument to overwrite existing destination texts.

The resource translator processes translation records and known binary text properties, including nested blocks, using the property rules described below for tree translation. Binary text is assumed to be in the selected source language. translation_translate_rsc:copy_translation/5 copies source texts and saves the resource without calling a translation service. m_translation:remove_translation/3 removes a language from the saved resource.

For an in-memory property map, m_translation:add_translation_map/5 returns {ok, TranslatedMap} or an error without saving a resource. m_translation:remove_translation_map/2 similarly removes languages from a map.

Editing texts in the admin

The resource translation overview opens a single-field dialog when an editable text is clicked. Body properties and properties ending in _html use z_editor; short plain texts use an input and other plain texts use a textarea. Sites can override the blocks in _translation_edit_field.tpl. The rich-text dialog inherits the page's editor profile (advanced in the admin) and works with any enabled editor module.

Saving changes only the selected property and language, preserving other languages and block structure. Named block fields are supported; structured survey answer lists, unnamed blocks and unsafe properties remain read-only. Missing translations open empty, without falling back to a different language. The page slug is editable only when the resource has Customize page slug enabled (custom_slug). This condition is checked when opening and saving the dialog.

The translation overview, admin and frontend-admin edit forms warn when a newer resource version is observed. A successful translation dialog save acknowledges its new version in the overview only if the overview was current when the dialog opened. Saves never clear warnings about intervening external updates. Resource update notifications include version, including updates through m_rsc:touch/2. Media notifications also trigger a version check. Reconnects, window focus and periodic checks recover missed notifications. Notifications never advance the version of an unsaved form, and stale saves are rejected under a database row lock. Reloading asks for confirmation when the form has unsaved edits.

Media replacements initiated in the edit page send a private version acknowledgement to that browser. The form advances only from the exact pre-update version and keeps its unsaved edits; other tabs and already-stale forms still show the warning.

Video conversion marks resource updates with the update option {update_reason, media_processing}. Resource update notifications include previous_version and reason. The media_processing reason is emitted only when the resource properties are unchanged except for version, modification date and modifier. Otherwise the reason is update, including when upload observers change text or category. Editors and translation overviews acknowledge only the next consecutive processing version, preserving unsaved edits and existing conflicts. Missed notifications conservatively require reloading; polling alone cannot prove that a skipped version contained no text changes. Failed conversion and preview-only medium updates do not themselves increment the resource version.

The single-field edit dialog offers Translate from when a translation service and another nonempty source language are available. It translates the saved source property (including named-block fields) using the configured services, with HTML mode for rich text and text mode otherwise. The result replaces only the dialog's draft, after confirmation; Save still performs the normal version-checked update. Source languages use exact translations, not fallback. Failed requests and results arriving after further edits or closing the dialog do not overwrite the draft. Resource permissions, field eligibility and source/target languages are rechecked on the server. Automatic translation may send the source text to a remote service.

Custom admin form templates must include _admin_edit_version.tpl inside the form (pass version_form for a form id other than rscform). The z_edit_version hidden input is required for version-checked saves.

Bulk translation management

The admin Bulk update dialog offers Manage translations for the selected pages. It opens the language-count and translation dialog with automatic translation, copying, empty languages, and confirmed language removal. Only the selected pages are processed, without following menu or collection descendants. Duplicate selections count once, and removing a language preserves pages where it is the last language.

Opening the dialog requires edit access to every selected page. The server stores the selection for one day and restricts access to the user who created it. A sidejob processes the selection, rechecking edit access before each page, while the browser polls progress and displays a blocking modal. Starting a job only waits for startup, not completion. Reopening the same selection under the same user reconnects to its running job. Reload after completion to refresh the overview. These changes are saved immediately, independently of the bulk property update form.

Translating a tree

The Translate all pages button manages translations for an entire menu or collection. It is available in the frontend editor and at the bottom of the admin + Translate dialog for menus and pages with outgoing haspart connections. Sites can use the same feature for menu-based content such as timelines; the frontend editor's translate-all button acts on its tree_id.

A tree includes its root resource, nested menu entries, and resources connected through haspart. Each resource is counted and processed once, even when it occurs more than once or the tree contains a cycle. The dialog shows the union of the pages' languages, with the number of pages in each language. Resources without a language list count as using the site's default language.

Translation and language removal

The dialog offers the same translation options as the single-page editor:

• Automatic translation uses the configured translation service and marks translated content for review.
• Copy texts copies text from the selected source language to the destination language.
• Leave texts empty enables the destination language without copying or translating text.

Copying and automatic translation fill empty destination fields. Existing text is preserved unless the overwrite option is selected. Pages without the selected source language in their language list are skipped for these two methods. Empty source text is allowed when the source language is listed. Before starting, the dialog refreshes the page count and asks the user to confirm applying the selected options to that many pages. Changes are saved immediately; save or discard any pending editor changes before starting a tree operation.

Plain binary text is also translated, assuming it is in the resource's selected source language. The shared resource translator recognizes title, short_title, chapeau, summary, body, body_extra, date_remarks, prompt, explanation, matching, narrative, feedback, seo_title, seo_desc, seo_keywords, and properties ending in _html. This also applies inside nested blocks. These fields become translation records retaining the source text alongside the destination text. Other binary properties remain unchanged; existing translation records continue to use their explicit language entries.

Deleting a language requires a separate warning and confirmation. It removes that language from every editable page in the tree, except pages where it is the last language. Those pages are kept unchanged.

Permissions, progress, and completion

Opening the dialog or starting a job requires edit access to the root. Traversal requires visibility of every page. The worker checks edit permission again immediately before processing each resource and skips resources that are no longer editable. Normal resource-update permission checks also apply when saving changes.

Translation and removal run in a sidejob that is unique per tree and site on the current Erlang node. Startup returns a running status as soon as the worker registers, before traversing the tree or calling a translation service. The request does not wait for completion, and the tree job has no overall translation deadline. A second job for the same tree returns busy while the first is running.

The editor shows progress in a modal that cannot be dismissed, including when an editor is opened or reloaded during a running job. This is a UI-only lock: it does not add ACL restrictions or block unrelated server-side updates. Progress includes the total number of pages, pages processed, pages skipped, and pages that failed. The final dialog lists skipped pages with links to open their edit pages in a new tab. If an automatic translation service returns incomplete translations for a page, that page's texts are left unchanged and it is counted as failed. Changes already saved on other pages are retained. After completion or failure, the dialog offers a reload so the editor shows the saved content.

Model and template integration

The m_translation_tree model exposes these paths through templates and the model API:

A translation payload contains method (translate, copy, or empty), src, dst, optional overwrite, and confirmed: true. A removal payload contains method: remove, language, and confirmed: true.

The worker publishes progress on model/rsc/event/<root>/translation_tree. Browser clients subscribe through bridge/origin/model/rsc/event/<root>/translation_tree and can poll the status path to recover after reconnecting. Running status is owned by the worker; completed results are cached for one hour. The initial total is zero until tree traversal finishes.

Custom editors can include _translation_tree_button.tpl and _translation_tree_init.tpl, passing the root as tree_id. The initialization template also wires an existing button with id translate-all. The standard admin and frontend editors already include this initialization.

Accepted Events

This module handles the following notifier callbacks:

• observe_admin_menu: Add translation and language administration entries to the admin menu.
• observe_dispatch_rewrite: Removes the language from the path parts and sets it as the page language.
• observe_language_detect: Recognize the language of supplied text using n-gram profiles, optionally restricted to editable languages.
• observe_request_context: Check if the user has a preferred language (in the user's config) using z_context:get_cookie.
• observe_scomp_script_render: Add translation runtime variables to rendered client-side script blocks.
• observe_set_user_language: Persist a user language change to the authenticated user profile when allowed.
• observe_url_rewrite: Rewrite URLs to include/remove language prefixes according to translation settings.
• observe_user_context: Set user context language/timezone defaults based on the user profile resource.

Delegate callbacks:

• event/2 with postback messages: language_default, language_delete, language_status, set_language, toggle_url_rewrite, translation_generate, translation_reload.
• event/2 with submit messages: language_add, language_list.