Translation of IDs with flexible formatting and name matching.
Classes
|
Translate IDs to human-readable labels. |
|
Create a |
Bases: Generic[Translatable, NameType, SourceType, IdType]
Translate IDs to human-readable labels.
The recommended way of initializing Translator instances is the from_config() method. For configuration
file details, please refer to the Translator Configuration Files page.
The Translator is the main entry point for all translation tasks. Simplified translation process steps:
The
map_to_sourcesmethod performs name-to-source mapping (seeDirectionalMapping).The
fetchmethod extracts IDs to translate and retrieves data (seeTranslationMap).Finally, the
translatemethod applies the translations and returns to the caller.
fetcher – A Fetcher or ready-to-use translations.
fmt – String Format specification for translations.
mapper – A Mapper instance for binding names to sources.
default_fmt – Alternative Format to use instead of fmt for fallback translation of unknown IDs.
default_fmt_placeholders – Shared and/or source-specific default placeholder values for unknown IDs. See
InheritedKeysDict.make() for details.
Notes
Untranslatable IDs will be None by default if neither default_fmt nor default_fmt_placeholders is given.
Adding the maximal_untranslated_fraction option to translate() will raise an exceptions if too many IDs
are left untranslated. Note however that this verifiction step may be expensive.
Examples
A minimal example. For a more complete use case, see the DVD Rental Database example. Assume that we have data for people and animals as in the table below:
people: animals:
id | name | gender id | name | is_nice
------+---------+-------- ----+--------+---------
1991 | Richard | Male 0 | Tarzan | false
1999 | Sofia | Female 1 | Morris | true
1904 | Fred | Male 2 | Simba | true
In most real cases we’d fetch this table from somewhere. In this case, howeever, there’s so little data that we
can simply enumerate the components needed for translation ourselves to create a MemoryFetcher.
>>> from rics.translation import Translator
>>> translation_data = {
... 'animals': {'id': [0, 1, 2], 'name': ['Tarzan', 'Morris', 'Simba'], 'is_nice': [False, True, True]},
... 'people': {'id': [1999, 1991, 1904], 'name': ['Sofia', 'Richard', 'Fred']},
... }
>>> translator = Translator(translation_data, fmt='{id}:{name}[, nice={is_nice}]')
>>> data = {'animals': [0, 2], 'people': [1991, 1999]}
>>> for key, translated_table in translator.translate(data).items():
>>> print(f'Translations for {repr(key)}:')
>>> for translated_id in translated_table:
>>> print(f' {repr(translated_id)}')
Translations for 'animals':
'0:Tarzan, nice=False'
'2:Simba, nice=True'
Translations for 'people':
'1991:Richard'
'1999:Sofia'
Handling unknown IDs.
>>> default_fmt_placeholders = dict(
... default={'is_nice': 'Maybe?', 'name': "Bob"},
... specific={'animals': {'name': 'Fido'}},
>>> )
>>> useless_database = {
... 'animals': {'id': [], 'name': []},
... 'people': {'id': [], 'name': []}
>>> }
>>> translator = Translator(useless_database, default_fmt_placeholders=default_fmt_placeholders,
... fmt='{id}:{name}[, nice={is_nice}]')
>>> data = {'animals': [0], 'people': [0]}
>>> for key, translated_table in translator.translate(data).items():
>>> print(f'Translations for {repr(key)}:')
>>> for translated_id in translated_table:
>>> print(f' {repr(translated_id)}')
Translations for 'animals':
'0:Fido, nice=Maybe?'
Translations for 'people':
'0:Bob, nice=Maybe?'
Since we didn’t give an explicit default_fmt_placeholders, the regular fmt is used instead. Formats can be
plain strings, in which case tranlation will never explicitly fail unless the name itself fails to map and
Mapper.unmapped_values_action is set to ActionLevel.RAISE.
Create a Translator from TOML inputs.
path – Path to a TOML file, or a pre-parsed dict.
extra_fetchers – Path to TOML files defining additional fetchers. Useful for fetching from multiple sources or kinds of sources, for example locally stored files in conjunction with one or more databases. The fetchers are ranked by input order, with the fetcher defined in path being given the highest priority (rank 0).
A Translator instance.
See also
The Translator Configuration Files page.
Make a copy of this Translator.
share_fetcher – If True, the returned instance use the same Fetcher.
overrides – Keyword arguments to use when instantiating the copy. Options that aren’t given will be taken
from the current instance. See the Translator class documentation for possible choices.
A copy of this Translator with overrides applied.
NotImplementedError – If share_fetcher=False.
Translate IDs to human-readable strings.
translatable – A data structure to translate.
names – Explicit names to translate. Will try to derive form translatable if not given. May also be a
predicate which indicates (returns True for) derived names to keep.
ignore_names – Names not to translate. Always precedence over names, both explicit and derived. May
also be a predicate which indicates (returns True for) names to ignore.
inplace – If True, translation is performed in-place and this function returns None.
override_function – A callable with inputs (value, candidates, ids) that returns either None, the
source to use, or a split mapping {source: [ids_for_source..]} which forces IDs to be fetched from
different sources in spite of being labelled with the same name.
maximal_untranslated_fraction – The maximum fraction of IDs for which translation may fail before an error is raised. 1=disabled. Ignored in reverse mode.
reverse – If True, perform reverse translations back to IDs instead. Offline mode only.
A copy of translatable with IDs replaced by translations if inplace=False, otherwise None.
UntranslatableTypeError – If translatable is not translatable using any standard IOs.
AttributeError – If names are not given and cannot be derived from translatable.
MappingError – If required (explicitly given) names fail to map to a source.
ValueError – If maximal_untranslated_fraction is not a valid fraction.
TooManyFailedTranslationsError – If translation fails for more than maximal_untranslated_fraction of IDs.
ConnectionStatusError – If reverse=True while the Translator is online.
UnknownSourceError – If override_function returns a source which is not known to the Translator.
See also
The Mapper.apply() function, which performs both placeholder and name-to-source mapping.
Map names to translation sources.
translatable – A data structure to map names for.
names – Explicit names to translate. Will try to derive form translatable if not given. May also be a
predicate which indicates (returns True for) derived names to keep.
ignore_names – Names not to translate. Always precedence over names, both explicit and derived. May
also be a predicate which indicates (returns True for) names to ignore.
override_function – A callable with inputs (value, candidates, ids) that returns either None, the
source to use, or a split mapping {source: [ids_for_source..]} which forces IDs to be fetched
from different sources.
A mapping of names to translation sources. Returns None if mapping failed.
AttributeError – If names are not given and cannot be derived from translatable.
MappingError – If required (explicitly given) names fail to map to a source.
UnknownSourceError – If override_function returns a source which is not known to the Translator.
Fetch translations.
ConnectionStatusError – If disconnected from the fetcher, ie not online.
Return the Fetcher instance used to retrieve translations.
Return the Mapper instance used for name-to-source binding.
Return a TranslationMap of cached translations.
Restore a serialized Translator.
path – Path to a serialized Translator.
A Translator.
TypeError – If the object at path is not a Translator.
See also
The Translator.store() method.
Retrieve and store translations in memory.
translatable – Data from which IDs to fetch will be extracted. Fetch all IDs if None.
names – Explicit names to translate. Will try to derive form translatable if not given. May also be a
predicate which indicates (returns True for) derived names to keep.
ignore_names – Names _not_ to translate. Always takes precedence over names, both explicit and derived. May also be a predicate which indicates names to ignore.
delete_fetcher – If True, invoke Fetcher.close() and delete the fetcher after retrieving data. The
Translator will still function, but new data cannot be retrieved.
path – If given, serialize the Translator to disk after retrieving data.
Self, for chained assignment.
ForbiddenOperationError – If the fetcher does not permit the FETCH_ALL operation (only when translatable is
None).
MappingError – If a translatable is given, but no names to translate could be extracted.
Notes
The Translator is guaranteed to be serializable once offline. Fetchers often aren’t as they require
things like database connections to function. Serializability can be tested using the
rics.utility.misc.serializable() method.
See also
The Translator.restore() method.
Bases: Generic[NameType, SourceType, IdType]
Create a Translator from TOML inputs.
A callable taking (name, kwargs) which returns an AbstractFetcher.
A callable taking (kwargs) which returns a Mapper.
Create a Translator from a TOML file.
Modules
Integration for insertion and extraction of IDs and translations to and from various data structures. |
|
General errors for the translation suite. |
|
Factory functions for translation classes. |
|
Translation using external sources. |
|
Offline (in-memory) translation classes. |
|
Types used for translation. |