Translation of IDs with flexible formatting and name matching.
Classes
|
Translate IDs to human-readable labels. |
|
Create a |
Bases: Generic[Translatable, NameType, SourceType, IdType]
Translate IDs to human-readable labels.
The recommended way of initializing Translator instances is the from_config() method. For configuration
file details, please refer to the Translator Configuration Files page.
The Translator is the main entry point for all translation tasks. Simplified translation process steps:
The
mapmethod performs name-to-source mapping (seeDirectionalMapping).The
fetchmethod extracts IDs to translate and retrieves data (seeTranslationMap).Finally, the
translatemethod applies the translations and returns to the caller.
fetcher – A Fetcher or ready-to-use translations.
fmt – String Format specification for translations.
mapper – A Mapper instance for binding names to sources.
default_fmt – Alternative Format to use instead of fmt for fallback translation of unknown IDs.
default_fmt_placeholders – Shared and/or source-specific default placeholder values for unknown IDs. See
InheritedKeysDict.make() for details.
allow_name_inheritance – If True, enable name resolution fallback to the parent translatable when
translating with the attribute-option. Allows nameless pandas.Index instances to inherit the name of
a pandas.Series.
Notes
Untranslatable IDs will be None by default if neither default_fmt nor default_fmt_placeholders is given.
Adding the maximal_untranslated_fraction option to translate() will raise an exceptions if too many IDs
are left untranslated. Note however that this verifiction step may be expensive.
Examples
A minimal example. For a more complete use case, see the DVD Rental Database example. Assume that we have data for people and animals as in the table below:
people: animals:
id | name | gender id | name | is_nice
------+---------+-------- ----+--------+---------
1991 | Richard | Male 0 | Tarzan | false
1999 | Sofia | Female 1 | Morris | true
1904 | Fred | Male 2 | Simba | true
In most real cases we’d fetch this table from somewhere. In this case, howeever, there’s so little data that we
can simply enumerate the components needed for translation ourselves to create a MemoryFetcher.
>>> from rics.translation import Translator
>>> translation_data = {
... 'animals': {'id': [0, 1, 2], 'name': ['Tarzan', 'Morris', 'Simba'], 'is_nice': [False, True, True]},
... 'people': {'id': [1999, 1991, 1904], 'name': ['Sofia', 'Richard', 'Fred']},
... }
>>> translator = Translator(translation_data, fmt='{id}:{name}[, nice={is_nice}]')
>>> data = {'animals': [0, 2], 'people': [1991, 1999]}
>>> for key, translated_table in translator.translate(data).items():
>>> print(f'Translations for {repr(key)}:')
>>> for translated_id in translated_table:
>>> print(f' {repr(translated_id)}')
Translations for 'animals':
'0:Tarzan, nice=False'
'2:Simba, nice=True'
Translations for 'people':
'1991:Richard'
'1999:Sofia'
Handling unknown IDs.
>>> default_fmt_placeholders = dict(
... default={'is_nice': 'Maybe?', 'name': "Bob"},
... specific={'animals': {'name': 'Fido'}},
>>> )
>>> useless_database = {
... 'animals': {'id': [], 'name': []},
... 'people': {'id': [], 'name': []}
>>> }
>>> translator = Translator(useless_database, default_fmt_placeholders=default_fmt_placeholders,
... fmt='{id}:{name}[, nice={is_nice}]')
>>> data = {'animals': [0], 'people': [0]}
>>> for key, translated_table in translator.translate(data).items():
>>> print(f'Translations for {repr(key)}:')
>>> for translated_id in translated_table:
>>> print(f' {repr(translated_id)}')
Translations for 'animals':
'0:Fido, nice=Maybe?'
Translations for 'people':
'0:Bob, nice=Maybe?'
Since we didn’t give an explicit default_fmt_placeholders, the regular fmt is used instead. Formats can be
plain strings, in which case tranlation will never explicitly fail unless the name itself fails to map and
Mapper.unmapped_values_action is set to ActionLevel.RAISE.
Create a Translator from TOML inputs.
path – Path to a TOML file, or a pre-parsed dict.
extra_fetchers – Path to TOML files defining additional fetchers. Useful for fetching from multiple sources or kinds of sources, for example locally stored files in conjunction with one or more databases. The fetchers are ranked by input order, with the fetcher defined in path being given the highest priority (rank 0).
A Translator instance.
See also
The Translator Configuration Files page.
Make a copy of this Translator.
share_fetcher – If True, the returned instance use the same Fetcher.
overrides – Keyword arguments to use when instantiating the copy. Options that aren’t given will be taken
from the current instance. See the Translator class documentation for possible choices.
A copy of this Translator with overrides applied.
NotImplementedError – If share_fetcher=False.
Translate IDs to human-readable strings.
translatable – A data structure to translate.
names – Explicit names to translate. Derive from translatable if None.
ignore_names – Names not to translate, or a predicate (str) -> bool.
inplace – If True, translate in-place and return None.
override_function –
A callable (name, fetcher.sources, ids) -> ... returning one of
None (use regular mapping logic)
a source to use, or
a split mapping {source: [ids_for_source..]}. This forces IDs to be fetched from
different sources in spite of being labelled with the same name.
maximal_untranslated_fraction – The maximum fraction of IDs for which translation may fail before an error is raised. 1=disabled. Ignored in reverse mode.
reverse – If True, perform translations back to IDs. Offline mode only.
attribute – If given, translate translatable.attribute instead. If inplace=False, the translated
attribute will be assigned to translatable using
setattr(translatable, attribute, <translated-attribute>).
A copy of translated copy of translatable if inplace=False, otherwise None.
UntranslatableTypeError – If type(translatable) cannot be translated.
AttributeError – If names are not given and cannot be derived from translatable.
MappingError – If any required (explicitly given) names fail to map to a source.
MappingError – If name-to-source mapping is ambiguous.
ValueError – If maximal_untranslated_fraction is not a valid fraction.
TooManyFailedTranslationsError – If translation fails for more than maximal_untranslated_fraction of IDs.
ConnectionStatusError – If reverse=True while the Translator is online.
UnknownSourceError – If override_function returns a source which is not known.
See also
The Mapper.apply() function, which performs both placeholder and name-to-source mapping.
Map names to translation sources.
translatable – A data structure to map names for.
names – Explicit names to translate. Derive from translatable if None.
ignore_names – Names not to translate, or a predicate (str) -> bool.
override_function –
A callable (name, fetcher.sources, ids) -> ... returning one of
None (use regular mapping logic)
a source to use, or
a split mapping {source: [ids_for_source..]}. This forces IDs to be fetched from
different sources in spite of being labelled with the same name.
A mapping of names to translation sources. Returns None if mapping failed.
AttributeError – If names are not given and cannot be derived from translatable.
MappingError – If any required (explicitly given) names fail to map to a source.
MappingError – If name-to-source mapping is ambiguous.
UnknownSourceError – If override_function returns a source which is not known.
Returns raw match scores for name-to-source mapping. See map() for details.
Return translation sources.
Fetch translations.
A TranslationMap.
ConnectionStatusError – If disconnected from the fetcher, i.e. not online.
Return the Fetcher instance used to retrieve translations.
Return the Mapper instance used for name-to-source binding.
Return a TranslationMap of cached translations.
Restore a serialized Translator.
path – Path to a serialized Translator.
A Translator.
TypeError – If the object at path is not a Translator.
See also
The Translator.store() method.
Retrieve and store translations in memory.
translatable – Data from which IDs to fetch will be extracted. Fetch all IDs if None.
names – Explicit names to translate. Derive from translatable if None.
ignore_names – Names not to translate, or a predicate (str) -> bool.
delete_fetcher – If True, invoke Fetcher.close() and delete the fetcher after retrieving data. The
Translator will still function, but new data cannot be retrieved.
path – If given, serialize the Translator to disk after retrieving data.
Self, for chained assignment.
ForbiddenOperationError – If Fetcher.fetch_all() is disabled and translatable=None.
MappingError – If map() fails (only when translatable is given).
Notes
The Translator is guaranteed to be serializable() once offline. Fetchers often
aren’t as they require things like database connections to function.
See also
The Translator.restore() method.
Bases: Generic[NameType, SourceType, IdType]
Create a Translator from TOML inputs.
A callable (name, kwargs) -> AbstractFetcher. Overwrite attribute to customize.
A callable (kwargs) -> Mapper. Overwrite attribute to customize.
Create a Translator from a TOML file.
Modules
Integration for insertion and extraction of IDs and translations to and from various data structures. |
|
General errors for the translation suite. |
|
Factory functions for translation classes. |
|
Translation using external sources. |
|
Offline (in-memory) translation classes. |
|
Test implementations. |
|
Types used for translation. |