source: main/waeup.sirp/branches/ulif-fasttables/src/waeup/sirp/utils/converters.txt @ 10009

:mod:`waeup.sirp.utils.converters` -- Converters
************************************************

.. module:: waeup.sirp.utils.converters

Converters for :mod:`zope.schema` based data.

.. contents::


.. :doctest:
.. :layer: waeup.sirp.testing.WAeUPSIRPUnitTestLayer


Constants
=========

.. attribute:: NONE_STRING_VALUE

   This value determines, which string represents
   'non-values'. Current setting is:

      >>> from waeup.sirp.utils.converters import NONE_STRING_VALUE
      >>> NONE_STRING_VALUE
      ''

   If this value is found during conversion from strings, the value to
   set will become ``None``.


Adapters
========

The :mod:`waeup.sirp.utils.converters` module is basically a collection of
adapters for field types of :mod:`zope.schema`.

:class:`Converter`
------------------

.. class:: Converter(field)

   Base class for converter classes. You cannot create working
   instances of this class. Instead derive from it and implement the
   missing classes.

   `field` should be an object implementing some interface from
   :mod:`zope.schema` like :class:`zope.schema.TextLine`,
   `zope.schema.Choice` or similar.

   .. method:: provides(waeup.sirp.interfaces.ISchemaTypeConverter)

   .. method:: _convertValueFromString(string)

       Raises :exc:`NotImplementedError`.

       Called by :meth:`fromString()` to convert a given string to a
       value appropriate for the applied field type. The string might
       be ``None`` or ``missing_value`` from field definition.

       Override this method in derived classes to get a working
       converter.

   .. method:: _convertValueToString(value)

       Raises :exc:`NotImplementedError`.

       Called by :meth:`toString()` to convert a given value to a
       string suitable to be stored for instance in CSV files.

       Override this method in derived classes to get a working
       converter.

   .. method:: fromString(string[, strict=True])

       Compute a value for the context field from the given
       `string`. Be aware that this might fail for several reasons,
       for instance because the string cannot be converted to an
       integer (if the applied field is an Integer field), etc.

       Use `strict` to enable/disable validations of transformed
       data.

   .. method:: toString(value[, strict=True])

       Generate a string from ``value``, suitable to be stored in CSV
       files or similar. This method does preliminary checks, handles
       defaults and ``missing_value`` stuff and then calls
       :meth:`_convertValueToString(value)` to do the actual
       conversion.

       Use `strict` to enable/disable validations of transformed
       data.
       


:class:`TextConverter`
----------------------

.. class:: TextConverter(field)

   Create a converter for text fields.

   `field` must be an object implementing
   ``zope.schema.interfaces.IText``. This is true for instance for
   :class:`zope.schema.TextLine` fields and most other, purely
   text-related field types.

   Instances of this class are available as adapters from :mod:`IText`
   to :mod:`ISchemaTypeConverter`.

   .. method:: adapts(zope.schema.IText)

   .. method:: provides(waeup.sirp.interfaces.ISchemaTypeConverter)

   .. method:: fromString(string[, strict=True])

       Compute a unicode string value from the given `string`. Be
       aware that this might fail if no conversion of the delivered
       data is possible.

       Use `strict` to enable/disable validations of transformed
       data.

   .. method:: toString(value[, strict=True])

       Return `value` as string or ``None``.

       If `value` is ``None`` (or field's `missing_value`), ``None``
       is returned.

       If the value is not valid for the applied field an exception
       might be raised.

       Use `strict` to enable/disable validations of input data.


:class:`BoolConverter`
----------------------

.. class:: BoolConverter(field)

   Create a converter for boolean fields.

   `field` must be an object implementing
   ``zope.schema.interfaces.IBool``.

   Instances of this class are available as adapters from :mod:`IBool`
   to :mod:`ISchemaTypeConverter`.

   .. method:: adapts(zope.schema.IBool)

   .. method:: provides(waeup.sirp.interfaces.ISchemaTypeConverter)

   .. method:: fromString(string[, strict=True])

       Compute an integer number value from the given `string`. In
       general '1', 'true', and 'yes' are interpreted as ``True``,
       ``None`` is interpreted as 'no value' and all other values are
       interpreted as ``True``.

       Use `strict` to enable/disable validations of transformed
       data.

   .. method:: toString(value[, strict=True])

       Return `value` as string or ``None``.

       If `value` is ``None`` (or field's `missing_value`), ``None``
       is returned. Otherwise ``0`` or ``1`` are returned.

       If the value is not valid for the applied field (i.e. not an
       integer or `missing_value`) an exception
       might be raised.

       Use `strict` to enable/disable validations of input data.


:class:`IntConverter`
---------------------

.. class:: IntConverter(field)

   Create a converter for integer fields.

   `field` must be an object implementing
   ``zope.schema.interfaces.IInt``.

   Instances of this class are available as adapters from :mod:`IInt`
   to :mod:`ISchemaTypeConverter`.

   .. method:: adapts(zope.schema.IInt)

   .. method:: provides(waeup.sirp.interfaces.ISchemaTypeConverter)

   .. method:: fromString(string[, strict=True])

       Compute an integer number value from the given `string`. Be
       aware that this might fail if no conversion of the delivered
       data is possible.

       Use `strict` to enable/disable validations of transformed
       data.

   .. method:: toString(value[, strict=True])

       Return `value` as string or ``None``.

       If `value` is ``None`` (or field's `missing_value`), ``None``
       is returned.

       If the value is not valid for the applied field (i.e. not an
       integer or `missing_value`) an exception
       might be raised.

       Use `strict` to enable/disable validations of input data.



:class:`ChoiceConverter`
------------------------

.. class:: ChoiceConverter(field)

   Create a converter for choice fields.

   `field` must be an object implementing
   ``zope.schema.interfaces.IChoice``.

   Instances of this class are available as adapters from :mod:`IChoice`
   to :mod:`ISchemaTypeConverter`.

   .. method:: adapts(zope.schema.IChoice)

   .. method:: provides(waeup.sirp.interfaces.ISchemaTypeConverter)

   .. method:: fromString(string[, strict=False])

       Compute a value from the given `string`. Be aware that this
       might fail if no conversion of the delivered data is possible.

       As choices generally require a certain set of allowed values,
       only these values be allowed, in their tokenized form. The
       converter takes the given string as token and then tries to
       generate the appropriate value out of it (see examples below).

       Use `strict` to enable/disable validations of transformed
       data.

       As an invalid value will result in a `KeyError`, additional
       validation checks are disabled with this converter. This speeds
       up things as expensive lookups are avoided.

   .. method:: toString(value[, strict=False])

       Return `value` as string or ``None``.

       If `value` is ``None`` (or field's `missing_value`), ``None``
       is returned.

       If the value is not valid for the applied field (i.e. not an
       integer or `missing_value`) an exception
       might be raised.

       Use `strict` to enable/disable validations of input data.

       As an invalid value will result in a `ValueError`, additional
       validation checks are disabled with this converter. This speeds
       up things as expensive lookups are avoided.


:class:`DateConverter`
----------------------

.. autoclass:: waeup.sirp.utils.converters.DateConverter
   :members:
   :inherited-members:


Examples
========

Text field converters
---------------------

We create a field that also implements ``IText``, a regular
``TextLine`` field:

    >>> from zope.schema import TextLine
    >>> field = TextLine(title=u'Some Title',
    ...                  default=u'Default Value',
    ...                  required=True)

Now we can get a converter for this field by explicitly creating a
:class:`TextConverter` instance:

    >>> from waeup.sirp.utils.converters import TextConverter
    >>> converter = TextConverter(field)

Or we can just grab a registered adapter:

    >>> from waeup.sirp.interfaces import ISchemaTypeConverter
    >>> converter = ISchemaTypeConverter(field)

This will select the correct converter for us automatically.

    >>> converter
    <waeup.sirp.utils.converters.TextConverter object at 0x...>

Now we can convert strings to this type:

    >>> converter.fromString('blah')
    u'blah'

And back:

    >>> converter.toString(u'blah')
    'blah'

Okay, not very surprising. But the field definitions can help also
deliver values, if the given value is missing:

    >>> converter.fromString(None)
    u'Default Value'

``None`` is not an acceptable value for fields which are required but
provide no default:

    >>> field = TextLine(title=u'Some Title',
    ...                  required=True)

    >>> converter = ISchemaTypeConverter(field)
    >>> converter.fromString(None)
    Traceback (most recent call last):
    ...
    RequiredMissing

If we want to avoid this type of exception (and risk non-applicable
data to be stored), we can use the ``strict`` parameter of
``fromString()``:

    >>> converter.fromString(None, strict=False) is None
    True 


If a field is not required, we will get the ``missing_value`` type in
same case:

    >>> field = TextLine(title=u'Some Title',
    ...                  required=False)

    >>> converter = ISchemaTypeConverter(field)
    >>> converter.fromString(None) is None
    True

We can also set a value that indicates missing data (``None`` by default):

    >>> field = TextLine(title=u'Some Title',
    ...                  required=False,
    ...                  missing_value=u'<NO-VALUE>')

    >>> converter = ISchemaTypeConverter(field)
    >>> converter.fromString(None)
    u'<NO-VALUE>'

And put it back:

    >>> converter.toString(u'<NO-VALUE>') is None
    True

Bool field converters
---------------------

We create a field that also implements ``IBool``, a regular
``Bool`` field:

    >>> from zope.schema import Bool
    >>> field = Bool(title=u'Some truth',
    ...             default=True,
    ...             required=True)

Now we can get a converter for this field by explicitly creating a
:class:`BoolConverter` instance:

    >>> from waeup.sirp.utils.converters import BoolConverter
    >>> converter = BoolConverter(field)

Or we can just grab a registered adapter:

    >>> from waeup.sirp.interfaces import ISchemaTypeConverter
    >>> converter = ISchemaTypeConverter(field)

This will select the correct converter for us automatically.

    >>> converter
    <waeup.sirp.utils.converters.BoolConverter object at 0x...>

Now we can convert strings to this type:

    >>> converter.fromString('yes')
    True

    >>> converter.fromString('Yes')
    True

    >>> converter.fromString('yEs')
    True

    >>> converter.fromString('1')
    True

    >>> converter.fromString('True')
    True

    >>> converter.fromString('0')
    False

    >>> converter.fromString('no')
    False

    >>> converter.fromString('false')
    False


And back:

    >>> converter.toString(True)
    '1'

    >>> converter.toString(False)
    '0'

Okay, not very surprising. But the field definitions can help also
deliver values, if the given value is missing:

    >>> converter.fromString(None)
    True

``None`` is not an acceptable value for fields which are required but
provide no default:

    >>> field = Bool(title=u'Some Truth',
    ...             required=True)

    >>> converter = ISchemaTypeConverter(field)
    >>> converter.fromString(None)
    Traceback (most recent call last):
    ...
    RequiredMissing

If we want to avoid this type of exception (and risk non-applicable
data to be stored), we can use the ``strict`` parameter of
``fromString()``:

    >>> converter.fromString(None, strict=False) is None
    True

The same for the inverse operation:

    >>> converter.toString(None)
    Traceback (most recent call last):
    ...
    RequiredMissing

    >>> converter.toString(None, strict=False) is None
    True


If a field is not required, we will get the ``missing_value`` type in
same case:

    >>> field = Bool(title=u'Some Title',
    ...             required=False)

    >>> converter = ISchemaTypeConverter(field)
    >>> converter.fromString(None) is None
    True

    >>> converter.toString(None) is None
    True


Int field converters
--------------------

We create a field that also implements ``IInt``, a regular
``Int`` field:

    >>> from zope.schema import Int
    >>> field = Int(title=u'Some number',
    ...             default=12,
    ...             required=True)

Now we can get a converter for this field by explicitly creating a
:class:`IntConverter` instance:

    >>> from waeup.sirp.utils.converters import IntConverter
    >>> converter = IntConverter(field)

Or we can just grab a registered adapter:

    >>> from waeup.sirp.interfaces import ISchemaTypeConverter
    >>> converter = ISchemaTypeConverter(field)

This will select the correct converter for us automatically.

    >>> converter
    <waeup.sirp.utils.converters.IntConverter object at 0x...>

Now we can convert strings to this type:

    >>> converter.fromString('666')
    666

And back:

    >>> converter.toString(666)
    '666'

Okay, not very surprising. But the field definitions can help also
deliver values, if the given value is missing:

    >>> converter.fromString(None)
    12

``None`` is not an acceptable value for fields which are required but
provide no default:

    >>> field = Int(title=u'Some Title',
    ...             required=True)

    >>> converter = ISchemaTypeConverter(field)
    >>> converter.fromString(None)
    Traceback (most recent call last):
    ...
    RequiredMissing

If we want to avoid this type of exception (and risk non-applicable
data to be stored), we can use the ``strict`` parameter of
``fromString()``:

    >>> converter.fromString(None, strict=False) is None
    True

The same for the inverse operation:

    >>> converter.toString(None)
    Traceback (most recent call last):
    ...
    RequiredMissing

    >>> converter.toString(None, strict=False) is None
    True


If a field is not required, we will get the ``missing_value`` type in
same case:

    >>> field = Int(title=u'Some Title',
    ...             required=False)

    >>> converter = ISchemaTypeConverter(field)
    >>> converter.fromString(None) is None
    True

    >>> converter.toString(None) is None
    True


Choice field converters
-----------------------

Before choices really work in unit tests, we have to make sure, that
the following adapters are registered (this is normally done during
startup automatically):

    >>> from zc.sourcefactory.browser.source import FactoredTerms
    >>> from zc.sourcefactory.browser.token import (
    ...   fromString, fromUnicode, fromInteger, fromPersistent)
    >>> from zope.component import getGlobalSiteManager
    >>> gsm = getGlobalSiteManager()
    >>> gsm.registerAdapter(FactoredTerms)
    >>> gsm.registerAdapter(fromString)
    >>> gsm.registerAdapter(fromUnicode)
    >>> gsm.registerAdapter(fromInteger)
    >>> gsm.registerAdapter(fromPersistent)

We create a field that also implements ``IChoice``, a regular
``Choice`` field:

    >>> from zope.schema import Choice
    >>> field = Choice(title=u'Some number',
    ...                default=u'a',
    ...                values=['a', 'b', 'c'],
    ...                required=True)

Now we can get a converter for this field by explicitly creating a
:class:`ChoiceConverter` instance:

    >>> from waeup.sirp.utils.converters import ChoiceConverter
    >>> converter = ChoiceConverter(field)

Or we can just grab a registered adapter:

    >>> from waeup.sirp.interfaces import ISchemaTypeConverter
    >>> converter = ISchemaTypeConverter(field)

This will select the correct converter for us automatically.

    >>> converter
    <waeup.sirp.utils.converters.ChoiceConverter object at 0x...>

Now we can convert strings to this type:

    >>> converter.fromString('a')
    'a'

or values back to strings:

    >>> converter.toString('a')
    'a'

Non-listed values will not be accepted:

    >>> converter.fromString('nonsense')
    Traceback (most recent call last):
    ...
    LookupError: nonsense

Also non-string values will work:

    >>> from zope.schema import Choice
    >>> field = Choice(title=u'Some number',
    ...                default=2,
    ...                values=[1, 2, 3],
    ...                required=True)

    >>> converter = ISchemaTypeConverter(field)
    >>> converter.fromString('1')
    1

If we define an own source, then it will cause no problems:

    >>> from zc.sourcefactory.basic import BasicSourceFactory
    >>> class CustomSource(BasicSourceFactory):
    ...   def getValues(self):
    ...     return [1,2,3]
    ...    
    ...   def getTitle(self, value):
    ...     return "Number %s" % (value,)

    >>> field = Choice(title=u'Some number',
    ...                default=2,
    ...                source=CustomSource(),
    ...                required=True)

    >>> converter = ISchemaTypeConverter(field)
    >>> converter.fromString('1')
    1

Note, that when asking for conversion of Choice fields, you must
deliver the token value of the source/vocabulary. This might be the
same as the actual value as string, but it it might be different,
depending of the kind of source/vocabulary used.

Furthermore, when writing 'exporters', i.e. components that turn a
value into a string for usage in CSV files, then you have to save the
token value of a term.

Using the :meth:`toString()` method will deliver the token for us:

    >>> converter.toString(1)
    '1'

    >>> from waeup.sirp.interfaces import SimpleWAeUPVocabulary
    >>> field = Choice(
    ...   title = u'Favourite Dish',
    ...   default = 0,
    ...   vocabulary = SimpleWAeUPVocabulary(
    ...        ('N/A', 0), ('Pizza', 1),
    ...        ('Cake', 2), ('Muffins', 3)),
    ...   required = True,
    ...   )

    >>> converter = ISchemaTypeConverter(field)
    >>> converter.fromString('0')
    0

    >>> converter.toString(1)
    '1'

A more complex (but still realistic example) for conversion of
:mod:`zope.schema.Choice` fields follows. Here we have a special kind
of object, the `Cave` class, and get their `name` attribute as token.

    >>> class Cave(object):
    ...   name = 'Home sweet home'

    >>> def tokenizer(cave):
    ...   return cave.name

The tokenizer has to be registered as an Cave-to-IToken adapter to
keep :mod:`zc.sourcefactory` happy:

    >>> from zc.sourcefactory.interfaces import IToken
    >>> from zope.component import getGlobalSiteManager
    >>> gsm = getGlobalSiteManager()
    >>> gsm.registerAdapter(tokenizer, required=(Cave,), provided=IToken)

Now we finally can create two objects of `Cave` and make a field, that
gives the choice between these two objects:

    >>> obj1 = Cave()
    >>> obj1.name = 'Wilma'

    >>> obj2 = Cave()
    >>> obj2.name = 'Fred'

    >>> from zc.sourcefactory.basic import BasicSourceFactory
    >>> class CustomSource(BasicSourceFactory):
    ...   def getValues(self):
    ...     return [obj1, obj2]
    ...    
    ...   def getTitle(self, value):
    ...     return value.name

    >>> field = Choice(title=u'Some cave',
    ...                default=obj1,
    ...                source=CustomSource(),
    ...                required=True)

We get a converter for this field in the usual way. The
:meth:`waeup.sirp.utils.converters.fromString()` method will return one of
the objects if we feed it with ``'Wilma'`` or ``'Fred'``:

    >>> converter = ISchemaTypeConverter(field)
    >>> result = converter.fromString('Wilma')
    >>> result
    <Cave object at 0x...>

    >>> result is obj1, result.name
    (True, 'Wilma')

If we use the converter the other way round, it will call the
tokenizer above and deliver ``'Wilma'`` or ``'Fred'`` as they are the
tokens of the objects in question:

    >>> converter.toString(obj2)
    'Fred'

Date field converters
---------------------

We create a field that also implements ``IDate``, a regular
``Date`` field:

    >>> from zope.schema import Date
    >>> from datetime import date
    >>> field = Date(title=u'Some date',
    ...             default=date(2010, 4, 1),
    ...             required=True)

Now we can get a converter for this field by explicitly creating a
:class:`IntConverter` instance:

    >>> from waeup.sirp.utils.converters import DateConverter
    >>> converter = DateConverter(field)

Or we can just grab a registered adapter:

    >>> from waeup.sirp.interfaces import ISchemaTypeConverter
    >>> converter = ISchemaTypeConverter(field)

This will select the correct converter for us automatically.

    >>> converter
    <waeup.sirp.utils.converters.DateConverter object at 0x...>

Now we can convert strings to this type:

    >>> converter.fromString('2007-04-23')
    datetime.date(2007, 4, 23)

And back:

    >>> converter.toString(date(2007,4,23))
    '2007-04-23'

As string input we also accept 'DD/MM/YYYY':

    >>> converter.fromString('13/05/1945')
    datetime.date(1945, 5, 13)

and some more bizarre formats:

    >>> converter.fromString('1945/05/13')
    datetime.date(1945, 5, 13)

    >>> converter.fromString('1945/05/13 12:12:34 GMT+1')
    datetime.date(1945, 5, 13)

    >>> converter.fromString('1945/5/6')
    datetime.date(1945, 5, 6)

    >>> converter.fromString('06.05.1945')
    datetime.date(1945, 5, 6)

    >>> converter.fromString('6.5.1945')
    datetime.date(1945, 5, 6)

while other date formats will mean trouble:

    >>> converter.fromString('first of July 1960')
    Traceback (most recent call last):
    ...
    ValueError: Cannot convert to date: first of July 1960. Use YYYY-MM-DD.

Okay, not very surprising. But the field definitions can help also
deliver values, if the given value is missing:

    >>> converter.fromString(None)
    datetime.date(2010, 4, 1)

``None`` is not an acceptable value for fields which are required but
provide no default:

    >>> field = Date(title=u'Some Title',
    ...             required=True)
    >>> converter = ISchemaTypeConverter(field)
    >>> converter.fromString(None)
    Traceback (most recent call last):
    ...
    RequiredMissing

If we want to avoid this type of exception (and risk non-applicable
data to be stored), we can use the ``strict`` parameter of
``fromString()``:

    >>> converter.fromString(None, strict=False) is None
    True

The same for the inverse operation:

    >>> converter.toString(None)
    Traceback (most recent call last):
    ...
    RequiredMissing
    >>> converter.toString(None, strict=False) is None
    True


If a field is not required, we will get the ``missing_value`` type in
same case:

    >>> field = Date(title=u'Some Title',
    ...             required=False)
    >>> converter = ISchemaTypeConverter(field)
    >>> converter.fromString(None) is None
    True

    >>> converter.toString(None) is None
    True