Ian Ward's email:
first name at this domain
wardi on OFTC, freenode and github
In any web application user data must be translated from HTML form data to native types and database types, and back again. Django web applcations are no different.
The "right way" to handle custom types is to extend Django's widgets, form fields and model fields. However, understanding exactly how these types perform each step of the conversion can be confusing. This post will attempt to explain how the data is converted at each stage and offer some advice about creating custom widgets, form fields and model fields.
This article is based on Django 1.3 and assumes the reader has experience creating and using Django forms, models and validation.
This is the generic term I use to describe the most appropriate Python data type for a particular field's data. It might be a builtin type or it could be an instance of a class someone created.
The native type might also be a Unicode string. But even in this "simple" case it's important to remember that the string is not necessarily exactly what will be sent to the browser or the database. It will be escaped and encoded when sent to the browser, and it may be encoded differently when stored in the database.
The native type should be the type that is most convenient for a programmer to work with to manipulate the data, not what is most convenient for the database or web browser. eg. If you find you have HTML entities in your native type, you likely did something wrong.
This is the term I use to describe Django's extra step between HTML form data and native types.
When interpreting HTML form data it is first split into values that correspond to each form field.
In the simple case processed data is the value string from an
<input> tag. Less simple cases include check-box values turned into
False, select box values turned into lists of key strings and
MultiWidget values turned into lists of processed data from each of the widgets they contain.
Processed data must be in a form that it can represent what the user sent, even if it contains errors. When there are errors this data is used when sending a form back to the user to make corrections.
Django widgets are responsible for rendering HTML versions of native type or processed data values. These values are passed as the
value parameter to their
render() method. More complicated widgets may use Django's template library for rendering HTML. Simple widgets can just return strings with careful use of
mark_safe() to leave HTML tags intact.
render() method is also passed a
name string that is a suggested HTML
<input> tag name typically based on the Django form field name and form prefix. The
render() method should use this name as a base for all the HTML fields it creates.
render() method is typically called from Django's template library in a very cautious way that tends to hide errors. If your widget is missing from the page it's likely that
render() raised an exception. I've writen about how to work around Django hiding widget exceptions.
When rendering more than one input field a widget must provide a
value_from_datadict() method that will select the proper HTML form data from the all of the data sent and return processed data as the value of the field.
value_from_datadict() is not allowed to raise any validation errors, so there is a limit to how far conversion to processed data can go. If the data passed isn't valid one strategy is to return a special value that will be intercepted by the form field so that it can flag the validation error. See the
ClearableFileInput source for an example of returning a special value.
In general processed data is different than the corresponding native types. This makes widget classes quite tightly coupled to the form fields that they will work with: Widgets render from native types to HTML, but don't convert all the way back to native. If you use incompatible widget and form field types your user's data won't correctly round-trip from the first time a form is displayed to the next, and it may even cause unhandled exceptions in your application.
render()method is passed either native type or processed data as its
nameparameter is a suggested HTML name, or name-prefix, for your
Django form fields are responsible for validating processed data and converting it to native types. They also define a default Widget class and default HTML attributes to be passed to the Widget.
to_python() method is used for converting processed data to native types. If that conversion fails a
ValidationError may be raised. This call happens before all other normal field validation. The convention is to raise errors defined in a
.error_messages dict, which can be overridden in Field instances or subclasses.
A form field's
.widget class attribute points to the default Widget class to use for this form field.
widget_attrs() method is used by a Form Field to customize the behaviour of Widget instances. This method returns a dictionary of HTML attributes to pass to a widget instance.
widget_attrs() is one way to communicate extra information such as
max_length from the field to the widget.
widget_attrs()can be used to pass extra information or restrictions to widget instances
Model fields are responsible for validating native types and converting between native types and database data in both directions. Django's writing custom model fields documentation is very thorough. As above I will only draw attention to the places where data conversion happens.
get_prep_value() is used to convert a native type to a string for queries and saving to the database. If you are using a character type for your data in the database this is the only method you need to override. For other database column types you will need to override
get_db_prep_save(). This method may also return None to indicate a NULL value in the database.
to_python() methods have to take both native types and database types and convert them to native types. It is used for both validating model field values and when reading values out of the database. This method may raise a
ValidationError, just like form fields'
to_python() method, and by convention it should use an
.error_messages dict for errors raised.
to_python() methods tend to start by passing through
None values, then checking for expected native types and cleaning or passing them through, and finally assuming it has data from a database and converting that data to a native type.
get_prep_value()+friends convert native types to database types