Ian Ward

CKAN contributor/tech lead
Urwid author
PyRF contributor
Speedometer author

Contributing to Open Source
IASA E-Summit, 2014-05-16
Urwid Applications
Urwid Intro
Unfortunate Python
Django 1.1

Moving to Python 3
Article Tags


Ian Ward's email:
first name at this domain

wardi on OFTC, freenode and github

Locations of visitors to this page

Widgets, Form Fields and Model Fields Explained

Django Widget data flow
Posted on 2011-09-19.

In any web application user data must be translated from HTML form data to native types and database types, and back again. Django web applcations are no different.

The "right way" to handle custom types is to extend Django's widgets, form fields and model fields. However, understanding exactly how these types perform each step of the conversion can be confusing. This post will attempt to explain how the data is converted at each stage and offer some advice about creating custom widgets, form fields and model fields.

This article is based on Django 1.3 and assumes the reader has experience creating and using Django forms, models and validation.

Some Definitions

Native Types

This is the generic term I use to describe the most appropriate Python data type for a particular field's data. It might be a builtin type or it could be an instance of a class someone created.

The native type might also be a Unicode string. But even in this "simple" case it's important to remember that the string is not necessarily exactly what will be sent to the browser or the database. It will be escaped and encoded when sent to the browser, and it may be encoded differently when stored in the database.

The native type should be the type that is most convenient for a programmer to work with to manipulate the data, not what is most convenient for the database or web browser. eg. If you find you have HTML entities in your native type, you likely did something wrong.

Processed Data

This is the term I use to describe Django's extra step between HTML form data and native types.

When interpreting HTML form data it is first split into values that correspond to each form field. In the simple case processed data is the value string from an <input> tag. Less simple cases include check-box values turned into True or False, select box values turned into lists of key strings and MultiWidget values turned into lists of processed data from each of the widgets they contain.

Processed data must be in a form that it can represent what the user sent, even if it contains errors. When there are errors this data is used when sending a form back to the user to make corrections.


Django widgets are responsible for rendering HTML versions of native type or processed data values. These values are passed as the value parameter to their render() method. More complicated widgets may use Django's template library for rendering HTML. Simple widgets can just return strings with careful use of mark_safe() to leave HTML tags intact.

Widgets' render() method is also passed a name string that is a suggested HTML <input> tag name typically based on the Django form field name and form prefix. The render() method should use this name as a base for all the HTML fields it creates.

The render() method is typically called from Django's template library in a very cautious way that tends to hide errors. If your widget is missing from the page it's likely that render() raised an exception. I've writen about how to work around Django hiding widget exceptions.

When rendering more than one input field a widget must provide a value_from_datadict() method that will select the proper HTML form data from the all of the data sent and return processed data as the value of the field.

value_from_datadict() is not allowed to raise any validation errors, so there is a limit to how far conversion to processed data can go. If the data passed isn't valid one strategy is to return a special value that will be intercepted by the form field so that it can flag the validation error. See the ClearableFileInput source for an example of returning a special value.

In general processed data is different than the corresponding native types. This makes widget classes quite tightly coupled to the form fields that they will work with: Widgets render from native types to HTML, but don't convert all the way back to native. If you use incompatible widget and form field types your user's data won't correctly round-trip from the first time a form is displayed to the next, and it may even cause unhandled exceptions in your application.


Form Fields

Django form fields are responsible for validating processed data and converting it to native types. They also define a default Widget class and default HTML attributes to be passed to the Widget.

Validation and other form field usage is covered well in the Form Field docs. I also wrote a Django Forms Quick Reference that covers common form field operations.

The to_python() method is used for converting processed data to native types. If that conversion fails a ValidationError may be raised. This call happens before all other normal field validation. The convention is to raise errors defined in a .error_messages dict, which can be overridden in Field instances or subclasses.

A form field's .widget class attribute points to the default Widget class to use for this form field. The widget_attrs() method is used by a Form Field to customize the behaviour of Widget instances. This method returns a dictionary of HTML attributes to pass to a widget instance. widget_attrs() is one way to communicate extra information such as max_length from the field to the widget.


Model Fields

Model fields are responsible for validating native types and converting between native types and database data in both directions. Django's writing custom model fields documentation is very thorough. As above I will only draw attention to the places where data conversion happens.

get_prep_value() is used to convert a native type to a string for queries and saving to the database. If you are using a character type for your data in the database this is the only method you need to override. For other database column types you will need to override get_db_prep_value() and/or get_db_prep_save(). This method may also return None to indicate a NULL value in the database.

Model fields' to_python() methods have to take both native types and database types and convert them to native types. It is used for both validating model field values and when reading values out of the database. This method may raise a ValidationError, just like form fields' to_python() method, and by convention it should use an .error_messages dict for errors raised.

to_python() methods tend to start by passing through None values, then checking for expected native types and cleaning or passing them through, and finally assuming it has data from a database and converting that data to a native type.


Tags: Django Software Python