Video Encoding at 5 bytes/frame

Ian Ward · March 11, 2022

Python

This is part two of a two-part series. Read part one here.

In this post we use Python to encode the full Bad Apple video (3m39s) into 32 kilobytes. Our playback hardware has no CPU so our video “compression” is limited to the features of our HD44780-powered LCD display.

We create an illusion of a full bitmap display by carefully juggling 8 CGRAM characters across the 8 x 4 video area and lean on LCD display persistence.

Encoder Files

The encoder is made of a number of Python scripts, some that generate other Python scripts.

extract.py uses OpenCV to convert the badapple.mp4 input video to a bitmap format badapple.enc.gz tailored to our LCD display resolution and suitable for encoding.
lookup_table.py generates a lookup table binary ltable.bin and a Python module with mnemonics for values in the table baconsts.py.
encoder.py encodes badapple.enc.gz into a Python script that exports the video binary file video.bin using the table mnemonics from baconsts.py.

[update] The first post now describes lookup_table.py and generation of baconsts.py.

Initialization

The video is encoded into a Python script video.py that will generate the video binary file video.bin. On startup the LCD display needs to be initialized, so the script starts with:

from baconsts import *

with open('video.bin', 'wb') as f:
    f.write(
        INI + # function set: initial setup
        HID + # hidden cursor
        EIN + # entry incrementing, no shift
        CLR
    )
...

The INI command sets the LCD display into 8-bit mode. This can only be done once after initial powerup and is ignored afterwards. HID enables the display but disables the blinking and underline cursor modes. EIN disables screen shifting on output and makes the cursor increment to the next location after each character is written.

Frame Comparison

While encoding the video we output a frame comparison and some bookkeeping information on every new frame as comments in video.py. These comments let us more easily find bugs or places for improvement without needing to play the video on real hardware:

...
# ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣷⣄⠀ » ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣷⡀⠀ frame 3278
# ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⠙⣿⡇⢻⣿⡇⣿⣿⡇⣿⣿⡇ » ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⠙⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡆ bytes sent 16557
# ⣶⣶⡆⣶⣶⡆⣶⣶⡆⣶⣶⡆⠀⡖⠀⡶⠖⡆⣶⣶⡆⣶⣶⡆ » ⣶⣶⡆⣶⣶⡆⣶⣶⡆⣶⣶⡆⠀⡖⠀⡴⠖⠂⣶⣶⡆⣶⣶⡆ position E22
# ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⡏⠁⠀⠀⠂⠠⠾⠃⣿⣿⡇⣿⣿⡇ » ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⡉⠁⠀⠀⠂⠠⠶⠃⣿⣿⡇⣿⣿⡇ delta 24
# ⣭⣭⡅⣭⣭⡅⣭⣭⡅⣭⣭⠀⠀⠀⠀⣤⣬⡅⣭⣭⡅⣭⣭⡅ » ⣭⣭⡅⣭⣭⡅⣭⣭⡅⣭⣥⠀⠀⠀⠀⣤⣬⡅⣭⣭⡅⣭⣭⡅ ▴
# ⣿⣿⡇⣿⣿⡇⣿⣿⡇⠛⠋⠀⠀⠀⡀⣿⣿⡇⣿⣿⡇⣿⣿⡇ » ⣿⣿⡇⣿⣿⡇⣿⣿⡇⠛⠋⠀⠀⢀⡀⣿⣿⡇⣿⣿⡇⣿⣿⡇ cgram 8/8
# ⣛⣛⡃⣛⣛⡃⣛⠛⠁⠀⠀⠀⠀⠚⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃ » ⣛⣛⡃⣛⣛⡃⠛⠛⠃⠀⠀⠀⠘⠛⠃⣛⣛⡃⣛⣛⡃⣛⣛⡃ D24:CG5 E05:CG1 E03:CG6 D23:CG2 E04:CG0 D07:CG3 D04:CG7 E21:CG4
# ⣿⣿⡇⣿⣿⡇⠃⠀⠀⠀⠀⠀⠀⠀⠀⣿⣿⡇⣿⣿⡇⣿⣿⡇ » ⣿⣿⡇⣿⣿⡇⠀⠀⠀⠀⠀⠀⠀⠀⠀⣿⣿⡇⣿⣿⡇⣿⣿⡇ .
# ⠿⠿⠇⠿⠟⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠽⠿⠇⠿⠿⠇⠿⠿⠇ » ⠿⠿⠇⠿⠏⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠿⠿⠇⠿⠿⠇⠿⠿⠇ .
...

the left “image” is the ideal frame from badapple.enc.gz
the right “image” is what would be displayed on the LCD display at this point in time
frame is the frame number
bytes sent is the position within the file
position is the cursor position on the display
delta is the number of pixels different between ideal and display images
cgram is the number of CGRAM characters in use and the mapping of screen positions to CGRAM character indexes

These frame comparisons are responsible for video.py being more than 10MB so we’re not storing a copy in the same repo.

👉 Follow along by generating video.py on your own machine:

$ python encoder.py badapple.enc.gz > video.py

Drawing Blocks

Drawing a blank space or solid block character at any location in the video area is one of the simplest updates. If the cursor is not already in position we first send a cursor movement command e.g. E00 to move to the first character in the second row.

Next we send a space character b' ' to turn off all pixels in a 5x8 cell or a solid block character b'\xff' to turn them all on.

At the start of the video we need to draw an all-black screen (all pixels on) so the first few frames are spent drawing blocks:

...
    f.write(b'\xff')
    f.write(b'\xff')
    f.write(b'\xff')
    f.write(b'\xff')
    f.write(b'\xff')
    f.write(b'\xff')
# ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ » ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⠀⠀⠀⠀⠀⠀ frame 0
# ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ » ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⠀⠀⠀⠀⠀⠀ bytes sent 6
# ⣶⣶⡆⣶⣶⡆⣶⣶⡆⣶⣶⡆⣶⣶⡆⣶⣶⡆⣶⣶⡆⣶⣶⡆ » ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ position D06
# ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ » ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ delta 1040
# ⣭⣭⡅⣭⣭⡅⣭⣭⡅⣭⣭⡅⣭⣭⡅⣭⣭⡅⣭⣭⡅⣭⣭⡅ » ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ ▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴
# ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ » ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ cgram 0/8
# ⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃ » ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ 
# ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ » ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ .
# ⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇ » ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ .
    f.write(b'\xff')
    f.write(b'\xff')
    f.write(E00)
    f.write(b'\xff')
    f.write(b'\xff')
# ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ » ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ frame 1
# ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ » ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ bytes sent 11
# ⣶⣶⡆⣶⣶⡆⣶⣶⡆⣶⣶⡆⣶⣶⡆⣶⣶⡆⣶⣶⡆⣶⣶⡆ » ⣶⣶⡆⣶⣶⡆⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ position E02
# ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ » ⣿⣿⡇⣿⣿⡇⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ delta 880
# ⣭⣭⡅⣭⣭⡅⣭⣭⡅⣭⣭⡅⣭⣭⡅⣭⣭⡅⣭⣭⡅⣭⣭⡅ » ⠉⠉⠁⠉⠉⠁⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ ▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴
# ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ » ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ cgram 0/8
# ⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃ » ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ 
# ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ » ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ .
# ⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇ » ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ .
    f.write(b'\xff')
    f.write(b'\xff')
    f.write(b'\xff')
    f.write(b'\xff')
    f.write(b'\xff')
# ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ » ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ frame 2
# ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ » ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ bytes sent 16
# ⣶⣶⡆⣶⣶⡆⣶⣶⡆⣶⣶⡆⣶⣶⡆⣶⣶⡆⣶⣶⡆⣶⣶⡆ » ⣶⣶⡆⣶⣶⡆⣶⣶⡆⣶⣶⡆⣶⣶⡆⣶⣶⡆⣶⣶⡆⠀⠀⠀ position E07
# ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ » ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⠀⠀⠀ delta 680
# ⣭⣭⡅⣭⣭⡅⣭⣭⡅⣭⣭⡅⣭⣭⡅⣭⣭⡅⣭⣭⡅⣭⣭⡅ » ⠉⠉⠁⠉⠉⠁⠉⠉⠁⠉⠉⠁⠉⠉⠁⠉⠉⠁⠉⠉⠁⠀⠀⠀ ▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴
# ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ » ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ cgram 0/8
# ⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃ » ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ 
# ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ » ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ .
# ⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇ » ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ .
...

Clearing the screen

When enough of the screen pixels need to be switched from on to off we can clear the whole screen with a single CLR command:

...
# ⢿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ » ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ frame 789
# ⠀⠉⠃⠿⣿⡇⣿⣿⡇⣿⡟⠁⠙⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ » ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⡟⠁⠛⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ bytes sent 3989
# ⠀⠀⠀⠀⠀⠀⠲⢶⡆⡒⠀⡀⣶⣶⡆⣶⣶⡆⣶⣶⡆⣶⣶⡆ » ⣶⣶⡆⣶⣶⡆⣶⣶⡆⠂⠀⡀⣶⣶⡆⣶⣶⡆⣶⣶⡆⣶⣶⡆ position E23
# ⠀⠀⠀⠀⠀⠀⠀⠀⠁⠓⠶⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ » ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣶⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ delta 559
# ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠉⠩⠅⣭⣭⡅⣭⣭⡅⣭⣭⡅ » ⣭⣭⡅⣭⣭⡅⣭⣭⡅⣭⣭⡅⣭⣭⡅⣭⣭⡅⣭⣭⡅⣭⣭⡅ ▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴▴
# ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠉⠛⠇⣿⣿⡇⣿⣿⡇ » ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ cgram 3/8
# ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠈⠙⠃⢛⣛⡃ » ⠛⠛⠃⠛⠛⠃⠛⠛⠃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃ E03:CG4 D04:CG3 D03:CG0
# ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠉⠃ » ⠀⠀⠀⠀⠀⠀⠀⠀⠀⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ .
# ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ » ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇ .
    f.write(CLR)
    f.write(D05)
    f.write(b'\xff')
    f.write(b'\xff')
    f.write(b'\xff')
# ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠈⠛⠇⢿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ » ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⣿⣿⡇⣿⣿⡇⣿⣿⡇ frame 790
# ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢠⡇⡟⠙⠃⢿⣿⡇⣿⣿⡇⣿⣿⡇ » ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⣿⣿⡇⣿⣿⡇⣿⣿⡇ bytes sent 3994
# ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠠⣶⡆⠀⠀⠀⠀⠀⠂⠶⣶⡆⣶⣶⡆ » ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ position D08
# ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠈⠁⠻⢿⡇ » ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ delta 103
# ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠁ » ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ ▴▴▴▴▴▴
# ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ » ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ cgram 0/8
# ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ » ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
# ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ » ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ .
# ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ » ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ .
...

This can prevent falling too far behind on screen updates when large parts of the screen are changing, but is distracting if overused.

We strike a balance by only clearing the screen when at least half of the pixels displayed are wrong and the number of wrong pixels is at least 1.5 times the number of pixels that should be “on”.

Drawing Pixels

Drawing pixels on the screen requires:

moving the cursor to the CGRAM area e.g. C20 for the first line of CGRAM character CG2
filling 8 bytes of data, only the least significant 5 bits matter so we use commands b'@' (all 0s) through b'_' (all 1s) in the ASCII command range
moving the cursor to the desired location e.g. E27 for the bottom-right corner of the screen
writing the CGRAM character to display the CGRAM pattern at this location e.g. CG2

...
    f.write(C20) # assign CG2 to E27 (9 steps)
    f.write(b'@')
# ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⠃⠀⠀⠀⠀⠀⠀⠀⠀ » ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⠃⠀⠀⠀⠀⠀⠀⠀⠀ frame 23
# ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⠁⠀⠀⠀⠀⠀⠀⠀⠀⠀ » ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⠀⠀⠀⠀⠀⠀⠀⠀⠀ bytes sent 122
# ⣶⣶⡆⣶⣶⡆⣶⣶⡆⣶⣶⡆⣶⣶⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ » ⣶⣶⡆⣶⣶⡆⣶⣶⡆⣶⣶⡆⣶⣶⡆⠀⠀⠀⠀⠀⠀⠀⠀⠀ position C21
# ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⠀⠀⠀⠀⠀⠀⠀⠀⠀ » ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⠀⠀⠀⠀⠀⠀⠀⠀⠀ delta 22
# ⣭⣭⡅⣭⣭⡅⣭⣭⡅⣭⣭⡅⣭⣭⡅⣀⠀⠀⠀⠀⠀⠀⠀⠀ » ⣭⣭⡅⣭⣭⡅⣭⣭⡅⣭⣭⡅⣭⣭⡅⣤⣤⡄⠀⠀⠀⠀⠀⠀ ▴
# ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣷⡆⠀⠀⠀⠀⠀⠀ » ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⠀⠀⠀⠀⠀⠀ cgram 2/8
# ⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣃⠀⠀⠀⠀⠀ » ⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣀⠀⠀⠀⠀⠀ E26:CG1 D05:CG0
# ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⠀⠀⠀⠀⠀ » ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣆⠀⠀⠀⠀ .
# ⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠆⠀⠀⠀ » ⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠆⠀⠀⠀ .
    f.write(b'@')
    f.write(b'@')
    f.write(b'@')
    f.write(b'@')
    f.write(b'X')
# ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⡟⠀⠀⠀⠀⠀⠀⠀⠀ » ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⠃⠀⠀⠀⠀⠀⠀⠀⠀ frame 24
# ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⠇⠀⠀⠀⠀⠀⠀⠀⠀⠀ » ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⠀⠀⠀⠀⠀⠀⠀⠀⠀ bytes sent 127
# ⣶⣶⡆⣶⣶⡆⣶⣶⡆⣶⣶⡆⣶⡆⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀ » ⣶⣶⡆⣶⣶⡆⣶⣶⡆⣶⣶⡆⣶⣶⡆⠀⠀⠀⠀⠀⠀⠀⠀⠀ position C26
# ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⠟⠀⠀⠀⠀⠀⠀⠀⠀ » ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⠀⠀⠀⠀⠀⠀⠀⠀⠀ delta 35
# ⣭⣭⡅⣭⣭⡅⣭⣭⡅⣭⣭⡅⣭⣭⡅⣥⣤⡀⠀⠀⠀⠀⠀⠀ » ⣭⣭⡅⣭⣭⡅⣭⣭⡅⣭⣭⡅⣭⣭⡅⣤⣤⡄⠀⠀⠀⠀⠀⠀ ▴▴
# ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣦⠀⠀⠀⠀⠀ » ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⠀⠀⠀⠀⠀⠀ cgram 2/8
# ⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⡃⠀⠀⠀⠀ » ⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣀⠀⠀⠀⠀⠀ E26:CG1 D05:CG0
# ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣇⠀⠀⠀⠀ » ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣆⠀⠀⠀⠀ .
# ⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠦⠀⠀ » ⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠆⠀⠀⠀ .
    f.write(b'X')
    f.write(b'\\')
    f.write(E27)
    f.write(CG2)
    f.write(E26)
# ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⠃⠀⠀⠀⠀⠀⠀⠀ » ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⠃⠀⠀⠀⠀⠀⠀⠀⠀ frame 25
# ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⠁⠀⠀⠀⠀⠀⠀⠀⠀ » ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⠀⠀⠀⠀⠀⠀⠀⠀⠀ bytes sent 132
# ⣶⣶⡆⣶⣶⡆⣶⣶⡆⣶⣶⡆⣶⣖⡀⣀⡀⠀⠀⠀⠀⠀⠀⠀ » ⣶⣶⡆⣶⣶⡆⣶⣶⡆⣶⣶⡆⣶⣶⡆⠀⠀⠀⠀⠀⠀⠀⠀⠀ position E26
# ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⡃⠀⠀⠀⠀⠀⠀⠀ » ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⠀⠀⠀⠀⠀⠀⠀⠀⠀ delta 63
# ⣭⣭⡅⣭⣭⡅⣭⣭⡅⣭⣭⡅⣭⣭⡅⣭⣭⡄⣄⠀⠀⠀⠀⠀ » ⣭⣭⡅⣭⣭⡅⣭⣭⡅⣭⣭⡅⣭⣭⡅⣤⣤⡄⠀⠀⠀⠀⠀⠀ ▴▴▴
# ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣧⠀⠀⠀⠀ » ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⠀⠀⠀⠀⠀⠀ cgram 3/8
# ⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⠀⠀⠀⠀ » ⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣀⠀⠀⠀⠀⠀ E26:CG1 D05:CG0 E27:CG2
# ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡀⠀⠀⠀ » ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣆⠀⠀⠀⠀ .
# ⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠄⠀ » ⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠆⠿⠄⠀ .
...

Notice that we’ve spent about two frames (11 bytes) updating just a single character cell. That’s very slow if need to update the whole screen this way.

Optimization strategies

We mitigate the slow speed of pixel updates a few different ways:

do all screen clearing and block updates first
update based on the state a couple frames ahead so that the cell painted is correct when drawn
prefer to not update cells that are only a few pixels different than “all-on” or “all-off” (block updates are fine for these)
don’t update cells that will become all-on or all-off in a few frames, wasting our effort when the cell is cleared
choose the unassigned CGRAM character that has the smallest difference to save a few bytes updating pixel data
when updating a cell already assigned to a CGRAM character, update the pixel data in-place

Example of an in-place pixel update of CGRAM character CG0 at position E04 (the witch’s back and hat) in only 7 bytes:

...
# ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ » ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ frame 571
# ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ » ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ bytes sent 2889
# ⣶⣶⡆⣶⣶⡆⣶⣶⡆⣶⣄⠀⠐⢶⡆⣶⣶⡆⣶⣶⡆⣶⣶⡆ » ⣶⣶⡆⣶⣶⡆⣶⣶⡆⣶⣄⠀⠐⢶⡆⣶⣶⡆⣶⣶⡆⣶⣶⡆ position C66
# ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⠀⠐⠸⠇⠿⠿⡇⣿⣿⡇⣿⣿⡇ » ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣷⡁⠀⠘⠇⠿⠿⡇⣿⣿⡇⣿⣿⡇ delta 8
# ⣭⣭⡅⣭⣭⡅⣭⣤⡄⣤⣤⡄⣄⣠⡄⣀⣀⡀⣭⣭⡅⣭⣭⡅ » ⣭⣭⡅⣭⣭⡅⣭⣤⡄⣤⣤⡄⣄⣠⡄⣤⣀⡄⣭⣭⡅⣭⣭⡅
# ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ » ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ cgram 6/8
# ⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃ » ⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃ E05:CG2 D25:CG4 E04:CG0 E02:CG3 D24:CG7 E03:CG6
# ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ » ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ .
# ⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇ » ⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇ .
    f.write(b'^')
    f.write(b'D')
    f.write(C00) # update assigned CG0 at E04
    f.write(b'G')
    f.write(b'O')
# ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ » ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ frame 572
# ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ » ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ bytes sent 2894
# ⣶⣶⡆⣶⣶⡆⣶⣶⡆⣶⣄⠀⠐⢶⡆⣶⣶⡆⣶⣶⡆⣶⣶⡆ » ⣶⣶⡆⣶⣶⡆⣶⣶⡆⣶⣄⠀⠠⢶⡆⣶⣶⡆⣶⣶⡆⣶⣶⡆ position C02
# ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⠀⠐⠸⠇⡿⢿⡇⣿⣿⡇⣿⣿⡇ » ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣷⠁⠀⠘⠇⠿⠿⡇⣿⣿⡇⣿⣿⡇ delta 12
# ⣭⣭⡅⣭⣭⡅⣭⣤⡄⣤⣤⡄⣄⣠⡄⣀⣀⡀⣭⣭⡅⣭⣭⡅ » ⣭⣭⡅⣭⣭⡅⣭⣤⡄⣤⣥⡄⣄⣠⡄⣤⣀⡄⣭⣭⡅⣭⣭⡅
# ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ » ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ cgram 6/8
# ⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃ » ⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃ E05:CG2 D25:CG4 E02:CG3 D24:CG7 E03:CG6 E04:CG0
# ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ » ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ .
# ⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇ » ⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇ .
    f.write(b'G')
    f.write(C04)
    f.write(b'K')
    f.write(b'C')
    f.write(E05)
# ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ » ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ frame 573
# ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ » ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ bytes sent 2899
# ⣶⣶⡆⣶⣶⡆⣶⣶⡆⣶⣄⠀⠰⢶⡆⣶⣶⡆⣶⣶⡆⣶⣶⡆ » ⣶⣶⡆⣶⣶⡆⣶⣶⡆⣶⣄⠀⠠⣶⡆⣶⣶⡆⣶⣶⡆⣶⣶⡆ position E05
# ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣷⠁⠐⠸⠇⣿⣿⡇⣿⣿⡇⣿⣿⡇ » ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣷⠁⠐⠸⠇⠿⠿⡇⣿⣿⡇⣿⣿⡇ delta 11
# ⣭⣭⡅⣭⣭⡅⣥⣤⡄⣤⣥⡄⣄⣠⡄⣀⣀⡁⣭⣭⡅⣭⣭⡅ » ⣭⣭⡅⣭⣭⡅⣭⣤⡄⣤⣥⡄⣄⣠⡄⣤⣀⡄⣭⣭⡅⣭⣭⡅
# ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ » ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ cgram 6/8
# ⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃ » ⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃ E05:CG2 D25:CG4 E02:CG3 D24:CG7 E03:CG6 E04:CG0
# ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ » ⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ .
# ⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇ » ⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇ .
...

Up in the air

When pixel updates in the video are constrained to 8 character cells or less and all updates are in-place the illusion of full 8 x 4 character cell video is very convincing:

8 active character cells

But when more than 8 character cells need pixel updates we must start juggling CGRAM characters by evicting the oldest character cell and using its CGRAM character at a new location on screen for every pixel character cell update.

...
    f.write(D01) # evict CG5 at D01
    f.write(b'\xff')
    f.write(C50) # reassign CG5 to E02
    f.write(b'@')
    f.write(b'@')
# ⣿⣿⡇⣿⣿⡇⠞⠛⠃⠛⢻⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ » ⣿⣿⡇⣿⣿⡇⠿⠛⠃⠙⢛⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ frame 5494
# ⣿⣿⡇⡟⠁⠀⠀⠀⠀⢼⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ » ⣿⣿⡇⣿⣿⡇⠀⠀⠀⢶⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ bytes sent 27746
# ⣶⣶⡆⠀⠀⠀⠀⠀⠀⢀⣶⡆⣶⣶⡆⣶⣶⡆⣶⣶⡆⣶⣶⡆ » ⣶⣶⡆⠀⠀⠀⠀⠀⠀⢀⣶⡆⣶⣶⡆⣶⣶⡆⣶⣶⡆⣶⣶⡆ position C52
# ⣿⣿⠀⠀⢀⡆⣷⡄⠀⠺⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ » ⣿⣿⡇⠀⠀⠀⠀⠀⠀⠻⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ delta 66
# ⣭⣭⡄⣤⣭⡅⣭⠅⠀⠀⠈⠅⢩⣭⡅⣭⣭⡁⣭⣭⡅⣭⣭⡅ » ⣭⣭⡅⣤⣤⡄⡄⠀⠀⠀⠉⠅⣭⣭⡅⣭⣭⡅⣭⣭⡅⣭⣭⡅ ▴▴▴▴
# ⣿⣿⡇⣿⣿⡇⣿⠇⠀⠀⢴⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ » ⣿⣿⡇⣿⣿⡇⡷⠀⠀⠀⢴⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ cgram 7/8
# ⣛⣛⡃⣛⠛⠃⠁⠀⠀⠀⢘⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃ » ⣛⣛⡃⡛⠛⠃⠀⠀⠀⠀⣘⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃⣛⣛⡃ D02:CG6 D22:CG2 E23:CG7 D03:CG4 E03:CG1 E21:CG0 D23:CG3
# ⣿⣿⠇⠀⠀⠀⠀⠀⠀⠀⢸⡇⣿⣿⡃⣿⣿⡇⣿⣿⡇⣿⣿⠇ » ⣿⣿⡇⠀⠀⠀⠀⠀⠀⠀⢿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇⣿⣿⡇ .
# ⠿⠿⠇⠶⠤⠀⠀⠀⠀⠀⠸⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇ » ⠿⠿⠇⠶⠤⠄⠀⠀⠀⠀⠸⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇⠿⠿⠇ .
...

Our LCD character display has “display persistence” meaning it hangs on to what was previously displayed for a short time. This effect gives us some cover when juggling CGRAM characters:

More than 8 active character cells

But eviction adds another 2 bytes to each character cell update, so pixel updates can now take up to 13 bytes each. At 13 bytes each we can only update about 11 character cells per second.

With too many cells needing pixel updates not even display persistence will help us.

Cheating (just a little)

Some parts of the original Bad Apple video have more detail than can be reasonably be displayed by juggling 8 CGRAM characters across a 8 x 4 video area at 11 updates per second. For these parts of the video we edit the video beforehand to scale the frame to fill 6 x 3 cells instead of the full video area. This reduces the number of character cells that need juggling:

Shrunken video

We also edit some of the iconic scenes that are poorly rendered because of significant movement on screen and extremely low bitrate. For these scenes we reduce the motion to give our encoder a chance to display something recognizable.

Finally we save some space by trimming the all-black frames at the beginning and the end of the video. We trim 80 all-black frames in total and use the 400 bytes saved to give ourselves an extra byte for video data every 16 frames or so.

Down time

While most of the time the encoder is frantically trying to keep up with screen updates there are a few moments during the video where nothing is changing on the screen.

These brief pauses give us a chance to draw some text messages on the right side of the screen:

Also more than 8 active character cells

These messages stay visible until the next time the screen is cleared.

Wrapping up

Nicely done, you made it all the way to the end! 🍺

In this post we demonstrated encoding the full Bad Apple video into 32 kilobytes using Python. We made it look like we had a bitmap display by using display persistence on our HD44780-powered LCD and by carefully juggling 8 CGRAM characters across the 8 x 4 video area.

This project is an offshoot of a Homebrew CPU build that is being published as a video series on YouTube.

All the code for this project is available on github.