Blogged by buffi

I posted a ticket about it being seemingly impossible to use djangos db api (even with custom SQL) to handle binary data.
http://code.djangoproject.com/ticket/5135
and got the reply.

Storing binary data was unsafe before (what if your binary data contained a zero byte?), so it was kind of lucky — and unsupported — that it worked at all. It just works even less well know.

The real fix here is something like #2417 (adding a propery binary field type). The current workaround is to use base64 encoding (or base96 or some other binary->ascii encoding) on the data before storing it. There’s nothing we can do at the text field level, since we are assuming Unicode strings for text and databases obviously use an encoding when they store stuff, hence we have to convert between the encoding and Python Unicode objects.

Although I do agree with him about a new BinaryField being the best way to solve this, I also realize that this means that django does not have any nice way to handle binary data at all right now, and that it probably won’t have it any time soon :/
I will have to resort to ugly “external” MySQLdb hacks for now.

Using base64 or similar is of course a good choice if you write your tables from scratch, but interfacing with a legacy application makes it impossible without digging through a whole lot of source code, and modifying that application (which also makes it a non-generic solution that pretty much requires a ugly fork of that app to work).

One would really think that interfacing with external application should be big enough to at least be able to handle “unsupported” field types using custom SQL. I mean… there’s even a chapter in djangobook about using legacy databases / external apps.

I hope that a BinaryField will eventually be incorporated in django, but I’m not the right person to do it since I only have experience using mysql, and it should of couse be generic
Due to django using unicode all the way down to custom SQL, I bet there will be quite a few issues with creating one though.

Edit: Also, this is important.
This only deals with the SVN-release.
0.96 was releases before the merge with the unicode-branch, and custom SQL to blob fields work there.

buffi Programming & scripting, Python