Blogged by buffi

Whenever I develop anything really, I often start out by tackling problems by doing them with a really simple approach to avoid premature optimization, since that often makes stuff messy, as well as increasing the development time. When the project finally works the way it’s supposed to you can get always get your hands dirty and optimize it then.

With the new pici in django I realized that showing pictures was sloooooow, and sought out where the problem could be. I then found out that I did a whole lot of unneeded SQL-queries in the view for showing pictures.

Each picture has a link to the next/previous picture on pici, if it’s in a gallery it has a link to the next/previous picture in that gallery and if it’s not an anonymous upload then it also has links to the next/previous picture from that user. Finally there is a link to display a random picture from pici.

My first approach was, whenever displaying a picture, get the previous and next picture in each category (that was represented for that picture) and build the links to the URL’s that way. That meant that pic http://pici.se/25 got links to http://pici.se/24 and http://pici.se/26, a random link to http://pici.se/xxxxx and also other links depending if it was in a gallery or uploaded by a user. This is obviously really stupid. Since at most one link can be clicked the SQL-queries for the other links are unneeded.

The obvious fix is first to remove all of the queries from the “show picture view”. Then change all of the links to stuff like http://pici.se/25/next , http://pici.se/25/next_in_gal and so on. When a link is clicked, the view handles the SQL needed and redirects to the new picture. This avoid unneeded queries and increased performance a whole lot. Adding a http://pici.se/random URL for the randomizer was also quite nice… since there previously wasn’t a way to get a random picture without first visiting a picture, let the view query up a random picture and then click the link. Now just visiting the URL is enough

There is however a downside. Without running the premature SQL, there is no way to check if a picture is the last one in a gallery and so on. One could create a hybrid solution that simply does this check beforehand, but I chose to ignore this issue since it is seldom an issue in this case, and the performance boost was a whole lot. I will go back and revisit the view code for this in a while though to check for more improvements, but right now the pictures show up really fast again, and that is the priority

buffi Programming & scripting, Python, Uninteresting

pici which is an image hosting site I own is written in php, even though this is a language that I despise, hate and dislike. I figured I should do something about this and started reimplementing everything in mod_python which turned out quite ok, but when I was pretty much finished (except for some php-scripts that I left the way they were) I realized that even though my code was a LOT better than my messy php code, it wasn’t really optimal so I started researching web frameworks.

Not doing this before rewriting the entire page in mod_python was of course not very smart… was it?

Anyways, I started having a look at django and it’s really really nice. Been messing around a bit with it now for a few nights and I really can’t see any reason to switch to anything else now. I’m maybe 50% done with reimplementing pici in django and I just keep on getting amazed on how much prettier the code turns out when using the MVC-approach. Finally I have a nice clean separation between my templates and my backbone code, and I can let django deal with all the low level management (users, database-connections and so on).

This might actually lead to me doing a lot more on pici since I won’t have to feel physically ill due to being forced to develop using php any more

buffi Programming & scripting, Python, Uninteresting

I was bored the last few nights and played around a bit with python and pygame and wrote a small app that takes a directory and then draws a structure depending on the underlying directories and files.
I probably won’t get around to finish this so I’ll just post a few screens since some turned out pretty!

Color is decided based on the files extension and the radius of circle is based of the size of the file

Click them for large!

Structure om a python project (with file names)

My programming folder

Structure type one

Structure type two

buffi Programming & scripting, Python, Uninteresting

I have to disable public comments due to massive amounts of spam.
If you still want to comment something, feel free to register.

buffi Uninteresting

While developing a small application for windows in python I realized that it was rather hard to send strings between different programs. I eventually got it running so I thought I’d post them here as examples.

I found out that the best path to choose is by using the windows message event-thingies. I’m not really much of a windows user so I have no clue about how they actually work but at least I kind of know how to use them now.
Basically to pass a string between two applications, you pass the memory address of a structure containing a string among other things (a pointer that is) to the target window by using it’s window handle (hWnd).
This is probably rather easy in c++ but since python doesn’t really have pointers you have to use the ctypes, array and struct modules to construct/deconstruct the things to send.

To get this running you need the win32 modules (win32con, win32gui, win32api) as well as ctypes. I used wxpython for the GUI but you can use anything you want. If you want my examples to run you need it installed though…

Anyways, I tried to document the code heavily so that I won’t have to explain much more…
Here is the code for the reciever

################
######reciever.py
############

import wx
import win32gui
import win32con
import struct
import ctypes
import sys

#Make an app with a static text
app = wx.PySimpleApp()
frame = wx.Frame( None, -1, "Message Reciever", size = ( 200, 50 ) )
result = wx.StaticText(frame, -1, "No text yet!",wx.Point(10,10))

#Write the windows hWnd to a file for the other app to read
fil = open( "hWnd", "w" )
fil.write( "%d" % frame.GetHandle() )
fil.close()

#Custom message handler
def MyWndProc( hWnd, msg, wParam, lParam ):

    # if we get any message from other app
    if msg == win32con.WM_COPYDATA: 

        #get the struct-string from the memory address
        structure = ctypes.string_at( lParam, 12 )

        #unpack the data
        data = struct.unpack("IIP", structure)
        unused, str_len, str_adr = data

        #and finally get the real string from the memory
        my_string = ctypes.string_at( str_adr, str_len )

        #update the static text
        result.SetLabel( my_string )

    #handles program termination
    elif msg == win32con.WM_DESTROY:
        win32gui.DestroyWindow( frame.GetHandle() )
        win32gui.PostQuitMessage(0)

    return win32gui.CallWindowProc( oldWndProc, hWnd, msg, wParam, lParam )

#Use the custom handler
oldWndProc = win32gui.SetWindowLong(
         frame.GetHandle() ,
         win32con.GWL_WNDPROC ,
         MyWndProc )

#show it!
frame.Show()
app.MainLoop()

By default running this displays a small window containing a static text that looks like this

Then we have the code for the sender

################
######sender.py
############

import win32con
import win32api
import struct, array
import sys

# get text to send
if len( sys.argv ) > 1:
  text_to_send = sys.argv[1]
else:
  sys.exit( 1 )

#get the hWnd to send to
#we have stored it in a file
hwnd = int( open( "hWnd", "rb" ).read() )

#create a char array from the string
my_data = array.array( 'c', text_to_send )

#get the memory address to the array
data_ad, data_len = my_data.buffer_info()

#create a struct with length and address to the data
my_struct = array.array( 'c',  struct.pack( "IIP", data_len, data_len, data_ad ) )

#and make a pointer to it
struct_ad = my_struct.buffer_info()[0]

#now send it to the hWnd
win32api.SendMessage(
         hwnd ,
         win32con.WM_COPYDATA ,
         0 ,
         struct_ad )

This doesnt use any GUI. It is just a console application that takes an argument and sends it to the other app. If you’d like you could let this application take input through a fancy GUI or whatever.

Since the reciever writes its hWnd to a file that the sender reads it knows where to send the data. You could also find the window through other means in the windows API.

Anyways, calling the sender with an argument results in the text in the other window changing!

Pretty basic examples that shouldn’t be too hard to build upon. Please leave a comment if it was helpful!

buffi Programming & scripting, Python, Uninteresting

PS3!

Downloading Yellow Dog Linux as I write this…

buffi Uninteresting

My harddrive in my main computer died earlier today. Sucks a bit but I was able to save most of my data.

And this is the worst output from fsck I’ve ever seen.
I wonder what the hell I fixed.

buffi Uninteresting

“A blog is a user generated website where entries are made in journal style and displayed in a reverse chronological order.”
Ok then, I guess this is a blog. Wikipedia never lies.

I’ll do my best to actually posting something useful but don’t get your hopes up.

Also, since everyone else is using wordpress and I’m a huge fan of being mainstream, so will I.

buffi Uninteresting