Crimsonland Pak Format Again

Have been spending a little bit of time playing with Lua and an early project was to rewrite the Python script for reading a Crimsonland Pak file. So far I’m enjoying Lua and its language design, even with the very limited standard libraries.


BinaryFile = {} = function(file)
    return setmetatable({file=file}, {__index=BinaryFile})

BinaryFile.readint32 = function(self)
    local data = self.file:read(4)
    return (data:byte(4) * (256^3)) +
           (data:byte(3) * (256^2)) +
           (data:byte(2) * (256^1)) +
           (data:byte(1) * (256^0))

BinaryFile.readnulstring = function(self)
    local buf = ''
    while true do
        local ch = self.file:read(1)
        if ch:byte() == 0 then break end
        buf = buf .. ch
    return buf
end = function(self, ...)
    return self.file:read(...)

BinaryFile.skip = = function(self, ...)
    return self.file:seek(...)

local readindex = function(filename)
    local f  =, 'rb')
    local bf =

    -- Header
    assert(bf:readnulstring() == 'PAK')
    assert(bf:readnulstring() == 'V11')

    -- index offsets
    local indexstartoffset = bf:readint32()
    local indexendoffset   = bf:readint32()

    -- just to index
    bf:seek('set', indexstartoffset)

    -- number of entries
    local indexsize = bf:readint32()

    local index = {}

    for i=1, indexsize do
        table.insert(index, {
            name   = bf:readnulstring(),
            offset = bf:readint32(),
            length = bf:readint32()

        -- Unknown (junk?)


    return index

local index = readindex('D:\\Steam\\steamapps\\common\\Crimsonland\\data.pak')

for _,v in ipairs(index) do
    print(string.format('name:%s offset:%d length:%d',, v.offset, v.length))

Posted on

CloudFlare FlexSSL

Thanks to CloudFlare this website now sortof has a SSL cert. Lots of posts needed to be cleaned up with links and embedded content that forced http (rather than being agnostic). Was interesting looking to look back at the old posts (first was in Jan 2006!). So much awful code; but I’m happy that I recorded it somewhere.

Posted on

Who Hung up an Asterisk Call

Little snippet to work out which party hung up on a call in Asterisk.


; Detect which party hungup an Asterisk call
; If we (the caller) hangup after Dial() we jump directly to h
; Because of the 'g' option if the other party hangs up we continue on
exten => s,1,Set(whoHungUp=CALLER)
exten => s,n,Dial(IAX2/provider/${number},,g)
exten => s,n,Set(whoHungUp=OTHERPARTY)
exten => s,n,Hangup
exten => h,1,NoOp(whoHungUp ${whoHungUp})

Posted on

Crimsonland (Steam) Pak Format

A remake of Crimsonland was recently released on Steam (and soon PS4/Vita) and I thought I’d take a look at the pak data file. Didn’t turn out to be anything really interesting, just art assets, sounds, music, and some frontend lua. But was good fun (even if it was a simple format).

import os
import struct

Crimsonland (Steam) pak file format
Header format:
50 40 4B 00 56 31 31 00   ("PAK" NUL "V11" NUL)
int32 index offset     (eg: 28679361)
int32 end index offset (eg: 28885444)
Index Format:
int32 number of indexes?
Index_File format:
null terminated string
int32 absolute offset of file
int32 file length
unknown maybe always (?) equal to: FF 26 E2 50 20 00 00 00

class ClPak(object):
    def __init__(self, filename):
        self.file = open(filename, 'rb')

    def read_header(self):
        bytes =

        (magic_1, magic_2, index_start_offset, index_end_offset) = 
            struct.unpack('3sx3sxii', bytes)

        if magic_1 != 'PAK' or magic_2 != 'V11':
            raise Exception('Unknown file format')

        return index_start_offset, index_end_offset

    def read_index(self):
        start_offset, end_offset = self.read_header()
        bytes = - start_offset)

        # index_size = struct.unpack('i', bytes[:4])

        return self.read_index_file_details(bytes[4:])

    def read_index_file_details(self, bytes):

        details = {'name': '', 'offset': 0, 'length': 0, 'unknown': ''}

        sequence = iter(bytes)

        for char in sequence:
            if char != chr(0):
                details['name'] += char
                offset_bytes = + +
                length_bytes = + +

                (details['offset'],) = struct.unpack('i', offset_bytes)
                (details['length'],) = struct.unpack('i', length_bytes)

                for x in range(0, 8):
                    details['unknown'] +=

                yield details

                details = {'name': '', 'offset': 0, 'length': 0, 'unknown': ''}

    def dump_file(self, details, base_directory):
        dest_filename = base_directory + '/' + details['name']

        if not os.path.exists(os.path.dirname(dest_filename)):

        with open(dest_filename, 'wb') as dest:

if __name__ == '__main__':
    cl_pak = ClPak(r'D:\Steam\steamapps\common\Crimsonland\data.pak')
    for file_details in cl_pak.read_index():
        print file_details
        cl_pak.dump_file(file_details, 'd:/clpak_files')

Posted on

Global Hot Keys in Python for Windows

I used to think I had a basic understanding of Python. I’m not so sure anymore…

import ctypes
import ctypes.wintypes
import win32con

class GlobalHotKeys(object):
    Register a key using the register() method, or using the @register decorator
    Use listen() to start the message pump
    from globalhotkeys import GlobalHotKeys
    def hello_world():
        print 'Hello World'

    key_mapping = []
    user32 = ctypes.windll.user32

    MOD_ALT = win32con.MOD_ALT
    MOD_CTRL = win32con.MOD_CONTROL
    MOD_SHIFT = win32con.MOD_SHIFT
    MOD_WIN = win32con.MOD_WIN

    def register(cls, vk, modifier=0, func=None):
        vk is a windows virtual key code
         - can use ord('X') for A-Z, and 0-1 (note uppercase letter only)
         - or win32con.VK_* constants
         - for full list of VKs see:
        modifier is a win32con.MOD_* constant
        func is the function to run.  If False then break out of the message loop

        # Called as a decorator?
        if func is None:
            def register_decorator(f):
                cls.register(vk, modifier, f)
                return f
            return register_decorator
            cls.key_mapping.append((vk, modifier, func))

    def listen(cls):
        Start the message pump

        for index, (vk, modifiers, func) in enumerate(cls.key_mapping):
            if not cls.user32.RegisterHotKey(None, index, modifiers, vk):
                raise Exception('Unable to register hot key: ' + str(vk))

            msg = ctypes.wintypes.MSG()
            while cls.user32.GetMessageA(ctypes.byref(msg), None, 0, 0) != 0:
                if msg.message == win32con.WM_HOTKEY:
                    (vk, modifiers, func) = cls.key_mapping[msg.wParam]
                    if not func:


            for index, (vk, modifiers, func) in enumerate(cls.key_mapping):
                cls.user32.UnregisterHotKey(None, index)

    def _include_defined_vks(cls):
        for item in win32con.__dict__:
            item = str(item)
            if item[:3] == 'VK_':
                setattr(cls, item, win32con.__dict__[item])

    def _include_alpha_numeric_vks(cls):
        for key_code in (range(ord('A'), ord('Z')) + range(ord('0'), ord('9'))):
            setattr(cls, 'VK_' + chr(key_code), key_code)

# Not sure if this is really a good idea or not?
# It makes decorators look a little nicer, and the user doesn't have to explicitly use win32con (and we add missing VKs
# for A-Z, 0-9
# But there no auto-complete (as it's done at run time), and lint'ers hate it

from globalhotkeys import GlobalHotKeys

@GlobalHotKeys.register(GlobalHotKeys.VK_F1, GlobalHotKeys.MOD_SHIFT)
def hello_world():
    print "Hello World!"

def hello_world_2():
    print "Hello World again?"

# Q and ctrl will stop message loop
GlobalHotKeys.register(GlobalHotKeys.VK_Q, 0, False)
GlobalHotKeys.register(GlobalHotKeys.VK_C, GlobalHotKeys.MOD_CTRL, False)

# start main loop

Posted on

Phone Fun (Snom300)

Finds all Snom brand phones in a /24 and tells them to key in 6405 turn on
the hands free speaker.  (6405 is an extension that plays the stock monkies
sound from Asterisk)

import urllib2
import socket
import sys

def enumerate_snom_ips(base):
    original_timeout = socket.getdefaulttimeout()

    ips_to_check = [base + '.' + str(i) for i in range(2, 255)]
    snom_ips = []

    for ip in ips_to_check:
            data = urllib2.urlopen('http://' + ip + '/index.htm').read()
            if '<TITLE>snom 300</TITLE>' in data:
        except KeyboardInterrupt:


    return snom_ips

def send_command(ip, key):
    urllib2.urlopen('http://' + ip + '/command.htm?key=' + key).read()

def monkies(ip):
    for key in ['6', '4', '0', '5', 'SPEAKER']:
        send_command(ip, key)

print ' - Finding all Snom phones...'
for ip in enumerate_snom_ips('10.10.12'):
    print ip

Posted on

Python + matplotlib

After two hours solid of banging my head against a wall trying to get matplotlib do what I wanted, I was able to go from this LibreOffice graph:

To something a little nicer:

It’s still ugly, and confusing, but it’s a lot closer to what I wanted. The IPython environment is a interesting way of working with data too. I will have to play around with it more in the future.

Awful, awful Python code

(Really. Everything below is probably wrong)

%matplotlib inline
import matplotlib.pyplot as plt
from matplotlib.dates import strpdate2num
import matplotlib.dates as mdates
import numpy
import csv

filename = 'c:\\Users\Matthew\\adsl.csv'

rows = ("\t".join(i) for i in csv.reader(open(filename, 'r'), quotechar='"'))
converters = {0: strpdate2num('%Y/%m/%d %H:%M:%S')}
(x, sync_up, sync_down, snr_up, snr_down, attune_up, attune_down) = numpy.genfromtxt(rows, delimiter="\t", skip_header=1, converters=converters, unpack=True)

fig = plt.figure()

ax = fig.add_subplot(111)

# Configure x-ticks
# ax.set_xticks(x) # Tickmark + label at every plotted point
ax.xaxis.set_major_formatter(mdates.DateFormatter('%d/%m/%Y %H:%M'))

ax.plot_date(x, sync_up, '--', c='r', label='Sync(up)') # ls='-', marker='o')
ax.plot_date(x, sync_down, '--', c='g', label='Sync(down)') # ls='-', marker='o')
ax.set_title('ADSL2+ Line Quality')
ax.set_ylabel('Speed (kbps)')

ax.legend(bbox_to_anchor=(1.05, 1), loc=2)

highlight_start = x[212]
highlight_end = x[220]

ax.axvspan(highlight_start, highlight_end, facecolor='red', alpha=0.1)

ax2 = ax.twinx()
ax2.plot_date(x, snr_up,'-', c='r', label='SNR(up)')
ax2.plot_date(x, snr_down, '-', c='g', label='SNR(down)')
ax2.plot_date(x, attune_up, '-', c='c', label='Attenuation(up)')
ax2.plot_date(x, attune_down, '-', c='m', label='Attenuation(down)')
ax2.set_ylabel('Decibel (db)')

ax2.legend(bbox_to_anchor=(1.05, 0.80), loc=2)


Posted on

Tyrian Music

Tyrian is a vertical shooter by World Tree Games released in 1995 and published by Epic games. I have fond memories of the music (and game) but was having trouble finding a copy of the soundtrack as MP3s (etc).

Luckily this site has a copy of the original LDS (Loudness Sound System) files and using Foobar2k + AdPlug input component I was able to get them to play (note: the component expects the extension to be be ldsa). While I was at it, I also made a copy of them as MP3 files.

You can listen to the music below, or download a zip file of all the MP3s: (40MB)

Posted on

Website downtime

My website went down for around 1:30 to 2 hours today, which was a little odd. After it came back up I noticed the 15min load was > 30. Turns out someone tried to log into WordPress which isn’t that odd. It happens all the time, so much so that I run Limit Login Attempts plugin to apply a 20min timeout a IP address after 4 incorrect attempts (and then a 24 hour timeout after 16 attempts). This time though, it was a kind of a large attack.

In 118 minutes, 2337 different IP addresses tried to log into WordPress. Most were timed-out after 4 attempts, but due to oom-killer sometimes the timeout details weren’t added to the database and more than 4 attempts were made (Two ips made > 26 attempts each, though only 8 IPs made 7 or more attempts). Using a Geo IP database we can quickly get a breakdown of where the IPs came from:

Country Attempts
Russia 1786
Ukraine 1225
Vietnam 909
Thailand 817
Taiwan 613
Romania 380
Turkey 379
Bulgaria 342
Iran 278
Belarus 276
India 241
Poland 164
Serbia 159
Egypt 146
Hungary 137
Brazil 122
United States of America 121
Canada 115
South Africa 115
Other (73 other unique countries) 1762

The user agent for every attempt was “Mozilla/5.0 (Windows NT 6.1; rv:19.0) Gecko/20100101 Firefox/19.0″ which is a valid user agent for FF19 running Windows 7. I’m guessing it was someone’s small botnet. Though the distribution of the countries seems a bit off? Maybe it’s a compromised FF addon? If it continues I may need to switch to a whitelist for logins, or move the admin login page elsewhere as current login limit plugin isn’t really suited to an attack from such a large number of IP addresses.


__author__ = 'Matthew'

import pygeoip

ACCESS_FILENAME = 'access.log'

GEOIP = pygeoip.Database('GeoIP.dat')

ip_breakdown = {}
country_breakdown = {}

for line in open(ACCESS_FILENAME):
    if 'wp-login' not in line:

    ip = line[0:line.find(' ')]

    if not ip_breakdown.has_key(ip):
        ip_breakdown[ip] = 0

    ip_breakdown[ip] += 1


    info = GEOIP.lookup(ip)
    if not
        country = 'unknown'
        country =

    if not country_breakdown.has_key(country):
        country_breakdown[country] = 0

    country_breakdown[country] += 1

print 'Number ips:', len(ip_breakdown.keys())
print 'Number countries:', len(country_breakdown.keys())

for country in sorted(country_breakdown, key=country_breakdown.get, reverse=True):
    print country, country_breakdown[country]


Number ips: 2337
Number countries: 92
RU 1786
UA 1225
VN 909
TH 817
TW 613
RO 380
TR 379
BG 342
IR 278
BY 276
IN 241
PL 164
RS 159
EG 146
HU 137
BR 122
US 121
CN 115
SA 115
PK 93
HK 91
CZ 90
ID 82
KZ 78
GE 74
GB 73
SK 63
MD 59
DE 57
AE 56
GR 53
AZ 50
IQ 45
ES 44
BD 43
IL 43
CA 39
AR 36
NL 34
CO 34
CL 32
LV 32
MY 30
AT 29
IT 28
KG 22
BE 21
DK 19
JP 18
QA 17
ZA 15
LK 15
AU 15
LT 14
MX 13
MO 12
KH 12
PH 11
NO 11
PS 10
SE 10
MA 9
MN 9
CY 9
AM 9
BA 8
PY 8
FR 8
GH 7
EE 7
YE 7
DZ 7
PT 6
A2 6
OM 4
HR 4
EC 4
NZ 4
LB 4
LA 3
JO 2
PE 2
MK 2
MZ 2
BH 1
AO 1
ET 1
UZ 1
NP 1
CD 1
SY 1
SD 1

Posted on

MetaBright PHP Challenge

MetaBright is a quiz/community-challenges website focusing on programming and development. I recently found out about a new general PHP section and gave it a go. Overall it was a fun diversion, though a handful of questions were a little odd (like the hash size for RIPEMD vs Whirlpool vs GOST vs MD5). I managed to get to second place on the leader-board, and with a itsy-little bug managed to take first place (of course this is a new section so there isn’t much competition).

Posted on