Thursday, April 1, 2010

Copying dumps from Postgres to MonetDB

When moving bulk data from postgres to monetdb you can use the psql \copy directive and the mclient copy like so:

\copy gftp_packets to 'my dump file' with csv
copy into gram_packets from 'my dump file' delimiters ',','\n' ;

Booleans don't work, but a little python will fix it up:

fin = open('my dump file', 'r')
fout = open('my new dump file', 'w')
falserep = (x.replace(',f,',',0,') for x in fin)
truerep = (x.replace(',t,',',1,') for x in falserep)
for line in truerep: fout.write(line)

I was too lazy to fix the problem with successive booleans, so I just rinsed and repeated until all the t's and f's were 0's and 1's.