Thursday, July 17, 2008

[GENERAL] Reducing memory usage of insert into select operations?

Redhat 4
postgresql 8.3.3
Memory: 12GB

While doing a couple of operations of the type
insert into <table> select from <some other table>

The OS triggered the out of memory killer (oom-killer).

After some research and trial/error I found it was the inserts.
I see one of the inserts is using up 12GB!

How can I reduce the usage?
Postgresql.conf settings.
shared_buffers = 3GB
temp_buffers = 64MB # min 800kB
work_mem = 256MB # min 64kB
maintenance_work_mem = 1GB

Reducing work_mem would help?

The table I am selecting from has a few hundred million rows.
The table I am inserting into has partitions. I am benchmarking breaking up
a large table into smaller partitions.

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

No comments: