In order to improve the performance, I made the system look ahead in the source, in groups of a thousand entries, so instead of running: SELECT * FROM table WHERE field = 'something'; a thousand times, we now run: SELECT * FROM table WHERE field IN ('something', 'something else'...); with a thousand things in the IN. Very simple query. It does run faster than the individual queries, but it still takes quite a while. Here is an example query:
Have you considered temporary tables? Use COPY to put everything you want to query into a temporary table, then SELECT to join the results, and pull all of the results, doing additional processing (UPDATE) as you pull?
Cheers,
mark
-- Mark Mielke <mark@mielke.cc>
No comments:
Post a Comment