> Recent discussions with the PostGIS hackers led me to think about ways
> to reduce overhead when the same TOAST value is repeatedly detoasted.
> In the example shown here
> http://archives.postgresql.org/pgsql-hackers/2008-06/msg00384.php
> 90% of the runtime is being consumed by repeated detoastings of a single
> datum. That is certainly an outlier case, but we've heard complaints of
> TOAST being slow before. The solution I'm about to propose should fix
> this, and as a bonus it will reduce the problem of memory leakage when
> a detoasted value doesn't get pfreed.
>
> What I am imagining is that the tuple toaster will maintain a cache of
> recently-detoasted values...
> Comments, better ideas? Anyone think this is too much trouble to take
> for the problem?
You've not covered the idea that we just alter the execution so we just
detoast once. If we tried harder to reduce the number of detoastings
then we would benefit all of the cases you mention, including internal
decompression. We would use memory, yes, but then so would a cache of
recently detoasted values.
If we see that the index scan key is toastable/ed then the lowest levels
of the plan can create an expanded copy of the tuple and pass that
upwards. We may need to do this in a longer lived context and explicitly
free previous tuples to avoid memory bloat, but we'd have the same
memory usage and same memory freeing issues as with caching. It just
seems more direct and more obvious, especially since it is just an
internal version of the workaround, which was to create a function to
perform early detoasting. Maybe this could be done inside the IndexScan
node when a tuple arrives with toasted values(s) for the scan key
attribute(s).
I presume there's various reasons why you've ruled that out, but with
such a complex proposal it seems worth revisiting the alternatives, even
if just to document them for the archives.
--
Simon Riggs
PostgreSQL Training, Services and Support
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
No comments:
Post a Comment