Thanks for the reasoned reply. As you saw from point #2 in my comments, I
think you should do this feature. I hope this answers Josh Berkus' concern
about my comments.
You make a very interesting comment which seems to go to the heart of this
design approach:
> About the only thing that would make me want to consider row-based
> replication in core would be if we determine that read-only slave
> queries are impractical atop a WAL-log-shipping implementation.
It's possible I'm misunderstanding some of the implementation issues, but it
is striking that the detailed responses to your proposal list a number of
low-level dependencies between master and slave states when replicating WAL
records. It appears that you are designing a replication mechanism that
works effectively between a master and a relatively small number of "nearby"
slaves. This is clearly an important use case but it also seems clear that
the WAL approach is not a general-purpose approach to replication. In other
words, you'll incrementally get to that limited end point I describe. This
will still leave a lot to be desired on read scaling, not to mention many
other cases.
Hence my original comments. However, rather than harp on that further I
will open up a separate thread to describe a relatively small set of
extensions to PostgreSQL that would be enabling for a wide range of
replication applications. Contrary to popular opinion these extensions are
actually well understood at the theory level and have been implemented as
prototypes as well as in commercial patches multiple times in different
databases. Those of us who are deeply involved in replication deserve just
condemnation for not stepping up and getting our thoughts out on the table.
Meanwhile, I would be interested in your reaction to these thoughts on the
scope of the real-time WAL approach. There's obviously tremendous interest
in this feature. A general description that goes beyond the NTT slides
would be most helpful for further discussions.
Cheers, Robert
P.s., The NTT slides were really great. Takahiro and Masao deserve
congratulations on an absolutely first-rate presentation.
On 5/29/08 9:09 PM, "Tom Lane" <tgl@sss.pgh.pa.us> wrote:
> Andrew Sullivan <ajs@commandprompt.com> writes:
>> On Thu, May 29, 2008 at 12:05:18PM -0700, Robert Hodges wrote:
>>> people are starting to get religion on this issue I would strongly
>>> advocate a parallel effort to put in a change-set extraction API
>>> that would allow construction of comprehensive master/slave
>>> replication.
>
>> You know, I gave a talk in Ottawa just last week about how the last
>> effort to develop a comprehensive API for replication failed.
>
> Indeed, core's change of heart on this issue was largely driven by
> Andrew's talk and subsequent discussion. We had more or less been
> waiting for the various external replication projects to tell us
> what they wanted in this line, and it was only the realization that
> no such thing was likely to happen that forced us to think seriously
> about what could be done within the core project.
>
> As I said originally, we have no expectation that the proposed features
> will displace the existing replication projects for "high end"
> replication problems ... and I'd characterize all of Robert's concerns
> as "high end" problems. We are happy to let those be solved outside
> the core project.
>
> About the only thing that would make me want to consider row-based
> replication in core would be if we determine that read-only slave
> queries are impractical atop a WAL-log-shipping implementation.
> Which could happen; in fact I think that's the main risk of the
> proposed development plan. But I also think that the near-term
> steps of the plan are worth doing anyway, for various other reasons,
> and so we won't be out too much effort if the plan fails.
>
> regards, tom lane
>
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
