Monday, August 4, 2008

Re: [GENERAL] recovery via base + WAL replay failure

Rob Adams wrote:
> I'm trying to demonstrate recovery using the "continuous archiving"
> backup technique. I'm using 8.3 on Windows.
>
> I made a base backup while the postgres was running using the following
> batch file:
>
> --------------------------
> psql -d test_database -U user_name -c "SELECT pg_start_backup('test');"
>
> 7za a -tzip "C:\backup\base.zip" "C:\Program Files\PostgreSQL\8.3\data"
>
> psql -d test_database -U user_name -c "SELECT pg_stop_backup();"
> --------------------------
>
> No indication of any errors. However, I am unable to recover. Here are
> my steps:
>
> 1. Stop service, replace the data directory w/ the base backup.
> 2. Create recovery.conf with this setting:
> restore_command = 'copy C:\backup\%f "%p"'
> 3. Try to start service
>
> After about 90 seconds, the dos prompt displays:
> "(postgres 8.3) service could not be started"
> "The service did not report an error"
>
> Does anyone know what I am probably doing wrong?

You'll need to look at the PostgreSQL logs to see what they say.

But I'm curious about the 7za step. It gives no errors or warnings?
IIRC, you need to use VSS on Windows to avoid sharing violations when
opening the files unless a specific open mode is used. It could be that
7za uses that by default, but it's also possible tha tit's not actually
backing up all files...

//Magnus

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

[pgsql-it-generale] == Postgres Weekly News - 3 agosto 2008 ==

Quando leggerete questo, la commitfest di luglio sarà finita.
Tante nuove funzionalità sono state introdotte.
Nella commitfest di settembre prestate attenzione alle Common Table
Expressions e alle Windowing Functions.

== Postgres News prodotti ==

Rilasciato Npgsql2 RC1.
http://www.npgsql.org

Rilasciato pgbouncer 1.2.1.
http://pgfoundry.org/projects/pgbouncer/

== PostgreSQL news locali ==

Il pgDay a San Francisco ci sarà il 5 agosto. Schedule:
http://pugs.postgresql.org/node/447
Registratevi qui:
http://www.linuxworldexpo.com/live/12/ehall//SN460564

Il Prato Linux User Group terrà dei talk su PostgreSQL talks a Settembre.
Il calendario in italiano si trova all'url:
http://www.prato.linux.it/serate_a_tema_2008

Sponsorizzate il PGDay Europeo!
http://www.pgday.org/en/sponsors/campaign

E' iniziato il call for papers per il PGDay europeo.
http://www.pgday.org/en/call4papers

Il PGCon Brasile 2008 ci sara' il 26 e 27 settembre 2008 a Unicamp, Campinas.
http://pgcon.postgresql.org.br/index.en.html

Il PgDay.fr ci sarà il 4 ottobrea a Toulouse. Il Call for Papers è aperto:
http://www.postgresqlfr.org/?q=node/1686
Per registrarsi:
http://www.pgday.fr/doku.php/inscription

Il PGDay.(IT|EU) 2008 ci sarà il 17 e 18 ottobre a Prato.
http://www.pgday.org/it/

== News su PostgreSQL ==

Planet PostgreSQL: http://www.planetpostgresql.org/

General Bits, archivi e nuovi articoli occasionali:
http://www.varlena.com/GeneralBits/

PostgreSQL Weekly News è stato spedito questa settimana grazie a David Fetter.

Per segnalare news e annunci invia un email in inglese entro le ore 15,
fuso orario della costa orientale degli U.S.A, di domenica.

Per segnalazioni in inglese david@fetter.org, per segnalazioni in
Tedesco pwn@pgug.de, per segnalazioni in italiano pwn@itpug.org


== Patch applicate ==

Tom Lane committed:

- Update 8.1 and 8.0 plpython to work with Python 2.5. This backports
several fixes made during the 8.2 development cycle, but not
backported at the time for lack of confidence in the new coding. I
didn't touch 7.4 because it has more problems than this: the
configure probe for Python fails.

- Add a new, improved version of citext as a contrib module. David E.
Wheeler.

- Replace the hard-wired type knowledge in TypeCategory() and
IsPreferredType() with system catalog lookups, as was foreseen to be
necessary almost since their creation. Instead put the information
into two new pg_type columns, typcategory and typispreferred. Add
support for setting these when creating a user-defined base type.
The category column is just a "char" (i.e. a poor man's enum),
allowing a crude form of user extensibility of the category list:
just use an otherwise-unused character. This seems sufficient for
foreseen uses, but we could upgrade to having an actual category
catalog someday, if there proves to be a huge demand for custom type
categories. In this patch I have attempted to hew exactly to the
behavior of the previous hardwired logic, except for introducing new
type categories for arrays, composites, and enums. In particular
the default preferred state for user-defined types remains TRUE.
That seems worth revisiting, but it should be done as a separate
patch from introducing the infrastructure. Likewise, any adjustment
of the standard set of categories should be done separately.

- Adjust citext to make use of the new ability to declare its type
category: by putting it into the standard string category, we cause
casts from citext to text to be recognized as "preferred" casts.
This eliminates the need for creation of alias functions and
operators that only serve to prevent ambiguous-function errors; get
rid of the ones that were in the original commit.

- Flip the default typispreferred setting from true to false. This
affects only type categories in which the previous coding made
*every* type preferred; so there is no change in effective behavior,
because the function resolution rules only do something different
when faced with a choice between preferred and non-preferred types
in the same category. It just seems safer and less surprising to
have CREATE TYPE default to non-preferred status ...

- Allow I/O conversion casts to be applied to or from any type that is
a member of the STRING type category, thereby opening up the
mechanism for user-defined types. This is mainly for the benefit of
citext, though; there aren't likely to be a lot of types that are
all general-purpose character strings. Per discussion with David
Wheeler.

- Require superuser privilege to create base types (but not
composites, enums, or domains). This was already effectively
required because you had to own the I/O functions, and the I/O
functions pretty much have to be written in C since we don't let PL
functions take or return cstring. But given the possible security
consequences of a malicious type definition, it seems prudent to
enforce superuser requirement directly. Per recent discussion.

- Fix parser so that we don't modify the user-written ORDER BY list in
order to represent DISTINCT or DISTINCT ON. This gets rid of a
longstanding annoyance that a view or rule using SELECT DISTINCT
will be dumped out with an overspecified ORDER BY list, and is one
small step along the way to decoupling DISTINCT and ORDER BY enough
so that hash-based implementation of DISTINCT will be possible. In
passing, improve transformDistinctClause so that it doesn't reject
duplicate DISTINCT ON items, as was reported by Steve Midgley a
couple weeks ago.

- Rearrange the querytree representation of ORDER BY/GROUP BY/DISTINCT
items as per my recent proposal: 1. Fold SortClause and GroupClause
into a single node type SortGroupClause. We were already relying on
them to be struct-equivalent, so using two node tags wasn't
accomplishing much except to get in the way of comparing items with
equal(). 2. Add an "eqop" field to SortGroupClause to carry the
associated equality operator. This is cheap for the parser to get
at the same time it's looking up the sort operator, and storing it
eliminates the need for repeated not-so-cheap lookups during
planning. In future this will also let us represent GROUP/DISTINCT
operations on datatypes that have hash opclasses but no btree
opclasses (ie, they have equality but no natural sort order). The
previous representation simply didn't work for that, since its only
indicator of comparison semantics was a sort operator. 3. Add a
hasDistinctOn boolean to struct Query to explicitly record whether
the distinctClause came from DISTINCT or DISTINCT ON. This allows
removing some complicated and not 100% bulletproof code that
attempted to figure that out from the distinctClause alone. This
patch doesn't in itself create any new capability, but it's
necessary infrastructure for future attempts to use hash-based
grouping for DISTINCT and UNION/INTERSECT/EXCEPT.

- In pgsql/src/test/regress/pg_regress.c, fix copy-and-pasteo that's
causing pg_regress to lie about which file it can't read when the
--temp-config argument is bad. Noted while wondering why buildfarm
member dungbeetle is failing ... this isn't why, but it is why the
error report isn't very helpful ...

- In pgsql/src/backend/catalog/pg_type.c, tighten up the sanity checks
in TypeCreate(): pass-by-value types must have a size that is one of
the supported values, not just anything <= sizeof(Datum).
Cross-check the alignment specification against size as well.

- Make GROUP BY work properly for datatypes that only support hashing
and not sorting. The infrastructure for this was all in place
already; it's only necessary to fix the planner to not assume that
sorting is always an available option.

Magnus Hagander committed:

- In pgsql/doc/src/sgml/install-win32.sgml, document which versions of
ActivePerl and ActiveTcl are required for building on MSVC, and that
the free distribution is enough (no need for the enterprise
version). Per gripe from Martin Zaun.

- In pgsql/doc/src/sgml/install-win32.sgml, clean up reference to
config.pl so it makes sense not only in SGML source but in the
actual web/pdf viewer...

- Move ident authentication code into auth.c along with the other
authenciation routines, leaving hba.c to deal only with processing
the HBA specific files.

- In pgsql/src/backend/libpq/auth.c, rearrange the code in auth.c so
that all functions for a single authentication method is grouped
together in a reasonably similar way, keeping the "global shared
functions" together in their own section as well. Makes it a lot
easier to find your way around the code.

Bruce Momjian committed:

- Add URL for TODO: "Consider decreasing the I/O caused by updating
tuple hint bits."

Alvaro Herrera committed:

- Add a few more DTrace probes to the backend. Robert Lor.

- Cope with Tcl versions that do not create a tclsh symlink to the
version- numbered program. Per persistent buildfarm failures. Tom
Lane.

== Patch rigettate (per ora) ==
Nessuno è stato scontentato questa settimana :-)

== Patch in attesa ==

Tatsuo Ishii sent in another revision of Yoshiuki Asaba's CTE patch.

Hitoshi Harada sent in another revision of his windowing functions
patch.

Jan Urbanski sent in two more revisions of his patch to add
selectivity functions for text search types.

KaiGai Kohei sent in another revision of his SE-PostgreSQL patch.

Abhijit Menon-Sen sent in two more revisions of his \ef (edit
function) patch for psql along with a patch to add
pg_get_functiondef() as an SQL-callable function.

Robert Lor sent in two more revisions of his DTrace probes patch.

ITAGAKI Takahiro sent in a patch to add STORAGE and reloptions to
CREATE TABLE ... LIKE.

Alvaro Herrera sent in two updates to Robert Lor's DTrace probes
patch.

ITAGAKI Takahiro sent in a patch to use NDirectFileRead and
NDirectFileWrite statistics counters for counting reads and writes in
BufFile.

Heikki Linnakangas sent in new revisions of his Relation Forks and FSM
rewrite patches, the latter depending on the former.


--
(all opinions expressed are my own)
Federico Campoli
PostgreSQL Consulting -> PGHost http://www.pghost.eu

--
Sent via pgsql-it-generale mailing list (pgsql-it-generale@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-it-generale

Re: [HACKERS] Mini improvement: statement_cost_limit

On Sun, Aug 03, 2008 at 10:57:55PM -0400, Robert Treat wrote:
>
> ISTR that what ended up killing the enthusiasm for this was that most people
> realized that this GUC was just a poor tool to take a stab at solving other
> problems (ie. rate limiting cpu for queries).

I'm not concerned with that, I want developers to have feed back on costs in
a way that is obvious.

> > > I think a variation on this could be very useful in development and test
> > > environments. Suppose it raised a warning or notice if the cost was over
> > > the limit. Then one could set a limit of a few million on the development
> > > and test servers and developers would at least have a clue that they
> > > needed to look at explain for that query. As it is now, one can exhort
> > > them to run explain, but it has no effect. Instead we later see queries
> > > killed by a 24 hour timeout with estimated costs ranging from "until they
> > > unplug the machine and dump it" to "until the sun turns into a red
> > > giant".
> >
> > Great argument. So that's 4 in favour at least.
> >
>
> Not such a great argument. Cost models on development servers can and often
> are quite different from those on production, so you might be putting an
> artifical limit on top of your developers.

We load the production dumps into our dev environment, which are the same
hardware spec, so the costs should be identical.

> I still think it is worth revisiting what problems people are trying to solve,
> and see if there are better tools they can be given to solve them. Barring
> that, I suppose a crude solution is better than nothing, though I fear people
> might point at the crude solution as a good enough solution to justify not
> working on better solutions.

Alerting developers and QA to potentially costly queries would help solve
some of the probems we are trying to solve. Better tools are welcome, an
argument that the good is the enemy of the best so we should be content with
nothing is not.

-dg

--
David Gould daveg@sonic.net 510 536 1443 510 282 0869
If simplicity worked, the world would be overrun with insects.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] unnecessary code in_bt_split

On Sun, 2008-08-03 at 19:44 -0400, Tom Lane wrote:
> Zdenek Kotala <Zdenek.Kotala@Sun.COM> writes:
> > I found that _bt_split function calls PageGetTempPage, but next call is
> > _bt_page_init which clear all contents anyway. Is there any reason to call
> > PageGetTempPage instead of palloc?
>
> Not violating a perfectly good abstraction?
>
> I agree that PageGetTempPage isn't amazingly efficient, but internal
> refactoring would halve its cost; and if you have some evidence that
> there's a real performance issue then we could think about adjusting
> the temp-page API to allow _bt_pageinit to be combined with it. But
> I have a real problem with hacking up _bt_split so that it will call
> PageRestoreTempPage on something it didn't get from PageGetTempPage.
>
> Considering the WAL and regular I/O that will be induced by a split,
> I kinda doubt this is even worth worrying about anyway...

Improving this should help, since the existing page is write locked
during _bt_split. The I/O won't happen at the point that these blocks
are critical contention points.

--
Simon Riggs www.2ndQuadrant.com
PostgreSQL Training, Services and Support


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Initial Unsigned Integer data type performance test results.

Hello All,

I wanted to pass some performance data on to the group regarding the
unsigned integer
data types I am working on. I tested on two systems running Ubuntu
Hardy. The first system
is an 8 x 2.66GHz x86-64 processor system. The second system is a 2 x
533 celeron i386
system. For this test I disabled autovaccuum and manually ran the
vacuum during times that
were not timed. I also added some sleep and sync commands to try and
stabalize the test
results. The integer and unsigned integer test results were also
interleaved to compensate for
other system activities. Finally, I have not done a lot of bench
marking, so feedback and
improvement ideas are welcome!

I benchmarked four tests:
32-bit int2 vs uint2 -- My expectation is these results
would be approximately
-- equal. The uint2 casts
up to an int4 which is still 32-bits.

32-bit int4 vs uint4 -- My expectation is <> operator
should be approximately
-- equal since I explicitly
added int4 <> uint4 operators.

64-bit int2 vs uint2 -- My expectation is these results
would be approximately
-- equal. The uint2 casts
up to an int4 which is less than
-- the 64-bit processor bus width.

64-bit int4 vs uint4 -- My expectation is these results
would be approximately
-- equal. The uint4 casts
up to an int8 which is the processor
-- bus width.

Here is the definition of each column (times are displayed in seconds):

Rows -- The number of rows added to the table.
Test -- The test number (I ran each benchmark 3 times).
S Load -- Time to load the table with signed integers.
U Load -- Time to load the table with unsigned integers.
S <> OP -- Time to perform an <> operator on each row in the
table (signed integers)
U <> OP -- Time to perform an <> operator on each row in the
table (unsigned integers)
S & OP -- Time to perform an AND operator on each row in
the table (signed integers)
U & OP -- Time to perform an AND operator on each row in
the table (unsigned integers)


I still have some more investigating to do into the results of this
data (i.e. why is the signed <> OP cheaper for int2 than uint2 types,
etc),
but I wanted to share this performance data with the community. Rough
calculations show about a 4% performance hit for 32-bit processors
to use the int8 type and a 6% performance hit for 64-bit processors.
This seems like a reasonable trade-off for the reduced storage space
I am after, and the reduction in operators Greg and Tom were concerned
about. If there are no objections from the community, I will plan to
complete my implementation of Tom's proposal at:
http://archives.postgresql.org/pgsql-hackers/2008-07/msg01204.php.

Thanks!

- Ryan

P.S. The code I have so far is not ready for review. It was
implemented quickly to get performance numbers to determine if Tom's
proposal
would be acceptable by the community. I can send it out if people are
interested, but I am planning on submitting it to a future commit fest
once I have it cleaned up and better tested.

Benchmark Data:
==================================

32-bit int2 vs. uint2

Rows Test S Load U Load S <> OP U <> OP S
& OP U & OP
--------------------------------------------------------------------------------------
100 1 000.002888 000.002151 000.010881 000.014691
000.011124 000.011000
100 2 000.002780 000.002127 000.011729 000.011611
000.012014 000.011925
100 3 000.002747 000.002085 000.010193 000.010318
000.010588 000.010576

1000 1 000.003201 000.003870 000.037837 000.037360
000.032064 000.032478
1000 2 000.003259 000.003912 000.033495 000.036281
000.032502 000.035195
1000 3 000.003201 000.003913 000.039156 000.035592
000.032405 000.040543

10000 1 000.024683 000.021306 000.255958 000.329045
000.255887 000.283782
10000 2 000.020214 000.021224 000.260252 000.290933
000.281468 000.255171
10000 3 000.020371 000.020940 000.276401 000.264791
000.257598 000.257258

100000 1 001.669571 001.687523 002.591442 002.682428
003.410724 003.490362
100000 2 001.682251 001.702598 003.379377 002.855622
002.549476 002.583431
100000 3 001.693429 001.684732 002.546024 002.641240
002.540556 003.366534

500000 1 010.138317 011.014532 015.707597 015.699991
015.394598 015.502639
500000 2 010.042176 010.179163 015.290994 015.407479
015.332925 016.321578
500000 3 010.047930 010.206489 015.016276 015.430527
015.201759 015.411601

1000000 1 020.762680 022.145950 030.338606 030.997060
030.971243 031.191622
1000000 2 020.645634 021.480215 031.222100 031.044017
031.489913 031.145342
1000000 3 020.908854 021.612863 031.735141 031.378030
033.323409 031.141103

5000000 1 108.538334 109.016628 151.732077 174.010438
183.003246 191.651814
5000000 2 109.040097 108.891102 152.354661 167.138746
183.423955 193.882820
5000000 3 108.302860 109.358920 151.745248 165.654813
182.462140 178.345020

10000000 1 217.707537 220.407657 313.047263 360.890670
415.392358 413.828828
10000000 2 217.200579 219.571731 307.769243 362.820028
404.587188 412.644460
10000000 3 218.502164 225.425290 308.132310 351.845443
401.132601 411.143661


32-bit int4 vs. uint4

Rows Test S Load U Load S <> OP U <> OP S
& OP U & OP
--------------------------------------------------------------------------------------
100 1 000.001379 000.004707 000.017991 000.004928
000.005532 000.023357
100 2 000.001192 000.008546 000.007557 000.004901
000.020157 000.008509
100 3 000.001211 000.009639 000.007729 000.004634
000.020458 000.008506

1000 1 000.003032 000.024415 000.049310 000.038847
000.028273 000.033758
1000 2 000.003094 000.023647 000.053854 000.029543
000.027895 000.030389
1000 3 000.003089 000.023558 000.049282 000.031464
000.028082 000.030534

10000 1 000.022861 000.206507 000.450483 000.294142
000.270429 000.285299
10000 2 000.020130 000.218892 000.452674 000.265420
000.294738 000.280119
10000 3 000.020216 000.207972 000.498028 000.264987
000.269354 000.307225

100000 1 001.654366 002.222839 003.158899 002.711246
002.797589 003.223520
100000 2 001.635678 002.117398 003.184728 002.687387
002.732426 002.908151
100000 3 001.641013 002.228674 003.122391 002.737217
002.726568 002.854913

500000 1 010.517882 011.035098 015.867863 015.819520
016.651296 017.366898
500000 2 010.301478 010.890427 015.898800 015.694182
016.374195 017.139048
500000 3 010.308812 011.336613 016.010874 015.694566
016.172569 017.050617

1000000 1 021.748016 022.032479 031.718041 031.377681
032.836890 034.495359
1000000 2 021.668741 022.764860 031.585905 031.485409
032.597464 034.408479
1000000 3 021.797422 022.287219 031.501989 031.310628
032.891124 034.352729

5000000 1 112.363704 113.290859 156.780144 157.629888
169.741430 178.738164
5000000 2 111.912800 113.728177 155.814622 157.592556
169.063996 177.893192
5000000 3 113.204484 113.338839 155.942430 157.708378
168.438944 177.544739

10000000 1 225.834990 227.816448 314.965601 318.707127
343.067000 355.863962
10000000 2 227.519247 227.054232 315.291132 318.536013
340.437748 356.954030
10000000 3 226.436991 227.728902 316.953600 319.432702
341.731871 356.278195


64-bit int2 vs. uint2

Rows Test S Load U Load S <> OP U <> OP S
& OP U & OP
--------------------------------------------------------------------------------------
100 1 000.000264 000.000378 000.005095 000.004019
000.004019 000.003967
100 2 000.000234 000.000149 000.005168 000.007987
000.008077 000.008046
100 3 000.000283 000.000422 000.000898 000.001001
000.003046 000.003973

1000 1 000.000472 000.000406 000.005944 000.004000
000.008402 000.006523
1000 2 000.000472 000.000614 000.003220 000.007445
000.008007 000.007899
1000 3 000.000434 000.000522 000.004744 000.003945
000.003904 000.003978

10000 1 000.003583 000.002897 000.027068 000.024923
000.026969 000.027948
10000 2 000.002681 000.002879 000.023959 000.028943
000.028847 000.024163
10000 3 000.002713 000.002826 000.027868 000.024297
000.027626 000.027936

100000 1 000.147438 000.158207 000.181544 000.281516
000.216321 000.256955
100000 2 000.122763 000.158806 000.262845 000.262513
000.206650 000.242393
100000 3 000.165053 000.137694 000.199290 000.200812
000.190772 000.231532

500000 1 000.861558 001.857679 001.201773 001.141055
001.146796 001.196645
500000 2 000.751839 001.769231 001.124357 001.096142
001.154715 001.156669
500000 3 000.757383 000.746437 001.046433 001.043038
001.092662 001.188915

1000000 1 001.624921 001.501079 002.219032 002.221471
002.351225 002.259619
1000000 2 001.635181 001.490060 002.132329 002.179557
002.230080 002.230306
1000000 3 001.642645 001.578584 002.204351 002.204351
002.271596 002.251025

5000000 1 008.175122 007.608278 012.993807 024.737981
032.636002 023.532173
5000000 2 008.402995 007.788409 022.455597 022.722125
023.375628 024.364674
5000000 3 008.146802 007.560563 011.000594 023.479316
023.127872 023.647729

10000000 1 027.417696 025.444743 035.633955 044.665248
056.828230 053.103862
10000000 2 026.629527 015.461252 033.425061 047.231670
058.463149 059.917299
10000000 3 026.714370 015.133970 033.187095 045.359560
059.858288 060.539286


64-bit int4 vs. uint4

Rows Test S Load U Load S <> OP U <> OP S
& OP U & OP
--------------------------------------------------------------------------------------
100 1 000.000283 000.000146 000.000887 000.000648
000.003931 000.000884
100 2 000.000285 000.000174 000.000884 000.000690
000.000652 000.000725
100 3 000.000225 000.000137 000.000657 000.003389
000.000738 000.000720

1000 1 000.000507 000.000409 000.002872 000.002924
000.003109 000.005706
1000 2 000.000458 000.000386 000.005279 000.003111
000.003007 000.002984
1000 3 000.000526 000.000458 000.003005 000.003102
000.007094 000.003343

10000 1 000.003591 000.002837 000.026670 000.025760
000.025237 000.026846
10000 2 000.002800 000.002690 000.023960 000.029733
000.031866 000.026354
10000 3 000.002718 000.002645 000.023977 000.024667
000.024566 000.034090

100000 1 000.149074 000.134729 000.212927 000.231054
000.236246 000.267968
100000 2 000.133630 000.116737 000.224492 000.218364
000.197834 000.213061
100000 3 000.142504 000.111011 000.239096 000.227658
000.213063 000.246546

500000 1 000.758492 000.716535 001.122045 001.088708
001.189956 001.211962
500000 2 000.771229 000.762520 001.084013 001.131540
001.171574 001.171354
500000 3 000.784780 000.694077 001.110563 001.207835
001.148557 001.239808

1000000 1 001.538802 001.519817 002.143364 002.182663
002.347049 002.496311
1000000 2 001.588864 001.438750 002.196512 002.236763
002.378072 002.443330
1000000 3 001.565230 001.535085 002.127375 002.468330
002.325026 002.470508

5000000 1 008.095642 007.514390 020.867962 011.222382
024.817883 025.539325
5000000 2 008.279959 007.822827 023.514454 024.288419
025.346532 025.263223
5000000 3 008.313125 007.794659 023.427362 023.892352
026.224463 024.725105

10000000 1 025.903194 027.249769 048.426016 045.101048
051.549006 052.169178
10000000 2 025.365287 026.901960 046.175783 045.509540
049.219715 052.168185
10000000 3 027.882902 028.379166 047.237950 048.097603
049.311239 051.762996

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Mini improvement: statement_cost_limit

On Sun, 2008-08-03 at 22:57 -0400, Robert Treat wrote:

> I still think it is worth revisiting what problems people are trying
> to solve, and see if there are better tools they can be given to solve
> them. Barring that, I suppose a crude solution is better than
> nothing, though I fear people might point at the crude solution as a
> good enough solution to justify not working on better solutions.

I advocate solutions to the problems of users I've worked with.

My preference is to help people in the next release, then improve from
there. We need to work with what we have.

In this case, an existing solution has been found.

--
Simon Riggs www.2ndQuadrant.com
PostgreSQL Training, Services and Support


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [pgsql-de-allgemein] Erlaubte Zeichen in Datenbanknamen

udono schrieb:
>>> Gibt es noch weitere erlaubte Zeichen oder andere Sachen die ich bei
>>> der Prüfung beachten muss?

Ich glaube, Deine Beschreibung (63 alphanumerische Zeichen) ist nicht
korrekt: es sind 64 *Bytes* (inklusive dem 0-Byte am Ende), und alle
Zeichen sind möglich.

Siehe bpchar_name in src/backend/utils/adt/varchar.c.

Allerdings ist es intelligent, die Eingabe auf Kleinbuchstaben, Ziffern und
Unterstriche zu beschränken.

>> IIRC nicht, aber ich versteh nicht ganz, warum Du den Namen einer DB als
>> Benutzereingabe prüfen willst/mußt. Der DB-Name ist doch eher, ähm,
>> statisch.
>
> Ja, bei der Software handelt es sich um ein ERP (Enterprice Resource
> Planning) Tool für kleine und mittelgroße Unternehmen.
> Wenn man eine neue einzelne Firma anlegt, dann werden alle Daten in
> eine Postgresdatenbank eingefügt. Deren Name muss der Administrator
> angeben um sie neu erstellen zu lassen. Wir haben diesen Dialog in
> das GUI gelegt, um die Konfiguration zu vereinfachen.
> Bei der Eingabe des Datenbank-Namens wird gleich auf Validität
> geprüft. (Also es sind nur alpha-nummerische Zeichen erlaubt und
> Unterstrich, das erste Zeichen muss ein Buchstabe sein, die Anzahl
> der eingegebenen Zeichen ist auf 63 begrenzt.) Quasi
> Fehlervermeidung, statt Fehlermeldung...

Ein Bedenken gegen das Design möchte ich kundtun:

Das bedeutet, daß das Programm mit einem User mit hohen Privilegien
(CREATEDB oder gar SUPERUSER) an die Datenbank verbunden sein muß.

Vielen Leuten macht das vielleicht nichts aus, aber es gibt Menschen
(und Unternehmen), für die sowas ein Ausschließungsgrund für eine
Software ist.

Warum kann das nicht einfach ein neues Schema statt einer neuen
Datenbank sein?

Liebe Grüße,
Laurenz Albe

--
Sent via pgsql-de-allgemein mailing list (pgsql-de-allgemein@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-de-allgemein

Sunday, August 3, 2008

Re: [pgsql-es-ayuda] retornar varias filas de un pl pgsql

2008/8/3 Ana María Mandiola Fernández <amandiola@gmail.com>:
> HOLa !!
> soy nuevo en lenguaje de bdatos, alguien que me ayude con una funcion que
> devuelva mas de un registro de una tabla, he intentado con record,
> refcursores y ninguna me sale , a lo mas tiro una fila pero tengo 8 filas,
> registros tuplas, en mi tabla prestadora, como lo hago??' tengo esta funcion
> q devuelve una sola fila.
>

nuevo, eh? leer el manual no hace daño ;)

> CREATE OR REPLACE FUNCTION p5()
> RETURNS record AS
> $BODY$DECLARE
> algo RECORD;
> BEGIN
>
> SELECT INTO algo * FROM prestadora;
>
> return algo;
> END;$BODY$
> LANGUAGE 'plpgsql'
>

CREATE OR REPLACE FUNCTION p5() RETURNS SETOF prestadora AS
$BODY$

SELECT INTO algo * FROM prestadora;

$BODY$
LANGUAGE 'sql'

ahora que si lo quieres hacer en plpgsql (posiblemente porque quieras
procesar datos antes de devolverlos)

CREATE OR REPLACE FUNCTION p5() RETURNS SETOF prestadora AS
$BODY$
DECLARE
algo RECORD;
BEGIN
for algo in SELECT * FROM prestadora loop
return next algo;
end loop;
return;
END;
$BODY$
LANGUAGE 'plpgsql'

o en 8.3

CREATE OR REPLACE FUNCTION p5() RETURNS SETOF prestadora AS
$BODY$
BEGIN
return query select * from prestadora;
END;
$BODY$
LANGUAGE 'plpgsql'

o con cursores o de 1001 formas mas...

la clave estuvo en el SETOF

--
Atentamente,
Jaime Casanova
Soporte y capacitación de PostgreSQL
Guayaquil - Ecuador
Cel. (593) 87171157
--
TIP 8: explain analyze es tu amigo

Re: [GENERAL] bytea encode performance issues

Tom Lane wrote:
> Could we see EXPLAIN ANALYZE, not EXPLAIN? Without actual facts
> to work from, any suggestions would be mere guesswork.

This was taken immediately after a vacuum analyze on the database.

"HashAggregate (cost=41596.68..41596.84 rows=16 width=764) (actual
time=488263.802..488263.837 rows=40 loops=1)"
" -> Nested Loop (cost=0.00..41596.60 rows=16 width=764) (actual
time=23375.445..488260.311 rows=40 loops=1)"
" -> Nested Loop (cost=0.00..41463.32 rows=16 width=780)
(actual time=23375.344..488231.994 rows=40 loops=1)"
" -> Seq Scan on dbmail_messageblks k
(cost=0.00..39193.21 rows=259 width=764) (actual time=30.662..486585.126
rows=2107 loops=1)"
" Filter: ((is_header = 0::smallint) AND
(encode(messageblk, 'escape'::text) ~~ '%Yossi%'::text))"
" -> Index Scan using dbmail_messages_2 on dbmail_messages
m (cost=0.00..8.75 rows=1 width=16) (actual time=0.777..0.777 rows=0
loops=2107)"
" Index Cond: (m.physmessage_id = k.physmessage_id)"
" Filter: ((mailbox_idnr = 8) AND (status = ANY
('{0,1}'::integer[])))"
" -> Index Scan using dbmail_physmessage_pkey on
dbmail_physmessage p (cost=0.00..8.32 rows=1 width=8) (actual
time=0.701..0.703 rows=1 loops=40)"
" Index Cond: (k.physmessage_id = p.id)"
"Total runtime: 488264.192 ms"


> Also, what can you tell us about the sizes of the messageblk
> strings (max and avg would be interesting)?
>
select max(length(messageblk)),avg(length(messageblk)) from
dbmail_messageblks
MAX AVG
532259; 48115.630147120314

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: [pgsql-es-ayuda] Insertar

siempre manda una copia a la lista

2008/8/3 Luis Martínez <luisedgardomartinezlinares@gmail.com>:
>>
>> ¿Estoy usando una sentencia asi y me da error insert into
>> prueba2.prueba2.prueba select * from prueba1.public.prueba1? y obtengo el
>> siguiente error
>
> ERROR: cross-database references are not implemented:
> "prueba2.prueba2.prueba2"
> Estado SQL:0A000
> donde uso la declaracion de <base de datos>.<schema>.<tabla> quisiera saber
> que hago mal

me parece bastante claro, no puedes usar sentencias entre bases de
datos... para eso debes usar dblink o plproxy en modo de conexion

http://www.postgresql.org/docs/8.3/static/dblink.html
https://developer.skype.com/SkypeGarage/DbProjects/PlProxy


--
Atentamente,
Jaime Casanova
Soporte y capacitación de PostgreSQL
Guayaquil - Ecuador
Cel. (593) 87171157
--
TIP 4: No hagas 'kill -9' a postmaster

Re: [GENERAL] [OT] newsreader issue? (Was: bytea encode performance issues)

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I got the response by email, but it was also addressed to the mailing
list. My response was also an email sent to the mailing list, not to the
newsgroup. I got one other response, that I do not see in the newsgroup.
And I will reply to it also using the mailing list.

Sim

Lew wrote:
> Sim Zacks wrote:
>> (quoting someone:)
>>> That LIKE operator is probably your problem. An unbounded LIKE like that
>>> (with a wildcard on both sides) means no index can be used, hence you
>>> get a sequential scan.
>
> Was the message to which you responded posted to the newsgroup? It
> isn't appearing in my newsreader.
>
> Who wrote the message you quoted (you failed to cite the source)?
>

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkiWk20ACgkQjDX6szCBa+pwzwCffdE3KZAg0f2TjUiq+gFCOrML
HM4An2wcV9G9aAE+94DH6Vwc6deMIHB4
=N0v/
-----END PGP SIGNATURE-----

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: [HACKERS] Mini improvement: statement_cost_limit

Josh Berkus wrote:
> Tom,
>
>
>> Wasn't this exact proposal discussed and rejected awhile back?
>>
>
> We rejected Greenplum's much more invasive resource manager, because it
> created a large performance penalty on small queries whether or not it was
> turned on. However, I don't remember any rejection of an idea as simple
> as a cost limit rejection.
>
> This would, IMHO, be very useful for production instances of PostgreSQL.
> The penalty for mis-rejection of a poorly costed query is much lower than
> the penalty for having a bad query eat all your CPU.
>
>
Greenplum's introduced a way to creating a cost "threshold" a bit like
the way Simon was going to do "shared" work_mem. It did 2 things:

1/ Counted the cost of an about-to-be run query against the threshold,
and made the query wait if it would exhaust it
2/ Aborted the query if its cost was greater than the threshold

Initially there was quite a noticeable performance penalty with it
enabled - but as the guy working on it (me) redid bits and pieces then
penalty decreased massively. Note that in all cases, disabling the
feature meant there was no penalty.

The latest variant of the code is around in the Bizgres repository
(src/backend/utils/resscheduler I think) - some bits might be worth
looking at!

Best wishes

Mark

P.s : I'm not working for Greenplum now.

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[pgsql-es-ayuda] retornar varias filas de un pl pgsql

HOLa !!
soy nuevo en lenguaje de bdatos, alguien que me ayude con una funcion que devuelva mas de un registro de una tabla, he intentado con record, refcursores y ninguna me sale , a lo mas tiro una fila pero tengo 8 filas, registros tuplas, en mi tabla prestadora, como lo hago??' tengo esta funcion q devuelve una sola fila.

CREATE OR REPLACE FUNCTION p5()
RETURNS record AS
$BODY$DECLARE
algo RECORD;
BEGIN

SELECT INTO algo * FROM prestadora;

return algo;
END;$BODY$
LANGUAGE 'plpgsql'


ahora como lo hago para que la funcion me devuelva el SELECT * FROM prestadora; completo??
de antemano gracias!!

Re: [ADMIN] client_encoding error on load


On Sun, Aug 3, 2008 at 9:38 PM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
"Andrew Goodnough" <agoodno@gmail.com> writes:
> I'm using these commands to perform a dump and load.
> pg_dump -c lakevie_typo | gzip > $HOME/backups/typo_$DATE.sql.gz
> gzip < $HOME/backups/typo_$DATE.sql.gz |psql -d lakevie_typobak -f -

Surely the second step must start with gunzip?

                       regards, tom lane

 
Yeah, that's gotta be it.  Dumb error, thanks.

Andy


Re: [ADMIN] client_encoding error on load

It will better you export and import through network by using following command.
psql -U < schemaname >  < databasename>   --schema=schemaname | psql  -U postgres  -h < IP adddress of system where you are importing data> <database name>
----- Original Message -----
Sent: Monday, August 04, 2008 7:29 AM
Subject: [ADMIN] client_encoding error on load

I'm using these commands to perform a dump and load.

pg_dump -c lakevie_typo | gzip > $HOME/backups/typo_$DATE.sql.gz
gzip < $HOME/backups/typo_$DATE.sql.gz |psql -d lakevie_typobak -f -

The dump is fine but the load only works if I first extract the sql, then run it.  Running it using the pipe to psql, I get:

psql:<stdin>:71: ERROR:  invalid byte sequence for encoding "UTF8": 0x8b
HINT:  This error can also happen if the byte sequence does not match the encoding expected by the server, which is controlled by "client_encoding".

yet, doing a 'psql -l' shows this database to be UTF-8.  And the dump file says:

SET client_encoding = 'UTF8';


My goal is to get the two line dump/load working so I don't have any manual steps and sql files laying around.  Any ideas?

Andy

Re: [HACKERS] small bug in hlCover

On Mon, 2008-08-04 at 00:36 -0300, Euler Taveira de Oliveira wrote:
> Sushant Sinha escreveu:
> > I think there is a slight bug in hlCover function in wparser_def.c
> >
> The bug is not in the hlCover. In prsd_headline, if we didn't find a
> suitable bestlen (i.e. >= 0), than it includes up to document length or
> *maxWords* (here is the bug). I'm attaching a small patch that fixes it
> and some comment typos. Please apply it to 8_3_STABLE too.

Well hlCover purpose is to find a cover and for the document '1 2 3 4 5
6 7 8 9 10' and the query '1'::tsquery, a cover exists. So it should
point it out.

On my source I see that prsd_headline marks only min_words which seems
like the right thing to do.

-Sushant.

>
> euler=# select ts_headline('1 2 3 4 5 6 7 8 9 10','1'::tsquery,
> 'MinWords=5');
> ts_headline
> -----------------------------
> <b>1</b> 2 3 4 5 6 7 8 9 10
> (1 registro)
>
> euler=# select ts_headline('1 2 3 4 5 6 7 8 9 10','1'::tsquery);
> ts_headline
> -----------------------------
> <b>1</b> 2 3 4 5 6 7 8 9 10
> (1 registro)
>
>


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Instructions for adding new catalog

"Gustavo Tonini" <gustavotonini@gmail.com> writes:
> Well, just for documenting the process...
> Adding new postgres catalog in 2 little steps:

> 1)Write catalog header file and save it into "src/include/catalog"
> directory. Hint: copy format from other catalog headers.
> 2)Add your header file name to variable POSTGRES_BKI_SRCS in file
> "src/backend/catalog/Makefile"

As long as we're documenting ...

* Documenting a new catalog in catalogs.sgml is not considered optional.
* Most likely, your catalog needs some indexes, which should be declared
in catalog/indexing.h.
* Of course, the reason you are defining a new system catalog is that
the C code is going to know about it. So there is a whole lot of
other stuff you probably need to write: object insertion code,
object deletion code, lookup code (maybe a new syscache or two).
Traditionally the lowest-level insertion/deletion code goes into
catalog/pg_yourcatalog.c, while slightly higher-level code such as
the implementation of new utility commands to manage this new kind
of object will go elsewhere.

That's before you get to any actual new functionality...

regards, tom lane

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Mini improvement: statement_cost_limit

On Sunday 03 August 2008 15:12:22 Simon Riggs wrote:
> On Sun, 2008-08-03 at 00:44 -0700, daveg wrote:
> > On Sat, Aug 02, 2008 at 09:30:08PM +0200, Hans-Jürgen Schönig wrote:
> > > On Aug 2, 2008, at 8:38 PM, Tom Lane wrote:
> > > >Andrew Dunstan <andrew@dunslane.net> writes:
> > > >>Hans-Jürgen Schönig wrote:
> > > >>>i introduced a GUC called statement_cost_limit which can be used to
> > > >>>error out if a statement is expected to be too expensive.
> > > >>
> > > >>You clearly have far more faith in the cost estimates than I do.
> > > >
> > > >Wasn't this exact proposal discussed and rejected awhile back?
> > >
> > > i don't remember precisely.
> > > i have seen it on simon's wiki page and it is something which would
> > > have been useful in some cases in the past.
>
> I still support it. Regrettably, many SQL developers introduce product
> joins and other unintentional errors. Why let problem queries through?
> Security-wise they're great Denial of Service attacks, bringing the
> server to its knees better than most ways I know, in conjunction with a
> nice hefty work_mem setting. 27 table product joins: memory, CPU, I/O
> and diskspace resources used all in a simple killer query.
>

ISTR that what ended up killing the enthusiasm for this was that most people
realized that this GUC was just a poor tool to take a stab at solving other
problems (ie. rate limiting cpu for queries).

> If anybody thinks costs are inaccurate, don't use it. Or better still
> improve the cost models. It isn't any harder or easier to find a useful
> value than it is to use statement_timeout. What's the difference between
> picking an arbitrary time and an arbitrary cost? You need to alter the
> value according to people's complaints in both cases.
>

I think the original argument for statement_timeout was that long running
queries were known to cause have wrt vacuum strategies (remember, that one
has been in the back end a long time). ISTR some recent threds on -hackers
questioning whether statement_timeout should be eliminated itself.

> > I think a variation on this could be very useful in development and test
> > environments. Suppose it raised a warning or notice if the cost was over
> > the limit. Then one could set a limit of a few million on the development
> > and test servers and developers would at least have a clue that they
> > needed to look at explain for that query. As it is now, one can exhort
> > them to run explain, but it has no effect. Instead we later see queries
> > killed by a 24 hour timeout with estimated costs ranging from "until they
> > unplug the machine and dump it" to "until the sun turns into a red
> > giant".
>
> Great argument. So that's 4 in favour at least.
>

Not such a great argument. Cost models on development servers can and often
are quite different from those on production, so you might be putting an
artifical limit on top of your developers.

> A compromise would be to have log_min_statement_cost (or
> warn_min_statement_cost) which will at least help find these problems in
> testing before we put things live, but that still won't help with
> production issues.
>
> Another alternative would be to have a plugin that can examine the plan
> immediately after planner executes, so you can implement this yourself,
> plus some other possibilities.
>

I still think it is worth revisiting what problems people are trying to solve,
and see if there are better tools they can be given to solve them. Barring
that, I suppose a crude solution is better than nothing, though I fear people
might point at the crude solution as a good enough solution to justify not
working on better solutions.

--
Robert Treat
Build A Brighter LAMP :: Linux Apache {middleware} PostgreSQL

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [pgsql-es-ayuda] Insertar



2008/8/3 Jaime Casanova <jcasanov@systemguards.com.ec>
On Sun, Aug 3, 2008 at 2:16 PM, Luis Martínez
<luisedgardomartinezlinares@gmail.com> wrote:
> Buenas amigos
> Mi pregunta es ¿Puedo insertar valores du na tabla en BD1 a Otra tabla de
> BD2? si es asi como lo hago gracias por la atencion

si las dos bases de datos son postgres, podrias probar plproxy


--
Atentamente,
Jaime Casanova
Soporte y capacitación de PostgreSQL
Guayaquil - Ecuador
Cel. (593) 87171157

Como es eso del plproxy,  y lo de las conexiones dblink
--
Ing. Luis E Martínez L
telf 0412-962-46-13

Re: [pgsql-es-ayuda] Insertar

On Sun, Aug 3, 2008 at 2:16 PM, Luis Martínez
<luisedgardomartinezlinares@gmail.com> wrote:
> Buenas amigos
> Mi pregunta es ¿Puedo insertar valores du na tabla en BD1 a Otra tabla de
> BD2? si es asi como lo hago gracias por la atencion

si las dos bases de datos son postgres, podrias probar plproxy


--
Atentamente,
Jaime Casanova
Soporte y capacitación de PostgreSQL
Guayaquil - Ecuador
Cel. (593) 87171157
--
TIP 10: no uses HTML en tu pregunta, seguro que quien responda no podrá leerlo

Re: [GENERAL] pgsql-general@postgresql.org.

Hi.

Sorry, it was not included in release.
please see,
http://winpg.jp/~saito/pg_work/OSSP_win32/

Regards,
Hiroshi Saito

>Hi all,
>
>I installed postgresql-8.3.3-1 for win2000 with UUID-OSSP. Following
>the documentation I issued this query:
>
>SELECT uuid_generate_v3(uuid_ns_url(), 'http://www.postgresql.org');
>
>but the DB returns with this error message:
>
>ERROR: function uuid_ns_url() does not exist
>SQL state: 42883
>Hint: No function matches the given name and argument types. You might
>need to add explicit type casts.
>Character: 25
>
>Any hint to use UUID within my database tirggers?
>
>Thank you,
>Laci
>
>
>--
>Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
>To make changes to your subscription:
>http://www.postgresql.org/mailpref/pgsql-general

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: [pgsql-de-allgemein] In Funktion prüfen ob Zeile existiert

Oliver Baer <oliver.baer@presswatch.de> wrote:

> [...]
> Ich stelle mir einfach eine IF abfrage vor:

> IF insert schlägt fehl THEN update ...

Aufgrund einer Abneigung gegen Exceptions schreibe ich so
etwas meist in der Form:

| UPDATE Table SET Something = 'something' WHERE ID = 'id';
| IF NOT FOUND THEN
| INSERT INTO Table (ID, Something) VALUES ('id', 'something');
| END IF;

Tim

--
Sent via pgsql-de-allgemein mailing list (pgsql-de-allgemein@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-de-allgemein

Re: [GENERAL] uuid

On Aug 3, 2008, at 1:18 PM, grafl@datatrans.hu wrote:
> ERROR: function uuid_ns_url() does not exist

Remember to install the functions in your database using the SQL file
in the contrib/uuid-ossp directory, uuid-ossp.sql.

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

UUID in windows was Re: [GENERAL] pgsql-general@postgresql.org.

Pretty sure the uuid stuff only works in non-windows OSes.

Oh, and next time, use a more descriptive subject line for better /
quicker responses.

On Sun, Aug 3, 2008 at 2:18 PM, <grafl@datatrans.hu> wrote:
> Hi all,
>
> I installed postgresql-8.3.3-1 for win2000 with UUID-OSSP. Following the
> documentation I issued this query:
>
> SELECT uuid_generate_v3(uuid_ns_url(), 'http://www.postgresql.org');
>
> but the DB returns with this error message:
>
> ERROR: function uuid_ns_url() does not exist
> SQL state: 42883
> Hint: No function matches the given name and argument types. You might need
> to add explicit type casts.
> Character: 25
>
> Any hint to use UUID within my database tirggers?
>
> Thank you,
> Laci
>
>
> --
> Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-general
>

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: [pgsql-de-allgemein] In Funktion prüfen ob Zeile existiert

2008/7/31, Oliver Baer <oliver.baer@presswatch.de>:
> Hallo,
>
> ich habe hier einen teil einer Postgresfunktion welche eine Tabelle updated.
>
> ...
> LOOP
> -- RAISE NOTICE 'notify_id %', asset_notify.notify_id;
> -- RAISE NOTICE 'asset_id %', NEW.asset_id;
> -- RAISE NOTICE 'label_id %', asset_notify.label_id;
> INSERT INTO asset.asset_labels (label_id, asset_id, state,
> login_id) values (asset_notify.label_id, NEW.asset_id, -2,
> asset_notify.login_id);
> END LOOP;
> ...
>
> Das problem welches ich gerade habe ist folgendes.
>
> Es kann sein das mit das der insert auf die tabelle fehlschlägt weil
> dort schon etwas drinsteht (unique auf asset_id, label_id). Das würde
> ich gerne abfangen, das heisst wenn der insert fehlschlägt soll er ein
> update machen.
>
> Leider kenne ich mit Funktionen in Postgres nicht so gut aus, vielleicht
> hilft mir ja hier jemand auf die sprünge...
>
> Ich stelle mir einfach eine IF abfrage vor:
>
> IF insert schlägt fehl THEN update ...
>
> Danke schonmal,

Mir würde da als erstes Exceptions einfallen:
http://www.postgresql.org/docs/8.3/interactive/plpgsql-control-structures.html#PLPGSQL-ERROR-TRAPPING

Vielleicht hilft Dir das ja schon mal weiter.

>
> Oli
>
Robert

--
Sent via pgsql-de-allgemein mailing list (pgsql-de-allgemein@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-de-allgemein

[HACKERS] unnecessary code in_bt_split

I found that _bt_split function calls PageGetTempPage, but next call is
_bt_page_init which clear all contents anyway. Is there any reason to call
PageGetTempPage instead of palloc?

Original code:

00796 leftpage = PageGetTempPage(origpage, sizeof(BTPageOpaqueData));
00797 rightpage = BufferGetPage(rbuf);
00798
00799 _bt_pageinit(leftpage, BufferGetPageSize(buf));

Suggested code:

00796 leftpage = palloc(PageGetSize(origpage));
00797 rightpage = BufferGetPage(rbuf);
00798
00799 _bt_pageinit(leftpage, BufferGetPageSize(buf));


Any idea?

thanks Zdenek

--
Zdenek Kotala Sun Microsystems
Prague, Czech Republic http://sun.com/postgresql


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [pgsql-es-ayuda] Insertar

On Sun, Aug 3, 2008 at 3:16 PM, Luis Martínez
<luisedgardomartinezlinares@gmail.com> wrote:
> Buenas amigos
> Mi pregunta es ¿Puedo insertar valores du na tabla en BD1 a Otra tabla de
> BD2? si es asi como lo hago gracias por la atencion

Hola, dblink soluciona tu pregunta. La documentación:

http://www.postgresql.org/docs/current/static/dblink.html

Alguien ya pregunto esto y en este post se le contesto:

http://archives.postgresql.org/pgsql-es-ayuda/2008-05/msg00612.php

--
Marco Antonio Frias Butrón
Slackware Linux User
Linux Registered User #356229
--
TIP 9: visita nuestro canal de IRC #postgresql-es en irc.freenode.net

Re: [HACKERS] Mini improvement: statement_cost_limit

On Sun, 2008-08-03 at 22:09 +0200, Hans-Jürgen Schönig wrote:

> > Another alternative would be to have a plugin that can examine the
> > plan
> > immediately after planner executes, so you can implement this
> > yourself,
> > plus some other possibilities.
> >

> this would be really fancy.
> how could a plugin like that look like?

Hmm...thinks: exactly like the existing planner_hook().

So, rewrite this as a planner hook and submit as a contrib module.

--
Simon Riggs www.2ndQuadrant.com
PostgreSQL Training, Services and Support


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] patch: Add a separate TRUNCATE permission

On Tue, 2008-07-29 at 09:03 -0400, Stephen Frost wrote:
> * Peter Eisentraut (peter_e@gmx.net) wrote:
> > Am Tuesday, 29. July 2008 schrieb Stephen Frost:
> > > I'd certainly like to see a truncate permission, I wrote a patch for it
> > > myself back in January of 2006. That thread starts here:
> > >
> > > http://archives.postgresql.org/pgsql-patches/2006-01/msg00028.php
> > >
> > > I think someone else submitted a patch for it last year too, actually.

It was raised in January and rejected again then. Glad to see it raised
again here. I believe Tom's previous concerns about allowing truncate
permissions were related to the potentially increased number of truncate
commands this would cause and the need to tune invalidation messages.
That's done now.

> > Well, that certainly indicates some demand.
> >
> > I think we should worry about adding more bits when we need them. It's
> > certainly possible to add more bits, so it is not like we need to save the
> > ones we have forever.
>
> I would agree with this. Others?

I've no problem with running out of bits. At this rate, we have enough
some for some years yet. But I don't see too many additional permissions
coming along anyhow.

--
Simon Riggs www.2ndQuadrant.com
PostgreSQL Training, Services and Support


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[GENERAL] pgsql-general@postgresql.org.

Hi all,

I installed postgresql-8.3.3-1 for win2000 with UUID-OSSP. Following
the documentation I issued this query:

SELECT uuid_generate_v3(uuid_ns_url(), 'http://www.postgresql.org');

but the DB returns with this error message:

ERROR: function uuid_ns_url() does not exist
SQL state: 42883
Hint: No function matches the given name and argument types. You might
need to add explicit type casts.
Character: 25

Any hint to use UUID within my database tirggers?

Thank you,
Laci


--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: [HACKERS] Mini improvement: statement_cost_limit

hello ...


>
> I still support it. Regrettably, many SQL developers introduce product
> joins and other unintentional errors. Why let problem queries through?


i think the killer is that we don't have to wait until the query dies
with a statement_timeout.
it is ways more elegant to kill things before they have already eaten
too many cycles.
one thing which is important as well: statement_cost_limit does not
kill queries which have just been waiting for a lock.
this makes things slightly more predictable.


> Security-wise they're great Denial of Service attacks, bringing the
> server to its knees better than most ways I know, in conjunction
> with a
> nice hefty work_mem setting. 27 table product joins: memory, CPU, I/O
> and diskspace resources used all in a simple killer query.
>


i am not too concerned about DNS, i have to admit.
i would rather see it as a way to make developers do better things.


> If anybody thinks costs are inaccurate, don't use it. Or better still
> improve the cost models. It isn't any harder or easier to find a
> useful
> value than it is to use statement_timeout. What's the difference
> between
> picking an arbitrary time and an arbitrary cost? You need to alter the
> value according to people's complaints in both cases.


the cost model is good enough to see if something is good or bad.
this is basically all we want to do here --- killing all evil.


> *snip*


>
>
> A compromise would be to have log_min_statement_cost (or
> warn_min_statement_cost) which will at least help find these
> problems in
> testing before we put things live, but that still won't help with
> production issues.
>


definitely. a good idea as well - but people will hardly read it, i
guess :(.


> Another alternative would be to have a plugin that can examine the
> plan
> immediately after planner executes, so you can implement this
> yourself,
> plus some other possibilities.
>


this would be really fancy.
how could a plugin like that look like?

hans

--
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt
Web: www.postgresql-support.de


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[pgsql-es-ayuda] Insertar

Buenas amigos
Mi pregunta es ¿Puedo insertar valores du na tabla en BD1 a Otra tabla de BD2? si es asi como lo hago gracias por la atencion
--
Ing. Luis E Martínez L
telf 0412-962-46-13

Re: [HACKERS] small bug in hlCover

Has any one noticed this?

-Sushant.

On Wed, 2008-07-16 at 23:01 -0400, Sushant Sinha wrote:
> I think there is a slight bug in hlCover function in wparser_def.c
>
> If there is only one query item and that is the first word in the text,
> then hlCover does not returns any cover. This is evident in this example
> when ts_headline only generates the min_words:
>
> testdb=# select ts_headline('1 2 3 4 5 6 7 8 9 10','1'::tsquery,
> 'MinWords=5');
> ts_headline
> ------------------
> <b>1</b> 2 3 4 5
> (1 row)
>
> The problem is that *q is initialized to 0 which is a legitimate value
> for a cover. So I have attached a patch that fixes it and after applying
> the patch here is the result.
>
> testdb=# select ts_headline('1 2 3 4 5 6 7 8 9 10','1'::tsquery,
> 'MinWords=5');
> ts_headline
> -----------------------------
> <b>1</b> 2 3 4 5 6 7 8 9 10
> (1 row)
>
> -Sushant.


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Mini improvement: statement_cost_limit

On Sun, 2008-08-03 at 00:44 -0700, daveg wrote:
> On Sat, Aug 02, 2008 at 09:30:08PM +0200, Hans-Jürgen Schönig wrote:
> > On Aug 2, 2008, at 8:38 PM, Tom Lane wrote:
> >
> > >Andrew Dunstan <andrew@dunslane.net> writes:
> > >>Hans-Jürgen Schönig wrote:
> > >>>i introduced a GUC called statement_cost_limit which can be used to
> > >>>error out if a statement is expected to be too expensive.
> > >
> > >>You clearly have far more faith in the cost estimates than I do.
> > >
> > >Wasn't this exact proposal discussed and rejected awhile back?
> > >
> > i don't remember precisely.
> > i have seen it on simon's wiki page and it is something which would
> > have been useful in some cases in the past.

I still support it. Regrettably, many SQL developers introduce product
joins and other unintentional errors. Why let problem queries through?
Security-wise they're great Denial of Service attacks, bringing the
server to its knees better than most ways I know, in conjunction with a
nice hefty work_mem setting. 27 table product joins: memory, CPU, I/O
and diskspace resources used all in a simple killer query.

If anybody thinks costs are inaccurate, don't use it. Or better still
improve the cost models. It isn't any harder or easier to find a useful
value than it is to use statement_timeout. What's the difference between
picking an arbitrary time and an arbitrary cost? You need to alter the
value according to people's complaints in both cases.

> I think a variation on this could be very useful in development and test
> environments. Suppose it raised a warning or notice if the cost was over
> the limit. Then one could set a limit of a few million on the development
> and test servers and developers would at least have a clue that they needed
> to look at explain for that query. As it is now, one can exhort them to
> run explain, but it has no effect. Instead we later see queries killed
> by a 24 hour timeout with estimated costs ranging from "until they unplug
> the machine and dump it" to "until the sun turns into a red giant".

Great argument. So that's 4 in favour at least.

A compromise would be to have log_min_statement_cost (or
warn_min_statement_cost) which will at least help find these problems in
testing before we put things live, but that still won't help with
production issues.

Another alternative would be to have a plugin that can examine the plan
immediately after planner executes, so you can implement this yourself,
plus some other possibilities.

--
Simon Riggs www.2ndQuadrant.com
PostgreSQL Training, Services and Support


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

[COMMITTERS] pgsql: Make GROUP BY work properly for datatypes that only support

Log Message:
-----------
Make GROUP BY work properly for datatypes that only support hashing and not
sorting. The infrastructure for this was all in place already; it's only
necessary to fix the planner to not assume that sorting is always an available
option.

Modified Files:
--------------
pgsql/src/backend/optimizer/plan:
planmain.c (r1.107 -> r1.108)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/optimizer/plan/planmain.c?r1=1.107&r2=1.108)
planner.c (r1.236 -> r1.237)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/optimizer/plan/planner.c?r1=1.236&r2=1.237)
pgsql/src/backend/parser:
parse_clause.c (r1.172 -> r1.173)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/parser/parse_clause.c?r1=1.172&r2=1.173)

--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Re: [pgsql-es-ayuda] Resultado de now()

On Sun, Aug 3, 2008 at 10:34 AM, luis alfonso cañas
<lucho.canas@gmail.com> wrote:
> Hola a todos.
>
> Requiero el conocimiento de ustedes para deteminar la causa por la cual la
> funcion now() me devuelve siempre la misma hora.
>
yo creo que lo que realmente necesitas es leer el manual:
http://www.postgresql.org/docs/8.3/static/functions-datetime.html#FUNCTIONS-DATETIME-CURRENT


Todas estas devuelven valores basandose en la fecha de *inicio de la
transaccion* actual:
CURRENT_DATE
CURRENT_TIME
CURRENT_TIMESTAMP
CURRENT_TIME(precision)
CURRENT_TIMESTAMP(precision)
LOCALTIME
LOCALTIMESTAMP
LOCALTIME(precision)
LOCALTIMESTAMP(precision)
now()
transaction_timestamp()

Esta devuelve valores basandose en la fecha de *inicio de la sentencia* actual
statement_timestamp()

Estas devuelven las fechas reales actuales (según el reloj de tu
maquina me parece):
clock_timestamp()
timeofday()

Para lo que tu quieres deberias usar statement_timestamp() o clock_timestamp()

--
Atentamente,
Jaime Casanova
Soporte y capacitación de PostgreSQL
Guayaquil - Ecuador
Cel. (593) 87171157
--
TIP 7: no olvides aumentar la configuración del "free space map"

Re: [GENERAL] Advice on implementing counters in postgreSQL

On Sun, Aug 3, 2008 at 1:50 AM, Marco Bizzarri <marco.bizzarri@gmail.com> wrote:
> On Sun, Aug 3, 2008 at 1:40 AM, Berend Tober <btober@ct.metrocast.net> wrote:
>>
>>
>> The way I understand the documentation at
>>
>> "http://www.postgresql.org/docs/8.3/static/transaction-iso.html"
>>
>> and
>>
>> 'http://www.postgresql.org/docs/current/static/explicit-locking.html',
>>
>> you should not have to use the serial isolation level.
>>
>> I would define the counter table so as to hold the last-used value, rather
>> that the "next" value, and then do the UPDATE first.
>>
>> As a consequence, assuming all this happens within a transaction of course,
>> the SELECT FOR UPDATE syntax is not required either because the UPDATE will
>> grab a lock on the row and block other updates until the transaction is
>> finished. That is, concurrency is protected and you don't have to restart
>> any transactions because subsequent transactions will just wait until the
>> first one finishes due to nature of the lock automatically acquired by the
>> initial UPDATE statement.
>>
>
> Yes, I'm considering moving away from serializable; the problem is
> that I have to explore all the implications of this on my code. Up to
> now, I wrote considering a serializable level, so I think I should do
> quite a review to be sure about it.

A fairly simple test shows that you can do this in read committed:

S1: # show transaction_isolation;
read committed

(setup a table for the value)
# create table t (i int);
# insert into t values (5);


S1: # begin;
S1: # update t set i=i+1;
S2: # update t set i=i+1;
(S2 now waits for S1)
S1: # select i from t;
6
S1: # commit;
(S2 now can continue...)
S2: # select i from t;
7
S2: # commit;

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

[pgsql-es-ayuda] Resultado de now()

Hola a todos.
 
Requiero el conocimiento de ustedes para deteminar la causa por la cual la funcion now() me devuelve siempre la misma hora.
 
Tengo una funcion que su ejecucion toma varios minutos, por que efectua muchas actividades.
 
Antes de cada actividad inserto un registro en una tabla de bitacora.
Ejemplo
 
insert into bitacora (actividad,fecha) values('Actividad nro 1',now());
....
insert into bitacora (actividad,fecha) values('Actividad nro 2',now());
 
....
insert into bitacora (actividad,fecha) values('Actividad nro 3',now());
 
Al verificar  la tabla de bitacora una vez termino la ejecucion, encuentro que la todos los registros tienen el mismo dato en el campo fecha.
fecha es del tipo timestamp without time zone.
 
Por que razon sucede esto?
 
Que puedo hacer para que en cada operacion de insert en la bitacora me almacene la hora exacta?
De que otra manera podria obtener los mismos resultados?
 
Esto lo requiero para poder determinar que actividades son las que estan tomando mas tiempo y asi poderlas custumizar.
 
Muchas gracias de antemando.
 
Att
Luis Alfonso Cañas
Colombia.
 
 

[COMMITTERS] pgsql: Tighten up the sanity checks in TypeCreate(): pass-by-value types

Log Message:
-----------
Tighten up the sanity checks in TypeCreate(): pass-by-value types must have
a size that is one of the supported values, not just anything <= sizeof(Datum).
Cross-check the alignment specification against size as well.

Modified Files:
--------------
pgsql/src/backend/catalog:
pg_type.c (r1.120 -> r1.121)
(http://anoncvs.postgresql.org/cvsweb.cgi/pgsql/src/backend/catalog/pg_type.c?r1=1.120&r2=1.121)

--
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Re: [GENERAL] bytea encode performance issues

Sim Zacks <sim@compulab.co.il> writes:
> The explain of the query is:

Could we see EXPLAIN ANALYZE, not EXPLAIN? Without actual facts
to work from, any suggestions would be mere guesswork.

Also, what can you tell us about the sizes of the messageblk
strings (max and avg would be interesting)?

regards, tom lane

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] bytea encode performance issues

On Aug 3, 2008, at 2:36 PM, Sim Zacks wrote:

> The LIKE operator is likely the problem, but it is a critical part
> of an
> email application. Searches are done by, "Show me all emails
> containing
> the following word."
>
> I've tried using TSearch2's full text index. It made the query 50%
> faster, taking 5 minutes. This is still not even close to the less
> then
> a minute in MySQL.
>
>>
>> That LIKE operator is probably your problem. An unbounded LIKE
>> like that
>> (with a wildcard on both sides) means no index can be used, hence you
>> get a sequential scan.
>>
>> There are apparently some possibilities with the new GIN indexes (or
>> maybe even using GIST), but I haven't had an opportunity to try those
>> yet. There were some messages about just that on this list recently.
>>
>
> I don't think a functional index (or anything other then a FTI) would
> accomplish anything, being that I am doing unbounded Likes.

That's why I suggested to use a text field instead of bytea.
IIRC, You can have an index on word triplets and use tsearch. I don't
have intimate knowledge on how that works though, hopefully other
people will chime in here.

Without the need to convert each row before comparing it, and with an
appropriate index, that should significantly speed up your queries.

>> If you create an index, make sure you create a _functional_ index
>> over
>> ENCODE(messageblk, 'escape').
>>
>
> Email is binary when it contains attachments. I actually planned on
> using an ASCII encoding, but the dbmail people specifically said
> not to.
> I don't know if they were speaking from experience, or because ASCII
> sounds bad.

It shouldn't be; those attachments are MIME or UU encoded, are they not?
Don't confuse ASCII and SQLASCII. The latter accepts characters from
any encoding, which is probably what you want.

>> Since when is e-mail binary data? I don't quite see why you'd use a
>> bytea field instead of text. If your problem is character
>> encoding, then
>> just don't store that ("encode" the DB using SQLASCII).
>>
>
> As I mentioned, this is the system that came with dbmail. It runs on
> both PostGresql and MySQL, so they may have done some compatibility
> things. There are 4 statuses possible, 0,1,2,3 if you use the database
> through the software then a 99 could never appear there.

The software isn't the only client that might connect to the database.
It is usually bad practice to put data constraint logic in the client
instead of in the database. Especially since in client code there are
usually multiple sections of code that have to deal with those
constraints, which tends to result in small differences in their
handling.

Next to that, if those statuses would have a proper foreign key
constraint, it would be very easy to add labels to each status in a
way they would make a bit more sense than 0, 1, 2, 3.
I expect the label would be a sufficient foreign key by itself
though, no need for those silly numbers.

Well, there's probably not much you can do about that, being just a
user of dbMail.

>> I notice some other oddities in that query/design. Why is is_header a
>> smallint instead of a boolean? I'm assuming this is for compatibility
>> with an other database?
>>
>> Why use status IN (0, 1) instead of more descriptive keys? Is it even
>> constrained this way, or could arbitrary numbers end up as status
>> (say
>> 99) and if so, what happens to those messages?

You have a very odd way of quoting. You don't top post as such, but
you top post in context... Haven't seen that one before. Usually
people reply _below_ a section, not above it ;) It made my reply a
bit harder to write.

Alban Hertroys

--
If you can't see the forest for the trees,
cut the trees and you'll see there is no forest.


!DSPAM:737,4895b34b243488085013917!

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: [HACKERS] Parsing of pg_hba.conf and authentication inconsistencies

* Magnus Hagander (magnus@hagander.net) wrote:
> Stephen Frost wrote:
> > A little extra code in the backend is well worth fixing this foot-gun.
> > The concerns raised so far have been "who will write it?" and "what if
> > it has bugs?". Neither of these are particularly compelling arguments
> > when you've already offered to write and bug-test it (right, Magnus? :).
>
> Toms main argument has been that it would move the code *from* the
> backend and into the *postmaster*, which is much more sensitive.

erm, yes, sorry I wasn't being clear. Brain moving faster than fingers
sometimes. :)

> And yes, I've offered to write the code. I take this as an offer from
> you to bug-test it :-)

Indeed, as long as I'm bug-testing the Kerberos mappings at the same
time... ;) Seriously though, I'd be happy to review and test the code.

Thanks,

Stephen

Re: [HACKERS] Parsing of pg_hba.conf and authentication inconsistencies

* Magnus Hagander (magnus@hagander.net) wrote:
> I think it'd be reasonable to refuse starting if the config is *known
> broken* (such as containing lines that are unparseable, or that contain
> completely invalid tokens), whereas you'd start if they just contain
> things that are "probably wrong". But picking from your previous
> examples of "more advanced checks", there are lots of cases where
> things like overlapping CIDR address ranges are perfectly valid, so I
> don't think we could even throw a warning for that - unless there's a
> separate flag to enable/disable warnings for such a thing.

Agreed. Making sure the config can parse is different from parsable but
non-sensible. It's ridiculously easy to mistakenly add a line w/ a
single character on it or something equally bad when saving a file
that's being modified by hand. That's a simple check that should be
done on re-hup and the broken config shouldn't be put in place.

I certainly agree that we should *also* have a way to just check the
config, so that can be built into init scripts and whatnot. I don't
think having one precludes having the other, and I'm pretty confident we
could find a way to not duplicate the code and have things be clean.

Thanks,

Stephen

Re: [HACKERS] Parsing of pg_hba.conf and authentication inconsistencies

Stephen Frost wrote:
> * Magnus Hagander (magnus@hagander.net) wrote:
>> For pg_hba.conf, I don't see that as a very big problem, really. It
>> doesn't (and shouldn't) modify any "external" variables, so it should be
>> as simple as parsing the new file into a completely separate
>> list-of-structs and only if it's all correct switch the main pointer
>> (and free the old struct).
>
> I'm in agreement with this approach. Allowing a config which won't
> parse properly to completely break access to a running database is
> terrible. It just doesn't come across to me as being all that difficult
> or complex code for pg_hba.conf.

That's my thoughts as well, which may be off of course ;-)


>> Yes, I still think we should do the "simple parsing" step at HUP time.
>> That doesn't mean that it wouldn't be a good idea to have one of these
>> check-config options that can look for conflicting options *as well*, of
>> course. But I'm getting the feeling I'm on the losing side of the debate
>> here...
>
> A little extra code in the backend is well worth fixing this foot-gun.
> The concerns raised so far have been "who will write it?" and "what if
> it has bugs?". Neither of these are particularly compelling arguments
> when you've already offered to write and bug-test it (right, Magnus? :).

Toms main argument has been that it would move the code *from* the
backend and into the *postmaster*, which is much more sensitive.

And yes, I've offered to write the code. I take this as an offer from
you to bug-test it :-)

//Magnus

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Parsing of pg_hba.conf and authentication inconsistencies

* Magnus Hagander (magnus@hagander.net) wrote:
> For pg_hba.conf, I don't see that as a very big problem, really. It
> doesn't (and shouldn't) modify any "external" variables, so it should be
> as simple as parsing the new file into a completely separate
> list-of-structs and only if it's all correct switch the main pointer
> (and free the old struct).

I'm in agreement with this approach. Allowing a config which won't
parse properly to completely break access to a running database is
terrible. It just doesn't come across to me as being all that difficult
or complex code for pg_hba.conf.

> Yes, I still think we should do the "simple parsing" step at HUP time.
> That doesn't mean that it wouldn't be a good idea to have one of these
> check-config options that can look for conflicting options *as well*, of
> course. But I'm getting the feeling I'm on the losing side of the debate
> here...

A little extra code in the backend is well worth fixing this foot-gun.
The concerns raised so far have been "who will write it?" and "what if
it has bugs?". Neither of these are particularly compelling arguments
when you've already offered to write and bug-test it (right, Magnus? :).

Thanks,

Stephen

Re: [GENERAL] bytea encode performance issues

The LIKE operator is likely the problem, but it is a critical part of an
email application. Searches are done by, "Show me all emails containing
the following word."

I've tried using TSearch2's full text index. It made the query 50%
faster, taking 5 minutes. This is still not even close to the less then
a minute in MySQL.

>
> That LIKE operator is probably your problem. An unbounded LIKE like that
> (with a wildcard on both sides) means no index can be used, hence you
> get a sequential scan.
>
> There are apparently some possibilities with the new GIN indexes (or
> maybe even using GIST), but I haven't had an opportunity to try those
> yet. There were some messages about just that on this list recently.
>

I don't think a functional index (or anything other then a FTI) would
accomplish anything, being that I am doing unbounded Likes.

> If you create an index, make sure you create a _functional_ index over
> ENCODE(messageblk, 'escape').
>

Email is binary when it contains attachments. I actually planned on
using an ASCII encoding, but the dbmail people specifically said not to.
I don't know if they were speaking from experience, or because ASCII
sounds bad.

> Since when is e-mail binary data? I don't quite see why you'd use a
> bytea field instead of text. If your problem is character encoding, then
> just don't store that ("encode" the DB using SQLASCII).
>

As I mentioned, this is the system that came with dbmail. It runs on
both PostGresql and MySQL, so they may have done some compatibility
things. There are 4 statuses possible, 0,1,2,3 if you use the database
through the software then a 99 could never appear there.

>
>
> I notice some other oddities in that query/design. Why is is_header a
> smallint instead of a boolean? I'm assuming this is for compatibility
> with an other database?
>
> Why use status IN (0, 1) instead of more descriptive keys? Is it even
> constrained this way, or could arbitrary numbers end up as status (say
> 99) and if so, what happens to those messages?
>
> Alban Hertroys
>
> --
> If you can't see the forest for the trees,
> cut the trees and you'll see there is no forest.
>
>
> !DSPAM:824,48958f30243481673380013!
>
>

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] bytea encode performance issues

On Aug 3, 2008, at 12:12 PM, Sim Zacks wrote:

> This is the query that is used (I know it is not as efficient as it
> could be, but this is the query it comes with):
>
> SELECT m.message_idnr,k.messageblk FROM dbmail_messageblks k JOIN
> dbmail_physmessage p ON k.physmessage_id = p.id JOIN dbmail_messages m
> ON p.id = m.physmessage_id WHERE mailbox_idnr = 8 AND status IN (0,1 )
> AND k.is_header = '0' GROUP BY m.message_idnr,k.messageblk HAVING
> ENCODE(k.messageblk::bytea,'escape') LIKE '%John%'

That LIKE operator is probably your problem. An unbounded LIKE like
that (with a wildcard on both sides) means no index can be used,
hence you get a sequential scan.

There are apparently some possibilities with the new GIN indexes (or
maybe even using GIST), but I haven't had an opportunity to try those
yet. There were some messages about just that on this list recently.

If you create an index, make sure you create a _functional_ index
over ENCODE(messageblk, 'escape').

> The messageblk field is a bytea in postgresql and a longblob in
> mysql.
> The only difference in the query is the MySQL does not need the encode
> function.

Since when is e-mail binary data? I don't quite see why you'd use a
bytea field instead of text. If your problem is character encoding,
then just don't store that ("encode" the DB using SQLASCII).

> Can someone make a suggestion for tuning the database?

An explain analyze would have been a bit more useful, but the biggest
issue is probably the seqscan.

> The explain of the query is:
> "HashAggregate (cost=43648.11..43648.85 rows=74 width=753)"
> " -> Nested Loop (cost=42999.83..43647.74 rows=74 width=753)"
> " -> Merge Join (cost=42999.83..43043.35 rows=74 width=769)"
> " Merge Cond: (k.physmessage_id = m.physmessage_id)"
> " -> Sort (cost=39264.12..39267.59 rows=1388
> width=753)"
> " Sort Key: k.physmessage_id"
> " -> Seq Scan on dbmail_messageblks k
> (cost=0.00..39191.68 rows=1388 width=753)"
> " Filter: ((is_header = 0::smallint) AND
> (encode(messageblk, 'escape'::text) ~~ '%John%'::text))"

Here is your problem, a sequential scan over a presumably large
table. It's either caused by the LIKE expression or by the lack of a
functional index on messageblk, or both.

If you change the type of the messageblk field to text you won't need
a functional index anymore (although that only saves time on index
creation and inserts/updates).

> " -> Sort (cost=3735.71..3754.59 rows=7552 width=16)"
> " Sort Key: m.physmessage_id"
> " -> Bitmap Heap Scan on dbmail_messages m
> (cost=385.98..3249.26 rows=7552 width=16)"
> " Recheck Cond: ((mailbox_idnr = 8) AND
> (status
> = ANY ('{0,1}'::integer[])))"
> " -> Bitmap Index Scan on dbmail_messages_8
> (cost=0.00..384.10 rows=7552 width=0)"
> " Index Cond: ((mailbox_idnr = 8) AND
> (status = ANY ('{0,1}'::integer[])))"
> " -> Index Scan using dbmail_physmessage_pkey on
> dbmail_physmessage p (cost=0.00..8.15 rows=1 width=8)"
> " Index Cond: (k.physmessage_id = p.id)"


I notice some other oddities in that query/design. Why is is_header a
smallint instead of a boolean? I'm assuming this is for compatibility
with an other database?

Why use status IN (0, 1) instead of more descriptive keys? Is it even
constrained this way, or could arbitrary numbers end up as status
(say 99) and if so, what happens to those messages?

Alban Hertroys

--
If you can't see the forest for the trees,
cut the trees and you'll see there is no forest.


!DSPAM:737,48958f34243483105918576!

--
Sent via pgsql-general mailing list (pgsql-general@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general