{
  "version": 1,
  "source": "pitch-finder pg19-not-in runs, each from session/gateway/exobench_calls.jsonl: G2 20261113_025021_phase2-3-cliff-agent-rc2, S2 20261113_015648_phase1-s11-agent-rc2, M2 20261113_161016_1m-equivalent-scripted-rc2, G1 20261112_231953_phase2-3-cliff-agent, S1 20261112_224834_phase1-s11-agent. Local ExoBench 1.4.26, captured 2026-11-12 and 2026-11-13.",
  "note": "Supplemental data for postgres-19-not-in-10000x-faster-or-2000x-slower.mdx. Each mcp-call block is one call captured by a logging gateway in front of a local ExoBench 1.4.26 server (tag v1.4.26) running PostgreSQL 17.11, 18.6 and 19rc2 in Docker on one laptop; runs G1 and S1 ran 19rc1 in place of 19rc2. Server defaults unless a card's own preBenchmarkSql says otherwise: work_mem 4MB, hash_mem_multiplier 2, JIT on in 17.11 and 18.6 and off in PG19. The three G1 cards ran SET jit = off, rc1CampaignWorkMem19 ran SET work_mem = '8MB', and million19 ran SET statement_timeout = '120s'. name, input and result are the gateway log's tool, arguments and response text, byte for byte, correlationId included; plans, timings and buffers are unedited, and the captions are authored. An AI agent designed and ran the calls in G2, S2, G1 and S1; the three M2 calls were scripted by the author. ExoBench ran locally, so these calls have no hosted replay. The chart's points are the Execution Time lines of cards gap17, nullableRescan17, nullable18, nullableRescan18, gap19, gapBeyond19 and antiJoin19, one execution each. Blocks by key, as run and seq: cliff17 G2 seq 12; campaign17 S2 seq 9; campaign19 S2 seq 10; nullableNoNulls19 S2 seq 33; gap19 G2 seq 26; gap17 G2 seq 32; campaignGap19rc1 S1 seq 32; campaign18 S2 seq 11; campaignFilter19 S2 seq 34; campaignNotExists17 S2 seq 21; campaignTimingOff17 S2 seq 55; campaignTimingOff18 S2 seq 56; campaignTimingOff19 S2 seq 57; antiJoin19 G2 seq 16; gapBeyond19 G2 seq 27; nullable18 G2 seq 35; nullableRescan17 G2 seq 38; nullableRescan18 G2 seq 41; cliff18Hashed G2 seq 14; cliff18Rescan G2 seq 15; notExists17 G2 seq 20; notIn19Repeated G2 seq 87; notExists19Repeated G2 seq 88; filtered19 G2 seq 24; ordersNullable19 G2 seq 29; outerFilter19 G2 seq 86; million19 G2 seq 78; equal17 M2 seq 1; equal18 M2 seq 2; equal19 M2 seq 3; rc1Hashed19 G1 seq 25; rc1Rescan19 G1 seq 57; rc1Hashed17 G1 seq 27; rc1Campaign18 S1 seq 41; rc1CampaignWorkMem19 S1 seq 42.",
  "blocks": {
    "cliff17": {
      "type": "mcp-call",
      "name": "benchmarkSql",
      "input": {
        "schemaTemplate": "CREATE TABLE orders (id int PRIMARY KEY, customer_id int NOT NULL, total numeric(10,2) NOT NULL);\nINSERT INTO orders SELECT i, i, (i % 1000) + 0.99 FROM generate_series(1, ${O}) i;\nCREATE TABLE banned (customer_id int NOT NULL);\nINSERT INTO banned SELECT i * 3 FROM generate_series(1, ${B}) i;\nVACUUM ANALYZE;",
        "query": "EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT count(*), sum(total) FROM orders WHERE customer_id NOT IN (SELECT customer_id FROM banned)",
        "scalePoints": [
          {
            "O": 10000,
            "B": 262144
          },
          {
            "O": 10000,
            "B": 262145
          }
        ],
        "databaseType": "POSTGRES",
        "seed": 4242,
        "waitTimeoutSeconds": 100,
        "correlationId": "34110ee7-8c59-4f5a-8afe-029834d88f50",
        "exobenchServerVersion": "1.4.26"
      },
      "result": "{\"timestamp\":\"2026-11-13T02:59:11.895523032Z\",\"databaseType\":\"postgres\",\"sql\":\"EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT count(*), sum(total) FROM orders WHERE customer_id NOT IN (SELECT customer_id FROM banned)\",\"warmCacheIterations\":0,\"seed\":4242,\"seedSource\":\"explicit\",\"scalePointCount\":2,\"totalComputeSeconds\":169.91899999999998,\"results\":[{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":262144},\"output\":\"QUERY PLAN\\nAggregate  (cost=4641.80..4641.81 rows=1 width=40) (actual rows=1 loops=1)\\n  Buffers: shared hit=1215\\n  ->  Seq Scan on orders  (cost=4436.80..4616.80 rows=5000 width=6) (actual rows=6667 loops=1)\\n        Filter: (NOT (ANY (customer_id = (hashed SubPlan 1).col1)))\\n        Rows Removed by Filter: 3333\\n        Buffers: shared hit=1215\\n        SubPlan 1\\n          ->  Seq Scan on banned  (cost=0.00..3781.44 rows=262144 width=4) (actual rows=262144 loops=1)\\n                Buffers: shared hit=1160\\nPlanning:\\n  Buffers: shared hit=26\\nPlanning Time: 0.103 ms\\nExecution Time: 94.516 ms\",\"runtimeDurationMs\":95,\"rowsAreCapped\":false,\"computeSeconds\":0.523},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":262145},\"output\":\"QUERY PLAN\\nAggregate  (cost=33862892.50..33862892.51 rows=1 width=40) (actual rows=1 loops=1)\\n  Buffers: shared hit=1215, temp read=3003672 written=449\\n  ->  Seq Scan on orders  (cost=0.00..33862867.50 rows=5000 width=6) (actual rows=6667 loops=1)\\n        Filter: (NOT (ANY (customer_id = (SubPlan 1).col1)))\\n        Rows Removed by Filter: 3333\\n        Buffers: shared hit=1215, temp read=3003672 written=449\\n        SubPlan 1\\n          ->  Materialize  (cost=0.00..6117.18 rows=262145 width=4) (actual rows=175328 loops=10000)\\n                Buffers: shared hit=1160, temp read=3003672 written=449\\n                ->  Seq Scan on banned  (cost=0.00..3781.45 rows=262145 width=4) (actual rows=262145 loops=1)\\n                      Buffers: shared hit=1160\\nPlanning:\\n  Buffers: shared hit=26\\nPlanning Time: 0.143 ms\\nJIT:\\n  Functions: 7\\n  Options: Inlining true, Optimization true, Expressions true, Deforming true\\nExecution Time: 169016.541 ms\",\"runtimeDurationMs\":169018,\"rowsAreCapped\":false,\"computeSeconds\":169.396}],\"workerId\":\"2d0989d5-0634-44aa-9a28-57a30661ace6\",\"correlationId\":\"34110ee7-8c59-4f5a-8afe-029834d88f50\",\"correlationIdInstruction\":\"Echo correlationId on subsequent ExoBench tool calls. Tracking only — it does NOT preserve DB state.\",\"version\":\"1.4.26\"}",
      "isError": false,
      "caption": "NOT IN on PostgreSQL 17.11 with both columns declared NOT NULL, 10,000 orders against 262,144 banned ids and then one more: one execution each, EXPLAIN (ANALYZE, TIMING OFF, BUFFERS)",
      "planCaptions": {
        "10K / 262.1K#1": "At 262,144 banned ids PG17 runs the NOT IN as a hashed SubPlan: one Seq Scan of banned fills a hash table, and each order is checked against it with one hash lookup. One execution, 94.516 ms, keeping 6,667 orders.",
        "10K / 262.1K#2": "One more banned id, 262,145, and the word hashed disappears from the Filter: the SubPlan now reads a Materialize from temp files on disk, rescanned 10,000 times, once per order. The one execution takes 169,016.541 ms."
      }
    },
    "campaign17": {
      "type": "mcp-call",
      "name": "benchmarkSql",
      "input": {
        "schemaTemplate": "-- contact ids come in one block per tenant: tenant 1 has 20,000 contacts, tenant 2 10,000, tenant t about 12,000 / t\nCREATE TABLE tenant_sizes AS SELECT t AS tenant_id, CASE WHEN t = 1 THEN 20000 WHEN t = 2 THEN 10000 ELSE greatest(5, 12000 / t) END AS n FROM generate_series(1, ${TENANTS}) t;\nCREATE TABLE tenant_blocks AS SELECT tenant_id, n, (sum(n) OVER (ORDER BY tenant_id) - n + 1)::int AS first_id FROM tenant_sizes;\nCREATE TABLE contacts (id INT PRIMARY KEY, tenant_id INT NOT NULL, email TEXT NOT NULL, first_name TEXT, status TEXT NOT NULL);\nINSERT INTO contacts SELECT b.first_id + k, b.tenant_id, 'c' || (b.first_id + k) || '@example.com', 'Ann',\n  CASE WHEN random() < 0.9 THEN 'subscribed' ELSE 'unsubscribed' END\nFROM tenant_blocks b, generate_series(0, b.n - 1) k;\n\n-- a year of sends: tenant 1 sends a third of them, tenant 2 a sixth, the rest follow a power law;\n-- each tenant mails its engaged segment (every fifth contact) over and over\nCREATE TABLE sends (id INT PRIMARY KEY, tenant_id INT NOT NULL, contact_id INT NOT NULL, sent_at TIMESTAMPTZ NOT NULL);\nINSERT INTO sends SELECT g.i, b.tenant_id, b.first_id + 5 * floor(random() * (b.n / 5))::int, NOW() - random() * interval '365 days'\nFROM (SELECT i, CASE WHEN r < 0.33 THEN 1 WHEN r < 0.495 THEN 2 ELSE 3 + floor((${TENANTS} - 2) * power(random(), 2))::int END AS t\n      FROM (SELECT i, random() AS r FROM generate_series(1, ${SENDS}) i) a) g\nJOIN tenant_blocks b ON b.tenant_id = g.t;\nCREATE INDEX contacts_tenant_id ON contacts (tenant_id, id);\nCREATE INDEX sends_tenant_sent ON sends (tenant_id, sent_at) INCLUDE (contact_id);\nCREATE INDEX sends_contact ON sends (contact_id);\nDROP TABLE tenant_blocks;\nDROP TABLE tenant_sizes;\nVACUUM ANALYZE;",
        "query": "EXPLAIN (ANALYZE, BUFFERS) SELECT c.id, c.email, c.first_name\nFROM contacts c\nWHERE c.tenant_id = 1\n  AND c.status = 'subscribed'\n  AND c.id NOT IN (SELECT s.contact_id\n                   FROM sends s\n                   WHERE s.tenant_id = 1\n                     AND s.sent_at >= now() - 365 * interval '1 day')\nORDER BY c.id\nLIMIT 10",
        "scalePoints": [
          {
            "TENANTS": 2000,
            "SENDS": 2000000
          }
        ],
        "databaseType": "POSTGRES",
        "warmCache": "2",
        "repetitions": "10",
        "seed": 4242,
        "correlationId": "b1b9f6ff-e6dd-43cb-bbb6-3969c7d6083d",
        "exobenchServerVersion": "1.4.26"
      },
      "result": "{\"timestamp\":\"2026-11-13T02:07:14.538311653Z\",\"databaseType\":\"postgres\",\"sql\":\"EXPLAIN (ANALYZE, BUFFERS) SELECT c.id, c.email, c.first_name\\nFROM contacts c\\nWHERE c.tenant_id = 1\\n  AND c.status = 'subscribed'\\n  AND c.id NOT IN (SELECT s.contact_id\\n                   FROM sends s\\n                   WHERE s.tenant_id = 1\\n                     AND s.sent_at >= now() - 365 * interval '1 day')\\nORDER BY c.id\\nLIMIT 10\",\"warmCacheIterations\":2,\"repetitions\":10,\"seed\":4242,\"seedSource\":\"explicit\",\"scalePointCount\":1,\"totalComputeSeconds\":28.302,\"results\":[{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"TENANTS\":2000,\"SENDS\":2000000},\"output\":\"QUERY PLAN\\nStatistics over 10 runs: avg = mean, stddev = sample standard deviation, omitted where 0.\\nLimit  (avg_cost=0.85..378662.57 avg_rows=10 avg_width=26) (actual avg_time=302.953..1329.724 stddev_time=61.511..212.136 avg_rows=10 avg_loops=1)\\n  Buffers: shared avg_hit=3256, temp avg_read=10148 avg_written=1127\\n  ->  Index Scan using contacts_tenant_id on contacts c  (avg_cost=0.85..346551203.46 avg_rows=9152 avg_width=26) (actual avg_time=297.417..1324.180 stddev_time=60.365..211.416 avg_rows=10 avg_loops=1)\\n        Index Cond: (tenant_id = 1)\\n        Filter: ((status = 'subscribed'::text) AND (NOT (ANY (id = (SubPlan 1).col1))))\\n        Rows Removed by Filter: avg=3\\n        Buffers: shared avg_hit=3256, temp avg_read=10148 avg_written=1127\\n        SubPlan 1\\n          ->  Materialize  (avg_cost=0.43..32334.78 avg_rows=663734 avg_width=4) (actual avg_time=0.006..69.390 stddev_time=0.001..11.207 avg_rows=550587 avg_loops=12)\\n                Buffers: shared avg_hit=3252, temp avg_read=10148 avg_written=1127\\n                ->  Index Only Scan using sends_tenant_sent on sends s  (avg_cost=0.43..26423.11 avg_rows=663734 avg_width=4) (actual avg_time=0.029..100.858 stddev_time=0.007..17.816 avg_rows=659439 avg_loops=1)\\n                      Index Cond: ((tenant_id = 1) AND (sent_at >= (now() - '365 days'::interval)))\\n                      Heap Fetches: avg=0\\n                      Buffers: shared avg_hit=3252\\nPlanning Time: avg=0.13 ms stddev=0.03 n=10 samples=0.14,0.14,0.16,0.11,0.18,0.11,0.11,0.11,0.11,0.11\\nJIT:\\n  Functions: avg=12\\n  Options: Inlining false, Optimization false, Expressions true, Deforming true\\n  Timing: Generation avg=0.000 ms, Inlining avg=0.000 ms, Optimization avg=0.299 ms stddev=0.065 ms, Emission avg=5.262 ms stddev=1.237 ms, Total avg=6.162 ms stddev=1.385 ms\\nExecution Time: avg=1331.79 ms stddev=212.33 n=10 samples=1446.46,1643.01,1363.66,1601.3,1419.8,1410.05,1069.81,1130.23,1173.79,1059.77\",\"runtimeDurationMs\":1333,\"rowsAreCapped\":false,\"computeSeconds\":28.302}],\"workerId\":\"39470321-67b9-4525-99d4-f950115dff9b\",\"correlationId\":\"ca7e1101-46c8-457d-8e95-1a21ee8b9ab8\",\"correlationIdInstruction\":\"Echo correlationId on subsequent ExoBench tool calls. Tracking only — it does NOT preserve DB state.\",\"version\":\"1.4.26\"}",
      "isError": false,
      "caption": "The campaign-preview NOT IN on PostgreSQL 17.11, tenant 1 over the last 365 days, both columns NOT NULL: 10 runs after two warm-ups, EXPLAIN (ANALYZE, BUFFERS) with per-node timing",
      "planCaptions": {
        "2K / 2M": "PG17 keeps the NOT IN as a plain SubPlan: it stores the 659,439 matching sends rows in a Materialize and rescans it for each subscribed contact it checks, 12 times per run on average, with 10,148 temp blocks read. Mean Execution Time over 10 runs is 1,331.79 ms, stddev 212.33 ms."
      }
    },
    "campaign19": {
      "type": "mcp-call",
      "name": "benchmarkSql",
      "input": {
        "schemaTemplate": "-- contact ids come in one block per tenant: tenant 1 has 20,000 contacts, tenant 2 10,000, tenant t about 12,000 / t\nCREATE TABLE tenant_sizes AS SELECT t AS tenant_id, CASE WHEN t = 1 THEN 20000 WHEN t = 2 THEN 10000 ELSE greatest(5, 12000 / t) END AS n FROM generate_series(1, ${TENANTS}) t;\nCREATE TABLE tenant_blocks AS SELECT tenant_id, n, (sum(n) OVER (ORDER BY tenant_id) - n + 1)::int AS first_id FROM tenant_sizes;\nCREATE TABLE contacts (id INT PRIMARY KEY, tenant_id INT NOT NULL, email TEXT NOT NULL, first_name TEXT, status TEXT NOT NULL);\nINSERT INTO contacts SELECT b.first_id + k, b.tenant_id, 'c' || (b.first_id + k) || '@example.com', 'Ann',\n  CASE WHEN random() < 0.9 THEN 'subscribed' ELSE 'unsubscribed' END\nFROM tenant_blocks b, generate_series(0, b.n - 1) k;\n\n-- a year of sends: tenant 1 sends a third of them, tenant 2 a sixth, the rest follow a power law;\n-- each tenant mails its engaged segment (every fifth contact) over and over\nCREATE TABLE sends (id INT PRIMARY KEY, tenant_id INT NOT NULL, contact_id INT NOT NULL, sent_at TIMESTAMPTZ NOT NULL);\nINSERT INTO sends SELECT g.i, b.tenant_id, b.first_id + 5 * floor(random() * (b.n / 5))::int, NOW() - random() * interval '365 days'\nFROM (SELECT i, CASE WHEN r < 0.33 THEN 1 WHEN r < 0.495 THEN 2 ELSE 3 + floor((${TENANTS} - 2) * power(random(), 2))::int END AS t\n      FROM (SELECT i, random() AS r FROM generate_series(1, ${SENDS}) i) a) g\nJOIN tenant_blocks b ON b.tenant_id = g.t;\nCREATE INDEX contacts_tenant_id ON contacts (tenant_id, id);\nCREATE INDEX sends_tenant_sent ON sends (tenant_id, sent_at) INCLUDE (contact_id);\nCREATE INDEX sends_contact ON sends (contact_id);\nDROP TABLE tenant_blocks;\nDROP TABLE tenant_sizes;\nVACUUM ANALYZE;",
        "query": "EXPLAIN (ANALYZE, BUFFERS) SELECT c.id, c.email, c.first_name\nFROM contacts c\nWHERE c.tenant_id = 1\n  AND c.status = 'subscribed'\n  AND c.id NOT IN (SELECT s.contact_id\n                   FROM sends s\n                   WHERE s.tenant_id = 1\n                     AND s.sent_at >= now() - 365 * interval '1 day')\nORDER BY c.id\nLIMIT 10",
        "scalePoints": [
          {
            "TENANTS": 2000,
            "SENDS": 2000000
          }
        ],
        "databaseType": "POSTGRES_19",
        "warmCache": "2",
        "repetitions": "10",
        "seed": 4242,
        "correlationId": "ca7e1101-46c8-457d-8e95-1a21ee8b9ab8",
        "exobenchServerVersion": "1.4.26"
      },
      "result": "{\"timestamp\":\"2026-11-13T02:07:49.947265551Z\",\"databaseType\":\"postgres_19\",\"sql\":\"EXPLAIN (ANALYZE, BUFFERS) SELECT c.id, c.email, c.first_name\\nFROM contacts c\\nWHERE c.tenant_id = 1\\n  AND c.status = 'subscribed'\\n  AND c.id NOT IN (SELECT s.contact_id\\n                   FROM sends s\\n                   WHERE s.tenant_id = 1\\n                     AND s.sent_at >= now() - 365 * interval '1 day')\\nORDER BY c.id\\nLIMIT 10\",\"warmCacheIterations\":2,\"repetitions\":10,\"seed\":4242,\"seedSource\":\"explicit\",\"scalePointCount\":1,\"totalComputeSeconds\":11.693,\"results\":[{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"TENANTS\":2000,\"SENDS\":2000000},\"output\":\"QUERY PLAN\\nStatistics over 10 runs: avg = mean, stddev = sample standard deviation, omitted where 0.\\nLimit  (avg_cost=0.84..9.46 avg_rows=10 avg_width=26) (actual avg_time=0.026..0.044 stddev_time=0.007..0.016 avg_rows=10 avg_loops=1)\\n  Buffers: shared avg_hit=42\\n  Disabled: false\\n  ->  Nested Loop Anti Join  (avg_cost=0.84..13470.56 avg_rows=15630 avg_width=26) (actual avg_time=0.026..0.043 stddev_time=0.007..0.016 avg_rows=10 avg_loops=1)\\n        Buffers: shared avg_hit=42\\n        Disabled: false\\n        ->  Index Scan using contacts_tenant_id on contacts c  (avg_cost=0.42..2530.02 avg_rows=18255 avg_width=26) (actual avg_time=0.017..0.020 stddev_time=0.005..0.006 avg_rows=12 avg_loops=1)\\n              Index Cond: (tenant_id = 1)\\n              Filter: (status = 'subscribed'::text)\\n              Rows Removed by Filter: avg=1\\n              Buffers: shared avg_hit=4\\n              Disabled: false\\n              Index Searches: avg=1\\n        ->  Index Scan using sends_contact on sends s  (avg_cost=0.43..6.67 avg_rows=42 avg_width=4) (actual avg_time=0.002..0.002 stddev_time=0.001..0.001 avg_rows=0.17 avg_loops=12)\\n              Index Cond: (contact_id = c.id)\\n              Filter: ((tenant_id = 1) AND (sent_at >= (now() - '365 days'::interval)))\\n              Buffers: shared avg_hit=38\\n              Disabled: false\\n              Index Searches: avg=12\\nPlanning:\\n  Buffers: shared avg_hit=14\\nPlanning Time: avg=0.2 ms stddev=0.04 n=10 samples=0.22,0.24,0.16,0.16,0.24,0.2,0.19,0.17,0.28,0.17\\nExecution Time: avg=0.06 ms stddev=0.02 n=10 samples=0.1,0.06,0.05,0.05,0.08,0.05,0.07,0.05,0.08,0.05\",\"runtimeDurationMs\":1,\"rowsAreCapped\":false,\"computeSeconds\":11.693}],\"workerId\":\"0d3c74f2-d309-4705-a7fa-19f5d83bc328\",\"correlationId\":\"a50489ad-267f-4f29-8ca4-9f12bbc5976d\",\"correlationIdInstruction\":\"Echo correlationId on subsequent ExoBench tool calls. Tracking only — it does NOT preserve DB state.\",\"version\":\"1.4.26\"}",
      "isError": false,
      "caption": "The campaign-preview NOT IN on PostgreSQL 19rc2, tenant 1 over the last 365 days, both columns NOT NULL: 10 runs after two warm-ups, EXPLAIN (ANALYZE, BUFFERS) with per-node timing",
      "planCaptions": {
        "2K / 2M": "19rc2 rewrites the NOT IN into a Nested Loop Anti Join: it walks tenant 1's subscribed contacts in id order and probes the sends_contact index once per contact, 12 probes per run on average, until the LIMIT 10 is met. It reads 42 shared buffers per run, and the mean Execution Time over 10 runs is 0.06 ms, stddev 0.02 ms."
      }
    },
    "nullableNoNulls19": {
      "type": "mcp-call",
      "name": "benchmarkSql",
      "input": {
        "schemaTemplate": "-- contact ids come in one block per tenant: tenant 1 has 20,000 contacts, tenant 2 10,000, tenant t about 12,000 / t\nCREATE TABLE tenant_sizes AS SELECT t AS tenant_id, CASE WHEN t = 1 THEN 20000 WHEN t = 2 THEN 10000 ELSE greatest(5, 12000 / t) END AS n FROM generate_series(1, ${TENANTS}) t;\nCREATE TABLE tenant_blocks AS SELECT tenant_id, n, (sum(n) OVER (ORDER BY tenant_id) - n + 1)::int AS first_id FROM tenant_sizes;\nCREATE TABLE contacts (id INT PRIMARY KEY, tenant_id INT NOT NULL, email TEXT NOT NULL, first_name TEXT, status TEXT NOT NULL);\nINSERT INTO contacts SELECT b.first_id + k, b.tenant_id, 'c' || (b.first_id + k) || '@example.com', 'Ann',\n  CASE WHEN random() < 0.9 THEN 'subscribed' ELSE 'unsubscribed' END\nFROM tenant_blocks b, generate_series(0, b.n - 1) k;\n\n-- a year of sends: tenant 1 sends a third of them, tenant 2 a sixth, the rest follow a power law;\n-- each tenant mails its engaged segment (every fifth contact) over and over\nCREATE TABLE sends (id INT PRIMARY KEY, tenant_id INT NOT NULL, contact_id INT, sent_at TIMESTAMPTZ NOT NULL);\nINSERT INTO sends SELECT g.i, b.tenant_id, b.first_id + 5 * floor(random() * (b.n / 5))::int, NOW() - random() * interval '365 days'\nFROM (SELECT i, CASE WHEN r < 0.33 THEN 1 WHEN r < 0.495 THEN 2 ELSE 3 + floor((${TENANTS} - 2) * power(random(), 2))::int END AS t\n      FROM (SELECT i, random() AS r FROM generate_series(1, ${SENDS}) i) a) g\nJOIN tenant_blocks b ON b.tenant_id = g.t;\nCREATE INDEX contacts_tenant_id ON contacts (tenant_id, id);\nCREATE INDEX sends_tenant_sent ON sends (tenant_id, sent_at) INCLUDE (contact_id);\nCREATE INDEX sends_contact ON sends (contact_id);\nDROP TABLE tenant_blocks;\nDROP TABLE tenant_sizes;\nVACUUM ANALYZE;",
        "query": "EXPLAIN (ANALYZE, BUFFERS) SELECT c.id, c.email, c.first_name\nFROM contacts c\nWHERE c.tenant_id = 1\n  AND c.status = 'subscribed'\n  AND c.id NOT IN (SELECT s.contact_id\n                   FROM sends s\n                   WHERE s.tenant_id = 1\n                     AND s.sent_at >= now() - 365 * interval '1 day')\nORDER BY c.id\nLIMIT 10",
        "scalePoints": [
          {
            "TENANTS": 2000,
            "SENDS": 2000000
          }
        ],
        "databaseType": "POSTGRES_19",
        "warmCache": "2",
        "repetitions": "10",
        "seed": 4242,
        "correlationId": "584e13a8-ad1d-46c8-9c24-435a7b165457",
        "exobenchServerVersion": "1.4.26"
      },
      "result": "{\"timestamp\":\"2026-11-13T02:22:25.264377193Z\",\"databaseType\":\"postgres_19\",\"sql\":\"EXPLAIN (ANALYZE, BUFFERS) SELECT c.id, c.email, c.first_name\\nFROM contacts c\\nWHERE c.tenant_id = 1\\n  AND c.status = 'subscribed'\\n  AND c.id NOT IN (SELECT s.contact_id\\n                   FROM sends s\\n                   WHERE s.tenant_id = 1\\n                     AND s.sent_at >= now() - 365 * interval '1 day')\\nORDER BY c.id\\nLIMIT 10\",\"warmCacheIterations\":2,\"repetitions\":10,\"seed\":4242,\"seedSource\":\"explicit\",\"scalePointCount\":1,\"totalComputeSeconds\":20.476,\"results\":[{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"TENANTS\":2000,\"SENDS\":2000000},\"output\":\"QUERY PLAN\\nStatistics over 10 runs: avg = mean, stddev = sample standard deviation, omitted where 0.\\nLimit  (avg_cost=0.85..373695.44 avg_rows=10 avg_width=26) (actual avg_time=206.355..799.837 stddev_time=9.592..24.220 avg_rows=10 avg_loops=1)\\n  Buffers: shared avg_hit=3256, temp avg_read=10148 avg_written=1127\\n  Disabled: false\\n  ->  Index Scan using contacts_tenant_id on contacts c  (avg_cost=0.85..337147258.00 avg_rows=9022 avg_width=26) (actual avg_time=206.354..799.830 stddev_time=9.592..24.219 avg_rows=10 avg_loops=1)\\n        Index Cond: (tenant_id = 1)\\n        Filter: ((status = 'subscribed'::text) AND (NOT (ANY (id = (SubPlan any_1).col1))))\\n        Rows Removed by Filter: avg=3\\n        Buffers: shared avg_hit=3256, temp avg_read=10148 avg_written=1127\\n        Disabled: false\\n        Index Searches: avg=1\\n        SubPlan any_1\\n          ->  Materialize  (avg_cost=0.43..31982.78 avg_rows=656534 avg_width=4) (actual avg_time=0.005..42.216 stddev_time=0.001..1.228 avg_rows=550586.92 avg_loops=12)\\n                Storage: Disk  Maximum Storage: avg=9016kB\\n                Buffers: shared avg_hit=3252, temp avg_read=10148 avg_written=1127\\n                Disabled: false\\n                ->  Index Only Scan using sends_tenant_sent on sends s  (avg_cost=0.43..26135.11 avg_rows=656534 avg_width=4) (actual avg_time=0.021..70.287 stddev_time=0.005..2.313 avg_rows=659439 avg_loops=1)\\n                      Index Cond: ((tenant_id = 1) AND (sent_at >= (now() - '365 days'::interval)))\\n                      Heap Fetches: avg=0\\n                      Buffers: shared avg_hit=3252\\n                      Disabled: false\\n                      Index Searches: avg=1\\nPlanning Time: avg=0.13 ms stddev=0.02 n=10 samples=0.12,0.12,0.12,0.12,0.12,0.12,0.18,0.12,0.12,0.12\\nExecution Time: avg=801.19 ms stddev=24.21 n=10 samples=797.9,790.28,840.42,785.73,829.86,770.15,779.52,813.04,780.54,824.5\",\"runtimeDurationMs\":800,\"rowsAreCapped\":false,\"computeSeconds\":20.476}],\"workerId\":\"6059602c-e3ae-4c32-8173-c5d776355001\",\"correlationId\":\"0b3e8520-1d46-416e-b28e-facc380dfbc6\",\"correlationIdInstruction\":\"Echo correlationId on subsequent ExoBench tool calls. Tracking only — it does NOT preserve DB state.\",\"version\":\"1.4.26\"}",
      "isError": false,
      "caption": "The campaign-preview NOT IN on PostgreSQL 19rc2, tenant 1 over the last 365 days, with sends.contact_id declared without NOT NULL, though every row the schema inserts has one: 10 runs after two warm-ups, EXPLAIN (ANALYZE, BUFFERS) with per-node timing",
      "planCaptions": {
        "2K / 2M": "With sends.contact_id declared nullable, 19rc2 does not rewrite the NOT IN and runs the old plain SubPlan: a Materialize of the 659,439 matching sends rows, stored on disk at 9,016 kB and rescanned 12 times per run on average. Mean Execution Time over 10 runs is 801.19 ms, stddev 24.21 ms."
      }
    },
    "gap19": {
      "type": "mcp-call",
      "name": "benchmarkSql",
      "input": {
        "schemaTemplate": "CREATE TABLE orders (id int PRIMARY KEY, customer_id int NOT NULL, total numeric(10,2) NOT NULL);\nINSERT INTO orders SELECT i, i, (i % 1000) + 0.99 FROM generate_series(1, ${O}) i;\nCREATE TABLE banned (customer_id int);\nINSERT INTO banned SELECT i * 3 FROM generate_series(1, ${B}) i;\nVACUUM ANALYZE;",
        "query": "EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT count(*), sum(total) FROM orders WHERE customer_id NOT IN (SELECT customer_id FROM banned)",
        "scalePoints": [
          {
            "O": 10000,
            "B": 100000
          },
          {
            "O": 10000,
            "B": 154198
          },
          {
            "O": 10000,
            "B": 154199
          },
          {
            "O": 10000,
            "B": 262144
          }
        ],
        "databaseType": "POSTGRES_19",
        "seed": 4242,
        "waitTimeoutSeconds": 110,
        "correlationId": "34110ee7-8c59-4f5a-8afe-029834d88f50",
        "exobenchServerVersion": "1.4.26"
      },
      "result": "{\"timestamp\":\"2026-11-13T03:21:33.393809579Z\",\"databaseType\":\"postgres_19\",\"sql\":\"EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT count(*), sum(total) FROM orders WHERE customer_id NOT IN (SELECT customer_id FROM banned)\",\"warmCacheIterations\":0,\"seed\":4242,\"seedSource\":\"explicit\",\"scalePointCount\":4,\"totalComputeSeconds\":296.586,\"results\":[{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":100000},\"output\":\"QUERY PLAN\\nAggregate  (cost=1898.00..1898.01 rows=1 width=40) (actual rows=1.00 loops=1)\\n  Buffers: shared hit=498\\n  ->  Seq Scan on orders  (cost=1693.00..1873.00 rows=5000 width=6) (actual rows=6667.00 loops=1)\\n        Filter: (NOT (ANY (customer_id = (hashed SubPlan any_1).col1)))\\n        Rows Removed by Filter: 3333\\n        Buffers: shared hit=498\\n        SubPlan any_1\\n          ->  Seq Scan on banned  (cost=0.00..1443.00 rows=100000 width=4) (actual rows=100000.00 loops=1)\\n                Buffers: shared hit=443\\nPlanning:\\n  Buffers: shared hit=32\\nPlanning Time: 0.152 ms\\nExecution Time: 24.285 ms\",\"runtimeDurationMs\":24,\"rowsAreCapped\":false,\"computeSeconds\":0.224},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":154198},\"output\":\"QUERY PLAN\\nAggregate  (cost=2815.48..2815.49 rows=1 width=40) (actual rows=1.00 loops=1)\\n  Buffers: shared hit=738\\n  ->  Seq Scan on orders  (cost=2610.47..2790.47 rows=5000 width=6) (actual rows=6667.00 loops=1)\\n        Filter: (NOT (ANY (customer_id = (hashed SubPlan any_1).col1)))\\n        Rows Removed by Filter: 3333\\n        Buffers: shared hit=738\\n        SubPlan any_1\\n          ->  Seq Scan on banned  (cost=0.00..2224.98 rows=154198 width=4) (actual rows=154198.00 loops=1)\\n                Buffers: shared hit=683\\nPlanning:\\n  Buffers: shared hit=32\\nPlanning Time: 0.118 ms\\nExecution Time: 39.764 ms\",\"runtimeDurationMs\":40,\"rowsAreCapped\":false,\"computeSeconds\":0.288},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":154199},\"output\":\"QUERY PLAN\\nAggregate  (cost=19922617.50..19922617.51 rows=1 width=40) (actual rows=1.00 loops=1)\\n  Buffers: shared hit=738, temp read=1770462 written=264\\n  ->  Seq Scan on orders  (cost=0.00..19922592.50 rows=5000 width=6) (actual rows=6667.00 loops=1)\\n        Filter: (NOT (ANY (customer_id = (SubPlan any_1).col1)))\\n        Rows Removed by Filter: 3333\\n        Buffers: shared hit=738, temp read=1770462 written=264\\n        SubPlan any_1\\n          ->  Materialize  (cost=0.00..3598.98 rows=154199 width=4) (actual rows=103360.08 loops=10000)\\n                Storage: Disk  Maximum Storage: 4096kB\\n                Buffers: shared hit=683, temp read=1770462 written=264\\n                ->  Seq Scan on banned  (cost=0.00..2224.99 rows=154199 width=4) (actual rows=154199.00 loops=1)\\n                      Buffers: shared hit=683\\nPlanning:\\n  Buffers: shared hit=33\\nPlanning Time: 0.200 ms\\nExecution Time: 101734.255 ms\",\"runtimeDurationMs\":101453,\"rowsAreCapped\":false,\"computeSeconds\":101.709},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":262144},\"output\":\"QUERY PLAN\\nAggregate  (cost=33857805.00..33857805.01 rows=1 width=40) (actual rows=1.00 loops=1)\\n  Buffers: shared hit=1215, temp read=2997006 written=448\\n  ->  Seq Scan on orders  (cost=0.00..33857780.00 rows=5000 width=6) (actual rows=6667.00 loops=1)\\n        Filter: (NOT (ANY (customer_id = (SubPlan any_1).col1)))\\n        Rows Removed by Filter: 3333\\n        Buffers: shared hit=1215, temp read=2997006 written=448\\n        SubPlan any_1\\n          ->  Materialize  (cost=0.00..6116.16 rows=262144 width=4) (actual rows=175327.02 loops=10000)\\n                Storage: Disk  Maximum Storage: 4096kB\\n                Buffers: shared hit=1160, temp read=2997006 written=448\\n                ->  Seq Scan on banned  (cost=0.00..3781.44 rows=262144 width=4) (actual rows=262144.00 loops=1)\\n                      Buffers: shared hit=1160\\nPlanning:\\n  Buffers: shared hit=32\\nPlanning Time: 0.210 ms\\nExecution Time: 194564.214 ms\",\"runtimeDurationMs\":194026,\"rowsAreCapped\":false,\"computeSeconds\":194.365}],\"workerId\":\"f6530a8e-c361-43b3-bfc0-52b8ff7b8aa9\",\"correlationId\":\"34110ee7-8c59-4f5a-8afe-029834d88f50\",\"correlationIdInstruction\":\"Echo correlationId on subsequent ExoBench tool calls. Tracking only — it does NOT preserve DB state.\",\"version\":\"1.4.26\"}",
      "isError": false,
      "caption": "NOT IN on PostgreSQL 19rc2 with banned.customer_id declared without NOT NULL, 10,000 orders against four list sizes from 100,000 to 262,144 banned ids: one execution each, EXPLAIN (ANALYZE, TIMING OFF, BUFFERS)",
      "planCaptions": {
        "10K / 100K": "At 100,000 banned ids 19rc2 leaves the NOT IN as it is, since the list column is nullable, and runs the old hashed SubPlan over one Seq Scan of banned. One execution, 24.285 ms, keeping 6,667 of the orders.",
        "10K / 154.2K#1": "At 154,198 banned ids, the largest list 19rc2 hashes here, the Filter still reads hashed SubPlan any_1. One execution, 39.764 ms, 738 shared buffer hits.",
        "10K / 154.2K#2": "One more banned id, 154,199, and 19rc2 stops hashing: the SubPlan reads a Materialize stored on disk, rescanned 10,000 times, once per order. The one execution takes 101,734.255 ms.",
        "10K / 262.1K": "At 262,144 banned ids, a list PG17 and PG18 still hash, 19rc2 runs the same plain SubPlan over a Materialize on disk, with 2,997,006 temp blocks read. One execution, 194,564.214 ms."
      }
    },
    "gap17": {
      "type": "mcp-call",
      "name": "benchmarkSql",
      "input": {
        "schemaTemplate": "CREATE TABLE orders (id int PRIMARY KEY, customer_id int NOT NULL, total numeric(10,2) NOT NULL);\nINSERT INTO orders SELECT i, i, (i % 1000) + 0.99 FROM generate_series(1, ${O}) i;\nCREATE TABLE banned (customer_id int);\nINSERT INTO banned SELECT i * 3 FROM generate_series(1, ${B}) i;\nVACUUM ANALYZE;",
        "query": "EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT count(*), sum(total) FROM orders WHERE customer_id NOT IN (SELECT customer_id FROM banned)",
        "scalePoints": [
          {
            "O": 10000,
            "B": 100000
          },
          {
            "O": 10000,
            "B": 154198
          },
          {
            "O": 10000,
            "B": 154199
          },
          {
            "O": 10000,
            "B": 262144
          }
        ],
        "databaseType": "POSTGRES",
        "seed": 4242,
        "waitTimeoutSeconds": 110,
        "correlationId": "34110ee7-8c59-4f5a-8afe-029834d88f50",
        "exobenchServerVersion": "1.4.26"
      },
      "result": "{\"timestamp\":\"2026-11-13T03:44:09.336112282Z\",\"databaseType\":\"postgres\",\"sql\":\"EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT count(*), sum(total) FROM orders WHERE customer_id NOT IN (SELECT customer_id FROM banned)\",\"warmCacheIterations\":0,\"seed\":4242,\"seedSource\":\"explicit\",\"scalePointCount\":4,\"totalComputeSeconds\":1.2699999999999998,\"results\":[{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":100000},\"output\":\"QUERY PLAN\\nAggregate  (cost=1898.00..1898.01 rows=1 width=40) (actual rows=1 loops=1)\\n  Buffers: shared hit=498\\n  ->  Seq Scan on orders  (cost=1693.00..1873.00 rows=5000 width=6) (actual rows=6667 loops=1)\\n        Filter: (NOT (ANY (customer_id = (hashed SubPlan 1).col1)))\\n        Rows Removed by Filter: 3333\\n        Buffers: shared hit=498\\n        SubPlan 1\\n          ->  Seq Scan on banned  (cost=0.00..1443.00 rows=100000 width=4) (actual rows=100000 loops=1)\\n                Buffers: shared hit=443\\nPlanning:\\n  Buffers: shared hit=26\\nPlanning Time: 0.099 ms\\nExecution Time: 28.683 ms\",\"runtimeDurationMs\":29,\"rowsAreCapped\":false,\"computeSeconds\":0.235},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":154198},\"output\":\"QUERY PLAN\\nAggregate  (cost=2815.48..2815.49 rows=1 width=40) (actual rows=1 loops=1)\\n  Buffers: shared hit=738\\n  ->  Seq Scan on orders  (cost=2610.47..2790.47 rows=5000 width=6) (actual rows=6667 loops=1)\\n        Filter: (NOT (ANY (customer_id = (hashed SubPlan 1).col1)))\\n        Rows Removed by Filter: 3333\\n        Buffers: shared hit=738\\n        SubPlan 1\\n          ->  Seq Scan on banned  (cost=0.00..2224.98 rows=154198 width=4) (actual rows=154198 loops=1)\\n                Buffers: shared hit=683\\nPlanning:\\n  Buffers: shared hit=26\\nPlanning Time: 0.122 ms\\nExecution Time: 52.754 ms\",\"runtimeDurationMs\":54,\"rowsAreCapped\":false,\"computeSeconds\":0.302},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":154199},\"output\":\"QUERY PLAN\\nAggregate  (cost=2815.49..2815.50 rows=1 width=40) (actual rows=1 loops=1)\\n  Buffers: shared hit=738\\n  ->  Seq Scan on orders  (cost=2610.49..2790.49 rows=5000 width=6) (actual rows=6667 loops=1)\\n        Filter: (NOT (ANY (customer_id = (hashed SubPlan 1).col1)))\\n        Rows Removed by Filter: 3333\\n        Buffers: shared hit=738\\n        SubPlan 1\\n          ->  Seq Scan on banned  (cost=0.00..2224.99 rows=154199 width=4) (actual rows=154199 loops=1)\\n                Buffers: shared hit=683\\nPlanning:\\n  Buffers: shared hit=26\\nPlanning Time: 0.104 ms\\nExecution Time: 51.777 ms\",\"runtimeDurationMs\":52,\"rowsAreCapped\":false,\"computeSeconds\":0.307},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":262144},\"output\":\"QUERY PLAN\\nAggregate  (cost=4641.80..4641.81 rows=1 width=40) (actual rows=1 loops=1)\\n  Buffers: shared hit=1215\\n  ->  Seq Scan on orders  (cost=4436.80..4616.80 rows=5000 width=6) (actual rows=6667 loops=1)\\n        Filter: (NOT (ANY (customer_id = (hashed SubPlan 1).col1)))\\n        Rows Removed by Filter: 3333\\n        Buffers: shared hit=1215\\n        SubPlan 1\\n          ->  Seq Scan on banned  (cost=0.00..3781.44 rows=262144 width=4) (actual rows=262144 loops=1)\\n                Buffers: shared hit=1160\\nPlanning:\\n  Buffers: shared hit=26\\nPlanning Time: 0.097 ms\\nExecution Time: 80.391 ms\",\"runtimeDurationMs\":81,\"rowsAreCapped\":false,\"computeSeconds\":0.426}],\"workerId\":\"6d67ec16-74ce-46ca-ba36-4cab3810def0\",\"correlationId\":\"34110ee7-8c59-4f5a-8afe-029834d88f50\",\"correlationIdInstruction\":\"Echo correlationId on subsequent ExoBench tool calls. Tracking only — it does NOT preserve DB state.\",\"version\":\"1.4.26\"}",
      "isError": false,
      "caption": "NOT IN on PostgreSQL 17.11 with banned.customer_id declared without NOT NULL, 10,000 orders against four list sizes up to 262,144 banned ids: one execution each, EXPLAIN (ANALYZE, TIMING OFF, BUFFERS)",
      "planCaptions": {
        "10K / 100K": "At 100,000 banned ids PG17 runs the NOT IN as a hashed SubPlan over one Seq Scan of banned. One execution, 28.683 ms, keeping 6,667 orders.",
        "10K / 154.2K#1": "At 154,198 banned ids PG17 still hashes the list. One execution, 52.754 ms, 738 shared buffer hits.",
        "10K / 154.2K#2": "At 154,199 banned ids, the size where 19rc2 stops hashing this same list, PG17 keeps the hashed SubPlan. One execution, 51.777 ms, for the same 6,667 orders.",
        "10K / 262.1K": "At 262,144 banned ids, the largest list PG17 hashes, the hashed SubPlan reads 1,215 shared buffers. One execution, 80.391 ms."
      }
    },
    "cliffChart": {
      "type": "chart",
      "chartType": "split-scale",
      "x": [
        100000,
        154198,
        154199,
        262144,
        262145,
        500000
      ],
      "xTicks": [
        100000,
        154199,
        262145,
        500000
      ],
      "xLabel": "Banned ids in the NOT IN list, to scale (10,000 orders)",
      "xUnit": "banned ids",
      "splitAt": 1000,
      "upper": {
        "title": "Rereads the list for every order (seconds)",
        "unit": "s",
        "divisor": 1000,
        "max": 360,
        "ticks": [
          0,
          100,
          200,
          300
        ]
      },
      "lower": {
        "title": "Hashes the list, or anti-joins (milliseconds)",
        "unit": "ms",
        "divisor": 1,
        "max": 100,
        "ticks": [
          0,
          25,
          50,
          75,
          100
        ]
      },
      "series": [
        {
          "label": "PG17, nullable list",
          "shortLabel": "PG17 nullable",
          "data": [
            28.683,
            52.754,
            51.777,
            80.391,
            165261.775,
            314769.094
          ],
          "color": "#6250d6",
          "style": "dashed"
        },
        {
          "label": "PG18, nullable list",
          "shortLabel": "PG18 nullable",
          "data": [
            36.507,
            44.124,
            44.066,
            79.451,
            162743.662,
            309423.976
          ],
          "color": "#9085e9",
          "style": "solid"
        },
        {
          "label": "PG19, nullable list",
          "shortLabel": "PG19 nullable",
          "data": [
            24.285,
            39.764,
            101734.255,
            194564.214,
            182753.052,
            345100.858
          ],
          "color": "#d95926",
          "style": "solid"
        },
        {
          "label": "PG19, both columns NOT NULL",
          "shortLabel": "PG19 NOT NULL",
          "data": [
            10.729,
            15.605,
            16.598,
            27.021,
            25.135,
            46.841
          ],
          "color": "#1D9E75",
          "style": "solid"
        }
      ],
      "annotations": [
        {
          "series": "PG19, nullable list",
          "x": 154199,
          "text": "+1 id: 40 ms → 102 s"
        }
      ],
      "caption": "NOT IN execution time as the banned list grows, 10,000 orders, one execution per point with EXPLAIN (ANALYZE, TIMING OFF, BUFFERS): PostgreSQL 17.11, 18.6 and 19rc2 with banned.customer_id declared without NOT NULL, and 19rc2 with both columns NOT NULL.",
      "subtitle": "Both panels are linear and share one x-axis drawn to scale: runs that reread the list above, in seconds, and runs that hash the list or anti-join below, in milliseconds. A dotted line is a jump from one panel to the other.",
      "dataAsProse": "Execution time per point, one execution each, 10,000 orders. PG17, nullable list: 28.683 ms at 100,000 banned ids, 52.754 ms at 154,198, 51.777 ms at 154,199 and 80.391 ms at 262,144 on the hashed SubPlan, then 165,261.775 ms at 262,145 and 314,769.094 ms at 500,000 on the plain SubPlan. PG18, nullable list: 36.507 ms, 44.124 ms, 44.066 ms and 79.451 ms hashed at those four sizes, then 162,743.662 ms at 262,145 and 309,423.976 ms at 500,000 on the plain SubPlan. PG19, nullable list: 24.285 ms at 100,000 and 39.764 ms at 154,198 hashed, then 101,734.255 ms at 154,199, 194,564.214 ms at 262,144, 182,753.052 ms at 262,145 and 345,100.858 ms at 500,000 on the plain SubPlan. PG19, both columns NOT NULL, a Hash Right Anti Join at every size: 10.729 ms, 15.605 ms, 16.598 ms, 27.021 ms, 25.135 ms and 46.841 ms.",
      "summaryCaption": "With a nullable list, 19rc2 jumps to the seconds panel at 154,199 banned ids while PG17 and PG18 keep hashing through 262,144, and 19rc2's anti join on NOT NULL columns never leaves the milliseconds panel."
    },
    "campaignGap19rc1": {
      "type": "mcp-call",
      "name": "benchmarkSql",
      "input": {
        "schemaTemplate": "-- contact ids come in one block per tenant: tenant 1 has 20,000 contacts, tenant 2 10,000, tenant t about 12,000 / t\nCREATE TABLE tenant_sizes AS SELECT t AS tenant_id, CASE WHEN t = 1 THEN 20000 WHEN t = 2 THEN 10000 ELSE greatest(5, 12000 / t) END AS n FROM generate_series(1, ${TENANTS}) t;\nCREATE TABLE tenant_blocks AS SELECT tenant_id, n, (sum(n) OVER (ORDER BY tenant_id) - n + 1)::int AS first_id FROM tenant_sizes;\nCREATE TABLE contacts (id INT PRIMARY KEY, tenant_id INT NOT NULL, email TEXT NOT NULL, first_name TEXT, status TEXT NOT NULL);\nINSERT INTO contacts SELECT b.first_id + k, b.tenant_id, 'c' || (b.first_id + k) || '@example.com', 'Ann',\n  CASE WHEN random() < 0.9 THEN 'subscribed' ELSE 'unsubscribed' END\nFROM tenant_blocks b, generate_series(0, b.n - 1) k;\n\n-- a year of sends: tenant 1 sends a third of them, tenant 2 a sixth, the rest follow a power law;\n-- each tenant mails its engaged segment (every fifth contact) over and over\nCREATE TABLE sends (id INT PRIMARY KEY, tenant_id INT NOT NULL, contact_id INT, sent_at TIMESTAMPTZ NOT NULL);\nINSERT INTO sends SELECT g.i, b.tenant_id, b.first_id + 5 * floor(random() * (b.n / 5))::int, NOW() - random() * interval '365 days'\nFROM (SELECT i, CASE WHEN r < 0.33 THEN 1 WHEN r < 0.495 THEN 2 ELSE 3 + floor((${TENANTS} - 2) * power(random(), 2))::int END AS t\n      FROM (SELECT i, random() AS r FROM generate_series(1, ${SENDS}) i) a) g\nJOIN tenant_blocks b ON b.tenant_id = g.t;\nCREATE INDEX contacts_tenant_id ON contacts (tenant_id, id);\nCREATE INDEX sends_tenant_sent ON sends (tenant_id, sent_at) INCLUDE (contact_id);\nCREATE INDEX sends_contact ON sends (contact_id);\nDROP TABLE tenant_blocks;\nDROP TABLE tenant_sizes;\nVACUUM ANALYZE;",
        "query": "EXPLAIN (ANALYZE, BUFFERS) SELECT c.id, c.email, c.first_name\nFROM contacts c\nWHERE c.tenant_id = 1\n  AND c.status = 'subscribed'\n  AND c.id NOT IN (SELECT s.contact_id\n                   FROM sends s\n                   WHERE s.tenant_id = 1\n                     AND s.sent_at >= now() - 90 * interval '1 day')\nORDER BY c.id\nLIMIT 10",
        "scalePoints": [
          {
            "TENANTS": 2000,
            "SENDS": 2000000
          }
        ],
        "databaseType": "POSTGRES_19",
        "warmCache": "3",
        "repetitions": "10",
        "seed": 4242,
        "waitTimeoutSeconds": 110,
        "correlationId": "32aa55e8-81d9-46a8-a9dd-aa8493389b21",
        "exobenchServerVersion": "1.4.26"
      },
      "result": "{\"timestamp\":\"2026-11-12T23:05:11.613735605Z\",\"databaseType\":\"postgres_19\",\"sql\":\"EXPLAIN (ANALYZE, BUFFERS) SELECT c.id, c.email, c.first_name\\nFROM contacts c\\nWHERE c.tenant_id = 1\\n  AND c.status = 'subscribed'\\n  AND c.id NOT IN (SELECT s.contact_id\\n                   FROM sends s\\n                   WHERE s.tenant_id = 1\\n                     AND s.sent_at >= now() - 90 * interval '1 day')\\nORDER BY c.id\\nLIMIT 10\",\"warmCacheIterations\":3,\"repetitions\":10,\"seed\":4242,\"seedSource\":\"explicit\",\"scalePointCount\":1,\"totalComputeSeconds\":15.548,\"results\":[{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"TENANTS\":2000,\"SENDS\":2000000},\"output\":\"QUERY PLAN\\nStatistics over 10 runs: avg = mean, stddev = sample standard deviation, omitted where 0.\\nLimit  (avg_cost=0.85..91253.47 avg_rows=10 avg_width=26) (actual avg_time=62.012..247.777 stddev_time=9.859..43.866 avg_rows=10 avg_loops=1)\\n  Buffers: shared avg_hit=807, temp avg_read=2502 avg_written=278\\n  Disabled: false\\n  ->  Index Scan using contacts_tenant_id on contacts c  (avg_cost=0.85..83012509.22 avg_rows=9097 avg_width=26) (actual avg_time=62.011..247.769 stddev_time=9.859..43.868 avg_rows=10 avg_loops=1)\\n        Index Cond: (tenant_id = 1)\\n        Filter: ((status = 'subscribed'::text) AND (NOT (ANY (id = (SubPlan any_1).col1))))\\n        Rows Removed by Filter: avg=3\\n        Buffers: shared avg_hit=807, temp avg_read=2502 avg_written=278\\n        Disabled: false\\n        Index Searches: avg=1\\n        SubPlan any_1\\n          ->  Materialize  (avg_cost=0.43..7829.73 avg_rows=160692 avg_width=4) (actual avg_time=0.005..12.765 stddev_time=0.001..2.049 avg_rows=135247 avg_loops=12)\\n                Storage: Disk  Maximum Storage: avg=4096kB\\n                Buffers: shared avg_hit=803, temp avg_read=2502 avg_written=278\\n                Disabled: false\\n                ->  Index Only Scan using sends_tenant_sent on sends s  (avg_cost=0.43..6398.27 avg_rows=160692 avg_width=4) (actual avg_time=0.021..20.379 stddev_time=0.007..2.386 avg_rows=162209 avg_loops=1)\\n                      Index Cond: ((tenant_id = 1) AND (sent_at >= (now() - '90 days'::interval)))\\n                      Heap Fetches: avg=0\\n                      Buffers: shared avg_hit=803\\n                      Disabled: false\\n                      Index Searches: avg=1\\nPlanning Time: avg=0.17 ms stddev=0.04 n=10 samples=0.19,0.18,0.12,0.24,0.17,0.22,0.15,0.17,0.17,0.12\\nExecution Time: avg=248.15 ms stddev=43.88 n=10 samples=201.53,200.18,198.43,313.82,281.41,213.56,228.21,285.18,276.31,282.9\",\"runtimeDurationMs\":248,\"rowsAreCapped\":false,\"computeSeconds\":15.548}],\"workerId\":\"33617442-961f-443d-84a4-d149acb5193d\",\"correlationId\":\"b374c260-bf78-4711-ade8-d3115e877725\",\"correlationIdInstruction\":\"Echo correlationId on subsequent ExoBench tool calls. Tracking only — it does NOT preserve DB state.\",\"version\":\"1.4.26\"}",
      "isError": false,
      "caption": "The campaign-preview NOT IN on PostgreSQL 19rc1, tenant 1 over the last 90 days, with sends.contact_id declared without NOT NULL: 10 runs after three warm-ups, EXPLAIN (ANALYZE, BUFFERS) with per-node timing",
      "planCaptions": {
        "2K / 2M": "19rc1 does not rewrite this NOT IN, since the column is nullable, and does not hash it either: the planner estimates 160,692 rows for the 90-day subquery, and the SubPlan reads a Materialize stored on disk, rescanned 12 times per run on average. The subquery returns 162,209 rows, and the mean Execution Time over 10 runs is 248.15 ms, stddev 43.88 ms."
      }
    },
    "campaign18": {
      "type": "mcp-call",
      "name": "benchmarkSql",
      "input": {
        "schemaTemplate": "-- contact ids come in one block per tenant: tenant 1 has 20,000 contacts, tenant 2 10,000, tenant t about 12,000 / t\nCREATE TABLE tenant_sizes AS SELECT t AS tenant_id, CASE WHEN t = 1 THEN 20000 WHEN t = 2 THEN 10000 ELSE greatest(5, 12000 / t) END AS n FROM generate_series(1, ${TENANTS}) t;\nCREATE TABLE tenant_blocks AS SELECT tenant_id, n, (sum(n) OVER (ORDER BY tenant_id) - n + 1)::int AS first_id FROM tenant_sizes;\nCREATE TABLE contacts (id INT PRIMARY KEY, tenant_id INT NOT NULL, email TEXT NOT NULL, first_name TEXT, status TEXT NOT NULL);\nINSERT INTO contacts SELECT b.first_id + k, b.tenant_id, 'c' || (b.first_id + k) || '@example.com', 'Ann',\n  CASE WHEN random() < 0.9 THEN 'subscribed' ELSE 'unsubscribed' END\nFROM tenant_blocks b, generate_series(0, b.n - 1) k;\n\n-- a year of sends: tenant 1 sends a third of them, tenant 2 a sixth, the rest follow a power law;\n-- each tenant mails its engaged segment (every fifth contact) over and over\nCREATE TABLE sends (id INT PRIMARY KEY, tenant_id INT NOT NULL, contact_id INT NOT NULL, sent_at TIMESTAMPTZ NOT NULL);\nINSERT INTO sends SELECT g.i, b.tenant_id, b.first_id + 5 * floor(random() * (b.n / 5))::int, NOW() - random() * interval '365 days'\nFROM (SELECT i, CASE WHEN r < 0.33 THEN 1 WHEN r < 0.495 THEN 2 ELSE 3 + floor((${TENANTS} - 2) * power(random(), 2))::int END AS t\n      FROM (SELECT i, random() AS r FROM generate_series(1, ${SENDS}) i) a) g\nJOIN tenant_blocks b ON b.tenant_id = g.t;\nCREATE INDEX contacts_tenant_id ON contacts (tenant_id, id);\nCREATE INDEX sends_tenant_sent ON sends (tenant_id, sent_at) INCLUDE (contact_id);\nCREATE INDEX sends_contact ON sends (contact_id);\nDROP TABLE tenant_blocks;\nDROP TABLE tenant_sizes;\nVACUUM ANALYZE;",
        "query": "EXPLAIN (ANALYZE, BUFFERS) SELECT c.id, c.email, c.first_name\nFROM contacts c\nWHERE c.tenant_id = 1\n  AND c.status = 'subscribed'\n  AND c.id NOT IN (SELECT s.contact_id\n                   FROM sends s\n                   WHERE s.tenant_id = 1\n                     AND s.sent_at >= now() - 365 * interval '1 day')\nORDER BY c.id\nLIMIT 10",
        "scalePoints": [
          {
            "TENANTS": 2000,
            "SENDS": 2000000
          }
        ],
        "databaseType": "POSTGRES_18",
        "warmCache": "2",
        "repetitions": "10",
        "seed": 4242,
        "correlationId": "a50489ad-267f-4f29-8ca4-9f12bbc5976d",
        "exobenchServerVersion": "1.4.26"
      },
      "result": "{\"timestamp\":\"2026-11-13T02:08:39.876239682Z\",\"databaseType\":\"postgres_18\",\"sql\":\"EXPLAIN (ANALYZE, BUFFERS) SELECT c.id, c.email, c.first_name\\nFROM contacts c\\nWHERE c.tenant_id = 1\\n  AND c.status = 'subscribed'\\n  AND c.id NOT IN (SELECT s.contact_id\\n                   FROM sends s\\n                   WHERE s.tenant_id = 1\\n                     AND s.sent_at >= now() - 365 * interval '1 day')\\nORDER BY c.id\\nLIMIT 10\",\"warmCacheIterations\":2,\"repetitions\":10,\"seed\":4242,\"seedSource\":\"explicit\",\"scalePointCount\":1,\"totalComputeSeconds\":25.839,\"results\":[{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"TENANTS\":2000,\"SENDS\":2000000},\"output\":\"QUERY PLAN\\nStatistics over 10 runs: avg = mean, stddev = sample standard deviation, omitted where 0.\\nLimit  (avg_cost=0.85..374566.94 avg_rows=10 avg_width=26) (actual avg_time=280.724..1106.140 stddev_time=51.887..57.159 avg_rows=10 avg_loops=1)\\n  Buffers: shared avg_hit=3256, temp avg_read=10148 avg_written=1127\\n  Disabled: false\\n  ->  Index Scan using contacts_tenant_id on contacts c  (avg_cost=0.85..333551105.44 avg_rows=8905 avg_width=26) (actual avg_time=275.674..1101.083 stddev_time=51.511..56.872 avg_rows=10 avg_loops=1)\\n        Index Cond: (tenant_id = 1)\\n        Filter: ((status = 'subscribed'::text) AND (NOT (ANY (id = (SubPlan 1).col1))))\\n        Rows Removed by Filter: avg=3\\n        Buffers: shared avg_hit=3256, temp avg_read=10148 avg_written=1127\\n        Disabled: false\\n        Index Searches: avg=1\\n        SubPlan 1\\n          ->  Materialize  (avg_cost=0.43..32020.13 avg_rows=657268 avg_width=4) (actual avg_time=0.006..57.908 stddev_time=0.001..3.794 avg_rows=550586.92 avg_loops=12)\\n                Storage: Disk  Maximum Storage: avg=9016kB\\n                Buffers: shared avg_hit=3252, temp avg_read=10148 avg_written=1127\\n                Disabled: false\\n                ->  Index Only Scan using sends_tenant_sent on sends s  (avg_cost=0.43..26165.79 avg_rows=657268 avg_width=4) (actual avg_time=0.031..87.993 stddev_time=0.010..13.204 avg_rows=659439 avg_loops=1)\\n                      Index Cond: ((tenant_id = 1) AND (sent_at >= (now() - '365 days'::interval)))\\n                      Heap Fetches: avg=0\\n                      Buffers: shared avg_hit=3252\\n                      Disabled: false\\n                      Index Searches: avg=1\\nPlanning Time: avg=0.14 ms stddev=0.02 n=10 samples=0.16,0.11,0.13,0.12,0.12,0.15,0.15,0.11,0.16,0.19\\nJIT:\\n  Functions: avg=12\\n  Options: Inlining false, Optimization false, Expressions true, Deforming true\\n  Timing: Generation avg=0.000 ms, Inlining avg=0.000 ms, Optimization avg=0.261 ms stddev=0.050 ms, Emission avg=4.818 ms stddev=0.466 ms, Total avg=5.702 ms stddev=0.561 ms\\nExecution Time: avg=1108 ms stddev=57.16 n=10 samples=1245.9,1048.48,1123.52,1115.45,1063.89,1095.79,1085.38,1142.59,1055.79,1103.25\",\"runtimeDurationMs\":1109,\"rowsAreCapped\":false,\"computeSeconds\":25.839}],\"workerId\":\"0524b759-4154-43cc-b3d5-8d7864ee10cb\",\"correlationId\":\"e5763b20-9a31-4457-be33-426baa68bc80\",\"correlationIdInstruction\":\"Echo correlationId on subsequent ExoBench tool calls. Tracking only — it does NOT preserve DB state.\",\"version\":\"1.4.26\"}",
      "isError": false,
      "caption": "The campaign-preview NOT IN on PostgreSQL 18.6, tenant 1 over the last 365 days, both columns NOT NULL: 10 runs after two warm-ups, EXPLAIN (ANALYZE, BUFFERS) with per-node timing",
      "planCaptions": {
        "2K / 2M": "PG18 runs the same plan shape as PG17: a plain SubPlan over a Materialize of the 659,439 matching sends rows, stored on disk at 9,016 kB and rescanned 12 times per run on average. Mean Execution Time over 10 runs is 1,108 ms, stddev 57.16 ms."
      }
    },
    "campaignFilter19": {
      "type": "mcp-call",
      "name": "benchmarkSql",
      "input": {
        "schemaTemplate": "-- contact ids come in one block per tenant: tenant 1 has 20,000 contacts, tenant 2 10,000, tenant t about 12,000 / t\nCREATE TABLE tenant_sizes AS SELECT t AS tenant_id, CASE WHEN t = 1 THEN 20000 WHEN t = 2 THEN 10000 ELSE greatest(5, 12000 / t) END AS n FROM generate_series(1, ${TENANTS}) t;\nCREATE TABLE tenant_blocks AS SELECT tenant_id, n, (sum(n) OVER (ORDER BY tenant_id) - n + 1)::int AS first_id FROM tenant_sizes;\nCREATE TABLE contacts (id INT PRIMARY KEY, tenant_id INT NOT NULL, email TEXT NOT NULL, first_name TEXT, status TEXT NOT NULL);\nINSERT INTO contacts SELECT b.first_id + k, b.tenant_id, 'c' || (b.first_id + k) || '@example.com', 'Ann',\n  CASE WHEN random() < 0.9 THEN 'subscribed' ELSE 'unsubscribed' END\nFROM tenant_blocks b, generate_series(0, b.n - 1) k;\n\n-- a year of sends: tenant 1 sends a third of them, tenant 2 a sixth, the rest follow a power law;\n-- each tenant mails its engaged segment (every fifth contact) over and over\nCREATE TABLE sends (id INT PRIMARY KEY, tenant_id INT NOT NULL, contact_id INT, sent_at TIMESTAMPTZ NOT NULL);\nINSERT INTO sends SELECT g.i, b.tenant_id, b.first_id + 5 * floor(random() * (b.n / 5))::int, NOW() - random() * interval '365 days'\nFROM (SELECT i, CASE WHEN r < 0.33 THEN 1 WHEN r < 0.495 THEN 2 ELSE 3 + floor((${TENANTS} - 2) * power(random(), 2))::int END AS t\n      FROM (SELECT i, random() AS r FROM generate_series(1, ${SENDS}) i) a) g\nJOIN tenant_blocks b ON b.tenant_id = g.t;\nCREATE INDEX contacts_tenant_id ON contacts (tenant_id, id);\nCREATE INDEX sends_tenant_sent ON sends (tenant_id, sent_at) INCLUDE (contact_id);\nCREATE INDEX sends_contact ON sends (contact_id);\nDROP TABLE tenant_blocks;\nDROP TABLE tenant_sizes;\nVACUUM ANALYZE;",
        "query": "EXPLAIN (ANALYZE, BUFFERS) SELECT c.id, c.email, c.first_name\nFROM contacts c\nWHERE c.tenant_id = 1\n  AND c.status = 'subscribed'\n  AND c.id NOT IN (SELECT s.contact_id\n                   FROM sends s\n                   WHERE s.tenant_id = 1\n                     AND s.sent_at >= now() - 365 * interval '1 day'\n                     AND s.contact_id IS NOT NULL)\nORDER BY c.id\nLIMIT 10",
        "scalePoints": [
          {
            "TENANTS": 2000,
            "SENDS": 2000000
          }
        ],
        "databaseType": "POSTGRES_19",
        "warmCache": "2",
        "repetitions": "10",
        "seed": 4242,
        "correlationId": "0b3e8520-1d46-416e-b28e-facc380dfbc6",
        "exobenchServerVersion": "1.4.26"
      },
      "result": "{\"timestamp\":\"2026-11-13T02:23:00.497599394Z\",\"databaseType\":\"postgres_19\",\"sql\":\"EXPLAIN (ANALYZE, BUFFERS) SELECT c.id, c.email, c.first_name\\nFROM contacts c\\nWHERE c.tenant_id = 1\\n  AND c.status = 'subscribed'\\n  AND c.id NOT IN (SELECT s.contact_id\\n                   FROM sends s\\n                   WHERE s.tenant_id = 1\\n                     AND s.sent_at >= now() - 365 * interval '1 day'\\n                     AND s.contact_id IS NOT NULL)\\nORDER BY c.id\\nLIMIT 10\",\"warmCacheIterations\":2,\"repetitions\":10,\"seed\":4242,\"seedSource\":\"explicit\",\"scalePointCount\":1,\"totalComputeSeconds\":11.674,\"results\":[{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"TENANTS\":2000,\"SENDS\":2000000},\"output\":\"QUERY PLAN\\nStatistics over 10 runs: avg = mean, stddev = sample standard deviation, omitted where 0.\\nLimit  (avg_cost=0.84..9.63 avg_rows=10 avg_width=26) (actual avg_time=0.029..0.048 stddev_time=0.014..0.017 avg_rows=10 avg_loops=1)\\n  Buffers: shared avg_hit=42\\n  Disabled: false\\n  ->  Nested Loop Anti Join  (avg_cost=0.84..13605.19 avg_rows=15494 avg_width=26) (actual avg_time=0.029..0.046 stddev_time=0.014..0.016 avg_rows=10 avg_loops=1)\\n        Buffers: shared avg_hit=42\\n        Disabled: false\\n        ->  Index Scan using contacts_tenant_id on contacts c  (avg_cost=0.42..2520.02 avg_rows=18132 avg_width=26) (actual avg_time=0.021..0.025 stddev_time=0.012..0.014 avg_rows=12 avg_loops=1)\\n              Index Cond: (tenant_id = 1)\\n              Filter: (status = 'subscribed'::text)\\n              Rows Removed by Filter: avg=1\\n              Buffers: shared avg_hit=4\\n              Disabled: false\\n              Index Searches: avg=1\\n        ->  Index Scan using sends_contact on sends s  (avg_cost=0.43..6.97 avg_rows=41 avg_width=4) (actual avg_time=0.001..0.001 avg_rows=0.17 avg_loops=12)\\n              Index Cond: ((contact_id = c.id) AND (contact_id IS NOT NULL))\\n              Filter: ((tenant_id = 1) AND (sent_at >= (now() - '365 days'::interval)))\\n              Buffers: shared avg_hit=38\\n              Disabled: false\\n              Index Searches: avg=12\\nPlanning:\\n  Buffers: shared avg_hit=14\\nPlanning Time: avg=0.21 ms stddev=0.04 n=10 samples=0.16,0.31,0.18,0.21,0.23,0.24,0.17,0.22,0.2,0.17\\nExecution Time: avg=0.06 ms stddev=0.02 n=10 samples=0.05,0.08,0.05,0.09,0.05,0.06,0.08,0.08,0.05,0.05\",\"runtimeDurationMs\":1,\"rowsAreCapped\":false,\"computeSeconds\":11.674}],\"workerId\":\"c2c5bffa-9916-45d4-8cb6-2b7cbc625598\",\"correlationId\":\"723b821c-10f1-49d9-bc4f-322a68869298\",\"correlationIdInstruction\":\"Echo correlationId on subsequent ExoBench tool calls. Tracking only — it does NOT preserve DB state.\",\"version\":\"1.4.26\"}",
      "isError": false,
      "caption": "The campaign-preview NOT IN on PostgreSQL 19rc2, tenant 1 over the last 365 days, with sends.contact_id declared without NOT NULL and AND s.contact_id IS NOT NULL inside the subquery: 10 runs after two warm-ups, EXPLAIN (ANALYZE, BUFFERS) with per-node timing",
      "planCaptions": {
        "2K / 2M": "The IS NOT NULL inside the subquery is enough for 19rc2 to rewrite the NOT IN into a Nested Loop Anti Join after all, and the filter shows up in the probe of sends_contact as Index Cond ((contact_id = c.id) AND (contact_id IS NOT NULL)), 12 probes per run on average. It reads 42 shared buffers per run, and the mean Execution Time over 10 runs is 0.06 ms, stddev 0.02 ms."
      }
    },
    "campaignNotExists18": {
      "type": "mcp-call",
      "name": "benchmarkSql",
      "input": {
        "schemaTemplate": "-- contact ids come in one block per tenant: tenant 1 has 20,000 contacts, tenant 2 10,000, tenant t about 12,000 / t\nCREATE TABLE tenant_sizes AS SELECT t AS tenant_id, CASE WHEN t = 1 THEN 20000 WHEN t = 2 THEN 10000 ELSE greatest(5, 12000 / t) END AS n FROM generate_series(1, ${TENANTS}) t;\nCREATE TABLE tenant_blocks AS SELECT tenant_id, n, (sum(n) OVER (ORDER BY tenant_id) - n + 1)::int AS first_id FROM tenant_sizes;\nCREATE TABLE contacts (id INT PRIMARY KEY, tenant_id INT NOT NULL, email TEXT NOT NULL, first_name TEXT, status TEXT NOT NULL);\nINSERT INTO contacts SELECT b.first_id + k, b.tenant_id, 'c' || (b.first_id + k) || '@example.com', 'Ann',\n  CASE WHEN random() < 0.9 THEN 'subscribed' ELSE 'unsubscribed' END\nFROM tenant_blocks b, generate_series(0, b.n - 1) k;\n\n-- a year of sends: tenant 1 sends a third of them, tenant 2 a sixth, the rest follow a power law;\n-- each tenant mails its engaged segment (every fifth contact) over and over\nCREATE TABLE sends (id INT PRIMARY KEY, tenant_id INT NOT NULL, contact_id INT NOT NULL, sent_at TIMESTAMPTZ NOT NULL);\nINSERT INTO sends SELECT g.i, b.tenant_id, b.first_id + 5 * floor(random() * (b.n / 5))::int, NOW() - random() * interval '365 days'\nFROM (SELECT i, CASE WHEN r < 0.33 THEN 1 WHEN r < 0.495 THEN 2 ELSE 3 + floor((${TENANTS} - 2) * power(random(), 2))::int END AS t\n      FROM (SELECT i, random() AS r FROM generate_series(1, ${SENDS}) i) a) g\nJOIN tenant_blocks b ON b.tenant_id = g.t;\nCREATE INDEX contacts_tenant_id ON contacts (tenant_id, id);\nCREATE INDEX sends_tenant_sent ON sends (tenant_id, sent_at) INCLUDE (contact_id);\nCREATE INDEX sends_contact ON sends (contact_id);\nDROP TABLE tenant_blocks;\nDROP TABLE tenant_sizes;\nVACUUM ANALYZE;",
        "query": "EXPLAIN (ANALYZE, BUFFERS) SELECT c.id, c.email, c.first_name\nFROM contacts c\nWHERE c.tenant_id = 1\n  AND c.status = 'subscribed'\n  AND NOT EXISTS (SELECT 1\n                  FROM sends s\n                  WHERE s.contact_id = c.id\n                    AND s.tenant_id = 1\n                    AND s.sent_at >= now() - 365 * interval '1 day')\nORDER BY c.id\nLIMIT 10",
        "scalePoints": [
          {
            "TENANTS": 2000,
            "SENDS": 2000000
          }
        ],
        "databaseType": "POSTGRES_18",
        "warmCache": "2",
        "repetitions": "10",
        "seed": 4242,
        "correlationId": "19372791-3102-4ec2-b594-9531cf5fe4de",
        "exobenchServerVersion": "1.4.26"
      },
      "result": "{\"timestamp\":\"2026-11-13T02:15:34.861322786Z\",\"databaseType\":\"postgres_18\",\"sql\":\"EXPLAIN (ANALYZE, BUFFERS) SELECT c.id, c.email, c.first_name\\nFROM contacts c\\nWHERE c.tenant_id = 1\\n  AND c.status = 'subscribed'\\n  AND NOT EXISTS (SELECT 1\\n                  FROM sends s\\n                  WHERE s.contact_id = c.id\\n                    AND s.tenant_id = 1\\n                    AND s.sent_at >= now() - 365 * interval '1 day')\\nORDER BY c.id\\nLIMIT 10\",\"warmCacheIterations\":2,\"repetitions\":10,\"seed\":4242,\"seedSource\":\"explicit\",\"scalePointCount\":1,\"totalComputeSeconds\":12.746,\"results\":[{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"TENANTS\":2000,\"SENDS\":2000000},\"output\":\"QUERY PLAN\\nStatistics over 10 runs: avg = mean, stddev = sample standard deviation, omitted where 0.\\nLimit  (avg_cost=0.84..9.47 avg_rows=10 avg_width=26) (actual avg_time=0.036..0.067 stddev_time=0.009..0.018 avg_rows=10 avg_loops=1)\\n  Buffers: shared avg_hit=42\\n  Disabled: false\\n  ->  Nested Loop Anti Join  (avg_cost=0.84..13421.75 avg_rows=15552 avg_width=26) (actual avg_time=0.035..0.065 stddev_time=0.009..0.018 avg_rows=10 avg_loops=1)\\n        Buffers: shared avg_hit=42\\n        Disabled: false\\n        ->  Index Scan using contacts_tenant_id on contacts c  (avg_cost=0.42..2525.92 avg_rows=18169 avg_width=26) (actual avg_time=0.025..0.031 stddev_time=0.008..0.009 avg_rows=12 avg_loops=1)\\n              Index Cond: (tenant_id = 1)\\n              Filter: (status = 'subscribed'::text)\\n              Rows Removed by Filter: avg=1\\n              Buffers: shared avg_hit=4\\n              Disabled: false\\n              Index Searches: avg=1\\n        ->  Index Scan using sends_contact on sends s  (avg_cost=0.43..6.67 avg_rows=42 avg_width=4) (actual avg_time=0.002..0.002 stddev_time=0.001..0.001 avg_rows=0.17 avg_loops=12)\\n              Index Cond: (contact_id = c.id)\\n              Filter: ((tenant_id = 1) AND (sent_at >= (now() - '365 days'::interval)))\\n              Buffers: shared avg_hit=38\\n              Disabled: false\\n              Index Searches: avg=12\\nPlanning:\\n  Buffers: shared avg_hit=14\\nPlanning Time: avg=0.26 ms stddev=0.07 n=10 samples=0.26,0.23,0.27,0.31,0.39,0.3,0.3,0.19,0.22,0.15\\nExecution Time: avg=0.09 ms stddev=0.03 n=10 samples=0.09,0.08,0.08,0.12,0.12,0.1,0.14,0.06,0.08,0.05\",\"runtimeDurationMs\":1,\"rowsAreCapped\":false,\"computeSeconds\":12.746}],\"workerId\":\"a63bc4f7-b34b-4a40-9fa0-ca2f1f0a3f17\",\"correlationId\":\"3d427d42-f54a-44bd-a355-5ddb65779fd0\",\"correlationIdInstruction\":\"Echo correlationId on subsequent ExoBench tool calls. Tracking only — it does NOT preserve DB state.\",\"version\":\"1.4.26\"}",
      "isError": false,
      "caption": "The campaign-preview query written with NOT EXISTS on PostgreSQL 18.6, tenant 1 over the last 365 days, both columns NOT NULL: 10 runs after two warm-ups, EXPLAIN (ANALYZE, BUFFERS) with per-node timing",
      "planCaptions": {
        "2K / 2M": "Written as NOT EXISTS, the query is already a Nested Loop Anti Join on PG18: it probes the sends_contact index once per subscribed contact, 12 probes per run on average, and reads 42 shared buffers. Mean Execution Time over 10 runs is 0.09 ms, stddev 0.03 ms."
      }
    },
    "campaignNotExists17": {
      "type": "mcp-call",
      "name": "benchmarkSql",
      "input": {
        "schemaTemplate": "-- contact ids come in one block per tenant: tenant 1 has 20,000 contacts, tenant 2 10,000, tenant t about 12,000 / t\nCREATE TABLE tenant_sizes AS SELECT t AS tenant_id, CASE WHEN t = 1 THEN 20000 WHEN t = 2 THEN 10000 ELSE greatest(5, 12000 / t) END AS n FROM generate_series(1, ${TENANTS}) t;\nCREATE TABLE tenant_blocks AS SELECT tenant_id, n, (sum(n) OVER (ORDER BY tenant_id) - n + 1)::int AS first_id FROM tenant_sizes;\nCREATE TABLE contacts (id INT PRIMARY KEY, tenant_id INT NOT NULL, email TEXT NOT NULL, first_name TEXT, status TEXT NOT NULL);\nINSERT INTO contacts SELECT b.first_id + k, b.tenant_id, 'c' || (b.first_id + k) || '@example.com', 'Ann',\n  CASE WHEN random() < 0.9 THEN 'subscribed' ELSE 'unsubscribed' END\nFROM tenant_blocks b, generate_series(0, b.n - 1) k;\n\n-- a year of sends: tenant 1 sends a third of them, tenant 2 a sixth, the rest follow a power law;\n-- each tenant mails its engaged segment (every fifth contact) over and over\nCREATE TABLE sends (id INT PRIMARY KEY, tenant_id INT NOT NULL, contact_id INT NOT NULL, sent_at TIMESTAMPTZ NOT NULL);\nINSERT INTO sends SELECT g.i, b.tenant_id, b.first_id + 5 * floor(random() * (b.n / 5))::int, NOW() - random() * interval '365 days'\nFROM (SELECT i, CASE WHEN r < 0.33 THEN 1 WHEN r < 0.495 THEN 2 ELSE 3 + floor((${TENANTS} - 2) * power(random(), 2))::int END AS t\n      FROM (SELECT i, random() AS r FROM generate_series(1, ${SENDS}) i) a) g\nJOIN tenant_blocks b ON b.tenant_id = g.t;\nCREATE INDEX contacts_tenant_id ON contacts (tenant_id, id);\nCREATE INDEX sends_tenant_sent ON sends (tenant_id, sent_at) INCLUDE (contact_id);\nCREATE INDEX sends_contact ON sends (contact_id);\nDROP TABLE tenant_blocks;\nDROP TABLE tenant_sizes;\nVACUUM ANALYZE;",
        "query": "EXPLAIN (ANALYZE, BUFFERS) SELECT c.id, c.email, c.first_name\nFROM contacts c\nWHERE c.tenant_id = 1\n  AND c.status = 'subscribed'\n  AND NOT EXISTS (SELECT 1\n                  FROM sends s\n                  WHERE s.contact_id = c.id\n                    AND s.tenant_id = 1\n                    AND s.sent_at >= now() - 365 * interval '1 day')\nORDER BY c.id\nLIMIT 10",
        "scalePoints": [
          {
            "TENANTS": 2000,
            "SENDS": 2000000
          }
        ],
        "databaseType": "POSTGRES",
        "warmCache": "2",
        "repetitions": "10",
        "seed": 4242,
        "correlationId": "d3c86ddb-69e8-4693-94e0-cdd8c54a09ff",
        "exobenchServerVersion": "1.4.26"
      },
      "result": "{\"timestamp\":\"2026-11-13T02:15:02.379695599Z\",\"databaseType\":\"postgres\",\"sql\":\"EXPLAIN (ANALYZE, BUFFERS) SELECT c.id, c.email, c.first_name\\nFROM contacts c\\nWHERE c.tenant_id = 1\\n  AND c.status = 'subscribed'\\n  AND NOT EXISTS (SELECT 1\\n                  FROM sends s\\n                  WHERE s.contact_id = c.id\\n                    AND s.tenant_id = 1\\n                    AND s.sent_at >= now() - 365 * interval '1 day')\\nORDER BY c.id\\nLIMIT 10\",\"warmCacheIterations\":2,\"repetitions\":10,\"seed\":4242,\"seedSource\":\"explicit\",\"scalePointCount\":1,\"totalComputeSeconds\":11.577,\"results\":[{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"TENANTS\":2000,\"SENDS\":2000000},\"output\":\"QUERY PLAN\\nStatistics over 10 runs: avg = mean, stddev = sample standard deviation, omitted where 0.\\nLimit  (avg_cost=0.84..9.50 avg_rows=10 avg_width=26) (actual avg_time=0.036..0.059 stddev_time=0.014..0.020 avg_rows=10 avg_loops=1)\\n  Buffers: shared avg_hit=42\\n  ->  Nested Loop Anti Join  (avg_cost=0.84..13304.76 avg_rows=15380 avg_width=26) (actual avg_time=0.035..0.057 stddev_time=0.014..0.020 avg_rows=10 avg_loops=1)\\n        Buffers: shared avg_hit=42\\n        ->  Index Scan using contacts_tenant_id on contacts c  (avg_cost=0.42..2514.53 avg_rows=17975 avg_width=26) (actual avg_time=0.023..0.029 stddev_time=0.013..0.013 avg_rows=12 avg_loops=1)\\n              Index Cond: (tenant_id = 1)\\n              Filter: (status = 'subscribed'::text)\\n              Rows Removed by Filter: avg=1\\n              Buffers: shared avg_hit=4\\n        ->  Index Scan using sends_contact on sends s  (avg_cost=0.43..6.69 avg_rows=42 avg_width=4) (actual avg_time=0.002..0.002 stddev_time=0.001..0.001 avg_rows=0 avg_loops=12)\\n              Index Cond: (contact_id = c.id)\\n              Filter: ((tenant_id = 1) AND (sent_at >= (now() - '365 days'::interval)))\\n              Buffers: shared avg_hit=38\\nPlanning:\\n  Buffers: shared avg_hit=14\\nPlanning Time: avg=0.26 ms stddev=0.06 n=10 samples=0.3,0.28,0.26,0.15,0.17,0.27,0.36,0.28,0.25,0.29\\nExecution Time: avg=0.08 ms stddev=0.02 n=10 samples=0.08,0.07,0.05,0.05,0.05,0.1,0.1,0.1,0.11,0.09\",\"runtimeDurationMs\":1,\"rowsAreCapped\":false,\"computeSeconds\":11.577}],\"workerId\":\"d37a91d1-4dbf-4116-8b7f-118a4b1efbd0\",\"correlationId\":\"19372791-3102-4ec2-b594-9531cf5fe4de\",\"correlationIdInstruction\":\"Echo correlationId on subsequent ExoBench tool calls. Tracking only — it does NOT preserve DB state.\",\"version\":\"1.4.26\"}",
      "isError": false,
      "caption": "The campaign-preview query written with NOT EXISTS on PostgreSQL 17.11, tenant 1 over the last 365 days, both columns NOT NULL: 10 runs after two warm-ups, EXPLAIN (ANALYZE, BUFFERS) with per-node timing",
      "planCaptions": {
        "2K / 2M": "Written as NOT EXISTS, the query is already a Nested Loop Anti Join on PG17: it probes the sends_contact index once per subscribed contact, 12 probes per run on average, and reads 42 shared buffers. Mean Execution Time over 10 runs is 0.08 ms, stddev 0.02 ms."
      }
    },
    "campaignTimingOff17": {
      "type": "mcp-call",
      "name": "benchmarkSql",
      "input": {
        "schemaTemplate": "-- contact ids come in one block per tenant: tenant 1 has 20,000 contacts, tenant 2 10,000, tenant t about 12,000 / t\nCREATE TABLE tenant_sizes AS SELECT t AS tenant_id, CASE WHEN t = 1 THEN 20000 WHEN t = 2 THEN 10000 ELSE greatest(5, 12000 / t) END AS n FROM generate_series(1, ${TENANTS}) t;\nCREATE TABLE tenant_blocks AS SELECT tenant_id, n, (sum(n) OVER (ORDER BY tenant_id) - n + 1)::int AS first_id FROM tenant_sizes;\nCREATE TABLE contacts (id INT PRIMARY KEY, tenant_id INT NOT NULL, email TEXT NOT NULL, first_name TEXT, status TEXT NOT NULL);\nINSERT INTO contacts SELECT b.first_id + k, b.tenant_id, 'c' || (b.first_id + k) || '@example.com', 'Ann',\n  CASE WHEN random() < 0.9 THEN 'subscribed' ELSE 'unsubscribed' END\nFROM tenant_blocks b, generate_series(0, b.n - 1) k;\n\n-- a year of sends: tenant 1 sends a third of them, tenant 2 a sixth, the rest follow a power law;\n-- each tenant mails its engaged segment (every fifth contact) over and over\nCREATE TABLE sends (id INT PRIMARY KEY, tenant_id INT NOT NULL, contact_id INT NOT NULL, sent_at TIMESTAMPTZ NOT NULL);\nINSERT INTO sends SELECT g.i, b.tenant_id, b.first_id + 5 * floor(random() * (b.n / 5))::int, NOW() - random() * interval '365 days'\nFROM (SELECT i, CASE WHEN r < 0.33 THEN 1 WHEN r < 0.495 THEN 2 ELSE 3 + floor((${TENANTS} - 2) * power(random(), 2))::int END AS t\n      FROM (SELECT i, random() AS r FROM generate_series(1, ${SENDS}) i) a) g\nJOIN tenant_blocks b ON b.tenant_id = g.t;\nCREATE INDEX contacts_tenant_id ON contacts (tenant_id, id);\nCREATE INDEX sends_tenant_sent ON sends (tenant_id, sent_at) INCLUDE (contact_id);\nCREATE INDEX sends_contact ON sends (contact_id);\nDROP TABLE tenant_blocks;\nDROP TABLE tenant_sizes;\nVACUUM ANALYZE;",
        "query": "EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT c.id, c.email, c.first_name\nFROM contacts c\nWHERE c.tenant_id = 1\n  AND c.status = 'subscribed'\n  AND c.id NOT IN (SELECT s.contact_id\n                   FROM sends s\n                   WHERE s.tenant_id = 1\n                     AND s.sent_at >= now() - 365 * interval '1 day')\nORDER BY c.id\nLIMIT 10",
        "scalePoints": [
          {
            "TENANTS": 2000,
            "SENDS": 2000000
          }
        ],
        "databaseType": "POSTGRES",
        "warmCache": "2",
        "seed": 4242,
        "correlationId": "f1b8df8b-49c1-4219-bd9b-a79abe07a98c",
        "exobenchServerVersion": "1.4.26"
      },
      "result": "{\"timestamp\":\"2026-11-13T02:45:42.051445830Z\",\"databaseType\":\"postgres\",\"sql\":\"EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT c.id, c.email, c.first_name\\nFROM contacts c\\nWHERE c.tenant_id = 1\\n  AND c.status = 'subscribed'\\n  AND c.id NOT IN (SELECT s.contact_id\\n                   FROM sends s\\n                   WHERE s.tenant_id = 1\\n                     AND s.sent_at >= now() - 365 * interval '1 day')\\nORDER BY c.id\\nLIMIT 10\",\"warmCacheIterations\":2,\"seed\":4242,\"seedSource\":\"explicit\",\"scalePointCount\":1,\"totalComputeSeconds\":14.981,\"results\":[{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"TENANTS\":2000,\"SENDS\":2000000},\"output\":\"QUERY PLAN\\nLimit  (cost=0.85..380120.65 rows=10 width=26) (actual rows=10 loops=1)\\n  Buffers: shared hit=3256, temp read=10148 written=1127\\n  ->  Index Scan using contacts_tenant_id on contacts c  (cost=0.85..338876800.29 rows=8915 width=26) (actual rows=10 loops=1)\\n        Index Cond: (tenant_id = 1)\\n        Filter: ((status = 'subscribed'::text) AND (NOT (ANY (id = (SubPlan 1).col1))))\\n        Rows Removed by Filter: 3\\n        Buffers: shared hit=3256, temp read=10148 written=1127\\n        SubPlan 1\\n          ->  Materialize  (cost=0.43..32477.43 rows=666600 width=4) (actual rows=550587 loops=12)\\n                Buffers: shared hit=3252, temp read=10148 written=1127\\n                ->  Index Only Scan using sends_tenant_sent on sends s  (cost=0.43..26540.43 rows=666600 width=4) (actual rows=659439 loops=1)\\n                      Index Cond: ((tenant_id = 1) AND (sent_at >= (now() - '365 days'::interval)))\\n                      Heap Fetches: 0\\n                      Buffers: shared hit=3252\\nPlanning Time: 0.123 ms\\nJIT:\\n  Functions: 12\\n  Options: Inlining false, Optimization false, Expressions true, Deforming true\\nExecution Time: 814.469 ms\",\"runtimeDurationMs\":815,\"rowsAreCapped\":false,\"computeSeconds\":14.981}],\"workerId\":\"28afa4f4-19c8-416c-a3c5-5bf413313b6a\",\"correlationId\":\"03f7d65b-ba0b-477f-8883-6caccb79ce82\",\"correlationIdInstruction\":\"Echo correlationId on subsequent ExoBench tool calls. Tracking only — it does NOT preserve DB state.\",\"version\":\"1.4.26\"}",
      "isError": false,
      "caption": "The campaign-preview NOT IN on PostgreSQL 17.11, tenant 1 over the last 365 days, both columns NOT NULL: one timed execution after two warm-ups, EXPLAIN (ANALYZE, TIMING OFF, BUFFERS)",
      "planCaptions": {
        "2K / 2M": "With per-node timing off, PG17 runs the same plain SubPlan over a Materialize of the 659,439 matching sends rows, rescanned 12 times, with 10,148 temp blocks read. The one timed execution takes 814.469 ms, with JIT compiling 12 functions."
      }
    },
    "campaignTimingOff18": {
      "type": "mcp-call",
      "name": "benchmarkSql",
      "input": {
        "schemaTemplate": "-- contact ids come in one block per tenant: tenant 1 has 20,000 contacts, tenant 2 10,000, tenant t about 12,000 / t\nCREATE TABLE tenant_sizes AS SELECT t AS tenant_id, CASE WHEN t = 1 THEN 20000 WHEN t = 2 THEN 10000 ELSE greatest(5, 12000 / t) END AS n FROM generate_series(1, ${TENANTS}) t;\nCREATE TABLE tenant_blocks AS SELECT tenant_id, n, (sum(n) OVER (ORDER BY tenant_id) - n + 1)::int AS first_id FROM tenant_sizes;\nCREATE TABLE contacts (id INT PRIMARY KEY, tenant_id INT NOT NULL, email TEXT NOT NULL, first_name TEXT, status TEXT NOT NULL);\nINSERT INTO contacts SELECT b.first_id + k, b.tenant_id, 'c' || (b.first_id + k) || '@example.com', 'Ann',\n  CASE WHEN random() < 0.9 THEN 'subscribed' ELSE 'unsubscribed' END\nFROM tenant_blocks b, generate_series(0, b.n - 1) k;\n\n-- a year of sends: tenant 1 sends a third of them, tenant 2 a sixth, the rest follow a power law;\n-- each tenant mails its engaged segment (every fifth contact) over and over\nCREATE TABLE sends (id INT PRIMARY KEY, tenant_id INT NOT NULL, contact_id INT NOT NULL, sent_at TIMESTAMPTZ NOT NULL);\nINSERT INTO sends SELECT g.i, b.tenant_id, b.first_id + 5 * floor(random() * (b.n / 5))::int, NOW() - random() * interval '365 days'\nFROM (SELECT i, CASE WHEN r < 0.33 THEN 1 WHEN r < 0.495 THEN 2 ELSE 3 + floor((${TENANTS} - 2) * power(random(), 2))::int END AS t\n      FROM (SELECT i, random() AS r FROM generate_series(1, ${SENDS}) i) a) g\nJOIN tenant_blocks b ON b.tenant_id = g.t;\nCREATE INDEX contacts_tenant_id ON contacts (tenant_id, id);\nCREATE INDEX sends_tenant_sent ON sends (tenant_id, sent_at) INCLUDE (contact_id);\nCREATE INDEX sends_contact ON sends (contact_id);\nDROP TABLE tenant_blocks;\nDROP TABLE tenant_sizes;\nVACUUM ANALYZE;",
        "query": "EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT c.id, c.email, c.first_name\nFROM contacts c\nWHERE c.tenant_id = 1\n  AND c.status = 'subscribed'\n  AND c.id NOT IN (SELECT s.contact_id\n                   FROM sends s\n                   WHERE s.tenant_id = 1\n                     AND s.sent_at >= now() - 365 * interval '1 day')\nORDER BY c.id\nLIMIT 10",
        "scalePoints": [
          {
            "TENANTS": 2000,
            "SENDS": 2000000
          }
        ],
        "databaseType": "POSTGRES_18",
        "warmCache": "2",
        "seed": 4242,
        "correlationId": "03f7d65b-ba0b-477f-8883-6caccb79ce82",
        "exobenchServerVersion": "1.4.26"
      },
      "result": "{\"timestamp\":\"2026-11-13T02:46:14.667807192Z\",\"databaseType\":\"postgres_18\",\"sql\":\"EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT c.id, c.email, c.first_name\\nFROM contacts c\\nWHERE c.tenant_id = 1\\n  AND c.status = 'subscribed'\\n  AND c.id NOT IN (SELECT s.contact_id\\n                   FROM sends s\\n                   WHERE s.tenant_id = 1\\n                     AND s.sent_at >= now() - 365 * interval '1 day')\\nORDER BY c.id\\nLIMIT 10\",\"warmCacheIterations\":2,\"seed\":4242,\"seedSource\":\"explicit\",\"scalePointCount\":1,\"totalComputeSeconds\":14.393,\"results\":[{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"TENANTS\":2000,\"SENDS\":2000000},\"output\":\"QUERY PLAN\\nLimit  (cost=0.85..374888.64 rows=10 width=26) (actual rows=10.00 loops=1)\\n  Buffers: shared hit=3256, temp read=10148 written=1127\\n  ->  Index Scan using contacts_tenant_id on contacts c  (cost=0.85..335861971.01 rows=8959 width=26) (actual rows=10.00 loops=1)\\n        Index Cond: (tenant_id = 1)\\n        Filter: ((status = 'subscribed'::text) AND (NOT (ANY (id = (SubPlan 1).col1))))\\n        Rows Removed by Filter: 3\\n        Index Searches: 1\\n        Buffers: shared hit=3256, temp read=10148 written=1127\\n        SubPlan 1\\n          ->  Materialize  (cost=0.43..32123.78 rows=659334 width=4) (actual rows=550586.92 loops=12)\\n                Storage: Disk  Maximum Storage: 9016kB\\n                Buffers: shared hit=3252, temp read=10148 written=1127\\n                ->  Index Only Scan using sends_tenant_sent on sends s  (cost=0.43..26251.11 rows=659334 width=4) (actual rows=659439.00 loops=1)\\n                      Index Cond: ((tenant_id = 1) AND (sent_at >= (now() - '365 days'::interval)))\\n                      Heap Fetches: 0\\n                      Index Searches: 1\\n                      Buffers: shared hit=3252\\nPlanning Time: 0.117 ms\\nJIT:\\n  Functions: 12\\n  Options: Inlining false, Optimization false, Expressions true, Deforming true\\nExecution Time: 870.716 ms\",\"runtimeDurationMs\":872,\"rowsAreCapped\":false,\"computeSeconds\":14.393}],\"workerId\":\"74ba423b-8dc5-45fa-98ee-f752b4308de5\",\"correlationId\":\"5c90ac37-90e2-48c3-8142-f377eb71ba3f\",\"correlationIdInstruction\":\"Echo correlationId on subsequent ExoBench tool calls. Tracking only — it does NOT preserve DB state.\",\"version\":\"1.4.26\"}",
      "isError": false,
      "caption": "The campaign-preview NOT IN on PostgreSQL 18.6, tenant 1 over the last 365 days, both columns NOT NULL: one timed execution after two warm-ups, EXPLAIN (ANALYZE, TIMING OFF, BUFFERS)",
      "planCaptions": {
        "2K / 2M": "With per-node timing off, PG18 runs the same plain SubPlan over a Materialize stored on disk at 9,016 kB, rescanned 12 times, with 10,148 temp blocks read. The one timed execution takes 870.716 ms, with JIT compiling 12 functions."
      }
    },
    "campaignTimingOff19": {
      "type": "mcp-call",
      "name": "benchmarkSql",
      "input": {
        "schemaTemplate": "-- contact ids come in one block per tenant: tenant 1 has 20,000 contacts, tenant 2 10,000, tenant t about 12,000 / t\nCREATE TABLE tenant_sizes AS SELECT t AS tenant_id, CASE WHEN t = 1 THEN 20000 WHEN t = 2 THEN 10000 ELSE greatest(5, 12000 / t) END AS n FROM generate_series(1, ${TENANTS}) t;\nCREATE TABLE tenant_blocks AS SELECT tenant_id, n, (sum(n) OVER (ORDER BY tenant_id) - n + 1)::int AS first_id FROM tenant_sizes;\nCREATE TABLE contacts (id INT PRIMARY KEY, tenant_id INT NOT NULL, email TEXT NOT NULL, first_name TEXT, status TEXT NOT NULL);\nINSERT INTO contacts SELECT b.first_id + k, b.tenant_id, 'c' || (b.first_id + k) || '@example.com', 'Ann',\n  CASE WHEN random() < 0.9 THEN 'subscribed' ELSE 'unsubscribed' END\nFROM tenant_blocks b, generate_series(0, b.n - 1) k;\n\n-- a year of sends: tenant 1 sends a third of them, tenant 2 a sixth, the rest follow a power law;\n-- each tenant mails its engaged segment (every fifth contact) over and over\nCREATE TABLE sends (id INT PRIMARY KEY, tenant_id INT NOT NULL, contact_id INT, sent_at TIMESTAMPTZ NOT NULL);\nINSERT INTO sends SELECT g.i, b.tenant_id, b.first_id + 5 * floor(random() * (b.n / 5))::int, NOW() - random() * interval '365 days'\nFROM (SELECT i, CASE WHEN r < 0.33 THEN 1 WHEN r < 0.495 THEN 2 ELSE 3 + floor((${TENANTS} - 2) * power(random(), 2))::int END AS t\n      FROM (SELECT i, random() AS r FROM generate_series(1, ${SENDS}) i) a) g\nJOIN tenant_blocks b ON b.tenant_id = g.t;\nCREATE INDEX contacts_tenant_id ON contacts (tenant_id, id);\nCREATE INDEX sends_tenant_sent ON sends (tenant_id, sent_at) INCLUDE (contact_id);\nCREATE INDEX sends_contact ON sends (contact_id);\nDROP TABLE tenant_blocks;\nDROP TABLE tenant_sizes;\nVACUUM ANALYZE;",
        "query": "EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT c.id, c.email, c.first_name\nFROM contacts c\nWHERE c.tenant_id = 1\n  AND c.status = 'subscribed'\n  AND c.id NOT IN (SELECT s.contact_id\n                   FROM sends s\n                   WHERE s.tenant_id = 1\n                     AND s.sent_at >= now() - 365 * interval '1 day')\nORDER BY c.id\nLIMIT 10",
        "scalePoints": [
          {
            "TENANTS": 2000,
            "SENDS": 2000000
          }
        ],
        "databaseType": "POSTGRES_19",
        "warmCache": "2",
        "seed": 4242,
        "correlationId": "5c90ac37-90e2-48c3-8142-f377eb71ba3f",
        "exobenchServerVersion": "1.4.26"
      },
      "result": "{\"timestamp\":\"2026-11-13T02:46:46.015072449Z\",\"databaseType\":\"postgres_19\",\"sql\":\"EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT c.id, c.email, c.first_name\\nFROM contacts c\\nWHERE c.tenant_id = 1\\n  AND c.status = 'subscribed'\\n  AND c.id NOT IN (SELECT s.contact_id\\n                   FROM sends s\\n                   WHERE s.tenant_id = 1\\n                     AND s.sent_at >= now() - 365 * interval '1 day')\\nORDER BY c.id\\nLIMIT 10\",\"warmCacheIterations\":2,\"seed\":4242,\"seedSource\":\"explicit\",\"scalePointCount\":1,\"totalComputeSeconds\":13.351,\"results\":[{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"TENANTS\":2000,\"SENDS\":2000000},\"output\":\"QUERY PLAN\\nLimit  (cost=0.85..374509.47 rows=10 width=26) (actual rows=10.00 loops=1)\\n  Buffers: shared hit=3256, temp read=10148 written=1127\\n  ->  Index Scan using contacts_tenant_id on contacts c  (cost=0.85..337994027.32 rows=9025 width=26) (actual rows=10.00 loops=1)\\n        Index Cond: (tenant_id = 1)\\n        Filter: ((status = 'subscribed'::text) AND (NOT (ANY (id = (SubPlan any_1).col1))))\\n        Rows Removed by Filter: 3\\n        Index Searches: 1\\n        Buffers: shared hit=3256, temp read=10148 written=1127\\n        SubPlan any_1\\n          ->  Materialize  (cost=0.43..31942.13 rows=655668 width=4) (actual rows=550586.92 loops=12)\\n                Storage: Disk  Maximum Storage: 9016kB\\n                Buffers: shared hit=3252, temp read=10148 written=1127\\n                ->  Index Only Scan using sends_tenant_sent on sends s  (cost=0.43..26101.79 rows=655668 width=4) (actual rows=659439.00 loops=1)\\n                      Index Cond: ((tenant_id = 1) AND (sent_at >= (now() - '365 days'::interval)))\\n                      Heap Fetches: 0\\n                      Index Searches: 1\\n                      Buffers: shared hit=3252\\nPlanning Time: 0.144 ms\\nExecution Time: 765.528 ms\",\"runtimeDurationMs\":764,\"rowsAreCapped\":false,\"computeSeconds\":13.351}],\"workerId\":\"765b4230-e43b-4a44-9832-0b82f800a6e6\",\"correlationId\":\"6c325ac7-0996-498f-8d47-34ade5a8bab4\",\"correlationIdInstruction\":\"Echo correlationId on subsequent ExoBench tool calls. Tracking only — it does NOT preserve DB state.\",\"version\":\"1.4.26\"}",
      "isError": false,
      "caption": "The campaign-preview NOT IN on PostgreSQL 19rc2, tenant 1 over the last 365 days, with sends.contact_id declared without NOT NULL: one timed execution after two warm-ups, EXPLAIN (ANALYZE, TIMING OFF, BUFFERS)",
      "planCaptions": {
        "2K / 2M": "With per-node timing off and the column nullable, 19rc2 runs the plain SubPlan that PG17 and PG18 run here, over a Materialize stored on disk at 9,016 kB, rescanned 12 times, with the same 10,148 temp blocks read. The one timed execution takes 765.528 ms, and the plan shows no JIT, which is off by default in 19rc2."
      }
    },
    "antiJoin19": {
      "type": "mcp-call",
      "name": "benchmarkSql",
      "input": {
        "schemaTemplate": "CREATE TABLE orders (id int PRIMARY KEY, customer_id int NOT NULL, total numeric(10,2) NOT NULL);\nINSERT INTO orders SELECT i, i, (i % 1000) + 0.99 FROM generate_series(1, ${O}) i;\nCREATE TABLE banned (customer_id int NOT NULL);\nINSERT INTO banned SELECT i * 3 FROM generate_series(1, ${B}) i;\nVACUUM ANALYZE;",
        "query": "EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT count(*), sum(total) FROM orders WHERE customer_id NOT IN (SELECT customer_id FROM banned)",
        "scalePoints": [
          {
            "O": 10000,
            "B": 100000
          },
          {
            "O": 10000,
            "B": 154198
          },
          {
            "O": 10000,
            "B": 154199
          },
          {
            "O": 10000,
            "B": 262144
          },
          {
            "O": 10000,
            "B": 262145
          }
        ],
        "databaseType": "POSTGRES_19",
        "seed": 4242,
        "waitTimeoutSeconds": 110,
        "correlationId": "34110ee7-8c59-4f5a-8afe-029834d88f50",
        "exobenchServerVersion": "1.4.26"
      },
      "result": "{\"timestamp\":\"2026-11-13T03:14:45.580975393Z\",\"databaseType\":\"postgres_19\",\"sql\":\"EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT count(*), sum(total) FROM orders WHERE customer_id NOT IN (SELECT customer_id FROM banned)\",\"warmCacheIterations\":0,\"seed\":4242,\"seedSource\":\"explicit\",\"scalePointCount\":5,\"totalComputeSeconds\":1.5170000000000001,\"results\":[{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":100000},\"output\":\"QUERY PLAN\\nAggregate  (cost=2198.01..2198.02 rows=1 width=40) (actual rows=1.00 loops=1)\\n  Buffers: shared hit=498\\n  ->  Hash Right Anti Join  (cost=280.00..2198.00 rows=1 width=6) (actual rows=6667.00 loops=1)\\n        Hash Cond: (banned.customer_id = orders.customer_id)\\n        Buffers: shared hit=498\\n        ->  Seq Scan on banned  (cost=0.00..1443.00 rows=100000 width=4) (actual rows=100000.00 loops=1)\\n              Buffers: shared hit=443\\n        ->  Hash  (cost=155.00..155.00 rows=10000 width=10) (actual rows=10000.00 loops=1)\\n              Buckets: 16384  Batches: 1  Memory Usage: 558kB\\n              Buffers: shared hit=55\\n              ->  Seq Scan on orders  (cost=0.00..155.00 rows=10000 width=10) (actual rows=10000.00 loops=1)\\n                    Buffers: shared hit=55\\nPlanning:\\n  Buffers: shared hit=88\\nPlanning Time: 0.310 ms\\nExecution Time: 10.729 ms\",\"runtimeDurationMs\":12,\"rowsAreCapped\":false,\"computeSeconds\":0.241},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":154198},\"output\":\"QUERY PLAN\\nAggregate  (cost=3183.23..3183.24 rows=1 width=40) (actual rows=1.00 loops=1)\\n  Buffers: shared hit=738\\n  ->  Hash Right Anti Join  (cost=280.00..3183.22 rows=1 width=6) (actual rows=6667.00 loops=1)\\n        Hash Cond: (banned.customer_id = orders.customer_id)\\n        Buffers: shared hit=738\\n        ->  Seq Scan on banned  (cost=0.00..2224.98 rows=154198 width=4) (actual rows=154198.00 loops=1)\\n              Buffers: shared hit=683\\n        ->  Hash  (cost=155.00..155.00 rows=10000 width=10) (actual rows=10000.00 loops=1)\\n              Buckets: 16384  Batches: 1  Memory Usage: 558kB\\n              Buffers: shared hit=55\\n              ->  Seq Scan on orders  (cost=0.00..155.00 rows=10000 width=10) (actual rows=10000.00 loops=1)\\n                    Buffers: shared hit=55\\nPlanning:\\n  Buffers: shared hit=89\\nPlanning Time: 0.270 ms\\nExecution Time: 15.605 ms\",\"runtimeDurationMs\":16,\"rowsAreCapped\":false,\"computeSeconds\":0.265},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":154199},\"output\":\"QUERY PLAN\\nAggregate  (cost=3183.24..3183.25 rows=1 width=40) (actual rows=1.00 loops=1)\\n  Buffers: shared hit=738\\n  ->  Hash Right Anti Join  (cost=280.00..3183.24 rows=1 width=6) (actual rows=6667.00 loops=1)\\n        Hash Cond: (banned.customer_id = orders.customer_id)\\n        Buffers: shared hit=738\\n        ->  Seq Scan on banned  (cost=0.00..2224.99 rows=154199 width=4) (actual rows=154199.00 loops=1)\\n              Buffers: shared hit=683\\n        ->  Hash  (cost=155.00..155.00 rows=10000 width=10) (actual rows=10000.00 loops=1)\\n              Buckets: 16384  Batches: 1  Memory Usage: 558kB\\n              Buffers: shared hit=55\\n              ->  Seq Scan on orders  (cost=0.00..155.00 rows=10000 width=10) (actual rows=10000.00 loops=1)\\n                    Buffers: shared hit=55\\nPlanning:\\n  Buffers: shared hit=89\\nPlanning Time: 0.300 ms\\nExecution Time: 16.598 ms\",\"runtimeDurationMs\":18,\"rowsAreCapped\":false,\"computeSeconds\":0.266},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":262144},\"output\":\"QUERY PLAN\\nAggregate  (cost=5144.49..5144.50 rows=1 width=40) (actual rows=1.00 loops=1)\\n  Buffers: shared hit=1215\\n  ->  Hash Right Anti Join  (cost=280.00..5144.48 rows=1 width=6) (actual rows=6667.00 loops=1)\\n        Hash Cond: (banned.customer_id = orders.customer_id)\\n        Buffers: shared hit=1215\\n        ->  Seq Scan on banned  (cost=0.00..3781.44 rows=262144 width=4) (actual rows=262144.00 loops=1)\\n              Buffers: shared hit=1160\\n        ->  Hash  (cost=155.00..155.00 rows=10000 width=10) (actual rows=10000.00 loops=1)\\n              Buckets: 16384  Batches: 1  Memory Usage: 558kB\\n              Buffers: shared hit=55\\n              ->  Seq Scan on orders  (cost=0.00..155.00 rows=10000 width=10) (actual rows=10000.00 loops=1)\\n                    Buffers: shared hit=55\\nPlanning:\\n  Buffers: shared hit=88\\nPlanning Time: 0.253 ms\\nExecution Time: 27.021 ms\",\"runtimeDurationMs\":28,\"rowsAreCapped\":false,\"computeSeconds\":0.376},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":262145},\"output\":\"QUERY PLAN\\nAggregate  (cost=5144.50..5144.51 rows=1 width=40) (actual rows=1.00 loops=1)\\n  Buffers: shared hit=1215\\n  ->  Hash Right Anti Join  (cost=280.00..5144.49 rows=1 width=6) (actual rows=6667.00 loops=1)\\n        Hash Cond: (banned.customer_id = orders.customer_id)\\n        Buffers: shared hit=1215\\n        ->  Seq Scan on banned  (cost=0.00..3781.45 rows=262145 width=4) (actual rows=262145.00 loops=1)\\n              Buffers: shared hit=1160\\n        ->  Hash  (cost=155.00..155.00 rows=10000 width=10) (actual rows=10000.00 loops=1)\\n              Buckets: 16384  Batches: 1  Memory Usage: 558kB\\n              Buffers: shared hit=55\\n              ->  Seq Scan on orders  (cost=0.00..155.00 rows=10000 width=10) (actual rows=10000.00 loops=1)\\n                    Buffers: shared hit=55\\nPlanning:\\n  Buffers: shared hit=89\\nPlanning Time: 0.205 ms\\nExecution Time: 25.135 ms\",\"runtimeDurationMs\":25,\"rowsAreCapped\":false,\"computeSeconds\":0.369}],\"workerId\":\"46eae406-55f6-4f30-a22b-498cbf4e95e2\",\"correlationId\":\"34110ee7-8c59-4f5a-8afe-029834d88f50\",\"correlationIdInstruction\":\"Echo correlationId on subsequent ExoBench tool calls. Tracking only — it does NOT preserve DB state.\",\"version\":\"1.4.26\"}",
      "isError": false,
      "caption": "NOT IN on PostgreSQL 19rc2 with both columns declared NOT NULL, 10,000 orders against five list sizes up to 262,145 banned ids: one execution each, EXPLAIN (ANALYZE, TIMING OFF, BUFFERS)",
      "planCaptions": {
        "10K / 100K": "At 100,000 banned ids 19rc2 rewrites the NOT IN into a Hash Right Anti Join: it hashes the orders, 558 kB in one batch, and streams the banned list past the hash. One execution, 10.729 ms.",
        "10K / 154.2K#1": "At 154,198 banned ids the plan is the same Hash Right Anti Join, and since the hash holds the orders rather than the list, the list's length does not decide whether the hash fits. One execution, 15.605 ms.",
        "10K / 154.2K#2": "At 154,199 banned ids, where the nullable list drops to a plain SubPlan on 19rc2, this NOT NULL list stays a Hash Right Anti Join. One execution, 16.598 ms, 738 shared buffer hits.",
        "10K / 262.1K#1": "At 262,144 banned ids it is still the Hash Right Anti Join over the same 558 kB hash of the orders, with the longer list only streaming past it. One execution, 27.021 ms.",
        "10K / 262.1K#2": "At 262,145 banned ids, where PG17 and PG18 start rescanning the list once per order, 19rc2 keeps the same anti join. One execution, 25.135 ms, keeping the same 6,667 orders."
      }
    },
    "gapBeyond19": {
      "type": "mcp-call",
      "name": "benchmarkSql",
      "input": {
        "schemaTemplate": "CREATE TABLE orders (id int PRIMARY KEY, customer_id int NOT NULL, total numeric(10,2) NOT NULL);\nINSERT INTO orders SELECT i, i, (i % 1000) + 0.99 FROM generate_series(1, ${O}) i;\nCREATE TABLE banned (customer_id int);\nINSERT INTO banned SELECT i * 3 FROM generate_series(1, ${B}) i;\nVACUUM ANALYZE;",
        "query": "EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT count(*), sum(total) FROM orders WHERE customer_id NOT IN (SELECT customer_id FROM banned)",
        "scalePoints": [
          {
            "O": 10000,
            "B": 262145
          }
        ],
        "databaseType": "POSTGRES_19",
        "seed": 4242,
        "waitTimeoutSeconds": 110,
        "correlationId": "34110ee7-8c59-4f5a-8afe-029834d88f50",
        "exobenchServerVersion": "1.4.26"
      },
      "result": "{\"timestamp\":\"2026-11-13T03:24:48.385047724Z\",\"databaseType\":\"postgres_19\",\"sql\":\"EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT count(*), sum(total) FROM orders WHERE customer_id NOT IN (SELECT customer_id FROM banned)\",\"warmCacheIterations\":0,\"seed\":4242,\"seedSource\":\"explicit\",\"scalePointCount\":1,\"totalComputeSeconds\":182.603,\"results\":[{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":262145},\"output\":\"QUERY PLAN\\nAggregate  (cost=33862892.50..33862892.51 rows=1 width=40) (actual rows=1.00 loops=1)\\n  Buffers: shared hit=1215, temp read=3003672 written=449\\n  ->  Seq Scan on orders  (cost=0.00..33862867.50 rows=5000 width=6) (actual rows=6667.00 loops=1)\\n        Filter: (NOT (ANY (customer_id = (SubPlan any_1).col1)))\\n        Rows Removed by Filter: 3333\\n        Buffers: shared hit=1215, temp read=3003672 written=449\\n        SubPlan any_1\\n          ->  Materialize  (cost=0.00..6117.18 rows=262145 width=4) (actual rows=175327.68 loops=10000)\\n                Storage: Disk  Maximum Storage: 4096kB\\n                Buffers: shared hit=1160, temp read=3003672 written=449\\n                ->  Seq Scan on banned  (cost=0.00..3781.45 rows=262145 width=4) (actual rows=262145.00 loops=1)\\n                      Buffers: shared hit=1160\\nPlanning:\\n  Buffers: shared hit=32\\nPlanning Time: 0.122 ms\\nExecution Time: 182753.052 ms\",\"runtimeDurationMs\":182247,\"rowsAreCapped\":false,\"computeSeconds\":182.603}],\"workerId\":\"1a0f84ff-ab43-4849-887b-17d96622fe6e\",\"correlationId\":\"34110ee7-8c59-4f5a-8afe-029834d88f50\",\"correlationIdInstruction\":\"Echo correlationId on subsequent ExoBench tool calls. Tracking only — it does NOT preserve DB state.\",\"version\":\"1.4.26\"}",
      "isError": false,
      "caption": "NOT IN on PostgreSQL 19rc2 with banned.customer_id declared without NOT NULL, 10,000 orders against 262,145 banned ids: one execution, EXPLAIN (ANALYZE, TIMING OFF, BUFFERS)",
      "planCaptions": {
        "10K / 262.1K": "At 262,145 banned ids, the size where PG17 and PG18 stop hashing too, 19rc2 runs the plain SubPlan over a Materialize on disk, rescanned 10,000 times, once per order. The one execution takes 182,753.052 ms."
      }
    },
    "nullable18": {
      "type": "mcp-call",
      "name": "benchmarkSql",
      "input": {
        "schemaTemplate": "CREATE TABLE orders (id int PRIMARY KEY, customer_id int NOT NULL, total numeric(10,2) NOT NULL);\nINSERT INTO orders SELECT i, i, (i % 1000) + 0.99 FROM generate_series(1, ${O}) i;\nCREATE TABLE banned (customer_id int);\nINSERT INTO banned SELECT i * 3 FROM generate_series(1, ${B}) i;\nVACUUM ANALYZE;",
        "query": "EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT count(*), sum(total) FROM orders WHERE customer_id NOT IN (SELECT customer_id FROM banned)",
        "scalePoints": [
          {
            "O": 10000,
            "B": 100000
          },
          {
            "O": 10000,
            "B": 154198
          },
          {
            "O": 10000,
            "B": 154199
          },
          {
            "O": 10000,
            "B": 262144
          }
        ],
        "databaseType": "POSTGRES_18",
        "seed": 4242,
        "waitTimeoutSeconds": 110,
        "correlationId": "34110ee7-8c59-4f5a-8afe-029834d88f50",
        "exobenchServerVersion": "1.4.26"
      },
      "result": "{\"timestamp\":\"2026-11-13T03:44:30.656361593Z\",\"databaseType\":\"postgres_18\",\"sql\":\"EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT count(*), sum(total) FROM orders WHERE customer_id NOT IN (SELECT customer_id FROM banned)\",\"warmCacheIterations\":0,\"seed\":4242,\"seedSource\":\"explicit\",\"scalePointCount\":4,\"totalComputeSeconds\":1.398,\"results\":[{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":100000},\"output\":\"QUERY PLAN\\nAggregate  (cost=1898.00..1898.01 rows=1 width=40) (actual rows=1.00 loops=1)\\n  Buffers: shared hit=498\\n  ->  Seq Scan on orders  (cost=1693.00..1873.00 rows=5000 width=6) (actual rows=6667.00 loops=1)\\n        Filter: (NOT (ANY (customer_id = (hashed SubPlan 1).col1)))\\n        Rows Removed by Filter: 3333\\n        Buffers: shared hit=498\\n        SubPlan 1\\n          ->  Seq Scan on banned  (cost=0.00..1443.00 rows=100000 width=4) (actual rows=100000.00 loops=1)\\n                Buffers: shared hit=443\\nPlanning:\\n  Buffers: shared hit=26\\nPlanning Time: 0.158 ms\\nExecution Time: 36.507 ms\",\"runtimeDurationMs\":37,\"rowsAreCapped\":false,\"computeSeconds\":0.354},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":154198},\"output\":\"QUERY PLAN\\nAggregate  (cost=2815.48..2815.49 rows=1 width=40) (actual rows=1.00 loops=1)\\n  Buffers: shared hit=738\\n  ->  Seq Scan on orders  (cost=2610.47..2790.47 rows=5000 width=6) (actual rows=6667.00 loops=1)\\n        Filter: (NOT (ANY (customer_id = (hashed SubPlan 1).col1)))\\n        Rows Removed by Filter: 3333\\n        Buffers: shared hit=738\\n        SubPlan 1\\n          ->  Seq Scan on banned  (cost=0.00..2224.98 rows=154198 width=4) (actual rows=154198.00 loops=1)\\n                Buffers: shared hit=683\\nPlanning:\\n  Buffers: shared hit=26\\nPlanning Time: 0.108 ms\\nExecution Time: 44.124 ms\",\"runtimeDurationMs\":45,\"rowsAreCapped\":false,\"computeSeconds\":0.328},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":154199},\"output\":\"QUERY PLAN\\nAggregate  (cost=2815.49..2815.50 rows=1 width=40) (actual rows=1.00 loops=1)\\n  Buffers: shared hit=738\\n  ->  Seq Scan on orders  (cost=2610.49..2790.49 rows=5000 width=6) (actual rows=6667.00 loops=1)\\n        Filter: (NOT (ANY (customer_id = (hashed SubPlan 1).col1)))\\n        Rows Removed by Filter: 3333\\n        Buffers: shared hit=738\\n        SubPlan 1\\n          ->  Seq Scan on banned  (cost=0.00..2224.99 rows=154199 width=4) (actual rows=154199.00 loops=1)\\n                Buffers: shared hit=683\\nPlanning:\\n  Buffers: shared hit=26\\nPlanning Time: 0.152 ms\\nExecution Time: 44.066 ms\",\"runtimeDurationMs\":44,\"rowsAreCapped\":false,\"computeSeconds\":0.291},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":262144},\"output\":\"QUERY PLAN\\nAggregate  (cost=4641.80..4641.81 rows=1 width=40) (actual rows=1.00 loops=1)\\n  Buffers: shared hit=1215\\n  ->  Seq Scan on orders  (cost=4436.80..4616.80 rows=5000 width=6) (actual rows=6667.00 loops=1)\\n        Filter: (NOT (ANY (customer_id = (hashed SubPlan 1).col1)))\\n        Rows Removed by Filter: 3333\\n        Buffers: shared hit=1215\\n        SubPlan 1\\n          ->  Seq Scan on banned  (cost=0.00..3781.44 rows=262144 width=4) (actual rows=262144.00 loops=1)\\n                Buffers: shared hit=1160\\nPlanning:\\n  Buffers: shared hit=26\\nPlanning Time: 0.139 ms\\nExecution Time: 79.451 ms\",\"runtimeDurationMs\":80,\"rowsAreCapped\":false,\"computeSeconds\":0.425}],\"workerId\":\"dc6d744c-3d98-4361-99ed-ae65d3b71de2\",\"correlationId\":\"34110ee7-8c59-4f5a-8afe-029834d88f50\",\"correlationIdInstruction\":\"Echo correlationId on subsequent ExoBench tool calls. Tracking only — it does NOT preserve DB state.\",\"version\":\"1.4.26\"}",
      "isError": false,
      "caption": "NOT IN on PostgreSQL 18.6 with banned.customer_id declared without NOT NULL, 10,000 orders against four list sizes up to 262,144 banned ids: one execution each, EXPLAIN (ANALYZE, TIMING OFF, BUFFERS)",
      "planCaptions": {
        "10K / 100K": "At 100,000 banned ids PG18 hashes the nullable list in a hashed SubPlan over one Seq Scan of banned. One execution, 36.507 ms, keeping 6,667 orders.",
        "10K / 154.2K#1": "At 154,198 banned ids PG18 still hashes the list. One execution, 44.124 ms, 738 shared buffer hits.",
        "10K / 154.2K#2": "At 154,199 banned ids, where 19rc2 stops hashing this same list, PG18 keeps the hashed SubPlan. One execution, 44.066 ms, for the same 6,667 orders.",
        "10K / 262.1K": "At 262,144 banned ids, the largest list PG18 hashes, the hashed SubPlan reads 1,215 shared buffers. One execution, 79.451 ms."
      }
    },
    "nullableRescan17": {
      "type": "mcp-call",
      "name": "benchmarkSql",
      "input": {
        "schemaTemplate": "CREATE TABLE orders (id int PRIMARY KEY, customer_id int NOT NULL, total numeric(10,2) NOT NULL);\nINSERT INTO orders SELECT i, i, (i % 1000) + 0.99 FROM generate_series(1, ${O}) i;\nCREATE TABLE banned (customer_id int);\nINSERT INTO banned SELECT i * 3 FROM generate_series(1, ${B}) i;\nVACUUM ANALYZE;",
        "query": "EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT count(*), sum(total) FROM orders WHERE customer_id NOT IN (SELECT customer_id FROM banned)",
        "scalePoints": [
          {
            "O": 10000,
            "B": 262145
          },
          {
            "O": 10000,
            "B": 500000
          }
        ],
        "databaseType": "POSTGRES",
        "seed": 4242,
        "waitTimeoutSeconds": 110,
        "correlationId": "34110ee7-8c59-4f5a-8afe-029834d88f50",
        "exobenchServerVersion": "1.4.26"
      },
      "result": "{\"timestamp\":\"2026-11-13T03:53:46.099200408Z\",\"databaseType\":\"postgres\",\"sql\":\"EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT count(*), sum(total) FROM orders WHERE customer_id NOT IN (SELECT customer_id FROM banned)\",\"warmCacheIterations\":0,\"seed\":4242,\"seedSource\":\"explicit\",\"scalePointCount\":2,\"totalComputeSeconds\":480.981,\"results\":[{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":262145},\"output\":\"QUERY PLAN\\nAggregate  (cost=33862892.50..33862892.51 rows=1 width=40) (actual rows=1 loops=1)\\n  Buffers: shared hit=1215, temp read=3003672 written=449\\n  ->  Seq Scan on orders  (cost=0.00..33862867.50 rows=5000 width=6) (actual rows=6667 loops=1)\\n        Filter: (NOT (ANY (customer_id = (SubPlan 1).col1)))\\n        Rows Removed by Filter: 3333\\n        Buffers: shared hit=1215, temp read=3003672 written=449\\n        SubPlan 1\\n          ->  Materialize  (cost=0.00..6117.18 rows=262145 width=4) (actual rows=175328 loops=10000)\\n                Buffers: shared hit=1160, temp read=3003672 written=449\\n                ->  Seq Scan on banned  (cost=0.00..3781.45 rows=262145 width=4) (actual rows=262145 loops=1)\\n                      Buffers: shared hit=1160\\nPlanning:\\n  Buffers: shared hit=26\\nPlanning Time: 0.107 ms\\nJIT:\\n  Functions: 7\\n  Options: Inlining true, Optimization true, Expressions true, Deforming true\\nExecution Time: 165261.775 ms\",\"runtimeDurationMs\":165263,\"rowsAreCapped\":false,\"computeSeconds\":165.64},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":500000},\"output\":\"QUERY PLAN\\nAggregate  (cost=64585205.00..64585205.01 rows=1 width=40) (actual rows=1 loops=1)\\n  Buffers: shared hit=2268, temp read=5710068 written=855\\n  ->  Seq Scan on orders  (cost=0.00..64585180.00 rows=5000 width=6) (actual rows=6667 loops=1)\\n        Filter: (NOT (ANY (customer_id = (SubPlan 1).col1)))\\n        Rows Removed by Filter: 3333\\n        Buffers: shared hit=2268, temp read=5710068 written=855\\n        SubPlan 1\\n          ->  Materialize  (cost=0.00..11667.00 rows=500000 width=4) (actual rows=333906 loops=10000)\\n                Buffers: shared hit=2213, temp read=5710068 written=855\\n                ->  Seq Scan on banned  (cost=0.00..7213.00 rows=500000 width=4) (actual rows=500000 loops=1)\\n                      Buffers: shared hit=2213\\nPlanning:\\n  Buffers: shared hit=26\\nPlanning Time: 0.096 ms\\nJIT:\\n  Functions: 7\\n  Options: Inlining true, Optimization true, Expressions true, Deforming true\\nExecution Time: 314769.094 ms\",\"runtimeDurationMs\":314770,\"rowsAreCapped\":false,\"computeSeconds\":315.341}],\"workerId\":\"3c6af982-d5e9-40bb-80df-cfe51440eeb0\",\"correlationId\":\"34110ee7-8c59-4f5a-8afe-029834d88f50\",\"correlationIdInstruction\":\"Echo correlationId on subsequent ExoBench tool calls. Tracking only — it does NOT preserve DB state.\",\"version\":\"1.4.26\"}",
      "isError": false,
      "caption": "NOT IN on PostgreSQL 17.11 with banned.customer_id declared without NOT NULL, against 262,145 and 500,000 banned ids: one execution each, EXPLAIN (ANALYZE, TIMING OFF, BUFFERS)",
      "planCaptions": {
        "10K / 262.1K": "At 262,145 banned ids PG17 stops hashing the nullable list as well: the SubPlan reads a Materialize from temp files on disk, rescanned 10,000 times, once per order. The one execution takes 165,261.775 ms.",
        "10K / 500K": "At 500,000 banned ids the same plain SubPlan rescans a longer list, 333,906 stored ids per pass on average, all of it read back from temp files. The one execution takes 314,769.094 ms."
      }
    },
    "nullableRescan18": {
      "type": "mcp-call",
      "name": "benchmarkSql",
      "input": {
        "schemaTemplate": "CREATE TABLE orders (id int PRIMARY KEY, customer_id int NOT NULL, total numeric(10,2) NOT NULL);\nINSERT INTO orders SELECT i, i, (i % 1000) + 0.99 FROM generate_series(1, ${O}) i;\nCREATE TABLE banned (customer_id int);\nINSERT INTO banned SELECT i * 3 FROM generate_series(1, ${B}) i;\nVACUUM ANALYZE;",
        "query": "EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT count(*), sum(total) FROM orders WHERE customer_id NOT IN (SELECT customer_id FROM banned)",
        "scalePoints": [
          {
            "O": 10000,
            "B": 262145
          },
          {
            "O": 10000,
            "B": 500000
          }
        ],
        "databaseType": "POSTGRES_18",
        "seed": 4242,
        "waitTimeoutSeconds": 110,
        "correlationId": "34110ee7-8c59-4f5a-8afe-029834d88f50",
        "exobenchServerVersion": "1.4.26"
      },
      "result": "{\"timestamp\":\"2026-11-13T04:18:08.983406032Z\",\"databaseType\":\"postgres_18\",\"sql\":\"EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT count(*), sum(total) FROM orders WHERE customer_id NOT IN (SELECT customer_id FROM banned)\",\"warmCacheIterations\":0,\"seed\":4242,\"seedSource\":\"explicit\",\"scalePointCount\":2,\"totalComputeSeconds\":473.13599999999997,\"results\":[{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":262145},\"output\":\"QUERY PLAN\\nAggregate  (cost=33862892.50..33862892.51 rows=1 width=40) (actual rows=1.00 loops=1)\\n  Buffers: shared hit=1215, temp read=3003672 written=449\\n  ->  Seq Scan on orders  (cost=0.00..33862867.50 rows=5000 width=6) (actual rows=6667.00 loops=1)\\n        Filter: (NOT (ANY (customer_id = (SubPlan 1).col1)))\\n        Rows Removed by Filter: 3333\\n        Buffers: shared hit=1215, temp read=3003672 written=449\\n        SubPlan 1\\n          ->  Materialize  (cost=0.00..6117.18 rows=262145 width=4) (actual rows=175327.68 loops=10000)\\n                Storage: Disk  Maximum Storage: 4096kB\\n                Buffers: shared hit=1160, temp read=3003672 written=449\\n                ->  Seq Scan on banned  (cost=0.00..3781.45 rows=262145 width=4) (actual rows=262145.00 loops=1)\\n                      Buffers: shared hit=1160\\nPlanning:\\n  Buffers: shared hit=26\\nPlanning Time: 0.108 ms\\nJIT:\\n  Functions: 7\\n  Options: Inlining true, Optimization true, Expressions true, Deforming true\\nExecution Time: 162743.662 ms\",\"runtimeDurationMs\":162744,\"rowsAreCapped\":false,\"computeSeconds\":163.138},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":500000},\"output\":\"QUERY PLAN\\nAggregate  (cost=64585205.00..64585205.01 rows=1 width=40) (actual rows=1.00 loops=1)\\n  Buffers: shared hit=2268, temp read=5710068 written=855\\n  ->  Seq Scan on orders  (cost=0.00..64585180.00 rows=5000 width=6) (actual rows=6667.00 loops=1)\\n        Filter: (NOT (ANY (customer_id = (SubPlan 1).col1)))\\n        Rows Removed by Filter: 3333\\n        Buffers: shared hit=2268, temp read=5710068 written=855\\n        SubPlan 1\\n          ->  Materialize  (cost=0.00..11667.00 rows=500000 width=4) (actual rows=333905.61 loops=10000)\\n                Storage: Disk  Maximum Storage: 6836kB\\n                Buffers: shared hit=2213, temp read=5710068 written=855\\n                ->  Seq Scan on banned  (cost=0.00..7213.00 rows=500000 width=4) (actual rows=500000.00 loops=1)\\n                      Buffers: shared hit=2213\\nPlanning:\\n  Buffers: shared hit=26\\nPlanning Time: 0.103 ms\\nJIT:\\n  Functions: 7\\n  Options: Inlining true, Optimization true, Expressions true, Deforming true\\nExecution Time: 309423.976 ms\",\"runtimeDurationMs\":309425,\"rowsAreCapped\":false,\"computeSeconds\":309.998}],\"workerId\":\"9cd262ac-dcb8-4791-883b-c246fa7893b7\",\"correlationId\":\"34110ee7-8c59-4f5a-8afe-029834d88f50\",\"correlationIdInstruction\":\"Echo correlationId on subsequent ExoBench tool calls. Tracking only — it does NOT preserve DB state.\",\"version\":\"1.4.26\"}",
      "isError": false,
      "caption": "NOT IN on PostgreSQL 18.6 with banned.customer_id declared without NOT NULL, against 262,145 and 500,000 banned ids: one execution each, EXPLAIN (ANALYZE, TIMING OFF, BUFFERS)",
      "planCaptions": {
        "10K / 262.1K": "At 262,145 banned ids PG18 stops hashing the nullable list: the SubPlan reads a Materialize stored on disk and rescanned 10,000 times, once per order. The one execution takes 162,743.662 ms.",
        "10K / 500K": "At 500,000 banned ids the Materialize still sits on disk, and each rescan reads 333,905.61 rows on average. The one execution takes 309,423.976 ms."
      }
    },
    "cliff18Hashed": {
      "type": "mcp-call",
      "name": "benchmarkSql",
      "input": {
        "schemaTemplate": "CREATE TABLE orders (id int PRIMARY KEY, customer_id int NOT NULL, total numeric(10,2) NOT NULL);\nINSERT INTO orders SELECT i, i, (i % 1000) + 0.99 FROM generate_series(1, ${O}) i;\nCREATE TABLE banned (customer_id int NOT NULL);\nINSERT INTO banned SELECT i * 3 FROM generate_series(1, ${B}) i;\nVACUUM ANALYZE;",
        "query": "EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT count(*), sum(total) FROM orders WHERE customer_id NOT IN (SELECT customer_id FROM banned)",
        "scalePoints": [
          {
            "O": 10000,
            "B": 100000
          },
          {
            "O": 10000,
            "B": 154198
          },
          {
            "O": 10000,
            "B": 154199
          },
          {
            "O": 10000,
            "B": 262144
          }
        ],
        "databaseType": "POSTGRES_18",
        "seed": 4242,
        "waitTimeoutSeconds": 110,
        "correlationId": "34110ee7-8c59-4f5a-8afe-029834d88f50",
        "exobenchServerVersion": "1.4.26"
      },
      "result": "{\"timestamp\":\"2026-11-13T03:05:58.333909402Z\",\"databaseType\":\"postgres_18\",\"sql\":\"EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT count(*), sum(total) FROM orders WHERE customer_id NOT IN (SELECT customer_id FROM banned)\",\"warmCacheIterations\":0,\"seed\":4242,\"seedSource\":\"explicit\",\"scalePointCount\":4,\"totalComputeSeconds\":1.313,\"results\":[{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":100000},\"output\":\"QUERY PLAN\\nAggregate  (cost=1898.00..1898.01 rows=1 width=40) (actual rows=1.00 loops=1)\\n  Buffers: shared hit=498\\n  ->  Seq Scan on orders  (cost=1693.00..1873.00 rows=5000 width=6) (actual rows=6667.00 loops=1)\\n        Filter: (NOT (ANY (customer_id = (hashed SubPlan 1).col1)))\\n        Rows Removed by Filter: 3333\\n        Buffers: shared hit=498\\n        SubPlan 1\\n          ->  Seq Scan on banned  (cost=0.00..1443.00 rows=100000 width=4) (actual rows=100000.00 loops=1)\\n                Buffers: shared hit=443\\nPlanning:\\n  Buffers: shared hit=26\\nPlanning Time: 0.132 ms\\nExecution Time: 24.079 ms\",\"runtimeDurationMs\":24,\"rowsAreCapped\":false,\"computeSeconds\":0.236},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":154198},\"output\":\"QUERY PLAN\\nAggregate  (cost=2815.48..2815.49 rows=1 width=40) (actual rows=1.00 loops=1)\\n  Buffers: shared hit=738\\n  ->  Seq Scan on orders  (cost=2610.47..2790.47 rows=5000 width=6) (actual rows=6667.00 loops=1)\\n        Filter: (NOT (ANY (customer_id = (hashed SubPlan 1).col1)))\\n        Rows Removed by Filter: 3333\\n        Buffers: shared hit=738\\n        SubPlan 1\\n          ->  Seq Scan on banned  (cost=0.00..2224.98 rows=154198 width=4) (actual rows=154198.00 loops=1)\\n                Buffers: shared hit=683\\nPlanning:\\n  Buffers: shared hit=26\\nPlanning Time: 0.125 ms\\nExecution Time: 42.746 ms\",\"runtimeDurationMs\":43,\"rowsAreCapped\":false,\"computeSeconds\":0.327},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":154199},\"output\":\"QUERY PLAN\\nAggregate  (cost=2815.49..2815.50 rows=1 width=40) (actual rows=1.00 loops=1)\\n  Buffers: shared hit=738\\n  ->  Seq Scan on orders  (cost=2610.49..2790.49 rows=5000 width=6) (actual rows=6667.00 loops=1)\\n        Filter: (NOT (ANY (customer_id = (hashed SubPlan 1).col1)))\\n        Rows Removed by Filter: 3333\\n        Buffers: shared hit=738\\n        SubPlan 1\\n          ->  Seq Scan on banned  (cost=0.00..2224.99 rows=154199 width=4) (actual rows=154199.00 loops=1)\\n                Buffers: shared hit=683\\nPlanning:\\n  Buffers: shared hit=26\\nPlanning Time: 0.107 ms\\nExecution Time: 43.270 ms\",\"runtimeDurationMs\":44,\"rowsAreCapped\":false,\"computeSeconds\":0.311},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":262144},\"output\":\"QUERY PLAN\\nAggregate  (cost=4641.80..4641.81 rows=1 width=40) (actual rows=1.00 loops=1)\\n  Buffers: shared hit=1215\\n  ->  Seq Scan on orders  (cost=4436.80..4616.80 rows=5000 width=6) (actual rows=6667.00 loops=1)\\n        Filter: (NOT (ANY (customer_id = (hashed SubPlan 1).col1)))\\n        Rows Removed by Filter: 3333\\n        Buffers: shared hit=1215\\n        SubPlan 1\\n          ->  Seq Scan on banned  (cost=0.00..3781.44 rows=262144 width=4) (actual rows=262144.00 loops=1)\\n                Buffers: shared hit=1160\\nPlanning:\\n  Buffers: shared hit=26\\nPlanning Time: 0.128 ms\\nExecution Time: 75.690 ms\",\"runtimeDurationMs\":77,\"rowsAreCapped\":false,\"computeSeconds\":0.439}],\"workerId\":\"300eb822-133e-4a82-8c20-5b52a1ade07a\",\"correlationId\":\"34110ee7-8c59-4f5a-8afe-029834d88f50\",\"correlationIdInstruction\":\"Echo correlationId on subsequent ExoBench tool calls. Tracking only — it does NOT preserve DB state.\",\"version\":\"1.4.26\"}",
      "isError": false,
      "caption": "NOT IN on PostgreSQL 18.6 with both columns declared NOT NULL, 10,000 orders against four list sizes up to 262,144 banned ids: one execution each, EXPLAIN (ANALYZE, TIMING OFF, BUFFERS)",
      "planCaptions": {
        "10K / 100K": "At 100,000 banned ids PG18 runs the NOT IN as a hashed SubPlan, not an anti join, even with both columns declared NOT NULL. One execution, 24.079 ms, keeping 6,667 orders.",
        "10K / 154.2K#1": "At 154,198 banned ids PG18 still hashes the list from one Seq Scan of banned. One execution, 42.746 ms, 738 shared buffer hits.",
        "10K / 154.2K#2": "At 154,199 banned ids the plan does not change on PG18: the same hashed SubPlan. One execution, 43.270 ms.",
        "10K / 262.1K": "At 262,144 banned ids, one row short of its cliff, PG18 still hashes, reading 1,215 shared buffers. One execution, 75.690 ms."
      }
    },
    "cliff18Rescan": {
      "type": "mcp-call",
      "name": "benchmarkSql",
      "input": {
        "schemaTemplate": "CREATE TABLE orders (id int PRIMARY KEY, customer_id int NOT NULL, total numeric(10,2) NOT NULL);\nINSERT INTO orders SELECT i, i, (i % 1000) + 0.99 FROM generate_series(1, ${O}) i;\nCREATE TABLE banned (customer_id int NOT NULL);\nINSERT INTO banned SELECT i * 3 FROM generate_series(1, ${B}) i;\nVACUUM ANALYZE;",
        "query": "EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT count(*), sum(total) FROM orders WHERE customer_id NOT IN (SELECT customer_id FROM banned)",
        "scalePoints": [
          {
            "O": 10000,
            "B": 262145
          },
          {
            "O": 10000,
            "B": 500000
          }
        ],
        "databaseType": "POSTGRES_18",
        "seed": 4242,
        "waitTimeoutSeconds": 110,
        "correlationId": "34110ee7-8c59-4f5a-8afe-029834d88f50",
        "exobenchServerVersion": "1.4.26"
      },
      "result": "{\"timestamp\":\"2026-11-13T03:14:34.090238208Z\",\"databaseType\":\"postgres_18\",\"sql\":\"EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT count(*), sum(total) FROM orders WHERE customer_id NOT IN (SELECT customer_id FROM banned)\",\"warmCacheIterations\":0,\"seed\":4242,\"seedSource\":\"explicit\",\"scalePointCount\":2,\"totalComputeSeconds\":507.58000000000004,\"results\":[{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":262145},\"output\":\"QUERY PLAN\\nAggregate  (cost=33862892.50..33862892.51 rows=1 width=40) (actual rows=1.00 loops=1)\\n  Buffers: shared hit=1215, temp read=3003672 written=449\\n  ->  Seq Scan on orders  (cost=0.00..33862867.50 rows=5000 width=6) (actual rows=6667.00 loops=1)\\n        Filter: (NOT (ANY (customer_id = (SubPlan 1).col1)))\\n        Rows Removed by Filter: 3333\\n        Buffers: shared hit=1215, temp read=3003672 written=449\\n        SubPlan 1\\n          ->  Materialize  (cost=0.00..6117.18 rows=262145 width=4) (actual rows=175327.68 loops=10000)\\n                Storage: Disk  Maximum Storage: 4096kB\\n                Buffers: shared hit=1160, temp read=3003672 written=449\\n                ->  Seq Scan on banned  (cost=0.00..3781.45 rows=262145 width=4) (actual rows=262145.00 loops=1)\\n                      Buffers: shared hit=1160\\nPlanning:\\n  Buffers: shared hit=26\\nPlanning Time: 0.114 ms\\nJIT:\\n  Functions: 7\\n  Options: Inlining true, Optimization true, Expressions true, Deforming true\\nExecution Time: 165878.249 ms\",\"runtimeDurationMs\":165879,\"rowsAreCapped\":false,\"computeSeconds\":166.23},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":500000},\"output\":\"QUERY PLAN\\nAggregate  (cost=64585205.00..64585205.01 rows=1 width=40) (actual rows=1.00 loops=1)\\n  Buffers: shared hit=2268, temp read=5710068 written=855\\n  ->  Seq Scan on orders  (cost=0.00..64585180.00 rows=5000 width=6) (actual rows=6667.00 loops=1)\\n        Filter: (NOT (ANY (customer_id = (SubPlan 1).col1)))\\n        Rows Removed by Filter: 3333\\n        Buffers: shared hit=2268, temp read=5710068 written=855\\n        SubPlan 1\\n          ->  Materialize  (cost=0.00..11667.00 rows=500000 width=4) (actual rows=333905.61 loops=10000)\\n                Storage: Disk  Maximum Storage: 6836kB\\n                Buffers: shared hit=2213, temp read=5710068 written=855\\n                ->  Seq Scan on banned  (cost=0.00..7213.00 rows=500000 width=4) (actual rows=500000.00 loops=1)\\n                      Buffers: shared hit=2213\\nPlanning:\\n  Buffers: shared hit=26\\nPlanning Time: 0.167 ms\\nJIT:\\n  Functions: 7\\n  Options: Inlining true, Optimization true, Expressions true, Deforming true\\nExecution Time: 340746.169 ms\",\"runtimeDurationMs\":340748,\"rowsAreCapped\":false,\"computeSeconds\":341.35}],\"workerId\":\"fe3b13a8-6d45-45a9-bce2-b542dca77255\",\"correlationId\":\"34110ee7-8c59-4f5a-8afe-029834d88f50\",\"correlationIdInstruction\":\"Echo correlationId on subsequent ExoBench tool calls. Tracking only — it does NOT preserve DB state.\",\"version\":\"1.4.26\"}",
      "isError": false,
      "caption": "NOT IN on PostgreSQL 18.6 with both columns declared NOT NULL, against 262,145 and 500,000 banned ids: one execution each, EXPLAIN (ANALYZE, TIMING OFF, BUFFERS)",
      "planCaptions": {
        "10K / 262.1K": "At 262,145 banned ids, one more than the largest list it hashes, PG18 switches to a plain SubPlan over a Materialize stored on disk and rescanned 10,000 times. One execution, 165,878.249 ms.",
        "10K / 500K": "At 500,000 banned ids the plain SubPlan rescans 333,905.61 stored ids per pass on average from its disk-backed Materialize. One execution, 340,746.169 ms."
      }
    },
    "notExists17": {
      "type": "mcp-call",
      "name": "benchmarkSql",
      "input": {
        "schemaTemplate": "CREATE TABLE orders (id int PRIMARY KEY, customer_id int NOT NULL, total numeric(10,2) NOT NULL);\nINSERT INTO orders SELECT i, i, (i % 1000) + 0.99 FROM generate_series(1, ${O}) i;\nCREATE TABLE banned (customer_id int NOT NULL);\nINSERT INTO banned SELECT i * 3 FROM generate_series(1, ${B}) i;\nVACUUM ANALYZE;",
        "query": "EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT count(*), sum(total) FROM orders o WHERE NOT EXISTS (SELECT 1 FROM banned b WHERE b.customer_id = o.customer_id)",
        "scalePoints": [
          {
            "O": 10000,
            "B": 100000
          },
          {
            "O": 10000,
            "B": 154198
          },
          {
            "O": 10000,
            "B": 154199
          },
          {
            "O": 10000,
            "B": 262144
          },
          {
            "O": 10000,
            "B": 262145
          }
        ],
        "databaseType": "POSTGRES",
        "seed": 4242,
        "waitTimeoutSeconds": 110,
        "correlationId": "34110ee7-8c59-4f5a-8afe-029834d88f50",
        "exobenchServerVersion": "1.4.26"
      },
      "result": "{\"timestamp\":\"2026-11-13T03:15:31.030629878Z\",\"databaseType\":\"postgres\",\"sql\":\"EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT count(*), sum(total) FROM orders o WHERE NOT EXISTS (SELECT 1 FROM banned b WHERE b.customer_id = o.customer_id)\",\"warmCacheIterations\":0,\"seed\":4242,\"seedSource\":\"explicit\",\"scalePointCount\":5,\"totalComputeSeconds\":1.603,\"results\":[{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":100000},\"output\":\"QUERY PLAN\\nAggregate  (cost=2198.01..2198.02 rows=1 width=40) (actual rows=1 loops=1)\\n  Buffers: shared hit=498\\n  ->  Hash Right Anti Join  (cost=280.00..2198.00 rows=1 width=6) (actual rows=6667 loops=1)\\n        Hash Cond: (b.customer_id = o.customer_id)\\n        Buffers: shared hit=498\\n        ->  Seq Scan on banned b  (cost=0.00..1443.00 rows=100000 width=4) (actual rows=100000 loops=1)\\n              Buffers: shared hit=443\\n        ->  Hash  (cost=155.00..155.00 rows=10000 width=10) (actual rows=10000 loops=1)\\n              Buckets: 16384  Batches: 1  Memory Usage: 558kB\\n              Buffers: shared hit=55\\n              ->  Seq Scan on orders o  (cost=0.00..155.00 rows=10000 width=10) (actual rows=10000 loops=1)\\n                    Buffers: shared hit=55\\nPlanning:\\n  Buffers: shared hit=78\\nPlanning Time: 0.228 ms\\nExecution Time: 14.730 ms\",\"runtimeDurationMs\":16,\"rowsAreCapped\":false,\"computeSeconds\":0.249},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":154198},\"output\":\"QUERY PLAN\\nAggregate  (cost=3183.23..3183.24 rows=1 width=40) (actual rows=1 loops=1)\\n  Buffers: shared hit=738\\n  ->  Hash Right Anti Join  (cost=280.00..3183.22 rows=1 width=6) (actual rows=6667 loops=1)\\n        Hash Cond: (b.customer_id = o.customer_id)\\n        Buffers: shared hit=738\\n        ->  Seq Scan on banned b  (cost=0.00..2224.98 rows=154198 width=4) (actual rows=154198 loops=1)\\n              Buffers: shared hit=683\\n        ->  Hash  (cost=155.00..155.00 rows=10000 width=10) (actual rows=10000 loops=1)\\n              Buckets: 16384  Batches: 1  Memory Usage: 558kB\\n              Buffers: shared hit=55\\n              ->  Seq Scan on orders o  (cost=0.00..155.00 rows=10000 width=10) (actual rows=10000 loops=1)\\n                    Buffers: shared hit=55\\nPlanning:\\n  Buffers: shared hit=78\\nPlanning Time: 0.251 ms\\nExecution Time: 20.228 ms\",\"runtimeDurationMs\":21,\"rowsAreCapped\":false,\"computeSeconds\":0.272},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":154199},\"output\":\"QUERY PLAN\\nAggregate  (cost=3183.24..3183.25 rows=1 width=40) (actual rows=1 loops=1)\\n  Buffers: shared hit=738\\n  ->  Hash Right Anti Join  (cost=280.00..3183.24 rows=1 width=6) (actual rows=6667 loops=1)\\n        Hash Cond: (b.customer_id = o.customer_id)\\n        Buffers: shared hit=738\\n        ->  Seq Scan on banned b  (cost=0.00..2224.99 rows=154199 width=4) (actual rows=154199 loops=1)\\n              Buffers: shared hit=683\\n        ->  Hash  (cost=155.00..155.00 rows=10000 width=10) (actual rows=10000 loops=1)\\n              Buckets: 16384  Batches: 1  Memory Usage: 558kB\\n              Buffers: shared hit=55\\n              ->  Seq Scan on orders o  (cost=0.00..155.00 rows=10000 width=10) (actual rows=10000 loops=1)\\n                    Buffers: shared hit=55\\nPlanning:\\n  Buffers: shared hit=78\\nPlanning Time: 0.174 ms\\nExecution Time: 21.754 ms\",\"runtimeDurationMs\":23,\"rowsAreCapped\":false,\"computeSeconds\":0.296},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":262144},\"output\":\"QUERY PLAN\\nAggregate  (cost=5144.49..5144.50 rows=1 width=40) (actual rows=1 loops=1)\\n  Buffers: shared hit=1215\\n  ->  Hash Right Anti Join  (cost=280.00..5144.48 rows=1 width=6) (actual rows=6667 loops=1)\\n        Hash Cond: (b.customer_id = o.customer_id)\\n        Buffers: shared hit=1215\\n        ->  Seq Scan on banned b  (cost=0.00..3781.44 rows=262144 width=4) (actual rows=262144 loops=1)\\n              Buffers: shared hit=1160\\n        ->  Hash  (cost=155.00..155.00 rows=10000 width=10) (actual rows=10000 loops=1)\\n              Buckets: 16384  Batches: 1  Memory Usage: 558kB\\n              Buffers: shared hit=55\\n              ->  Seq Scan on orders o  (cost=0.00..155.00 rows=10000 width=10) (actual rows=10000 loops=1)\\n                    Buffers: shared hit=55\\nPlanning:\\n  Buffers: shared hit=78\\nPlanning Time: 0.184 ms\\nExecution Time: 32.076 ms\",\"runtimeDurationMs\":33,\"rowsAreCapped\":false,\"computeSeconds\":0.379},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":262145},\"output\":\"QUERY PLAN\\nAggregate  (cost=5144.50..5144.51 rows=1 width=40) (actual rows=1 loops=1)\\n  Buffers: shared hit=1215\\n  ->  Hash Right Anti Join  (cost=280.00..5144.49 rows=1 width=6) (actual rows=6667 loops=1)\\n        Hash Cond: (b.customer_id = o.customer_id)\\n        Buffers: shared hit=1215\\n        ->  Seq Scan on banned b  (cost=0.00..3781.45 rows=262145 width=4) (actual rows=262145 loops=1)\\n              Buffers: shared hit=1160\\n        ->  Hash  (cost=155.00..155.00 rows=10000 width=10) (actual rows=10000 loops=1)\\n              Buckets: 16384  Batches: 1  Memory Usage: 558kB\\n              Buffers: shared hit=55\\n              ->  Seq Scan on orders o  (cost=0.00..155.00 rows=10000 width=10) (actual rows=10000 loops=1)\\n                    Buffers: shared hit=55\\nPlanning:\\n  Buffers: shared hit=78\\nPlanning Time: 0.226 ms\\nExecution Time: 40.291 ms\",\"runtimeDurationMs\":41,\"rowsAreCapped\":false,\"computeSeconds\":0.407}],\"workerId\":\"a72ecfd9-c388-4932-b6d5-71d578ec6a3d\",\"correlationId\":\"34110ee7-8c59-4f5a-8afe-029834d88f50\",\"correlationIdInstruction\":\"Echo correlationId on subsequent ExoBench tool calls. Tracking only — it does NOT preserve DB state.\",\"version\":\"1.4.26\"}",
      "isError": false,
      "caption": "The orders query written with NOT EXISTS on PostgreSQL 17.11, both columns declared NOT NULL, 10,000 orders against five list sizes up to 262,145 banned ids: one execution each, EXPLAIN (ANALYZE, TIMING OFF, BUFFERS)",
      "planCaptions": {
        "10K / 100K": "At 100,000 banned ids PG17 already plans NOT EXISTS as a Hash Right Anti Join: the orders go into a 558 kB hash and the banned list streams past it. One execution, 14.730 ms.",
        "10K / 154.2K#1": "At 154,198 banned ids the NOT EXISTS plan is unchanged, the same Hash Right Anti Join. One execution, 20.228 ms, 738 shared buffer hits.",
        "10K / 154.2K#2": "At 154,199 banned ids PG17's NOT EXISTS is still that anti join. One execution, 21.754 ms, keeping 6,667 orders.",
        "10K / 262.1K#1": "At 262,144 banned ids the NOT EXISTS anti join reads 1,215 shared buffers. One execution, 32.076 ms.",
        "10K / 262.1K#2": "At 262,145 banned ids, where PG17's NOT IN falls to rescanning the list once per order, NOT EXISTS stays a Hash Right Anti Join. One execution, 40.291 ms."
      }
    },
    "notExists18": {
      "type": "mcp-call",
      "name": "benchmarkSql",
      "input": {
        "schemaTemplate": "CREATE TABLE orders (id int PRIMARY KEY, customer_id int NOT NULL, total numeric(10,2) NOT NULL);\nINSERT INTO orders SELECT i, i, (i % 1000) + 0.99 FROM generate_series(1, ${O}) i;\nCREATE TABLE banned (customer_id int NOT NULL);\nINSERT INTO banned SELECT i * 3 FROM generate_series(1, ${B}) i;\nVACUUM ANALYZE;",
        "query": "EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT count(*), sum(total) FROM orders o WHERE NOT EXISTS (SELECT 1 FROM banned b WHERE b.customer_id = o.customer_id)",
        "scalePoints": [
          {
            "O": 10000,
            "B": 100000
          },
          {
            "O": 10000,
            "B": 154198
          },
          {
            "O": 10000,
            "B": 154199
          },
          {
            "O": 10000,
            "B": 262144
          },
          {
            "O": 10000,
            "B": 262145
          }
        ],
        "databaseType": "POSTGRES_18",
        "seed": 4242,
        "waitTimeoutSeconds": 110,
        "correlationId": "34110ee7-8c59-4f5a-8afe-029834d88f50",
        "exobenchServerVersion": "1.4.26"
      },
      "result": "{\"timestamp\":\"2026-11-13T03:15:33.475549284Z\",\"databaseType\":\"postgres_18\",\"sql\":\"EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT count(*), sum(total) FROM orders o WHERE NOT EXISTS (SELECT 1 FROM banned b WHERE b.customer_id = o.customer_id)\",\"warmCacheIterations\":0,\"seed\":4242,\"seedSource\":\"explicit\",\"scalePointCount\":5,\"totalComputeSeconds\":1.55,\"results\":[{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":100000},\"output\":\"QUERY PLAN\\nAggregate  (cost=2198.01..2198.02 rows=1 width=40) (actual rows=1.00 loops=1)\\n  Buffers: shared hit=498\\n  ->  Hash Right Anti Join  (cost=280.00..2198.00 rows=1 width=6) (actual rows=6667.00 loops=1)\\n        Hash Cond: (b.customer_id = o.customer_id)\\n        Buffers: shared hit=498\\n        ->  Seq Scan on banned b  (cost=0.00..1443.00 rows=100000 width=4) (actual rows=100000.00 loops=1)\\n              Buffers: shared hit=443\\n        ->  Hash  (cost=155.00..155.00 rows=10000 width=10) (actual rows=10000.00 loops=1)\\n              Buckets: 16384  Batches: 1  Memory Usage: 558kB\\n              Buffers: shared hit=55\\n              ->  Seq Scan on orders o  (cost=0.00..155.00 rows=10000 width=10) (actual rows=10000.00 loops=1)\\n                    Buffers: shared hit=55\\nPlanning:\\n  Buffers: shared hit=86\\nPlanning Time: 0.245 ms\\nExecution Time: 12.489 ms\",\"runtimeDurationMs\":13,\"rowsAreCapped\":false,\"computeSeconds\":0.206},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":154198},\"output\":\"QUERY PLAN\\nAggregate  (cost=3183.23..3183.24 rows=1 width=40) (actual rows=1.00 loops=1)\\n  Buffers: shared hit=738\\n  ->  Hash Right Anti Join  (cost=280.00..3183.22 rows=1 width=6) (actual rows=6667.00 loops=1)\\n        Hash Cond: (b.customer_id = o.customer_id)\\n        Buffers: shared hit=738\\n        ->  Seq Scan on banned b  (cost=0.00..2224.98 rows=154198 width=4) (actual rows=154198.00 loops=1)\\n              Buffers: shared hit=683\\n        ->  Hash  (cost=155.00..155.00 rows=10000 width=10) (actual rows=10000.00 loops=1)\\n              Buckets: 16384  Batches: 1  Memory Usage: 558kB\\n              Buffers: shared hit=55\\n              ->  Seq Scan on orders o  (cost=0.00..155.00 rows=10000 width=10) (actual rows=10000.00 loops=1)\\n                    Buffers: shared hit=55\\nPlanning:\\n  Buffers: shared hit=86\\nPlanning Time: 0.202 ms\\nExecution Time: 18.024 ms\",\"runtimeDurationMs\":19,\"rowsAreCapped\":false,\"computeSeconds\":0.299},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":154199},\"output\":\"QUERY PLAN\\nAggregate  (cost=3183.24..3183.25 rows=1 width=40) (actual rows=1.00 loops=1)\\n  Buffers: shared hit=738\\n  ->  Hash Right Anti Join  (cost=280.00..3183.24 rows=1 width=6) (actual rows=6667.00 loops=1)\\n        Hash Cond: (b.customer_id = o.customer_id)\\n        Buffers: shared hit=738\\n        ->  Seq Scan on banned b  (cost=0.00..2224.99 rows=154199 width=4) (actual rows=154199.00 loops=1)\\n              Buffers: shared hit=683\\n        ->  Hash  (cost=155.00..155.00 rows=10000 width=10) (actual rows=10000.00 loops=1)\\n              Buckets: 16384  Batches: 1  Memory Usage: 558kB\\n              Buffers: shared hit=55\\n              ->  Seq Scan on orders o  (cost=0.00..155.00 rows=10000 width=10) (actual rows=10000.00 loops=1)\\n                    Buffers: shared hit=55\\nPlanning:\\n  Buffers: shared hit=86\\nPlanning Time: 0.202 ms\\nExecution Time: 17.029 ms\",\"runtimeDurationMs\":18,\"rowsAreCapped\":false,\"computeSeconds\":0.279},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":262144},\"output\":\"QUERY PLAN\\nAggregate  (cost=5144.49..5144.50 rows=1 width=40) (actual rows=1.00 loops=1)\\n  Buffers: shared hit=1215\\n  ->  Hash Right Anti Join  (cost=280.00..5144.48 rows=1 width=6) (actual rows=6667.00 loops=1)\\n        Hash Cond: (b.customer_id = o.customer_id)\\n        Buffers: shared hit=1215\\n        ->  Seq Scan on banned b  (cost=0.00..3781.44 rows=262144 width=4) (actual rows=262144.00 loops=1)\\n              Buffers: shared hit=1160\\n        ->  Hash  (cost=155.00..155.00 rows=10000 width=10) (actual rows=10000.00 loops=1)\\n              Buckets: 16384  Batches: 1  Memory Usage: 558kB\\n              Buffers: shared hit=55\\n              ->  Seq Scan on orders o  (cost=0.00..155.00 rows=10000 width=10) (actual rows=10000.00 loops=1)\\n                    Buffers: shared hit=55\\nPlanning:\\n  Buffers: shared hit=86\\nPlanning Time: 0.207 ms\\nExecution Time: 29.547 ms\",\"runtimeDurationMs\":30,\"rowsAreCapped\":false,\"computeSeconds\":0.387},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":262145},\"output\":\"QUERY PLAN\\nAggregate  (cost=5144.50..5144.51 rows=1 width=40) (actual rows=1.00 loops=1)\\n  Buffers: shared hit=1215\\n  ->  Hash Right Anti Join  (cost=280.00..5144.49 rows=1 width=6) (actual rows=6667.00 loops=1)\\n        Hash Cond: (b.customer_id = o.customer_id)\\n        Buffers: shared hit=1215\\n        ->  Seq Scan on banned b  (cost=0.00..3781.45 rows=262145 width=4) (actual rows=262145.00 loops=1)\\n              Buffers: shared hit=1160\\n        ->  Hash  (cost=155.00..155.00 rows=10000 width=10) (actual rows=10000.00 loops=1)\\n              Buckets: 16384  Batches: 1  Memory Usage: 558kB\\n              Buffers: shared hit=55\\n              ->  Seq Scan on orders o  (cost=0.00..155.00 rows=10000 width=10) (actual rows=10000.00 loops=1)\\n                    Buffers: shared hit=55\\nPlanning:\\n  Buffers: shared hit=86\\nPlanning Time: 0.267 ms\\nExecution Time: 26.699 ms\",\"runtimeDurationMs\":27,\"rowsAreCapped\":false,\"computeSeconds\":0.379}],\"workerId\":\"7dd8969d-a602-4a43-bbf7-2effa1846a31\",\"correlationId\":\"34110ee7-8c59-4f5a-8afe-029834d88f50\",\"correlationIdInstruction\":\"Echo correlationId on subsequent ExoBench tool calls. Tracking only — it does NOT preserve DB state.\",\"version\":\"1.4.26\"}",
      "isError": false,
      "caption": "The orders query written with NOT EXISTS on PostgreSQL 18.6, both columns declared NOT NULL, 10,000 orders against five list sizes up to 262,145 banned ids: one execution each, EXPLAIN (ANALYZE, TIMING OFF, BUFFERS)",
      "planCaptions": {
        "10K / 100K": "At 100,000 banned ids PG18 plans NOT EXISTS as a Hash Right Anti Join: the orders are hashed and the banned list streams past them. One execution, 12.489 ms.",
        "10K / 154.2K#1": "At 154,198 banned ids the NOT EXISTS plan is unchanged, the same Hash Right Anti Join. One execution, 18.024 ms.",
        "10K / 154.2K#2": "At 154,199 banned ids PG18's NOT EXISTS is still that anti join. One execution, 17.029 ms, keeping 6,667 orders.",
        "10K / 262.1K#1": "At 262,144 banned ids the NOT EXISTS anti join is unchanged. One execution, 29.547 ms.",
        "10K / 262.1K#2": "At 262,145 banned ids, where PG18's NOT IN falls to rescanning the list once per order, NOT EXISTS stays a Hash Right Anti Join. One execution, 26.699 ms."
      }
    },
    "notIn19Repeated": {
      "type": "mcp-call",
      "name": "benchmarkSql",
      "input": {
        "schemaTemplate": "CREATE TABLE orders (id int PRIMARY KEY, customer_id int NOT NULL, total numeric(10,2) NOT NULL);\nINSERT INTO orders SELECT i, i, (i % 1000) + 0.99 FROM generate_series(1, ${O}) i;\nCREATE TABLE banned (customer_id int NOT NULL);\nINSERT INTO banned SELECT i * 3 FROM generate_series(1, ${B}) i;\nVACUUM ANALYZE;",
        "query": "EXPLAIN (ANALYZE, BUFFERS) SELECT count(*), sum(total) FROM orders WHERE customer_id NOT IN (SELECT customer_id FROM banned)",
        "scalePoints": [
          {
            "O": 10000,
            "B": 154199
          },
          {
            "O": 10000,
            "B": 500000
          }
        ],
        "databaseType": "POSTGRES_19",
        "warmCache": "0",
        "repetitions": "10",
        "seed": 4242,
        "correlationId": "34110ee7-8c59-4f5a-8afe-029834d88f50",
        "exobenchServerVersion": "1.4.26"
      },
      "result": "{\"timestamp\":\"2026-11-13T04:51:20.375897613Z\",\"databaseType\":\"postgres_19\",\"sql\":\"EXPLAIN (ANALYZE, BUFFERS) SELECT count(*), sum(total) FROM orders WHERE customer_id NOT IN (SELECT customer_id FROM banned)\",\"warmCacheIterations\":0,\"repetitions\":10,\"seed\":4242,\"seedSource\":\"explicit\",\"scalePointCount\":2,\"totalComputeSeconds\":1.5410000000000001,\"results\":[{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":154199},\"output\":\"QUERY PLAN\\nStatistics over 10 runs: avg = mean, stddev = sample standard deviation, omitted where 0.\\nAggregate  (avg_cost=3183.24..3183.25 avg_rows=1 avg_width=40) (actual avg_time=17.629..17.630 stddev_time=0.538..0.538 avg_rows=1 avg_loops=1)\\n  Buffers: shared avg_hit=738\\n  Disabled: false\\n  ->  Hash Right Anti Join  (avg_cost=280.00..3183.24 avg_rows=1 avg_width=6) (actual avg_time=16.589..17.094 stddev_time=0.561..0.542 avg_rows=6667 avg_loops=1)\\n        Hash Cond: (banned.customer_id = orders.customer_id)\\n        Buffers: shared avg_hit=738\\n        Disabled: false\\n        ->  Seq Scan on banned  (avg_cost=0.00..2224.99 avg_rows=154199 avg_width=4) (actual avg_time=0.005..4.956 stddev_time=0.004..0.164 avg_rows=154199 avg_loops=1)\\n              Buffers: shared avg_hit=683\\n              Disabled: false\\n        ->  Hash  (avg_cost=155.00..155.00 avg_rows=10000 avg_width=10) (actual avg_time=1.773..1.774 stddev_time=0.485..0.486 avg_rows=10000 avg_loops=1)\\n              Buckets: avg=16384  Batches: avg=1  Memory Usage: avg=558kB\\n              Buffers: shared avg_hit=55\\n              Disabled: false\\n              ->  Seq Scan on orders  (avg_cost=0.00..155.00 avg_rows=10000 avg_width=10) (actual avg_time=0.007..0.693 stddev_time=0.001..0.216 avg_rows=10000 avg_loops=1)\\n                    Buffers: shared avg_hit=55\\n                    Disabled: false\\nPlanning:\\n  Buffers: shared avg_hit=9.1 stddev_hit=28.8\\nPlanning Time: avg=0.12 ms stddev=0.08 n=10 samples=0.33,0.09,0.08,0.09,0.09,0.09,0.1,0.15,0.09,0.08\\nExecution Time: avg=17.66 ms stddev=0.54 n=10 samples=18.1,17.1,17.08,17.56,18.24,18.01,17.43,18.58,17.36,17.12\",\"runtimeDurationMs\":18,\"rowsAreCapped\":false,\"computeSeconds\":0.435},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":500000},\"output\":\"QUERY PLAN\\nStatistics over 10 runs: avg = mean, stddev = sample standard deviation, omitted where 0.\\nAggregate  (avg_cost=9468.01..9468.02 avg_rows=1 avg_width=40) (actual avg_time=52.262..52.264 stddev_time=3.372..3.372 avg_rows=1 avg_loops=1)\\n  Buffers: shared avg_hit=2268\\n  Disabled: false\\n  ->  Hash Right Anti Join  (avg_cost=280.00..9468.00 avg_rows=1 avg_width=6) (actual avg_time=51.205..51.720 stddev_time=3.373..3.371 avg_rows=6667 avg_loops=1)\\n        Hash Cond: (banned.customer_id = orders.customer_id)\\n        Buffers: shared avg_hit=2268\\n        Disabled: false\\n        ->  Seq Scan on banned  (avg_cost=0.00..7213.00 avg_rows=500000 avg_width=4) (actual avg_time=0.004..16.685 stddev_time=0.001..1.211 avg_rows=500000 avg_loops=1)\\n              Buffers: shared avg_hit=2213\\n              Disabled: false\\n        ->  Hash  (avg_cost=155.00..155.00 avg_rows=10000 avg_width=10) (actual avg_time=1.577..1.577 stddev_time=0.164..0.164 avg_rows=10000 avg_loops=1)\\n              Buckets: avg=16384  Batches: avg=1  Memory Usage: avg=558kB\\n              Buffers: shared avg_hit=55\\n              Disabled: false\\n              ->  Seq Scan on orders  (avg_cost=0.00..155.00 avg_rows=10000 avg_width=10) (actual avg_time=0.007..0.599 stddev_time=0.001..0.078 avg_rows=10000 avg_loops=1)\\n                    Buffers: shared avg_hit=55\\n                    Disabled: false\\nPlanning:\\n  Buffers: shared avg_hit=8.9 stddev_hit=28.1\\nPlanning Time: avg=0.12 ms stddev=0.06 n=10 samples=0.29,0.1,0.09,0.11,0.09,0.09,0.09,0.09,0.09,0.17\\nExecution Time: avg=52.29 ms stddev=3.37 n=10 samples=51.1,57.94,49.4,51.64,49.75,50.68,58.78,52.75,51.5,49.39\",\"runtimeDurationMs\":53,\"rowsAreCapped\":false,\"computeSeconds\":1.106}],\"workerId\":\"1921906f-909b-4b80-b4e2-95148fd9d84b\",\"correlationId\":\"34110ee7-8c59-4f5a-8afe-029834d88f50\",\"correlationIdInstruction\":\"Echo correlationId on subsequent ExoBench tool calls. Tracking only — it does NOT preserve DB state.\",\"version\":\"1.4.26\"}",
      "isError": false,
      "caption": "NOT IN on PostgreSQL 19rc2 with both columns declared NOT NULL, 10,000 orders against 154,199 and 500,000 banned ids: 10 runs, no warm-up, EXPLAIN (ANALYZE, BUFFERS) with per-node timing",
      "planCaptions": {
        "10K / 154.2K": "Over 10 runs at 154,199 banned ids, 19rc2 plans the NOT IN as a Hash Right Anti Join with the same cost and the same shared buffers per run as NOT EXISTS on this list. Mean Execution Time 17.66 ms, stddev 0.54 ms.",
        "10K / 500K": "At 500,000 banned ids the anti join's cost and buffer count again match NOT EXISTS on the same list. Mean Execution Time over 10 runs is 52.29 ms, stddev 3.37 ms."
      }
    },
    "notExists19Repeated": {
      "type": "mcp-call",
      "name": "benchmarkSql",
      "input": {
        "schemaTemplate": "CREATE TABLE orders (id int PRIMARY KEY, customer_id int NOT NULL, total numeric(10,2) NOT NULL);\nINSERT INTO orders SELECT i, i, (i % 1000) + 0.99 FROM generate_series(1, ${O}) i;\nCREATE TABLE banned (customer_id int NOT NULL);\nINSERT INTO banned SELECT i * 3 FROM generate_series(1, ${B}) i;\nVACUUM ANALYZE;",
        "query": "EXPLAIN (ANALYZE, BUFFERS) SELECT count(*), sum(total) FROM orders o WHERE NOT EXISTS (SELECT 1 FROM banned b WHERE b.customer_id = o.customer_id)",
        "scalePoints": [
          {
            "O": 10000,
            "B": 154199
          },
          {
            "O": 10000,
            "B": 500000
          }
        ],
        "databaseType": "POSTGRES_19",
        "warmCache": "0",
        "repetitions": "10",
        "seed": 4242,
        "correlationId": "34110ee7-8c59-4f5a-8afe-029834d88f50",
        "exobenchServerVersion": "1.4.26"
      },
      "result": "{\"timestamp\":\"2026-11-13T04:51:22.156948943Z\",\"databaseType\":\"postgres_19\",\"sql\":\"EXPLAIN (ANALYZE, BUFFERS) SELECT count(*), sum(total) FROM orders o WHERE NOT EXISTS (SELECT 1 FROM banned b WHERE b.customer_id = o.customer_id)\",\"warmCacheIterations\":0,\"repetitions\":10,\"seed\":4242,\"seedSource\":\"explicit\",\"scalePointCount\":2,\"totalComputeSeconds\":1.5499999999999998,\"results\":[{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":154199},\"output\":\"QUERY PLAN\\nStatistics over 10 runs: avg = mean, stddev = sample standard deviation, omitted where 0.\\nAggregate  (avg_cost=3183.24..3183.25 avg_rows=1 avg_width=40) (actual avg_time=18.244..18.248 stddev_time=1.189..1.188 avg_rows=1 avg_loops=1)\\n  Buffers: shared avg_hit=738\\n  Disabled: false\\n  ->  Hash Right Anti Join  (avg_cost=280.00..3183.24 avg_rows=1 avg_width=6) (actual avg_time=17.120..17.673 stddev_time=1.133..1.149 avg_rows=6667 avg_loops=1)\\n        Hash Cond: (b.customer_id = o.customer_id)\\n        Buffers: shared avg_hit=738\\n        Disabled: false\\n        ->  Seq Scan on banned b  (avg_cost=0.00..2224.99 avg_rows=154199 avg_width=4) (actual avg_time=0.007..5.063 stddev_time=0.005..0.198 avg_rows=154199 avg_loops=1)\\n              Buffers: shared avg_hit=683\\n              Disabled: false\\n        ->  Hash  (avg_cost=155.00..155.00 avg_rows=10000 avg_width=10) (actual avg_time=1.754..1.755 stddev_time=0.349..0.349 avg_rows=10000 avg_loops=1)\\n              Buckets: avg=16384  Batches: avg=1  Memory Usage: avg=558kB\\n              Buffers: shared avg_hit=55\\n              Disabled: false\\n              ->  Seq Scan on orders o  (avg_cost=0.00..155.00 avg_rows=10000 avg_width=10) (actual avg_time=0.009..0.688 stddev_time=0.007..0.158 avg_rows=10000 avg_loops=1)\\n                    Buffers: shared avg_hit=55\\n                    Disabled: false\\nPlanning:\\n  Buffers: shared avg_hit=8.9 stddev_hit=28.1\\nPlanning Time: avg=0.11 ms stddev=0.04 n=10 samples=0.22,0.08,0.09,0.08,0.13,0.08,0.08,0.1,0.11,0.08\\nExecution Time: avg=18.28 ms stddev=1.19 n=10 samples=17.82,18.73,18.12,21.04,19.4,16.96,17.6,17.97,17.32,17.81\",\"runtimeDurationMs\":19,\"rowsAreCapped\":false,\"computeSeconds\":0.485},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":500000},\"output\":\"QUERY PLAN\\nStatistics over 10 runs: avg = mean, stddev = sample standard deviation, omitted where 0.\\nAggregate  (avg_cost=9468.01..9468.02 avg_rows=1 avg_width=40) (actual avg_time=49.124..49.126 stddev_time=1.624..1.624 avg_rows=1 avg_loops=1)\\n  Buffers: shared avg_hit=2268\\n  Disabled: false\\n  ->  Hash Right Anti Join  (avg_cost=280.00..9468.00 avg_rows=1 avg_width=6) (actual avg_time=48.063..48.570 stddev_time=1.582..1.591 avg_rows=6667 avg_loops=1)\\n        Hash Cond: (b.customer_id = o.customer_id)\\n        Buffers: shared avg_hit=2268\\n        Disabled: false\\n        ->  Seq Scan on banned b  (avg_cost=0.00..7213.00 avg_rows=500000 avg_width=4) (actual avg_time=0.005..15.618 stddev_time=0.005..0.566 avg_rows=500000 avg_loops=1)\\n              Buffers: shared avg_hit=2213\\n              Disabled: false\\n        ->  Hash  (avg_cost=155.00..155.00 avg_rows=10000 avg_width=10) (actual avg_time=1.546..1.547 stddev_time=0.124..0.124 avg_rows=10000 avg_loops=1)\\n              Buckets: avg=16384  Batches: avg=1  Memory Usage: avg=558kB\\n              Buffers: shared avg_hit=55\\n              Disabled: false\\n              ->  Seq Scan on orders o  (avg_cost=0.00..155.00 avg_rows=10000 avg_width=10) (actual avg_time=0.009..0.593 stddev_time=0.007..0.062 avg_rows=10000 avg_loops=1)\\n                    Buffers: shared avg_hit=55\\n                    Disabled: false\\nPlanning:\\n  Buffers: shared avg_hit=8.8 stddev_hit=27.8\\nPlanning Time: avg=0.11 ms stddev=0.05 n=10 samples=0.24,0.08,0.12,0.09,0.11,0.09,0.08,0.08,0.08,0.08\\nExecution Time: avg=49.15 ms stddev=1.63 n=10 samples=52.91,49.09,48.25,48.06,48.48,48.48,51.27,47.97,48.28,48.74\",\"runtimeDurationMs\":50,\"rowsAreCapped\":false,\"computeSeconds\":1.065}],\"workerId\":\"8fc511c0-ca70-4ab4-b549-f55a27ec8dd2\",\"correlationId\":\"34110ee7-8c59-4f5a-8afe-029834d88f50\",\"correlationIdInstruction\":\"Echo correlationId on subsequent ExoBench tool calls. Tracking only — it does NOT preserve DB state.\",\"version\":\"1.4.26\"}",
      "isError": false,
      "caption": "NOT EXISTS on PostgreSQL 19rc2 with both columns declared NOT NULL, 10,000 orders against 154,199 and 500,000 banned ids: 10 runs, no warm-up, EXPLAIN (ANALYZE, BUFFERS) with per-node timing",
      "planCaptions": {
        "10K / 154.2K": "At 154,199 banned ids NOT EXISTS gets the Hash Right Anti Join 19rc2 builds for the rewritten NOT IN, at the same cost and the same shared buffers per run. Mean Execution Time over 10 runs is 18.28 ms, stddev 1.19 ms.",
        "10K / 500K": "At 500,000 banned ids NOT EXISTS and the rewritten NOT IN again share one plan, cost and buffer count. Mean Execution Time over 10 runs is 49.15 ms, stddev 1.63 ms."
      }
    },
    "filtered19": {
      "type": "mcp-call",
      "name": "benchmarkSql",
      "input": {
        "schemaTemplate": "CREATE TABLE orders (id int PRIMARY KEY, customer_id int NOT NULL, total numeric(10,2) NOT NULL);\nINSERT INTO orders SELECT i, i, (i % 1000) + 0.99 FROM generate_series(1, ${O}) i;\nCREATE TABLE banned (customer_id int);\nINSERT INTO banned SELECT i * 3 FROM generate_series(1, ${B}) i;\nVACUUM ANALYZE;",
        "query": "EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT count(*), sum(total) FROM orders WHERE customer_id NOT IN (SELECT customer_id FROM banned WHERE customer_id IS NOT NULL)",
        "scalePoints": [
          {
            "O": 10000,
            "B": 100000
          },
          {
            "O": 10000,
            "B": 154198
          },
          {
            "O": 10000,
            "B": 154199
          },
          {
            "O": 10000,
            "B": 262144
          },
          {
            "O": 10000,
            "B": 262145
          }
        ],
        "databaseType": "POSTGRES_19",
        "seed": 4242,
        "waitTimeoutSeconds": 110,
        "correlationId": "34110ee7-8c59-4f5a-8afe-029834d88f50",
        "exobenchServerVersion": "1.4.26"
      },
      "result": "{\"timestamp\":\"2026-11-13T03:16:35.942860035Z\",\"databaseType\":\"postgres_19\",\"sql\":\"EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT count(*), sum(total) FROM orders WHERE customer_id NOT IN (SELECT customer_id FROM banned WHERE customer_id IS NOT NULL)\",\"warmCacheIterations\":0,\"seed\":4242,\"seedSource\":\"explicit\",\"scalePointCount\":5,\"totalComputeSeconds\":1.5910000000000002,\"results\":[{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":100000},\"output\":\"QUERY PLAN\\nAggregate  (cost=2198.01..2198.02 rows=1 width=40) (actual rows=1.00 loops=1)\\n  Buffers: shared hit=498\\n  ->  Hash Right Anti Join  (cost=280.00..2198.00 rows=1 width=6) (actual rows=6667.00 loops=1)\\n        Hash Cond: (banned.customer_id = orders.customer_id)\\n        Buffers: shared hit=498\\n        ->  Seq Scan on banned  (cost=0.00..1443.00 rows=100000 width=4) (actual rows=100000.00 loops=1)\\n              Filter: (customer_id IS NOT NULL)\\n              Buffers: shared hit=443\\n        ->  Hash  (cost=155.00..155.00 rows=10000 width=10) (actual rows=10000.00 loops=1)\\n              Buckets: 16384  Batches: 1  Memory Usage: 558kB\\n              Buffers: shared hit=55\\n              ->  Seq Scan on orders  (cost=0.00..155.00 rows=10000 width=10) (actual rows=10000.00 loops=1)\\n                    Buffers: shared hit=55\\nPlanning:\\n  Buffers: shared hit=89\\nPlanning Time: 0.224 ms\\nExecution Time: 12.497 ms\",\"runtimeDurationMs\":13,\"rowsAreCapped\":false,\"computeSeconds\":0.216},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":154198},\"output\":\"QUERY PLAN\\nAggregate  (cost=3183.23..3183.24 rows=1 width=40) (actual rows=1.00 loops=1)\\n  Buffers: shared hit=738\\n  ->  Hash Right Anti Join  (cost=280.00..3183.22 rows=1 width=6) (actual rows=6667.00 loops=1)\\n        Hash Cond: (banned.customer_id = orders.customer_id)\\n        Buffers: shared hit=738\\n        ->  Seq Scan on banned  (cost=0.00..2224.98 rows=154198 width=4) (actual rows=154198.00 loops=1)\\n              Filter: (customer_id IS NOT NULL)\\n              Buffers: shared hit=683\\n        ->  Hash  (cost=155.00..155.00 rows=10000 width=10) (actual rows=10000.00 loops=1)\\n              Buckets: 16384  Batches: 1  Memory Usage: 558kB\\n              Buffers: shared hit=55\\n              ->  Seq Scan on orders  (cost=0.00..155.00 rows=10000 width=10) (actual rows=10000.00 loops=1)\\n                    Buffers: shared hit=55\\nPlanning:\\n  Buffers: shared hit=87\\nPlanning Time: 0.272 ms\\nExecution Time: 20.894 ms\",\"runtimeDurationMs\":22,\"rowsAreCapped\":false,\"computeSeconds\":0.267},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":154199},\"output\":\"QUERY PLAN\\nAggregate  (cost=3183.24..3183.25 rows=1 width=40) (actual rows=1.00 loops=1)\\n  Buffers: shared hit=738\\n  ->  Hash Right Anti Join  (cost=280.00..3183.24 rows=1 width=6) (actual rows=6667.00 loops=1)\\n        Hash Cond: (banned.customer_id = orders.customer_id)\\n        Buffers: shared hit=738\\n        ->  Seq Scan on banned  (cost=0.00..2224.99 rows=154199 width=4) (actual rows=154199.00 loops=1)\\n              Filter: (customer_id IS NOT NULL)\\n              Buffers: shared hit=683\\n        ->  Hash  (cost=155.00..155.00 rows=10000 width=10) (actual rows=10000.00 loops=1)\\n              Buckets: 16384  Batches: 1  Memory Usage: 558kB\\n              Buffers: shared hit=55\\n              ->  Seq Scan on orders  (cost=0.00..155.00 rows=10000 width=10) (actual rows=10000.00 loops=1)\\n                    Buffers: shared hit=55\\nPlanning:\\n  Buffers: shared hit=87\\nPlanning Time: 0.235 ms\\nExecution Time: 20.332 ms\",\"runtimeDurationMs\":22,\"rowsAreCapped\":false,\"computeSeconds\":0.352},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":262144},\"output\":\"QUERY PLAN\\nAggregate  (cost=5144.49..5144.50 rows=1 width=40) (actual rows=1.00 loops=1)\\n  Buffers: shared hit=1215\\n  ->  Hash Right Anti Join  (cost=280.00..5144.48 rows=1 width=6) (actual rows=6667.00 loops=1)\\n        Hash Cond: (banned.customer_id = orders.customer_id)\\n        Buffers: shared hit=1215\\n        ->  Seq Scan on banned  (cost=0.00..3781.44 rows=262144 width=4) (actual rows=262144.00 loops=1)\\n              Filter: (customer_id IS NOT NULL)\\n              Buffers: shared hit=1160\\n        ->  Hash  (cost=155.00..155.00 rows=10000 width=10) (actual rows=10000.00 loops=1)\\n              Buckets: 16384  Batches: 1  Memory Usage: 558kB\\n              Buffers: shared hit=55\\n              ->  Seq Scan on orders  (cost=0.00..155.00 rows=10000 width=10) (actual rows=10000.00 loops=1)\\n                    Buffers: shared hit=55\\nPlanning:\\n  Buffers: shared hit=88\\nPlanning Time: 0.352 ms\\nExecution Time: 27.767 ms\",\"runtimeDurationMs\":29,\"rowsAreCapped\":false,\"computeSeconds\":0.398},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":262145},\"output\":\"QUERY PLAN\\nAggregate  (cost=5144.50..5144.51 rows=1 width=40) (actual rows=1.00 loops=1)\\n  Buffers: shared hit=1215\\n  ->  Hash Right Anti Join  (cost=280.00..5144.49 rows=1 width=6) (actual rows=6667.00 loops=1)\\n        Hash Cond: (banned.customer_id = orders.customer_id)\\n        Buffers: shared hit=1215\\n        ->  Seq Scan on banned  (cost=0.00..3781.45 rows=262145 width=4) (actual rows=262145.00 loops=1)\\n              Filter: (customer_id IS NOT NULL)\\n              Buffers: shared hit=1160\\n        ->  Hash  (cost=155.00..155.00 rows=10000 width=10) (actual rows=10000.00 loops=1)\\n              Buckets: 16384  Batches: 1  Memory Usage: 558kB\\n              Buffers: shared hit=55\\n              ->  Seq Scan on orders  (cost=0.00..155.00 rows=10000 width=10) (actual rows=10000.00 loops=1)\\n                    Buffers: shared hit=55\\nPlanning:\\n  Buffers: shared hit=88\\nPlanning Time: 0.231 ms\\nExecution Time: 27.848 ms\",\"runtimeDurationMs\":29,\"rowsAreCapped\":false,\"computeSeconds\":0.358}],\"workerId\":\"09a277a0-a92d-4af5-b212-7c67c1d33c53\",\"correlationId\":\"34110ee7-8c59-4f5a-8afe-029834d88f50\",\"correlationIdInstruction\":\"Echo correlationId on subsequent ExoBench tool calls. Tracking only — it does NOT preserve DB state.\",\"version\":\"1.4.26\"}",
      "isError": false,
      "caption": "NOT IN on PostgreSQL 19rc2 with banned.customer_id declared without NOT NULL and WHERE customer_id IS NOT NULL inside the subquery, 10,000 orders against five list sizes up to 262,145 banned ids: one execution each, EXPLAIN (ANALYZE, TIMING OFF, BUFFERS)",
      "planCaptions": {
        "10K / 100K": "At 100,000 banned ids the IS NOT NULL inside the subquery is enough for 19rc2 to rewrite the NOT IN into a Hash Right Anti Join, with the filter applied in the Seq Scan on banned. One execution, 12.497 ms, keeping 6,667 orders.",
        "10K / 154.2K#1": "At 154,198 banned ids the filtered list is still an anti join against a hash of the orders. One execution, 20.894 ms.",
        "10K / 154.2K#2": "At 154,199 banned ids, where the same nullable list without the filter falls to a plain SubPlan on 19rc2, the filtered one stays a Hash Right Anti Join. One execution, 20.332 ms, 738 shared buffer hits.",
        "10K / 262.1K#1": "At 262,144 banned ids the filtered NOT IN is still the Hash Right Anti Join. One execution, 27.767 ms, 1,215 shared buffer hits.",
        "10K / 262.1K#2": "At 262,145 banned ids the plan does not change either: a Hash Right Anti Join with Filter (customer_id IS NOT NULL) on the banned scan. One execution, 27.848 ms."
      }
    },
    "ordersNullable19": {
      "type": "mcp-call",
      "name": "benchmarkSql",
      "input": {
        "schemaTemplate": "CREATE TABLE orders (id int PRIMARY KEY, customer_id int, total numeric(10,2) NOT NULL);\nINSERT INTO orders SELECT i, i, (i % 1000) + 0.99 FROM generate_series(1, ${O}) i;\nCREATE TABLE banned (customer_id int NOT NULL);\nINSERT INTO banned SELECT i * 3 FROM generate_series(1, ${B}) i;\nVACUUM ANALYZE;",
        "query": "EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT count(*), sum(total) FROM orders WHERE customer_id NOT IN (SELECT customer_id FROM banned)",
        "scalePoints": [
          {
            "O": 10000,
            "B": 100000
          },
          {
            "O": 10000,
            "B": 154198
          },
          {
            "O": 10000,
            "B": 154199
          },
          {
            "O": 10000,
            "B": 262144
          }
        ],
        "databaseType": "POSTGRES_19",
        "seed": 4242,
        "waitTimeoutSeconds": 110,
        "correlationId": "34110ee7-8c59-4f5a-8afe-029834d88f50",
        "exobenchServerVersion": "1.4.26"
      },
      "result": "{\"timestamp\":\"2026-11-13T03:35:35.978568432Z\",\"databaseType\":\"postgres_19\",\"sql\":\"EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT count(*), sum(total) FROM orders WHERE customer_id NOT IN (SELECT customer_id FROM banned)\",\"warmCacheIterations\":0,\"seed\":4242,\"seedSource\":\"explicit\",\"scalePointCount\":4,\"totalComputeSeconds\":282.56100000000004,\"results\":[{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":100000},\"output\":\"QUERY PLAN\\nAggregate  (cost=1898.00..1898.01 rows=1 width=40) (actual rows=1.00 loops=1)\\n  Buffers: shared hit=498\\n  ->  Seq Scan on orders  (cost=1693.00..1873.00 rows=5000 width=6) (actual rows=6667.00 loops=1)\\n        Filter: (NOT (ANY (customer_id = (hashed SubPlan any_1).col1)))\\n        Rows Removed by Filter: 3333\\n        Buffers: shared hit=498\\n        SubPlan any_1\\n          ->  Seq Scan on banned  (cost=0.00..1443.00 rows=100000 width=4) (actual rows=100000.00 loops=1)\\n                Buffers: shared hit=443\\nPlanning:\\n  Buffers: shared hit=33\\nPlanning Time: 0.185 ms\\nExecution Time: 25.934 ms\",\"runtimeDurationMs\":26,\"rowsAreCapped\":false,\"computeSeconds\":0.26},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":154198},\"output\":\"QUERY PLAN\\nAggregate  (cost=2815.48..2815.49 rows=1 width=40) (actual rows=1.00 loops=1)\\n  Buffers: shared hit=738\\n  ->  Seq Scan on orders  (cost=2610.47..2790.47 rows=5000 width=6) (actual rows=6667.00 loops=1)\\n        Filter: (NOT (ANY (customer_id = (hashed SubPlan any_1).col1)))\\n        Rows Removed by Filter: 3333\\n        Buffers: shared hit=738\\n        SubPlan any_1\\n          ->  Seq Scan on banned  (cost=0.00..2224.98 rows=154198 width=4) (actual rows=154198.00 loops=1)\\n                Buffers: shared hit=683\\nPlanning:\\n  Buffers: shared hit=32\\nPlanning Time: 0.274 ms\\nExecution Time: 43.766 ms\",\"runtimeDurationMs\":45,\"rowsAreCapped\":false,\"computeSeconds\":0.354},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":154199},\"output\":\"QUERY PLAN\\nAggregate  (cost=19922617.50..19922617.51 rows=1 width=40) (actual rows=1.00 loops=1)\\n  Buffers: shared hit=738, temp read=1770462 written=264\\n  ->  Seq Scan on orders  (cost=0.00..19922592.50 rows=5000 width=6) (actual rows=6667.00 loops=1)\\n        Filter: (NOT (ANY (customer_id = (SubPlan any_1).col1)))\\n        Rows Removed by Filter: 3333\\n        Buffers: shared hit=738, temp read=1770462 written=264\\n        SubPlan any_1\\n          ->  Materialize  (cost=0.00..3598.98 rows=154199 width=4) (actual rows=103360.08 loops=10000)\\n                Storage: Disk  Maximum Storage: 4096kB\\n                Buffers: shared hit=683, temp read=1770462 written=264\\n                ->  Seq Scan on banned  (cost=0.00..2224.99 rows=154199 width=4) (actual rows=154199.00 loops=1)\\n                      Buffers: shared hit=683\\nPlanning:\\n  Buffers: shared hit=33\\nPlanning Time: 0.146 ms\\nExecution Time: 116371.893 ms\",\"runtimeDurationMs\":116050,\"rowsAreCapped\":false,\"computeSeconds\":116.334},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":262144},\"output\":\"QUERY PLAN\\nAggregate  (cost=33857805.00..33857805.01 rows=1 width=40) (actual rows=1.00 loops=1)\\n  Buffers: shared hit=1215, temp read=2997006 written=448\\n  ->  Seq Scan on orders  (cost=0.00..33857780.00 rows=5000 width=6) (actual rows=6667.00 loops=1)\\n        Filter: (NOT (ANY (customer_id = (SubPlan any_1).col1)))\\n        Rows Removed by Filter: 3333\\n        Buffers: shared hit=1215, temp read=2997006 written=448\\n        SubPlan any_1\\n          ->  Materialize  (cost=0.00..6116.16 rows=262144 width=4) (actual rows=175327.02 loops=10000)\\n                Storage: Disk  Maximum Storage: 4096kB\\n                Buffers: shared hit=1160, temp read=2997006 written=448\\n                ->  Seq Scan on banned  (cost=0.00..3781.44 rows=262144 width=4) (actual rows=262144.00 loops=1)\\n                      Buffers: shared hit=1160\\nPlanning:\\n  Buffers: shared hit=34\\nPlanning Time: 0.207 ms\\nExecution Time: 165691.856 ms\",\"runtimeDurationMs\":165233,\"rowsAreCapped\":false,\"computeSeconds\":165.613}],\"workerId\":\"1d1b11fb-77f6-4098-be4f-1e717211fb19\",\"correlationId\":\"34110ee7-8c59-4f5a-8afe-029834d88f50\",\"correlationIdInstruction\":\"Echo correlationId on subsequent ExoBench tool calls. Tracking only — it does NOT preserve DB state.\",\"version\":\"1.4.26\"}",
      "isError": false,
      "caption": "NOT IN on PostgreSQL 19rc2 with orders.customer_id, the outer column, declared without NOT NULL and the banned list NOT NULL, 10,000 orders against four list sizes up to 262,144 banned ids: one execution each, EXPLAIN (ANALYZE, TIMING OFF, BUFFERS)",
      "planCaptions": {
        "10K / 100K": "With the orders column nullable, 19rc2 does not rewrite the NOT IN either: at 100,000 banned ids it runs a hashed SubPlan over one Seq Scan of banned. One execution, 25.934 ms.",
        "10K / 154.2K#1": "At 154,198 banned ids the NOT IN with a nullable orders column is still hashed on 19rc2. One execution, 43.766 ms, 738 shared buffer hits.",
        "10K / 154.2K#2": "At 154,199 banned ids it stops hashing at the same row as the nullable ban list: a plain SubPlan over a Materialize on disk, rescanned 10,000 times. One execution, 116,371.893 ms.",
        "10K / 262.1K": "At 262,144 banned ids the plain SubPlan rescans 175,327.02 stored ids per pass on average from disk. The one execution takes 165,691.856 ms."
      }
    },
    "outerFilter19": {
      "type": "mcp-call",
      "name": "benchmarkSql",
      "input": {
        "schemaTemplate": "CREATE TABLE orders (id int PRIMARY KEY, customer_id int, total numeric(10,2) NOT NULL);\nINSERT INTO orders SELECT i, i, (i % 1000) + 0.99 FROM generate_series(1, ${O}) i;\nCREATE TABLE banned (customer_id int NOT NULL);\nINSERT INTO banned SELECT i * 3 FROM generate_series(1, ${B}) i;\nVACUUM ANALYZE;",
        "query": "EXPLAIN SELECT count(*), sum(total) FROM orders WHERE customer_id IS NOT NULL AND customer_id NOT IN (SELECT customer_id FROM banned)",
        "scalePoints": [
          {
            "O": 10000,
            "B": 262144
          }
        ],
        "databaseType": "POSTGRES_19",
        "warmCache": "0",
        "seed": 4242,
        "correlationId": "34110ee7-8c59-4f5a-8afe-029834d88f50",
        "exobenchServerVersion": "1.4.26"
      },
      "result": "{\"timestamp\":\"2026-11-13T04:50:53.639146068Z\",\"databaseType\":\"postgres_19\",\"sql\":\"EXPLAIN SELECT count(*), sum(total) FROM orders WHERE customer_id IS NOT NULL AND customer_id NOT IN (SELECT customer_id FROM banned)\",\"warmCacheIterations\":0,\"seed\":4242,\"seedSource\":\"explicit\",\"scalePointCount\":1,\"totalComputeSeconds\":0.382,\"results\":[{\"kind\":\"simple_row_set\",\"scalePoint\":{\"O\":10000,\"B\":262144},\"output\":\"QUERY PLAN\\nAggregate  (cost=33857805.00..33857805.01 rows=1 width=40)\\n  ->  Seq Scan on orders  (cost=0.00..33857780.00 rows=5000 width=6)\\n        Filter: ((customer_id IS NOT NULL) AND (NOT (ANY (customer_id = (SubPlan any_1).col1))))\\n        SubPlan any_1\\n          ->  Materialize  (cost=0.00..6116.16 rows=262144 width=4)\\n                ->  Seq Scan on banned  (cost=0.00..3781.44 rows=262144 width=4)\",\"runtimeDurationMs\":1,\"rowsAreCapped\":false,\"computeSeconds\":0.382}],\"workerId\":\"2f275f6c-4940-48c0-9699-ca87eb8c676a\",\"correlationId\":\"34110ee7-8c59-4f5a-8afe-029834d88f50\",\"correlationIdInstruction\":\"Echo correlationId on subsequent ExoBench tool calls. Tracking only — it does NOT preserve DB state.\",\"version\":\"1.4.26\"}",
      "isError": false,
      "caption": "Plan only, not executed: NOT IN on PostgreSQL 19rc2 with orders.customer_id declared without NOT NULL and customer_id IS NOT NULL added to the outer WHERE, 10,000 orders against 262,144 banned ids, plain EXPLAIN",
      "planCaptions": {
        "10K / 262.1K": "Plan only, so no timing: with customer_id IS NOT NULL in the outer WHERE, 19rc2 still does not rewrite the NOT IN. The Seq Scan on orders checks the IS NOT NULL and then a SubPlan over a Materialize of the 262,144 banned ids, the plain form, with no anti join anywhere in the plan."
      }
    },
    "million19": {
      "type": "mcp-call",
      "name": "benchmarkSql",
      "input": {
        "schemaTemplate": "CREATE TABLE orders (id int PRIMARY KEY, customer_id int NOT NULL, total numeric(10,2) NOT NULL);\nINSERT INTO orders SELECT i, i, (i % 1000) + 0.99 FROM generate_series(1, ${O}) i;\nCREATE TABLE banned (customer_id int NOT NULL);\nINSERT INTO banned SELECT i * 3 FROM generate_series(1, ${B}) i;\nVACUUM ANALYZE;",
        "query": "EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT count(*), sum(total) FROM orders WHERE customer_id NOT IN (SELECT customer_id FROM banned)",
        "scalePoints": [
          {
            "O": 1000000,
            "B": 300000
          }
        ],
        "databaseType": "POSTGRES_19",
        "preBenchmarkSql": "SET statement_timeout = '120s';",
        "warmCache": "0",
        "seed": 4242,
        "correlationId": "34110ee7-8c59-4f5a-8afe-029834d88f50",
        "exobenchServerVersion": "1.4.26"
      },
      "result": "{\"timestamp\":\"2026-11-13T04:48:32.357761439Z\",\"databaseType\":\"postgres_19\",\"sql\":\"EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT count(*), sum(total) FROM orders WHERE customer_id NOT IN (SELECT customer_id FROM banned)\",\"warmCacheIterations\":0,\"seed\":4242,\"seedSource\":\"explicit\",\"scalePointCount\":1,\"totalComputeSeconds\":2.786,\"results\":[{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":1000000,\"B\":300000},\"output\":\"QUERY PLAN\\nFinalize Aggregate  (cost=21480.87..21480.88 rows=1 width=40) (actual rows=1.00 loops=1)\\n  Buffers: shared hit=6734\\n  ->  Gather  (cost=21480.64..21480.85 rows=2 width=40) (actual rows=3.00 loops=1)\\n        Workers Planned: 2\\n        Workers Launched: 2\\n        Buffers: shared hit=6734\\n        ->  Partial Aggregate  (cost=20480.64..20480.65 rows=1 width=40) (actual rows=1.00 loops=3)\\n              Buffers: shared hit=6734\\n              ->  Parallel Hash Anti Join  (cost=5298.59..19022.31 rows=291667 width=6) (actual rows=233333.33 loops=3)\\n                    Hash Cond: (orders.customer_id = banned.customer_id)\\n                    Buffers: shared hit=6734\\n                    ->  Parallel Seq Scan on orders  (cost=0.00..9572.67 rows=416667 width=10) (actual rows=333333.33 loops=3)\\n                          Buffers: shared hit=5406\\n                    ->  Parallel Hash  (cost=3092.71..3092.71 rows=176471 width=4) (actual rows=100000.00 loops=3)\\n                          Buckets: 524288  Batches: 1  Memory Usage: 15872kB\\n                          Buffers: shared hit=1328\\n                          ->  Parallel Seq Scan on banned  (cost=0.00..3092.71 rows=176471 width=4) (actual rows=100000.00 loops=3)\\n                                Buffers: shared hit=1328\\nPlanning:\\n  Buffers: shared hit=98\\nPlanning Time: 0.383 ms\\nExecution Time: 145.789 ms\",\"runtimeDurationMs\":146,\"rowsAreCapped\":false,\"computeSeconds\":2.786}],\"workerId\":\"d733b96f-f140-4332-80d0-20c5a47ee5ce\",\"correlationId\":\"34110ee7-8c59-4f5a-8afe-029834d88f50\",\"correlationIdInstruction\":\"Echo correlationId on subsequent ExoBench tool calls. Tracking only — it does NOT preserve DB state.\",\"version\":\"1.4.26\"}",
      "isError": false,
      "caption": "NOT IN on PostgreSQL 19rc2 with both columns declared NOT NULL, 1,000,000 orders against 300,000 banned ids, under SET statement_timeout = '120s': one execution, EXPLAIN (ANALYZE, TIMING OFF, BUFFERS)",
      "planCaptions": {
        "1M / 300K": "At 1,000,000 orders and 300,000 banned ids 19rc2 runs the NOT IN as a Parallel Hash Anti Join with two workers launched. The three processes share one hash of the banned list, built in a single batch, and the one execution takes 145.789 ms."
      }
    },
    "equal17": {
      "type": "mcp-call",
      "name": "benchmarkSql",
      "input": {
        "schemaTemplate": "CREATE TABLE orders (id int PRIMARY KEY, customer_id int NOT NULL, total numeric(10,2) NOT NULL);\nINSERT INTO orders SELECT i, i, (i % 1000) + 0.99 FROM generate_series(1, ${O}) i;\nCREATE TABLE banned (customer_id int NOT NULL);\nINSERT INTO banned SELECT i * 3 FROM generate_series(1, ${B}) i;\nVACUUM ANALYZE;",
        "query": "EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT count(*), sum(total) FROM orders WHERE customer_id NOT IN (SELECT customer_id FROM banned)",
        "scalePoints": [
          {
            "O": 1000000,
            "B": 100000
          },
          {
            "O": 1000000,
            "B": 262144
          }
        ],
        "databaseType": "POSTGRES",
        "warmCache": "0",
        "repetitions": "10",
        "seed": 4242,
        "exobenchServerVersion": "1.4.26"
      },
      "result": "{\"timestamp\":\"2026-11-13T16:10:39.125133954Z\",\"databaseType\":\"postgres\",\"sql\":\"EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT count(*), sum(total) FROM orders WHERE customer_id NOT IN (SELECT customer_id FROM banned)\",\"warmCacheIterations\":0,\"repetitions\":10,\"seed\":4242,\"seedSource\":\"explicit\",\"scalePointCount\":2,\"totalComputeSeconds\":8.775,\"results\":[{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":1000000,\"B\":100000},\"output\":\"QUERY PLAN\\nStatistics over 10 runs: avg = mean, stddev = sample standard deviation, omitted where 0.\\nFinalize Aggregate  (avg_cost=14349.23..14349.24 avg_rows=1 avg_width=40) (actual avg_time=154.868..160.455 stddev_time=9.973..9.964 avg_rows=1 avg_loops=1)\\n  Buffers: shared avg_hit=6735\\n  ->  Gather  (avg_cost=14349.00..14349.21 avg_rows=2 avg_width=40) (actual avg_time=154.134..160.443 stddev_time=10.016..9.964 avg_rows=3 avg_loops=1)\\n        Workers Planned: avg=2\\n        Workers Launched: avg=2\\n        Buffers: shared avg_hit=6735\\n        ->  Partial Aggregate  (avg_cost=13349.00..13349.01 avg_rows=1 avg_width=40) (actual avg_time=151.582..151.584 stddev_time=9.926..9.926 avg_rows=1 avg_loops=3)\\n              Buffers: shared avg_hit=6735\\n              ->  Parallel Seq Scan on orders  (avg_cost=1693.00..12307.33 avg_rows=208333 avg_width=6) (actual avg_time=34.559..115.013 stddev_time=5.594..8.453 avg_rows=300000 avg_loops=3)\\n                    Filter: (NOT (ANY (customer_id = (hashed SubPlan 1).col1)))\\n                    Rows Removed by Filter: avg=33333\\n                    Buffers: shared avg_hit=6735\\n                    SubPlan 1\\n                      ->  Seq Scan on banned  (avg_cost=0.00..1443.00 avg_rows=100000 avg_width=4) (actual avg_time=0.013..6.328 stddev_time=0.002..1.190 avg_rows=100000 avg_loops=3)\\n                            Buffers: shared avg_hit=1329\\nPlanning:\\n  Buffers: shared avg_hit=3.5 stddev_hit=11.1\\nPlanning Time: avg=0.08 ms stddev=0.04 n=10 samples=0.18,0.06,0.08,0.11,0.06,0.06,0.08,0.06,0.07,0.06\\nExecution Time: avg=161.04 ms stddev=10.01 n=10 samples=156.06,167.84,182.88,151.64,157.79,154.3,158.78,155.86,171.9,153.33\",\"runtimeDurationMs\":162,\"rowsAreCapped\":false,\"computeSeconds\":3.945},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":1000000,\"B\":262144},\"output\":\"QUERY PLAN\\nStatistics over 10 runs: avg = mean, stddev = sample standard deviation, omitted where 0.\\nFinalize Aggregate  (avg_cost=17093.03..17093.04 avg_rows=1 avg_width=40) (actual avg_time=223.184..229.085 stddev_time=38.046..38.028 avg_rows=1 avg_loops=1)\\n  Buffers: shared avg_hit=8886\\n  ->  Gather  (avg_cost=17092.80..17093.01 avg_rows=2 avg_width=40) (actual avg_time=220.889..229.072 stddev_time=37.911..38.028 avg_rows=3 avg_loops=1)\\n        Workers Planned: avg=2\\n        Workers Launched: avg=2\\n        Buffers: shared avg_hit=8886\\n        ->  Partial Aggregate  (avg_cost=16092.80..16092.81 avg_rows=1 avg_width=40) (actual avg_time=217.733..217.734 stddev_time=36.197..36.198 avg_rows=1 avg_loops=3)\\n              Buffers: shared avg_hit=8886\\n              ->  Parallel Seq Scan on orders  (avg_cost=4436.80..15051.13 avg_rows=208333 avg_width=6) (actual avg_time=96.880..186.851 stddev_time=26.264..33.095 avg_rows=245952 avg_loops=3)\\n                    Filter: (NOT (ANY (customer_id = (hashed SubPlan 1).col1)))\\n                    Rows Removed by Filter: avg=87381\\n                    Buffers: shared avg_hit=8886\\n                    SubPlan 1\\n                      ->  Seq Scan on banned  (avg_cost=0.00..3781.44 avg_rows=262144 avg_width=4) (actual avg_time=0.020..16.630 stddev_time=0.005..5.391 avg_rows=262144 avg_loops=3)\\n                            Buffers: shared avg_hit=3480\\nPlanning:\\n  Buffers: shared avg_hit=3.5 stddev_hit=11.1\\nPlanning Time: avg=0.08 ms stddev=0.04 n=10 samples=0.18,0.09,0.06,0.12,0.06,0.08,0.06,0.07,0.06,0.07\\nExecution Time: avg=230.75 ms stddev=38.05 n=10 samples=236.37,336.94,221.2,217.47,223.37,219.26,211.67,218.53,210.51,212.22\",\"runtimeDurationMs\":231,\"rowsAreCapped\":false,\"computeSeconds\":4.83}],\"workerId\":\"7ba8f6c9-d548-43a8-b72c-cbfb0882b5c1\",\"correlationId\":\"4dd293de-df86-4c8e-80cc-a51b806cd8ef\",\"correlationIdInstruction\":\"Echo correlationId on subsequent ExoBench tool calls. Tracking only — it does NOT preserve DB state.\",\"version\":\"1.4.26\"}",
      "isError": false,
      "caption": "Scripted by the author rather than the agent: NOT IN on PostgreSQL 17.11 with both columns declared NOT NULL, a million orders against 100,000 and 262,144 banned ids: 10 runs each, no warm-up, EXPLAIN (ANALYZE, TIMING OFF, BUFFERS)",
      "planCaptions": {
        "1M / 100K": "At a million orders and 100,000 banned ids PG17 runs a parallel Seq Scan of orders, and each of the three processes builds its own hashed SubPlan from a full scan of banned. Mean Execution Time over 10 runs is 161.04 ms, stddev 10.01 ms.",
        "1M / 262.1K": "At 262,144 banned ids each of the three processes still hashes the full list in its own SubPlan. Mean Execution Time over 10 runs is 230.75 ms, stddev 38.05 ms."
      }
    },
    "equal18": {
      "type": "mcp-call",
      "name": "benchmarkSql",
      "input": {
        "schemaTemplate": "CREATE TABLE orders (id int PRIMARY KEY, customer_id int NOT NULL, total numeric(10,2) NOT NULL);\nINSERT INTO orders SELECT i, i, (i % 1000) + 0.99 FROM generate_series(1, ${O}) i;\nCREATE TABLE banned (customer_id int NOT NULL);\nINSERT INTO banned SELECT i * 3 FROM generate_series(1, ${B}) i;\nVACUUM ANALYZE;",
        "query": "EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT count(*), sum(total) FROM orders WHERE customer_id NOT IN (SELECT customer_id FROM banned)",
        "scalePoints": [
          {
            "O": 1000000,
            "B": 100000
          },
          {
            "O": 1000000,
            "B": 262144
          }
        ],
        "databaseType": "POSTGRES_18",
        "warmCache": "0",
        "repetitions": "10",
        "seed": 4242,
        "exobenchServerVersion": "1.4.26"
      },
      "result": "{\"timestamp\":\"2026-11-13T16:10:48.317292478Z\",\"databaseType\":\"postgres_18\",\"sql\":\"EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT count(*), sum(total) FROM orders WHERE customer_id NOT IN (SELECT customer_id FROM banned)\",\"warmCacheIterations\":0,\"repetitions\":10,\"seed\":4242,\"seedSource\":\"explicit\",\"scalePointCount\":2,\"totalComputeSeconds\":8.972000000000001,\"results\":[{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":1000000,\"B\":100000},\"output\":\"QUERY PLAN\\nStatistics over 10 runs: avg = mean, stddev = sample standard deviation, omitted where 0.\\nFinalize Aggregate  (avg_cost=14349.23..14349.24 avg_rows=1 avg_width=40) (actual avg_time=141.995..148.034 stddev_time=6.045..6.395 avg_rows=1 avg_loops=1)\\n  Buffers: shared avg_hit=6735\\n  Disabled: false\\n  ->  Gather  (avg_cost=14349.00..14349.21 avg_rows=2 avg_width=40) (actual avg_time=141.344..148.023 stddev_time=6.021..6.395 avg_rows=3 avg_loops=1)\\n        Workers Planned: avg=2\\n        Workers Launched: avg=2\\n        Buffers: shared avg_hit=6735\\n        Disabled: false\\n        ->  Partial Aggregate  (avg_cost=13349.00..13349.01 avg_rows=1 avg_width=40) (actual avg_time=138.534..138.535 stddev_time=6.071..6.071 avg_rows=1 avg_loops=3)\\n              Buffers: shared avg_hit=6735\\n              Disabled: false\\n              ->  Parallel Seq Scan on orders  (avg_cost=1693.00..12307.33 avg_rows=208333 avg_width=6) (actual avg_time=30.267..102.349 stddev_time=1.759..4.331 avg_rows=300000 avg_loops=3)\\n                    Filter: (NOT (ANY (customer_id = (hashed SubPlan 1).col1)))\\n                    Rows Removed by Filter: avg=33333\\n                    Buffers: shared avg_hit=6735\\n                    Disabled: false\\n                    SubPlan 1\\n                      ->  Seq Scan on banned  (avg_cost=0.00..1443.00 avg_rows=100000 avg_width=4) (actual avg_time=0.018..5.439 stddev_time=0.004..0.284 avg_rows=100000 avg_loops=3)\\n                            Buffers: shared avg_hit=1329\\n                            Disabled: false\\nPlanning:\\n  Buffers: shared avg_hit=3.5 stddev_hit=11.1\\nPlanning Time: avg=0.08 ms stddev=0.03 n=10 samples=0.17,0.06,0.06,0.06,0.07,0.08,0.06,0.06,0.07,0.08\\nExecution Time: avg=148.39 ms stddev=6.38 n=10 samples=148.83,140.96,147.15,155.81,147.58,161.88,148.95,147.75,142.8,142.13\",\"runtimeDurationMs\":149,\"rowsAreCapped\":false,\"computeSeconds\":4.056},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":1000000,\"B\":262144},\"output\":\"QUERY PLAN\\nStatistics over 10 runs: avg = mean, stddev = sample standard deviation, omitted where 0.\\nFinalize Aggregate  (avg_cost=17093.03..17093.04 avg_rows=1 avg_width=40) (actual avg_time=210.844..216.728 stddev_time=12.698..12.740 avg_rows=1 avg_loops=1)\\n  Buffers: shared avg_hit=8886\\n  Disabled: false\\n  ->  Gather  (avg_cost=17092.80..17093.01 avg_rows=2 avg_width=40) (actual avg_time=209.085..216.717 stddev_time=12.714..12.741 avg_rows=3 avg_loops=1)\\n        Workers Planned: avg=2\\n        Workers Launched: avg=2\\n        Buffers: shared avg_hit=8886\\n        Disabled: false\\n        ->  Partial Aggregate  (avg_cost=16092.80..16092.81 avg_rows=1 avg_width=40) (actual avg_time=206.086..206.087 stddev_time=12.726..12.726 avg_rows=1 avg_loops=3)\\n              Buffers: shared avg_hit=8886\\n              Disabled: false\\n              ->  Parallel Seq Scan on orders  (avg_cost=4436.80..15051.13 avg_rows=208333 avg_width=6) (actual avg_time=91.135..175.820 stddev_time=7.250..10.531 avg_rows=245952 avg_loops=3)\\n                    Filter: (NOT (ANY (customer_id = (hashed SubPlan 1).col1)))\\n                    Rows Removed by Filter: avg=87381\\n                    Buffers: shared avg_hit=8886\\n                    Disabled: false\\n                    SubPlan 1\\n                      ->  Seq Scan on banned  (avg_cost=0.00..3781.44 avg_rows=262144 avg_width=4) (actual avg_time=0.019..15.391 stddev_time=0.004..1.598 avg_rows=262144 avg_loops=3)\\n                            Buffers: shared avg_hit=3480\\n                            Disabled: false\\nPlanning:\\n  Buffers: shared avg_hit=3.5 stddev_hit=11.1\\nPlanning Time: avg=0.09 ms stddev=0.05 n=10 samples=0.23,0.06,0.06,0.06,0.06,0.08,0.09,0.07,0.11,0.06\\nExecution Time: avg=217.72 ms stddev=12.84 n=10 samples=213.17,224.47,210.47,213.46,208.99,209.68,212.53,228.1,207.49,248.83\",\"runtimeDurationMs\":218,\"rowsAreCapped\":false,\"computeSeconds\":4.916}],\"workerId\":\"edce7fed-cc94-453d-a1b4-676f4df28338\",\"correlationId\":\"12c9f7a2-4f7c-4b66-aec4-0cfca8d1bb35\",\"correlationIdInstruction\":\"Echo correlationId on subsequent ExoBench tool calls. Tracking only — it does NOT preserve DB state.\",\"version\":\"1.4.26\"}",
      "isError": false,
      "caption": "Scripted by the author rather than the agent: NOT IN on PostgreSQL 18.6 with both columns declared NOT NULL, a million orders against 100,000 and 262,144 banned ids: 10 runs each, no warm-up, EXPLAIN (ANALYZE, TIMING OFF, BUFFERS)",
      "planCaptions": {
        "1M / 100K": "At a million orders and 100,000 banned ids PG18 runs the same parallel plan as PG17, a hashed SubPlan built in each of the three processes. Mean Execution Time over 10 runs is 148.39 ms, stddev 6.38 ms.",
        "1M / 262.1K": "At 262,144 banned ids PG18 still hashes the full list once per process. Mean Execution Time over 10 runs is 217.72 ms, stddev 12.84 ms."
      }
    },
    "equal19": {
      "type": "mcp-call",
      "name": "benchmarkSql",
      "input": {
        "schemaTemplate": "CREATE TABLE orders (id int PRIMARY KEY, customer_id int NOT NULL, total numeric(10,2) NOT NULL);\nINSERT INTO orders SELECT i, i, (i % 1000) + 0.99 FROM generate_series(1, ${O}) i;\nCREATE TABLE banned (customer_id int NOT NULL);\nINSERT INTO banned SELECT i * 3 FROM generate_series(1, ${B}) i;\nVACUUM ANALYZE;",
        "query": "EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT count(*), sum(total) FROM orders WHERE customer_id NOT IN (SELECT customer_id FROM banned)",
        "scalePoints": [
          {
            "O": 1000000,
            "B": 100000
          },
          {
            "O": 1000000,
            "B": 262144
          }
        ],
        "databaseType": "POSTGRES_19",
        "warmCache": "0",
        "repetitions": "10",
        "seed": 4242,
        "exobenchServerVersion": "1.4.26"
      },
      "result": "{\"timestamp\":\"2026-11-13T16:10:56.322546975Z\",\"databaseType\":\"postgres_19\",\"sql\":\"EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT count(*), sum(total) FROM orders WHERE customer_id NOT IN (SELECT customer_id FROM banned)\",\"warmCacheIterations\":0,\"repetitions\":10,\"seed\":4242,\"seedSource\":\"explicit\",\"scalePointCount\":2,\"totalComputeSeconds\":7.879,\"results\":[{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":1000000,\"B\":100000},\"output\":\"QUERY PLAN\\nStatistics over 10 runs: avg = mean, stddev = sample standard deviation, omitted where 0.\\nFinalize Aggregate  (avg_cost=20031.52..20031.53 avg_rows=1 avg_width=40) (actual avg_time=131.475..137.313 stddev_time=12.583..12.475 avg_rows=1 avg_loops=1)\\n  Buffers: shared avg_hit=6735\\n  Disabled: false\\n  ->  Gather  (avg_cost=20031.30..20031.51 avg_rows=2 avg_width=40) (actual avg_time=130.285..137.298 stddev_time=12.568..12.476 avg_rows=3 avg_loops=1)\\n        Workers Planned: avg=2\\n        Workers Launched: avg=2\\n        Buffers: shared avg_hit=6735\\n        Disabled: false\\n        ->  Partial Aggregate  (avg_cost=19031.30..19031.31 avg_rows=1 avg_width=40) (actual avg_time=127.353..127.356 stddev_time=12.513..12.514 avg_rows=1 avg_loops=3)\\n              Buffers: shared avg_hit=6735\\n              Disabled: false\\n              ->  Hash Anti Join  (avg_cost=2693.00..17156.29 avg_rows=375000 avg_width=6) (actual avg_time=16.515..100.093 stddev_time=2.125..9.461 avg_rows=300000 avg_loops=3)\\n                    Hash Cond: (orders.customer_id = banned.customer_id)\\n                    Buffers: shared avg_hit=6735\\n                    Disabled: false\\n                    ->  Parallel Seq Scan on orders  (avg_cost=0.00..9572.67 avg_rows=416667 avg_width=10) (actual avg_time=0.007..13.579 stddev_time=0.001..1.686 avg_rows=333333.33 avg_loops=3)\\n                          Buffers: shared avg_hit=5406\\n                          Disabled: false\\n                    ->  Hash  (avg_cost=1443.00..1443.00 avg_rows=100000 avg_width=4) (actual avg_time=15.938..15.939 stddev_time=2.109..2.110 avg_rows=100000 avg_loops=3)\\n                          Buckets: avg=131072  Batches: avg=1  Memory Usage: avg=4540kB\\n                          Buffers: shared avg_hit=1329\\n                          Disabled: false\\n                          ->  Seq Scan on banned  (avg_cost=0.00..1443.00 avg_rows=100000 avg_width=4) (actual avg_time=0.021..5.739 stddev_time=0.006..0.425 avg_rows=100000 avg_loops=3)\\n                                Buffers: shared avg_hit=1329\\n                                Disabled: false\\nPlanning:\\n  Buffers: shared avg_hit=9.8 stddev_hit=31.0\\nPlanning Time: avg=0.13 ms stddev=0.06 n=10 samples=0.29,0.09,0.11,0.11,0.15,0.1,0.15,0.1,0.13,0.1\\nExecution Time: avg=137.75 ms stddev=12.51 n=10 samples=136.59,143.91,134.5,134.65,132.65,126.21,170.76,128.6,135.18,134.5\",\"runtimeDurationMs\":138,\"rowsAreCapped\":false,\"computeSeconds\":3.931},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":1000000,\"B\":262144},\"output\":\"QUERY PLAN\\nStatistics over 10 runs: avg = mean, stddev = sample standard deviation, omitted where 0.\\nFinalize Aggregate  (avg_cost=21030.68..21030.69 avg_rows=1 avg_width=40) (actual avg_time=134.254..140.426 stddev_time=9.458..9.589 avg_rows=1 avg_loops=1)\\n  Buffers: shared avg_hit=6566\\n  Disabled: false\\n  ->  Gather  (avg_cost=21030.45..21030.66 avg_rows=2 avg_width=40) (actual avg_time=133.430..140.412 stddev_time=9.364..9.587 avg_rows=3 avg_loops=1)\\n        Workers Planned: avg=2\\n        Workers Launched: avg=2\\n        Buffers: shared avg_hit=6566\\n        Disabled: false\\n        ->  Partial Aggregate  (avg_cost=20030.45..20030.46 avg_rows=1 avg_width=40) (actual avg_time=130.463..130.465 stddev_time=9.151..9.151 avg_rows=1 avg_loops=3)\\n              Buffers: shared avg_hit=6566\\n              Disabled: false\\n              ->  Parallel Hash Anti Join  (avg_cost=4629.55..18493.25 avg_rows=307440 avg_width=6) (actual avg_time=19.756..108.000 stddev_time=2.341..7.174 avg_rows=245952 avg_loops=3)\\n                    Hash Cond: (orders.customer_id = banned.customer_id)\\n                    Buffers: shared avg_hit=6566\\n                    Disabled: false\\n                    ->  Parallel Seq Scan on orders  (avg_cost=0.00..9572.67 avg_rows=416667 avg_width=10) (actual avg_time=0.012..13.396 stddev_time=0.004..1.004 avg_rows=333333.33 avg_loops=3)\\n                          Buffers: shared avg_hit=5406\\n                          Disabled: false\\n                    ->  Parallel Hash  (avg_cost=2702.02..2702.02 avg_rows=154202 avg_width=4) (actual avg_time=19.046..19.047 stddev_time=2.331..2.331 avg_rows=87381.33 avg_loops=3)\\n                          Buckets: avg=262144  Batches: avg=1  Memory Usage: avg=12352kB stddev=21.333kB\\n                          Buffers: shared avg_hit=1160\\n                          Disabled: false\\n                          ->  Parallel Seq Scan on banned  (avg_cost=0.00..2702.02 avg_rows=154202 avg_width=4) (actual avg_time=0.008..2.910 stddev_time=0.003..0.448 avg_rows=87381.33 avg_loops=3)\\n                                Buffers: shared avg_hit=1160\\n                                Disabled: false\\nPlanning:\\n  Buffers: shared avg_hit=9.7 stddev_hit=30.7\\nPlanning Time: avg=0.12 ms stddev=0.04 n=10 samples=0.22,0.09,0.09,0.13,0.1,0.1,0.13,0.1,0.15,0.1\\nExecution Time: avg=140.46 ms stddev=9.59 n=10 samples=137.76,137.66,141.51,139.17,133.91,131.71,133.03,157.67,157.84,134.37\",\"runtimeDurationMs\":141,\"rowsAreCapped\":false,\"computeSeconds\":3.948}],\"workerId\":\"d475f5b4-dd8c-4bac-b45a-3c20594f0aab\",\"correlationId\":\"fdd01145-66be-4237-bce3-2c3cecbf9ce9\",\"correlationIdInstruction\":\"Echo correlationId on subsequent ExoBench tool calls. Tracking only — it does NOT preserve DB state.\",\"version\":\"1.4.26\"}",
      "isError": false,
      "caption": "Scripted by the author rather than the agent: NOT IN on PostgreSQL 19rc2 with both columns declared NOT NULL, a million orders against 100,000 and 262,144 banned ids: 10 runs each, no warm-up, EXPLAIN (ANALYZE, TIMING OFF, BUFFERS)",
      "planCaptions": {
        "1M / 100K": "At a million orders and 100,000 banned ids 19rc2 runs a Hash Anti Join in each of the three processes, each building its own hash of the whole banned list. Mean Execution Time over 10 runs is 137.75 ms, stddev 12.51 ms.",
        "1M / 262.1K": "At 262,144 banned ids 19rc2 switches to a Parallel Hash Anti Join, the three processes sharing one hash of the list. Mean Execution Time over 10 runs is 140.46 ms, stddev 9.59 ms."
      }
    },
    "rc1Hashed19": {
      "type": "mcp-call",
      "name": "benchmarkSql",
      "input": {
        "schemaTemplate": "CREATE TABLE orders (id int PRIMARY KEY, customer_id int NOT NULL, total numeric(10,2) NOT NULL);\nINSERT INTO orders SELECT i, i, (i % 1000) + 0.99 FROM generate_series(1, ${O}) i;\nCREATE TABLE banned (customer_id int);\nINSERT INTO banned SELECT i * 3 FROM generate_series(1, ${B}) i;\nVACUUM ANALYZE;",
        "query": "EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT count(*), sum(total) FROM orders WHERE customer_id NOT IN (SELECT customer_id FROM banned)",
        "scalePoints": [
          {
            "O": 10000,
            "B": 100000
          },
          {
            "O": 10000,
            "B": 154198
          }
        ],
        "databaseType": "POSTGRES_19",
        "preBenchmarkSql": "SET jit = off;",
        "seed": 4242,
        "correlationId": "a81ef270-6400-4b6e-81eb-9a487cc25fa4",
        "exobenchServerVersion": "1.4.26"
      },
      "result": "{\"timestamp\":\"2026-11-12T23:36:10.305622019Z\",\"databaseType\":\"postgres_19\",\"sql\":\"EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT count(*), sum(total) FROM orders WHERE customer_id NOT IN (SELECT customer_id FROM banned)\",\"warmCacheIterations\":0,\"seed\":4242,\"seedSource\":\"explicit\",\"scalePointCount\":2,\"totalComputeSeconds\":0.494,\"results\":[{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":100000},\"output\":\"QUERY PLAN\\nAggregate  (cost=1898.00..1898.01 rows=1 width=40) (actual rows=1.00 loops=1)\\n  Buffers: shared hit=498\\n  ->  Seq Scan on orders  (cost=1693.00..1873.00 rows=5000 width=6) (actual rows=6667.00 loops=1)\\n        Filter: (NOT (ANY (customer_id = (hashed SubPlan any_1).col1)))\\n        Rows Removed by Filter: 3333\\n        Buffers: shared hit=498\\n        SubPlan any_1\\n          ->  Seq Scan on banned  (cost=0.00..1443.00 rows=100000 width=4) (actual rows=100000.00 loops=1)\\n                Buffers: shared hit=443\\nPlanning:\\n  Buffers: shared hit=33\\nPlanning Time: 0.164 ms\\nExecution Time: 21.338 ms\",\"runtimeDurationMs\":22,\"rowsAreCapped\":false,\"computeSeconds\":0.214},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":154198},\"output\":\"QUERY PLAN\\nAggregate  (cost=2815.48..2815.49 rows=1 width=40) (actual rows=1.00 loops=1)\\n  Buffers: shared hit=738\\n  ->  Seq Scan on orders  (cost=2610.47..2790.47 rows=5000 width=6) (actual rows=6667.00 loops=1)\\n        Filter: (NOT (ANY (customer_id = (hashed SubPlan any_1).col1)))\\n        Rows Removed by Filter: 3333\\n        Buffers: shared hit=738\\n        SubPlan any_1\\n          ->  Seq Scan on banned  (cost=0.00..2224.98 rows=154198 width=4) (actual rows=154198.00 loops=1)\\n                Buffers: shared hit=683\\nPlanning:\\n  Buffers: shared hit=33\\nPlanning Time: 0.141 ms\\nExecution Time: 38.464 ms\",\"runtimeDurationMs\":39,\"rowsAreCapped\":false,\"computeSeconds\":0.28}],\"workerId\":\"24de36d8-880d-427b-9859-ca59f5e5417f\",\"correlationId\":\"a81ef270-6400-4b6e-81eb-9a487cc25fa4\",\"correlationIdInstruction\":\"Echo correlationId on subsequent ExoBench tool calls. Tracking only — it does NOT preserve DB state.\",\"version\":\"1.4.26\"}",
      "isError": false,
      "caption": "NOT IN on PostgreSQL 19rc1 with banned.customer_id declared without NOT NULL, 10,000 orders against 100,000 and 154,198 banned ids: one execution each, EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) after SET jit = off",
      "planCaptions": {
        "10K / 100K": "On the first release candidate the nullable list is not rewritten either: at 100,000 banned ids 19rc1 runs a hashed SubPlan, one execution in 21.338 ms with JIT off.",
        "10K / 154.2K": "At 154,198 banned ids 19rc1 still hashes the list, 738 shared buffer hits. One execution, 38.464 ms."
      }
    },
    "rc1Rescan19": {
      "type": "mcp-call",
      "name": "benchmarkSql",
      "input": {
        "schemaTemplate": "CREATE TABLE orders (id int PRIMARY KEY, customer_id int NOT NULL, total numeric(10,2) NOT NULL);\nINSERT INTO orders SELECT i, i, (i % 1000) + 0.99 FROM generate_series(1, ${O}) i;\nCREATE TABLE banned (customer_id int);\nINSERT INTO banned SELECT i * 3 FROM generate_series(1, ${B}) i;\nVACUUM ANALYZE;",
        "query": "EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT count(*), sum(total) FROM orders WHERE customer_id NOT IN (SELECT customer_id FROM banned)",
        "scalePoints": [
          {
            "O": 10000,
            "B": 154199
          },
          {
            "O": 10000,
            "B": 200000
          }
        ],
        "databaseType": "POSTGRES_19",
        "preBenchmarkSql": "SET jit = off;",
        "seed": 4242,
        "waitTimeoutSeconds": 110,
        "correlationId": "6de3ccc7-12a1-4183-8831-5e461ddd2515",
        "exobenchServerVersion": "1.4.26"
      },
      "result": "{\"timestamp\":\"2026-11-13T00:14:17.948744257Z\",\"databaseType\":\"postgres_19\",\"sql\":\"EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT count(*), sum(total) FROM orders WHERE customer_id NOT IN (SELECT customer_id FROM banned)\",\"warmCacheIterations\":0,\"seed\":4242,\"seedSource\":\"explicit\",\"scalePointCount\":2,\"totalComputeSeconds\":217.51,\"results\":[{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":154199},\"output\":\"QUERY PLAN\\nAggregate  (cost=19922617.50..19922617.51 rows=1 width=40) (actual rows=1.00 loops=1)\\n  Buffers: shared hit=738, temp read=1770462 written=264\\n  ->  Seq Scan on orders  (cost=0.00..19922592.50 rows=5000 width=6) (actual rows=6667.00 loops=1)\\n        Filter: (NOT (ANY (customer_id = (SubPlan any_1).col1)))\\n        Rows Removed by Filter: 3333\\n        Buffers: shared hit=738, temp read=1770462 written=264\\n        SubPlan any_1\\n          ->  Materialize  (cost=0.00..3598.98 rows=154199 width=4) (actual rows=103360.08 loops=10000)\\n                Storage: Disk  Maximum Storage: 4096kB\\n                Buffers: shared hit=683, temp read=1770462 written=264\\n                ->  Seq Scan on banned  (cost=0.00..2224.99 rows=154199 width=4) (actual rows=154199.00 loops=1)\\n                      Buffers: shared hit=683\\nPlanning:\\n  Buffers: shared hit=32\\nPlanning Time: 0.117 ms\\nExecution Time: 95669.969 ms\",\"runtimeDurationMs\":95406,\"rowsAreCapped\":false,\"computeSeconds\":95.659},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":200000},\"output\":\"QUERY PLAN\\nAggregate  (cost=25835205.00..25835205.01 rows=1 width=40) (actual rows=1.00 loops=1)\\n  Buffers: shared hit=940, temp read=2290410 written=342\\n  ->  Seq Scan on orders  (cost=0.00..25835180.00 rows=5000 width=6) (actual rows=6667.00 loops=1)\\n        Filter: (NOT (ANY (customer_id = (SubPlan any_1).col1)))\\n        Rows Removed by Filter: 3333\\n        Buffers: shared hit=940, temp read=2290410 written=342\\n        SubPlan any_1\\n          ->  Materialize  (cost=0.00..4667.00 rows=200000 width=4) (actual rows=133895.61 loops=10000)\\n                Storage: Disk  Maximum Storage: 4096kB\\n                Buffers: shared hit=885, temp read=2290410 written=342\\n                ->  Seq Scan on banned  (cost=0.00..2885.00 rows=200000 width=4) (actual rows=200000.00 loops=1)\\n                      Buffers: shared hit=885\\nPlanning:\\n  Buffers: shared hit=33\\nPlanning Time: 0.195 ms\\nExecution Time: 121886.209 ms\",\"runtimeDurationMs\":121550,\"rowsAreCapped\":false,\"computeSeconds\":121.851}],\"workerId\":\"e359a034-463d-4147-afe3-c528dafc00e5\",\"correlationId\":\"6de3ccc7-12a1-4183-8831-5e461ddd2515\",\"correlationIdInstruction\":\"Echo correlationId on subsequent ExoBench tool calls. Tracking only — it does NOT preserve DB state.\",\"version\":\"1.4.26\"}",
      "isError": false,
      "caption": "NOT IN on PostgreSQL 19rc1 with banned.customer_id declared without NOT NULL, 10,000 orders against 154,199 and 200,000 banned ids: one execution each, EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) after SET jit = off",
      "planCaptions": {
        "10K / 154.2K": "At 154,199 banned ids 19rc1 stops hashing, the same row where 19rc2 stops: the SubPlan reads a Materialize on disk, rescanned 10,000 times. One execution, 95,669.969 ms.",
        "10K / 200K": "At 200,000 banned ids 19rc1 stays on the plain SubPlan, with 2,290,410 temp blocks read. One execution, 121,886.209 ms."
      }
    },
    "rc1Hashed18": {
      "type": "mcp-call",
      "name": "benchmarkSql",
      "input": {
        "schemaTemplate": "CREATE TABLE orders (id int PRIMARY KEY, customer_id int NOT NULL, total numeric(10,2) NOT NULL);\nINSERT INTO orders SELECT i, i, (i % 1000) + 0.99 FROM generate_series(1, ${O}) i;\nCREATE TABLE banned (customer_id int);\nINSERT INTO banned SELECT i * 3 FROM generate_series(1, ${B}) i;\nVACUUM ANALYZE;",
        "query": "EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT count(*), sum(total) FROM orders WHERE customer_id NOT IN (SELECT customer_id FROM banned)",
        "scalePoints": [
          {
            "O": 10000,
            "B": 100000
          },
          {
            "O": 10000,
            "B": 154198
          },
          {
            "O": 10000,
            "B": 154199
          },
          {
            "O": 10000,
            "B": 200000
          },
          {
            "O": 10000,
            "B": 262144
          }
        ],
        "databaseType": "POSTGRES_18",
        "preBenchmarkSql": "SET jit = off;",
        "seed": 4242,
        "correlationId": "a81ef270-6400-4b6e-81eb-9a487cc25fa4",
        "exobenchServerVersion": "1.4.26"
      },
      "result": "{\"timestamp\":\"2026-11-12T23:36:47.265067894Z\",\"databaseType\":\"postgres_18\",\"sql\":\"EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT count(*), sum(total) FROM orders WHERE customer_id NOT IN (SELECT customer_id FROM banned)\",\"warmCacheIterations\":0,\"seed\":4242,\"seedSource\":\"explicit\",\"scalePointCount\":5,\"totalComputeSeconds\":1.603,\"results\":[{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":100000},\"output\":\"QUERY PLAN\\nAggregate  (cost=1898.00..1898.01 rows=1 width=40) (actual rows=1.00 loops=1)\\n  Buffers: shared hit=498\\n  ->  Seq Scan on orders  (cost=1693.00..1873.00 rows=5000 width=6) (actual rows=6667.00 loops=1)\\n        Filter: (NOT (ANY (customer_id = (hashed SubPlan 1).col1)))\\n        Rows Removed by Filter: 3333\\n        Buffers: shared hit=498\\n        SubPlan 1\\n          ->  Seq Scan on banned  (cost=0.00..1443.00 rows=100000 width=4) (actual rows=100000.00 loops=1)\\n                Buffers: shared hit=443\\nPlanning:\\n  Buffers: shared hit=26\\nPlanning Time: 0.130 ms\\nExecution Time: 24.398 ms\",\"runtimeDurationMs\":25,\"rowsAreCapped\":false,\"computeSeconds\":0.249},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":154198},\"output\":\"QUERY PLAN\\nAggregate  (cost=2815.48..2815.49 rows=1 width=40) (actual rows=1.00 loops=1)\\n  Buffers: shared hit=738\\n  ->  Seq Scan on orders  (cost=2610.47..2790.47 rows=5000 width=6) (actual rows=6667.00 loops=1)\\n        Filter: (NOT (ANY (customer_id = (hashed SubPlan 1).col1)))\\n        Rows Removed by Filter: 3333\\n        Buffers: shared hit=738\\n        SubPlan 1\\n          ->  Seq Scan on banned  (cost=0.00..2224.98 rows=154198 width=4) (actual rows=154198.00 loops=1)\\n                Buffers: shared hit=683\\nPlanning:\\n  Buffers: shared hit=26\\nPlanning Time: 0.117 ms\\nExecution Time: 45.454 ms\",\"runtimeDurationMs\":46,\"rowsAreCapped\":false,\"computeSeconds\":0.298},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":154199},\"output\":\"QUERY PLAN\\nAggregate  (cost=2815.49..2815.50 rows=1 width=40) (actual rows=1.00 loops=1)\\n  Buffers: shared hit=738\\n  ->  Seq Scan on orders  (cost=2610.49..2790.49 rows=5000 width=6) (actual rows=6667.00 loops=1)\\n        Filter: (NOT (ANY (customer_id = (hashed SubPlan 1).col1)))\\n        Rows Removed by Filter: 3333\\n        Buffers: shared hit=738\\n        SubPlan 1\\n          ->  Seq Scan on banned  (cost=0.00..2224.99 rows=154199 width=4) (actual rows=154199.00 loops=1)\\n                Buffers: shared hit=683\\nPlanning:\\n  Buffers: shared hit=26\\nPlanning Time: 0.147 ms\\nExecution Time: 42.129 ms\",\"runtimeDurationMs\":43,\"rowsAreCapped\":false,\"computeSeconds\":0.295},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":200000},\"output\":\"QUERY PLAN\\nAggregate  (cost=3590.00..3590.01 rows=1 width=40) (actual rows=1.00 loops=1)\\n  Buffers: shared hit=940\\n  ->  Seq Scan on orders  (cost=3385.00..3565.00 rows=5000 width=6) (actual rows=6667.00 loops=1)\\n        Filter: (NOT (ANY (customer_id = (hashed SubPlan 1).col1)))\\n        Rows Removed by Filter: 3333\\n        Buffers: shared hit=940\\n        SubPlan 1\\n          ->  Seq Scan on banned  (cost=0.00..2885.00 rows=200000 width=4) (actual rows=200000.00 loops=1)\\n                Buffers: shared hit=885\\nPlanning:\\n  Buffers: shared hit=26\\nPlanning Time: 0.138 ms\\nExecution Time: 53.194 ms\",\"runtimeDurationMs\":54,\"rowsAreCapped\":false,\"computeSeconds\":0.342},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":262144},\"output\":\"QUERY PLAN\\nAggregate  (cost=4641.80..4641.81 rows=1 width=40) (actual rows=1.00 loops=1)\\n  Buffers: shared hit=1215\\n  ->  Seq Scan on orders  (cost=4436.80..4616.80 rows=5000 width=6) (actual rows=6667.00 loops=1)\\n        Filter: (NOT (ANY (customer_id = (hashed SubPlan 1).col1)))\\n        Rows Removed by Filter: 3333\\n        Buffers: shared hit=1215\\n        SubPlan 1\\n          ->  Seq Scan on banned  (cost=0.00..3781.44 rows=262144 width=4) (actual rows=262144.00 loops=1)\\n                Buffers: shared hit=1160\\nPlanning:\\n  Buffers: shared hit=26\\nPlanning Time: 0.192 ms\\nExecution Time: 77.042 ms\",\"runtimeDurationMs\":78,\"rowsAreCapped\":false,\"computeSeconds\":0.419}],\"workerId\":\"fbb21d67-1d9d-4f92-a5e9-703047a0002b\",\"correlationId\":\"a81ef270-6400-4b6e-81eb-9a487cc25fa4\",\"correlationIdInstruction\":\"Echo correlationId on subsequent ExoBench tool calls. Tracking only — it does NOT preserve DB state.\",\"version\":\"1.4.26\"}",
      "isError": false,
      "caption": "NOT IN on PostgreSQL 18.6 from the 19rc1 run, banned.customer_id declared without NOT NULL, five list sizes up to 262,144 banned ids: one execution each, EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) after SET jit = off",
      "planCaptions": {
        "10K / 100K": "At 100,000 banned ids PG18 hashes the nullable list, a hashed SubPlan over one Seq Scan of banned. One execution with JIT off, 24.398 ms.",
        "10K / 154.2K#1": "At 154,198 banned ids PG18 still hashes the list, a hashed SubPlan over one Seq Scan of banned. One execution, 45.454 ms.",
        "10K / 154.2K#2": "At 154,199 banned ids, where 19rc1 switches to the plain SubPlan, PG18 keeps hashing. One execution, 42.129 ms.",
        "10K / 200K": "At 200,000 banned ids PG18 is still on the hashed SubPlan. One execution, 53.194 ms.",
        "10K / 262.1K": "At 262,144 banned ids, the largest list on this card, PG18 still hashes. One execution, 77.042 ms."
      }
    },
    "rc1Hashed17": {
      "type": "mcp-call",
      "name": "benchmarkSql",
      "input": {
        "schemaTemplate": "CREATE TABLE orders (id int PRIMARY KEY, customer_id int NOT NULL, total numeric(10,2) NOT NULL);\nINSERT INTO orders SELECT i, i, (i % 1000) + 0.99 FROM generate_series(1, ${O}) i;\nCREATE TABLE banned (customer_id int);\nINSERT INTO banned SELECT i * 3 FROM generate_series(1, ${B}) i;\nVACUUM ANALYZE;",
        "query": "EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT count(*), sum(total) FROM orders WHERE customer_id NOT IN (SELECT customer_id FROM banned)",
        "scalePoints": [
          {
            "O": 10000,
            "B": 100000
          },
          {
            "O": 10000,
            "B": 154198
          },
          {
            "O": 10000,
            "B": 154199
          },
          {
            "O": 10000,
            "B": 200000
          },
          {
            "O": 10000,
            "B": 262144
          }
        ],
        "databaseType": "POSTGRES",
        "preBenchmarkSql": "SET jit = off;",
        "seed": 4242,
        "correlationId": "a81ef270-6400-4b6e-81eb-9a487cc25fa4",
        "exobenchServerVersion": "1.4.26"
      },
      "result": "{\"timestamp\":\"2026-11-12T23:36:28.207528413Z\",\"databaseType\":\"postgres\",\"sql\":\"EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) SELECT count(*), sum(total) FROM orders WHERE customer_id NOT IN (SELECT customer_id FROM banned)\",\"warmCacheIterations\":0,\"seed\":4242,\"seedSource\":\"explicit\",\"scalePointCount\":5,\"totalComputeSeconds\":1.696,\"results\":[{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":100000},\"output\":\"QUERY PLAN\\nAggregate  (cost=1898.00..1898.01 rows=1 width=40) (actual rows=1 loops=1)\\n  Buffers: shared hit=498\\n  ->  Seq Scan on orders  (cost=1693.00..1873.00 rows=5000 width=6) (actual rows=6667 loops=1)\\n        Filter: (NOT (ANY (customer_id = (hashed SubPlan 1).col1)))\\n        Rows Removed by Filter: 3333\\n        Buffers: shared hit=498\\n        SubPlan 1\\n          ->  Seq Scan on banned  (cost=0.00..1443.00 rows=100000 width=4) (actual rows=100000 loops=1)\\n                Buffers: shared hit=443\\nPlanning:\\n  Buffers: shared hit=26\\nPlanning Time: 0.190 ms\\nExecution Time: 40.655 ms\",\"runtimeDurationMs\":42,\"rowsAreCapped\":false,\"computeSeconds\":0.249},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":154198},\"output\":\"QUERY PLAN\\nAggregate  (cost=2815.48..2815.49 rows=1 width=40) (actual rows=1 loops=1)\\n  Buffers: shared hit=738\\n  ->  Seq Scan on orders  (cost=2610.47..2790.47 rows=5000 width=6) (actual rows=6667 loops=1)\\n        Filter: (NOT (ANY (customer_id = (hashed SubPlan 1).col1)))\\n        Rows Removed by Filter: 3333\\n        Buffers: shared hit=738\\n        SubPlan 1\\n          ->  Seq Scan on banned  (cost=0.00..2224.98 rows=154198 width=4) (actual rows=154198 loops=1)\\n                Buffers: shared hit=683\\nPlanning:\\n  Buffers: shared hit=26\\nPlanning Time: 0.100 ms\\nExecution Time: 45.723 ms\",\"runtimeDurationMs\":46,\"rowsAreCapped\":false,\"computeSeconds\":0.294},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":154199},\"output\":\"QUERY PLAN\\nAggregate  (cost=2815.49..2815.50 rows=1 width=40) (actual rows=1 loops=1)\\n  Buffers: shared hit=738\\n  ->  Seq Scan on orders  (cost=2610.49..2790.49 rows=5000 width=6) (actual rows=6667 loops=1)\\n        Filter: (NOT (ANY (customer_id = (hashed SubPlan 1).col1)))\\n        Rows Removed by Filter: 3333\\n        Buffers: shared hit=738\\n        SubPlan 1\\n          ->  Seq Scan on banned  (cost=0.00..2224.99 rows=154199 width=4) (actual rows=154199 loops=1)\\n                Buffers: shared hit=683\\nPlanning:\\n  Buffers: shared hit=26\\nPlanning Time: 0.107 ms\\nExecution Time: 44.734 ms\",\"runtimeDurationMs\":45,\"rowsAreCapped\":false,\"computeSeconds\":0.298},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":200000},\"output\":\"QUERY PLAN\\nAggregate  (cost=3590.00..3590.01 rows=1 width=40) (actual rows=1 loops=1)\\n  Buffers: shared hit=940\\n  ->  Seq Scan on orders  (cost=3385.00..3565.00 rows=5000 width=6) (actual rows=6667 loops=1)\\n        Filter: (NOT (ANY (customer_id = (hashed SubPlan 1).col1)))\\n        Rows Removed by Filter: 3333\\n        Buffers: shared hit=940\\n        SubPlan 1\\n          ->  Seq Scan on banned  (cost=0.00..2885.00 rows=200000 width=4) (actual rows=200000 loops=1)\\n                Buffers: shared hit=885\\nPlanning:\\n  Buffers: shared hit=26\\nPlanning Time: 0.171 ms\\nExecution Time: 89.468 ms\",\"runtimeDurationMs\":90,\"rowsAreCapped\":false,\"computeSeconds\":0.423},{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"O\":10000,\"B\":262144},\"output\":\"QUERY PLAN\\nAggregate  (cost=4641.80..4641.81 rows=1 width=40) (actual rows=1 loops=1)\\n  Buffers: shared hit=1215\\n  ->  Seq Scan on orders  (cost=4436.80..4616.80 rows=5000 width=6) (actual rows=6667 loops=1)\\n        Filter: (NOT (ANY (customer_id = (hashed SubPlan 1).col1)))\\n        Rows Removed by Filter: 3333\\n        Buffers: shared hit=1215\\n        SubPlan 1\\n          ->  Seq Scan on banned  (cost=0.00..3781.44 rows=262144 width=4) (actual rows=262144 loops=1)\\n                Buffers: shared hit=1160\\nPlanning:\\n  Buffers: shared hit=26\\nPlanning Time: 0.131 ms\\nExecution Time: 80.796 ms\",\"runtimeDurationMs\":82,\"rowsAreCapped\":false,\"computeSeconds\":0.432}],\"workerId\":\"0f1f728c-46af-4c1a-8130-cc5e066bf5f4\",\"correlationId\":\"a81ef270-6400-4b6e-81eb-9a487cc25fa4\",\"correlationIdInstruction\":\"Echo correlationId on subsequent ExoBench tool calls. Tracking only — it does NOT preserve DB state.\",\"version\":\"1.4.26\"}",
      "isError": false,
      "caption": "NOT IN on PostgreSQL 17.11 from the 19rc1 run, banned.customer_id declared without NOT NULL, five list sizes up to 262,144 banned ids: one execution each, EXPLAIN (ANALYZE, TIMING OFF, BUFFERS) after SET jit = off",
      "planCaptions": {
        "10K / 100K": "At 100,000 banned ids PG17 hashes the nullable list here too, a hashed SubPlan over one Seq Scan of banned. One execution with JIT off, 40.655 ms.",
        "10K / 154.2K#1": "At 154,198 banned ids PG17 still hashes the list. One execution, 45.723 ms, 738 shared buffer hits.",
        "10K / 154.2K#2": "At 154,199 banned ids, where 19rc1 switches to the plain SubPlan, PG17 keeps hashing. One execution, 44.734 ms.",
        "10K / 200K": "At 200,000 banned ids PG17 is still on the hashed SubPlan, 940 shared buffer hits. One execution, 89.468 ms.",
        "10K / 262.1K": "At 262,144 banned ids, the largest list on this card, PG17 still hashes, reading 1,215 shared buffers. One execution, 80.796 ms."
      }
    },
    "rc1Campaign18": {
      "type": "mcp-call",
      "name": "benchmarkSql",
      "input": {
        "schemaTemplate": "-- contact ids come in one block per tenant: tenant 1 has 20,000 contacts, tenant 2 10,000, tenant t about 12,000 / t\nCREATE TABLE tenant_sizes AS SELECT t AS tenant_id, CASE WHEN t = 1 THEN 20000 WHEN t = 2 THEN 10000 ELSE greatest(5, 12000 / t) END AS n FROM generate_series(1, ${TENANTS}) t;\nCREATE TABLE tenant_blocks AS SELECT tenant_id, n, (sum(n) OVER (ORDER BY tenant_id) - n + 1)::int AS first_id FROM tenant_sizes;\nCREATE TABLE contacts (id INT PRIMARY KEY, tenant_id INT NOT NULL, email TEXT NOT NULL, first_name TEXT, status TEXT NOT NULL);\nINSERT INTO contacts SELECT b.first_id + k, b.tenant_id, 'c' || (b.first_id + k) || '@example.com', 'Ann',\n  CASE WHEN random() < 0.9 THEN 'subscribed' ELSE 'unsubscribed' END\nFROM tenant_blocks b, generate_series(0, b.n - 1) k;\n\n-- a year of sends: tenant 1 sends a third of them, tenant 2 a sixth, the rest follow a power law;\n-- each tenant mails its engaged segment (every fifth contact) over and over\nCREATE TABLE sends (id INT PRIMARY KEY, tenant_id INT NOT NULL, contact_id INT, sent_at TIMESTAMPTZ NOT NULL);\nINSERT INTO sends SELECT g.i, b.tenant_id, b.first_id + 5 * floor(random() * (b.n / 5))::int, NOW() - random() * interval '365 days'\nFROM (SELECT i, CASE WHEN r < 0.33 THEN 1 WHEN r < 0.495 THEN 2 ELSE 3 + floor((${TENANTS} - 2) * power(random(), 2))::int END AS t\n      FROM (SELECT i, random() AS r FROM generate_series(1, ${SENDS}) i) a) g\nJOIN tenant_blocks b ON b.tenant_id = g.t;\nCREATE INDEX contacts_tenant_id ON contacts (tenant_id, id);\nCREATE INDEX sends_tenant_sent ON sends (tenant_id, sent_at) INCLUDE (contact_id);\nCREATE INDEX sends_contact ON sends (contact_id);\nDROP TABLE tenant_blocks;\nDROP TABLE tenant_sizes;\nVACUUM ANALYZE;",
        "query": "EXPLAIN (ANALYZE, BUFFERS) SELECT c.id, c.email, c.first_name\nFROM contacts c\nWHERE c.tenant_id = 1\n  AND c.status = 'subscribed'\n  AND c.id NOT IN (SELECT s.contact_id\n                   FROM sends s\n                   WHERE s.tenant_id = 1\n                     AND s.sent_at >= now() - 90 * interval '1 day')\nORDER BY c.id\nLIMIT 10",
        "scalePoints": [
          {
            "TENANTS": 2000,
            "SENDS": 2000000
          }
        ],
        "databaseType": "POSTGRES_18",
        "warmCache": "3",
        "repetitions": "10",
        "seed": 4242,
        "waitTimeoutSeconds": 110,
        "correlationId": "c7527f46-4a31-4cd9-b801-308c2f7e3783",
        "exobenchServerVersion": "1.4.26"
      },
      "result": "{\"timestamp\":\"2026-11-12T23:08:36.344004447Z\",\"databaseType\":\"postgres_18\",\"sql\":\"EXPLAIN (ANALYZE, BUFFERS) SELECT c.id, c.email, c.first_name\\nFROM contacts c\\nWHERE c.tenant_id = 1\\n  AND c.status = 'subscribed'\\n  AND c.id NOT IN (SELECT s.contact_id\\n                   FROM sends s\\n                   WHERE s.tenant_id = 1\\n                     AND s.sent_at >= now() - 90 * interval '1 day')\\nORDER BY c.id\\nLIMIT 10\",\"warmCacheIterations\":3,\"repetitions\":10,\"seed\":4242,\"seedSource\":\"explicit\",\"scalePointCount\":1,\"totalComputeSeconds\":11.751,\"results\":[{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"TENANTS\":2000,\"SENDS\":2000000},\"output\":\"QUERY PLAN\\nStatistics over 10 runs: avg = mean, stddev = sample standard deviation, omitted where 0.\\nLimit  (avg_cost=6917.57..6920.42 avg_rows=10 avg_width=26) (actual avg_time=43.802..43.807 stddev_time=1.742..1.743 avg_rows=10 avg_loops=1)\\n  Buffers: shared avg_hit=807\\n  Disabled: false\\n  ->  Index Scan using contacts_tenant_id on contacts c  (avg_cost=6917.57..9487.59 avg_rows=9022 avg_width=26) (actual avg_time=43.800..43.805 stddev_time=1.742..1.743 avg_rows=10 avg_loops=1)\\n        Index Cond: (tenant_id = 1)\\n        Filter: ((NOT (ANY (id = (hashed SubPlan 1).col1))) AND (status = 'subscribed'::text))\\n        Rows Removed by Filter: avg=3\\n        Buffers: shared avg_hit=807\\n        Disabled: false\\n        Index Searches: avg=1\\n        SubPlan 1\\n          ->  Index Only Scan using sends_tenant_sent on sends s  (avg_cost=0.43..6508.63 avg_rows=163410 avg_width=4) (actual avg_time=0.014..19.965 stddev_time=0.003..0.672 avg_rows=162209 avg_loops=1)\\n                Index Cond: ((tenant_id = 1) AND (sent_at >= (now() - '90 days'::interval)))\\n                Heap Fetches: avg=0\\n                Buffers: shared avg_hit=803\\n                Disabled: false\\n                Index Searches: avg=1\\nPlanning Time: avg=0.14 ms stddev=0.04 n=10 samples=0.11,0.11,0.11,0.12,0.11,0.13,0.22,0.11,0.2,0.15\\nExecution Time: avg=43.83 ms stddev=1.74 n=10 samples=43.75,43.97,46.61,46.25,41.04,44.98,42.59,42.18,43.23,43.73\",\"runtimeDurationMs\":44,\"rowsAreCapped\":false,\"computeSeconds\":11.751}],\"workerId\":\"bba77e29-c7ec-4935-84fb-0f9b9a3f26d4\",\"correlationId\":\"10a7a755-0b1f-4f20-957a-cb846ca5224c\",\"correlationIdInstruction\":\"Echo correlationId on subsequent ExoBench tool calls. Tracking only — it does NOT preserve DB state.\",\"version\":\"1.4.26\"}",
      "isError": false,
      "caption": "The campaign-preview NOT IN on PostgreSQL 18.6 from the 19rc1 run, tenant 1 over the last 90 days, with sends.contact_id declared without NOT NULL: 10 runs after three warm-ups, EXPLAIN (ANALYZE, BUFFERS) with per-node timing",
      "planCaptions": {
        "2K / 2M": "PG18 hashes the 90-day subquery: the planner estimates 163,410 rows, the Index Only Scan returns 162,209, and the hashed SubPlan is built once per run with no temp reads. Mean Execution Time over 10 runs is 43.83 ms, stddev 1.74 ms."
      }
    },
    "rc1CampaignWorkMem19": {
      "type": "mcp-call",
      "name": "benchmarkSql",
      "input": {
        "schemaTemplate": "-- contact ids come in one block per tenant: tenant 1 has 20,000 contacts, tenant 2 10,000, tenant t about 12,000 / t\nCREATE TABLE tenant_sizes AS SELECT t AS tenant_id, CASE WHEN t = 1 THEN 20000 WHEN t = 2 THEN 10000 ELSE greatest(5, 12000 / t) END AS n FROM generate_series(1, ${TENANTS}) t;\nCREATE TABLE tenant_blocks AS SELECT tenant_id, n, (sum(n) OVER (ORDER BY tenant_id) - n + 1)::int AS first_id FROM tenant_sizes;\nCREATE TABLE contacts (id INT PRIMARY KEY, tenant_id INT NOT NULL, email TEXT NOT NULL, first_name TEXT, status TEXT NOT NULL);\nINSERT INTO contacts SELECT b.first_id + k, b.tenant_id, 'c' || (b.first_id + k) || '@example.com', 'Ann',\n  CASE WHEN random() < 0.9 THEN 'subscribed' ELSE 'unsubscribed' END\nFROM tenant_blocks b, generate_series(0, b.n - 1) k;\n\n-- a year of sends: tenant 1 sends a third of them, tenant 2 a sixth, the rest follow a power law;\n-- each tenant mails its engaged segment (every fifth contact) over and over\nCREATE TABLE sends (id INT PRIMARY KEY, tenant_id INT NOT NULL, contact_id INT, sent_at TIMESTAMPTZ NOT NULL);\nINSERT INTO sends SELECT g.i, b.tenant_id, b.first_id + 5 * floor(random() * (b.n / 5))::int, NOW() - random() * interval '365 days'\nFROM (SELECT i, CASE WHEN r < 0.33 THEN 1 WHEN r < 0.495 THEN 2 ELSE 3 + floor((${TENANTS} - 2) * power(random(), 2))::int END AS t\n      FROM (SELECT i, random() AS r FROM generate_series(1, ${SENDS}) i) a) g\nJOIN tenant_blocks b ON b.tenant_id = g.t;\nCREATE INDEX contacts_tenant_id ON contacts (tenant_id, id);\nCREATE INDEX sends_tenant_sent ON sends (tenant_id, sent_at) INCLUDE (contact_id);\nCREATE INDEX sends_contact ON sends (contact_id);\nDROP TABLE tenant_blocks;\nDROP TABLE tenant_sizes;\nVACUUM ANALYZE;",
        "query": "EXPLAIN (ANALYZE, BUFFERS) SELECT c.id, c.email, c.first_name\nFROM contacts c\nWHERE c.tenant_id = 1\n  AND c.status = 'subscribed'\n  AND c.id NOT IN (SELECT s.contact_id\n                   FROM sends s\n                   WHERE s.tenant_id = 1\n                     AND s.sent_at >= now() - 90 * interval '1 day')\nORDER BY c.id\nLIMIT 10",
        "scalePoints": [
          {
            "TENANTS": 2000,
            "SENDS": 2000000
          }
        ],
        "databaseType": "POSTGRES_19",
        "preBenchmarkSql": "SET work_mem = '8MB';",
        "warmCache": "3",
        "repetitions": "10",
        "seed": 4242,
        "waitTimeoutSeconds": 110,
        "correlationId": "10a7a755-0b1f-4f20-957a-cb846ca5224c",
        "exobenchServerVersion": "1.4.26"
      },
      "result": "{\"timestamp\":\"2026-11-12T23:09:08.797131866Z\",\"databaseType\":\"postgres_19\",\"sql\":\"EXPLAIN (ANALYZE, BUFFERS) SELECT c.id, c.email, c.first_name\\nFROM contacts c\\nWHERE c.tenant_id = 1\\n  AND c.status = 'subscribed'\\n  AND c.id NOT IN (SELECT s.contact_id\\n                   FROM sends s\\n                   WHERE s.tenant_id = 1\\n                     AND s.sent_at >= now() - 90 * interval '1 day')\\nORDER BY c.id\\nLIMIT 10\",\"warmCacheIterations\":3,\"repetitions\":10,\"seed\":4242,\"seedSource\":\"explicit\",\"scalePointCount\":1,\"totalComputeSeconds\":11.215,\"results\":[{\"kind\":\"postgres_explain_analyze\",\"scalePoint\":{\"TENANTS\":2000,\"SENDS\":2000000},\"output\":\"QUERY PLAN\\nStatistics over 10 runs: avg = mean, stddev = sample standard deviation, omitted where 0.\\nLimit  (avg_cost=6935.11..6937.98 avg_rows=10 avg_width=26) (actual avg_time=35.567..35.573 stddev_time=1.222..1.222 avg_rows=10 avg_loops=1)\\n  Buffers: shared avg_hit=807\\n  Disabled: false\\n  ->  Index Scan using contacts_tenant_id on contacts c  (avg_cost=6935.11..9493.21 avg_rows=8927 avg_width=26) (actual avg_time=35.566..35.572 stddev_time=1.222..1.223 avg_rows=10 avg_loops=1)\\n        Index Cond: (tenant_id = 1)\\n        Filter: ((NOT (ANY (id = (hashed SubPlan any_1).col1))) AND (status = 'subscribed'::text))\\n        Rows Removed by Filter: avg=3\\n        Buffers: shared avg_hit=807\\n        Disabled: false\\n        Index Searches: avg=1\\n        SubPlan any_1\\n          ->  Index Only Scan using sends_tenant_sent on sends s  (avg_cost=0.43..6525.11 avg_rows=163834 avg_width=4) (actual avg_time=0.015..16.498 stddev_time=0.005..0.326 avg_rows=162209 avg_loops=1)\\n                Index Cond: ((tenant_id = 1) AND (sent_at >= (now() - '90 days'::interval)))\\n                Heap Fetches: avg=0\\n                Buffers: shared avg_hit=803\\n                Disabled: false\\n                Index Searches: avg=1\\nPlanning Time: avg=0.13 ms stddev=0.02 n=10 samples=0.11,0.12,0.12,0.16,0.18,0.12,0.12,0.12,0.12,0.12\\nExecution Time: avg=35.6 ms stddev=1.22 n=10 samples=36.74,35.4,36.38,35.98,34.57,34.97,34.35,37.46,36.55,33.62\",\"runtimeDurationMs\":36,\"rowsAreCapped\":false,\"computeSeconds\":11.215}],\"workerId\":\"0f27a88b-f50a-4cdb-bd7f-8baad4926c7a\",\"correlationId\":\"1db65b72-115e-433a-a636-2da3f5760354\",\"correlationIdInstruction\":\"Echo correlationId on subsequent ExoBench tool calls. Tracking only — it does NOT preserve DB state.\",\"version\":\"1.4.26\"}",
      "isError": false,
      "caption": "The campaign-preview NOT IN on PostgreSQL 19rc1, tenant 1 over the last 90 days, with sends.contact_id declared without NOT NULL, after SET work_mem = '8MB': 10 runs after three warm-ups, EXPLAIN (ANALYZE, BUFFERS) with per-node timing",
      "planCaptions": {
        "2K / 2M": "Under SET work_mem = '8MB', 19rc1 hashes the 90-day subquery again: the planner estimates 163,834 rows, and the hashed SubPlan is built once per run with no temp reads. Mean Execution Time over 10 runs is 35.6 ms, stddev 1.22 ms."
      }
    }
  }
}
