V2EX = way to explore
V2EX 是一个关于分享和探索的地方
现在注册
已注册用户请  登录
V2EX  ›  sunrealzhang  ›  全部回复第 2 页 / 共 2 页
回复总数  23
1  2  
2023-12-13 17:54:41 +08:00
回复了 sunrealzhang 创建的主题 ClickHouse 求大佬优化一下 3000 万数据的 NOT IN 查询
@OOKAMI 感谢您的回复,实际上,clickhouse 对 EXISTS 语法不完全支持,特别是子查询中无法引用外部表和列,这会导致执行时提示 Missing columns: 'A.AAC001',这也是我将 oracle 中的 NOT EXISTS 改成 NOT IN 的原因
2023-12-13 17:50:51 +08:00
回复了 sunrealzhang 创建的主题 ClickHouse 求大佬优化一下 3000 万数据的 NOT IN 查询
@sss15 感谢您的回复,首先连接处应该是 ON t0.TID = A.AAC001 ,然后,AAC001 是这张表的联合主键之一,类型为 Int64 ,未关联上时值不是 NULL 而是 0 (这一点我也不知道为什么要这么实现,按理说未关联上不应该是 NULL 吗),然后我执行了 该查询,依然很慢
```
EXPLAIN indexes = 1 SELECT
COUNT( 1 ) AS "新参保人数"
FROM
AC02_TEMP AS A
LEFT JOIN ( SELECT B.AAC001 AS TID FROM AC02_TEMP AS B WHERE B.AAC030 < '2018-01-01 00:00:00' ) AS t0 ON t0.TID = A.AAC001
WHERE
A.AAB301 IN ( SELECT AAB301 FROM AA26 WHERE AAA148 = '130800' )
AND A.AAE200 = '41'
AND A.AAC031 = '1'
AND A.AAC030 >= '2018-01-01 00:00:00'
AND t0.TID = 0;
```

执行计划
```
CreatingSets (Create sets before main query execution)
Expression ((Projection + Before ORDER BY))
Aggregating
Expression (Before GROUP BY)
Filter (WHERE)
Join (JOIN FillRightFirst)
Filter (( + Before JOIN))
ReadFromMergeTree (default.AC02_TEMP)
Indexes:
PrimaryKey
Keys:
AAE200
" Condition: (AAE200 in ['41', '41'])"
Parts: 2/2
Granules: 4821/4821
Expression ((Joined actions + (Rename joined columns + (Projection + Before ORDER BY))))
ReadFromMergeTree (default.AC02_TEMP)
Indexes:
PrimaryKey
Condition: true
Parts: 2/2
Granules: 4821/4821
CreatingSet (Create set for subquery)
Expression ((Projection + Before ORDER BY))
ReadFromMergeTree (default.AA26)
Indexes:
PrimaryKey
Condition: true
Parts: 1/1
Granules: 1/1

```
2023-12-13 17:38:16 +08:00
回复了 sunrealzhang 创建的主题 ClickHouse 求大佬优化一下 3000 万数据的 NOT IN 查询
NOT IN 里的数据大概有两千多万
1  2  
关于   ·   帮助文档   ·   自助推广系统   ·   博客   ·   API   ·   FAQ   ·   实用小工具   ·   857 人在线   最高记录 6679   ·     Select Language
创意工作者们的社区
World is powered by solitude
VERSION: 3.9.8.5 · 22ms · UTC 21:48 · PVG 05:48 · LAX 14:48 · JFK 17:48
Developed with CodeLauncher
♥ Do have faith in what you're doing.