SQLAlchemy中的取消透视熔化：如何使用CROSS JOIN LATERAL

如何解决SQLAlchemy中的取消透视熔化：如何使用CROSS JOIN LATERAL

我正在努力将执行“ unpivoting”（或“融化” SQL表”）的SQL查询转换为SQLAlchemy DDL元素，或通过内置的SA核心元素将其表达

我的设置是什么

• Postgres> 12
• Python> 3.8
• Sqlalchemy> 1.3

查询的外观

我尝试取消透视表，在该表中，我的索引列应为idxa，...，idxn和值列valuea，... valuen ：

SELECT idxa,idxb,...,idxn
       s.column_name as "name",s.column_value as "value"
FROM mytable t
CROSS JOIN LATERAL (VALUES ('valuea',valuea),('valueb',valueb),('valuen',valuen))
                    s(column_name,column_value);

将表格从宽格式转换为长格式，即从

idxA | ... | idxN | valueA | ... | valueN
========================================
1    | ... |  3   |  4     | ... |   5

到

idxA | ... | idxN | name    | value
========================================
1    | ... |  3   |  valueA | 4
1    | ... |  3   |  valueN | 4

关于查询的功能，您可以尝试以下方法来说服自己

with tmp as (
    select * from (values (1,2,3,4),(5,6,7,8))
                   as t ("idxa","idxb","valuea","valueb"))
select idxa,s.column_name as "name",s.column_value as "value"
FROM tmp t
CROSS JOIN LATERAL (VALUES('valuea',valueb))
s(column_name,column_value);

我尝试了什么？

我对DDL的Ansatz是

import sqlalchemy as db
from sqlalchemy.sql.expression import FromClause
from sqlalchemy.ext.compiler import compiles

class UnCrossTab(FromClause):
    def __init__(self,idx_cols,value_cols):

        self.columns = idx_cols + [db.literal_column("col"),db.literal_column("name")]
        self.idx_cols = idx_cols
        self.value_cols = value_cols
        self.name = idx_cols[0].table.name

@compiles(UnCrossTab,"postgresql")
def visit_element(element,compiler,**kwargs):  # noqa: F811
    query = create_query_melt(element.idx_cols,element.value_cols)
    return query

def create_query_melt(idx_cols,value_cols):
    tablename = idx_cols[0].table
    idx_cols = ",".join(c.name for c in idx_cols)
    value_cols = ",".join(f"('{c.name}',{c.name})" for c in value_cols)
    s = f"""select {idx_cols},s.column_value as "value"
    FROM {tablename} t
    CROSS JOIN LATERAL (VALUES {value_cols})
    s(column_name,column_value);"""

    return s

假设我的数据库中有一个名为public.testmelt的表（在python中是hte对象ttest），然后出现以下错误

uc = UnCrossTab([ttest.c["idxa"],ttest.c["idxb"]],[ttest.c["valuea"],ttest.c["valueb"]])
pd.read_sql(db.select(["*"]).select_from(uc),con=engine)

sqlalchemy.exc.ProgrammingError: (psycopg2.errors.SyntaxError) syntax error at or near "select"
LINE 2: FROM select idxa,^
[SQL: SELECT * 
FROM select idxa,s.column_value as "value"
    FROM public.testmelt t
    CROSS JOIN LATERAL (VALUES ('valuea',valueb))
    s(column_name,column_value);]
(Background on this error at: http://sqlalche.me/e/13/f405)

它几乎在那儿，但不完全是！

有效的方法

我实现了使其在某种程度上可以与SA的核心元素一起工作：

def uncrosstab(idx_cols,value_cols,column_name="col",value_name="value"):
    v_clause = ",{c.name})" for c in value_cols)

    v = f"(VALUES {v_clause}) as foo ({column_name},{value_name})"
    values_field = db.text(v)
    sub_q = db.select([db.Column(column_name),db.Column(value_name)])\
            .select_from(values_field).lateral()
    query = db.select(idx_cols + list(sub_q.c))
    print(query)
    return query

pd.read_sql(uncrosstab([ttest.c["idxa"],ttest.c["valueb"]]),con=engine)

但是，由于涉及子查询以及db.text，因此感觉有点骇人听闻。

因此，我的问题是，是否有人可以帮助我将此操作编写为自定义DDL或将其转换为不涉及db.text的适当SA核心对象？