Nutz
  1. 主页/
  2. 问答
NutzCN Logo
问答 nutz 在阿里数据库RDS中不支持批量插入？？
发布于 2470天前 作者 qq_6f5e9a61 2161 次浏览 复制 上一个帖子 下一个帖子
标签: 无

在阿里数据库RDS中批量出错，添加阿里事务GTS后，出错，无法回滚

14 回复

阿里云事务gts？ 有日志可以看看吗？

2018-03-19 09:04:07.470 org.nutz.dao.DaoException: !Nutz SQL Error: 'DELETE FROM surf_wea_chn_min_all_reg_tab  WHERE D_RECORD_ID='20180313093300_A1363''
PreparedStatement: 
'DELETE FROM surf_wea_chn_min_all_reg_tab  WHERE D_RECORD_ID=?'
CaseMessage=[ca9d0e351400001-2][10.40.44.4:3306][rdb]ERR-CODE: [TDDL-4603][ERR_ACCROSS_DB_TRANSACTION] Transaction accross db is not supported in current transaction policy, transaction no
de is: RDB_1517898071163CXAHRDB_JTBM_0021_RDS, but this sql execute on: RDB_1517898071163CXAHRDB_JTBM_0047_RDS. More: [http://middleware.alibaba-inc.com/faq/faqByFaqCode.html?faqCode=TDDL-
4603]
2018-03-19 09:04:07.470 at org.nutz.dao.impl.sql.run.NutDaoExecutor.exec(NutDaoExecutor.java:104)
2018-03-19 09:04:07.470 at org.nutz.dao.DaoInterceptorChain.doChain(DaoInterceptorChain.java:66)
2018-03-19 09:04:07.470 at org.nutz.dao.impl.interceptor.DaoLogInterceptor.filter(DaoLogInterceptor.java:22)
2018-03-19 09:04:07.470 at org.nutz.dao.DaoInterceptorChain.doChain(DaoInterceptorChain.java:64)
2018-03-19 09:04:07.470 at org.nutz.dao.DaoInterceptorChain.invoke(DaoInterceptorChain.java:139)
2018-03-19 09:04:07.470 at org.nutz.dao.impl.sql.run.NutDaoRunner.runCallback(NutDaoRunner.java:158)
2018-03-19 09:04:07.471 at org.nutz.dao.impl.sql.run.NutDaoRunner._runWithoutTransaction(NutDaoRunner.java:125)
2018-03-19 09:04:07.471 at org.nutz.dao.impl.sql.run.NutDaoRunner._run(NutDaoRunner.java:92)
2018-03-19 09:04:07.471 at org.nutz.dao.impl.sql.run.NutDaoRunner.run(NutDaoRunner.java:81)
2018-03-19 09:04:07.471 at org.nutz.dao.impl.DaoSupport.run(DaoSupport.java:240)
2018-03-19 09:04:07.471 at org.nutz.dao.impl.DaoSupport._exec(DaoSupport.java:252)
2018-03-19 09:04:07.471 at org.nutz.dao.impl.NutDao.clear(NutDao.java:651)
2018-03-19 09:04:07.471 at org.cimiss2.dwp.core.etl.AbstractEtl.insertError(AbstractEtl.java:347)
2018-03-19 09:04:07.471 at org.cimiss2.dwp.core.etl.AbstractEtl.insert(AbstractEtl.java:300)
2018-03-19 09:04:07.471 at org.cimiss2.dwp.core.etl.AbstractEtl.subload(AbstractEtl.java:254)
2018-03-19 09:04:07.471 at org.cimiss2.dwp.core.etl.AbstractEtl.load(AbstractEtl.java:206)
2018-03-19 09:04:07.471 at org.cimiss2.dwp.Z_SURF_REG.REGBolt.execute(REGBolt.java:108)
2018-03-19 09:04:07.471 at backtype.storm.daemon.executor$fn__5694$tuple_action_fn__5696.invoke(executor.clj:690)
2018-03-19 09:04:07.471 at backtype.storm.daemon.executor$mk_task_receiver$fn__5615.invoke(executor.clj:436)
2018-03-19 09:04:07.471 at backtype.storm.disruptor$clojure_handler$reify__5189.onEvent(disruptor.clj:58)
2018-03-19 09:04:07.471 at backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:132)
2018-03-19 09:04:07.471 at backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:106)
2018-03-19 09:04:07.471 at backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80)
2018-03-19 09:04:07.471 at backtype.storm.daemon.executor$fn__5694$fn__5707$fn__5758.invoke(executor.clj:819)
2018-03-19 09:04:07.471 at backtype.storm.util$async_loop$fn__545.invoke(util.clj:479)
2018-03-19 09:04:07.471 at clojure.lang.AFn.run(AFn.java:22)
2018-03-19 09:04:07.472 at java.lang.Thread.run(Thread.java:745)
2018-03-19 09:04:07.472 Caused by: java.sql.SQLException: [ca9d0e351400001-2][10.40.44.4:3306][rdb]ERR-CODE: [TDDL-4603][ERR_ACCROSS_DB_TRANSACTION] Transaction accross db is not supported
 in current transaction policy, transaction node is: RDB_1517898071163CXAHRDB_JTBM_0021_RDS, but this sql execute on: RDB_1517898071163CXAHRDB_JTBM_0047_RDS. More: [http://middleware.aliba
ba-inc.com/faq/faqByFaqCode.html?faqCode=TDDL-4603]
2018-03-19 09:04:07.472 at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:957)
2018-03-19 09:04:07.472 at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3878)
2018-03-19 09:04:07.472 at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3814)
2018-03-19 09:04:07.472 at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2478)
2018-03-19 09:04:07.472 at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2625)
2018-03-19 09:04:07.472 at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2551)
2018-03-19 09:04:07.472 at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1861)
2018-03-19 09:04:07.472 at com.mysql.jdbc.PreparedStatement.execute(PreparedStatement.java:1192)
2018-03-19 09:04:07.472 at com.alibaba.druid.filter.FilterChainImpl.preparedStatement_execute(FilterChainImpl.java:3051)
2018-03-19 09:04:07.472 at com.alibaba.druid.filter.FilterEventAdapter.preparedStatement_execute(FilterEventAdapter.java:440)
2018-03-19 09:04:07.472 at com.alibaba.druid.filter.FilterChainImpl.preparedStatement_execute(FilterChainImpl.java:3049)
2018-03-19 09:04:07.472 at com.alibaba.druid.proxy.jdbc.PreparedStatementProxyImpl.execute(PreparedStatementProxyImpl.java:167)
2018-03-19 09:04:07.472 at com.alibaba.druid.pool.DruidPooledPreparedStatement.execute(DruidPooledPreparedStatement.java:498)
2018-03-19 09:04:07.472 at org.nutz.dao.impl.sql.run.NutDaoExecutor._runPreparedStatement(NutDaoExecutor.java:308)
2018-03-19 09:04:07.472 at org.nutz.dao.impl.sql.run.NutDaoExecutor.exec(NutDaoExecutor.java:90)
2018-03-19 09:04:07.472 ... 26 more
2018-03-19 09:04:07.472 区域站信息单条删除错误未插入更新:/CIMISS2/data/A/A.0001.0028.R001/201803/2018031309/Z_SURF_C_BEPK-REG_20180313093349_O_AWS_FTM_PQC.txt--A1363--20180313093300_A1363
2018-03-19 09:04:07.495 org.nutz.dao.DaoException: !Nutz SQL Error: 'DELETE FROM surf_wea_chn_min_all_reg_tab  WHERE D_RECORD_ID='20180313093300_A1364''
PreparedStatement: 
'DELETE FROM surf_wea_chn_min_all_reg_tab  WHERE D_RECORD_ID=?'
CaseMessage=[ca9d0e358000000-2][10.40.44.4:3306][rdb]ERR-CODE: [TDDL-4603][ERR_ACCROSS_DB_TRANSACTION] Transaction accross db is not supported in current transaction policy, transaction no
de is: RDB_1517898071163CXAHRDB_JTBM_0021_RDS, but this sql execute on: RDB_1517898071163CXAHRDB_JTBM_0003_RDS. More: [http://middleware.alibaba-inc.com/faq/faqByFaqCode.html?faqCode=TDDL-
4603]
2018-03-19 09:04:07.495 at org.nutz.dao.impl.sql.run.NutDaoExecutor.exec(NutDaoExecutor.java:104)
2018-03-19 09:04:07.495 at org.nutz.dao.DaoInterceptorChain.doChain(DaoInterceptorChain.java:66)
2018-03-19 09:04:07.495 at org.nutz.dao.impl.interceptor.DaoLogInterceptor.filter(DaoLogInterceptor.java:22)
2018-03-19 09:04:07.495 at org.nutz.dao.DaoInterceptorChain.doChain(DaoInterceptorChain.java:64)
2018-03-19 09:04:07.495 at org.nutz.dao.DaoInterceptorChain.invoke(DaoInterceptorChain.java:139)
2018-03-19 09:04:07.495 at org.nutz.dao.impl.sql.run.NutDaoRunner.runCallback(NutDaoRunner.java:158)
2018-03-19 09:04:07.495 at org.nutz.dao.impl.sql.run.NutDaoRunner._runWithoutTransaction(NutDaoRunner.java:125)
2018-03-19 09:04:07.495 at org.nutz.dao.impl.sql.run.NutDaoRunner._run(NutDaoRunner.java:92)
2018-03-19 09:04:07.495 at org.nutz.dao.impl.sql.run.NutDaoRunner.run(NutDaoRunner.java:81)
2018-03-19 09:04:07.495 at org.nutz.dao.impl.DaoSupport.run(DaoSupport.java:240)
2018-03-19 09:04:07.495 at org.nutz.dao.impl.DaoSupport._exec(DaoSupport.java:252)
2018-03-19 09:04:07.495 at org.nutz.dao.impl.NutDao.clear(NutDao.java:651)
2018-03-19 09:04:07.495 at org.cimiss2.dwp.core.etl.AbstractEtl.insertError(AbstractEtl.java:347)
2018-03-19 09:04:07.495 at org.cimiss2.dwp.core.etl.AbstractEtl.insert(AbstractEtl.java:300)
2018-03-19 09:04:07.495 at org.cimiss2.dwp.core.etl.AbstractEtl.subload(AbstractEtl.java:254)
2018-03-19 09:04:07.495 at org.cimiss2.dwp.core.etl.AbstractEtl.load(AbstractEtl.java:206)
2018-03-19 09:04:07.496 at org.cimiss2.dwp.Z_SURF_REG.REGBolt.execute(REGBolt.java:108)
2018-03-19 09:04:07.496 at backtype.storm.daemon.executor$fn__5694$tuple_action_fn__5696.invoke(executor.clj:690)
2018-03-19 09:04:07.496 at backtype.storm.daemon.executor$mk_task_receiver$fn__5615.invoke(executor.clj:436)
2018-03-19 09:04:07.496 at backtype.storm.disruptor$clojure_handler$reify__5189.onEvent(disruptor.clj:58)
2018-03-19 09:04:07.496 at backtype.storm.utils.DisruptorQueue.consumeBatchToCursor(DisruptorQueue.java:132)
2018-03-19 09:04:07.496 at backtype.storm.utils.DisruptorQueue.consumeBatchWhenAvailable(DisruptorQueue.java:106)
2018-03-19 09:04:07.496 at backtype.storm.disruptor$consume_batch_when_available.invoke(disruptor.clj:80)
2018-03-19 09:04:07.496 at backtype.storm.daemon.executor$fn__5694$fn__5707$fn__5758.invoke(executor.clj:819)
2018-03-19 09:04:07.496 at backtype.storm.util$async_loop$fn__545.invoke(util.clj:479)
2018-03-19 09:04:07.496 at clojure.lang.AFn.run(AFn.java:22)
2018-03-19 09:04:07.496 at java.lang.Thread.run(Thread.java:745)
2018-03-19 09:04:07.496 Caused by: java.sql.SQLException: [ca9d0e358000000-2][10.40.44.4:3306][rdb]ERR-CODE: [TDDL-4603][ERR_ACCROSS_DB_TRANSACTION] Transaction accross db is not supported
 in current transaction policy, transaction node is: RDB_1517898071163CXAHRDB_JTBM_0021_RDS, but this sql execute on: RDB_1517898071163CXAHRDB_JTBM_0003_RDS. More: [http://middleware.aliba
ba-inc.com/faq/faqByFaqCode.html?faqCode=TDDL-4603]
2018-03-19 09:04:07.496 at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:957)
2018-03-19 09:04:07.496 at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3878)
2018-03-19 09:04:07.496 at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3814)
2018-03-19 09:04:07.496 at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2478)
2018-03-19 09:04:07.496 at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2625)
2018-03-19 09:04:07.496 at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2551)
2018-03-19 09:04:07.496 at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1861)
2018-03-19 09:04:07.497 at com.mysql.jdbc.PreparedStatement.execute(PreparedStatement.java:1192)
2018-03-19 09:04:07.497 at com.alibaba.druid.filter.FilterChainImpl.preparedStatement_execute(FilterChainImpl.java:3051)
2018-03-19 09:04:07.497 at com.alibaba.druid.filter.FilterEventAdapter.preparedStatement_execute(FilterEventAdapter.java:440)
2018-03-19 09:04:07.497 at com.alibaba.druid.filter.FilterChainImpl.preparedStatement_execute(FilterChainImpl.java:3049)
2018-03-19 09:04:07.497 at com.alibaba.druid.proxy.jdbc.PreparedStatementProxyImpl.execute(PreparedStatementProxyImpl.java:167)
2018-03-19 09:04:07.497 at com.alibaba.druid.pool.DruidPooledPreparedStatement.execute(DruidPooledPreparedStatement.java:498)
2018-03-19 09:04:07.497 at org.nutz.dao.impl.sql.run.NutDaoExecutor._runPreparedStatement(NutDaoExecutor.java:308)
2018-03-19 09:04:07.497 at org.nutz.dao.impl.sql.run.NutDaoExecutor.exec(NutDaoExecutor.java:90)
2018-03-19 09:04:07.497 ... 26 more

逻辑是批量插入一批数据，如果出错，要把批量改为逐条插入，主键冲突要删除，在插入，现在gts一直报错，原因为gts的事务不是同一个，应该是没有rowback导致的吧，nutz批量出错的话，我把defaultAutoCommit置位false。没有回滚吗？

Trans.exec包裹起来试试？

也不行

https://help.aliyun.com/document_detail/55535.html?spm=a2c4g.11186623.6.557.jiyzlQ 是这个吧? 似乎需要额外的注解和代码进行配合

恩 已经加了， 没用nutz 之前，先批次回滚，在逐条操作是没问题的，好像就是没有回滚的问题

额，你手动做的回滚操作？还是说nutz没调用rollback呢？ 给点代码看看？

没用过阿里云gts，所以需要你说详细一点^_^

代码太多，发点伪代码吧

private void insert(List<Map<String, Object>> datas) {
        try {  
                Trans.exec(new Atom() {
                    public void run() {
                        rdb.fastInsert(datas);
                    }
                });
        } catch (Exception e2) {
            for (int i = 0; i < datas.size(); i++) {
                Map<String, Object> data = datas.get(i);
                try {
                    rdb.insert(data);
                } catch (Exception e1) {
                    insertError(data);
                }
            }
        }
    }

    private void insertError( Map<String, Object> data) {
                String table = (String) data.get(".table");
                String D_RECORD_ID = (String) data.get("D_RECORD_ID");
                    try {
                        aliGTS(dao);
                        Trans.exec(new Atom() {
                            public void run() {
                                rdb.clear(table, Cnd.where("D_RECORD_ID", "=", D_RECORD_ID));
                            }
                        });
                    } catch (Exception e) {
                        e.printStackTrace();
                    }
                    if (clear == 1) {
                        rdb.insert(data);//删除成功
                    } else {
                        errorNum.set(errorNum.get() + 1);
                    }
			}

是回滚操作错了吗

额, 看不出跟gts有什么关系啊...

应该是因为fastInsert批量插入失败，没有回滚导致的，我那样写会回滚吗

nutz 里有类似conn.rollback();conn.commit();这样的用法吗

有， Trans.begin/commit/rollback/close,还有NutTxDao，推荐先试试后者，文档有

添加回复
请先登陆
回到顶部