NutzCN Logo
问答 fastInsert批量插入慢的问题
发布于 2419天前 作者 wx_6pa8idmo7gr9iqmvap20 2130 次浏览 复制 上一个帖子 下一个帖子
标签:

新人,项目使用的版本为1.b.53
插入的数据量为百万级

@Table("TestInsert")
public class TestInsert {
    @Id
    private long id;

    @Column
    private String value;

    public TestInsert(){}

    public TestInsert(String value){
        this.value = value;
    }

    public long getId() {
        return id;
    }

    public void setId(long id) {
        this.id = id;
    }

    public String getValue() {
        return value;
    }

    public void setValue(String value) {
        this.value = value;
    }
}

调用fastInsert的代码如下

@Autowired
    private TransactionApi transactionApi;

    @Override
    public void fastInsert(final List<TestInsert> testInsertList) {
        transactionApi.runInTransaction(new Executable<Boolean>() {
            @Override
            public Boolean exec() {
                dao.fastInsert(testInsertList);
                return true;
            }
        });
    }

执行时间超长,大神能帮忙看下问题吗

16 回复

需要dao.js 里面 defaultAutoCommit :false

@qq_0f18c1df 事务下总是defaultAutoCommit=false, 所以没影响
这是我看之前的帖子回复
我使用事务了吧

<bean id="daoBlackhole" class="org.nutz.dao.impl.NutDao">
        <property name="dataSource" ref="dataSourceBlackhole"/>
        <property name="runner" ref="springDaoRunner"/>
    </bean>

    <tx:annotation-driven transaction-manager="txManagerBlackhole" />
    <bean id="txManagerBlackhole" class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
        <property name="dataSource" ref="dataSourceBlackhole"/>
    </bean>

哦,transactionApi是什么东西?

是不是打印日志耗时?

2017-09-08 12:53:14,018 WARN  [main] testInsertList:start, size()=2702116
2017-09-08 12:53:14,023 DEBUG [main] Creating new transaction with name [com.getfun.service.api.transaction.TransactionApiImpl.runInTransaction]: PROPAGATION_REQUIRED,ISOLATION_DEFAULT; '',-java.lang.Exception
2017-09-08 12:53:14,024 DEBUG [main] Acquired Connection [1781044784, URL=jdbc:mysql://10.0.104.72:3306/getfun_db?characterEncoding=utf-8, UserName=gf_test@10.0.104.76, MySQL Connector Java] for JDBC transaction
2017-09-08 12:53:14,026 DEBUG [main] Switching JDBC Connection [1781044784, URL=jdbc:mysql://10.0.104.72:3306/getfun_db?characterEncoding=utf-8, UserName=gf_test@10.0.104.76, MySQL Connector Java] to manual commit
2017-09-08 12:53:14,185 DEBUG [main] Fetching JDBC Connection from DataSource
2017-09-08 12:53:14,185 DEBUG [main] Registering transaction synchronization for JDBC Connection
2017-09-08 12:53:16,839 DEBUG [main] INSERT INTO TestInsert(value) VALUES(?) 


  .............................................
  !!!Too many data . Only display 50 lines , don't show the remaining record
  .............................................
  For example:> "INSERT INTO TestInsert(value) VALUES('4984325184') "

要不,上jprofiler, youkit之类的看看哪一个地方慢?

spring的事务会确保auto commit = false吗?

spring的不清楚。我先试下设置defaultAutoCommit :false
回复能修改或撤销吗。。

额,不支持修改,咋了?就一个ip地址死不了的

速度上去了,也没有异常。为啥mysql表里没数据。。只显示有200多M

设置了defaultAutoCommit :false
@Override
public void fastInsert(final List testInsertList) {
dao.fastInsert(testInsertList);
}
这样调用的

那是spring的事务,不清楚

添加回复
请先登陆
回到顶部