Nutz
  1. 主页/
  2. 问答
NutzCN Logo
问答 Netty集成nutz并发时，内存溢出的问题
发布于 2727天前 作者 qq_ef6310b4 2853 次浏览 复制 上一个帖子 下一个帖子
标签: 无

因为之前报过类似的错误，看网上的教程，虚拟机启动的时候加了参数：-Dio.netty.leakDetectionLevel=advanced
知道是ByteBuf内存溢出的问题，但是不知道怎么解决啊。求解啊
另外我的ByteBuf的使用方式是这样的：

在处理类，WebSocketServerHandler extends SimpleChannelInboundHandler的
protected void channelRead0(ChannelHandlerContext ctx, Object msg) {}这个方法中，使用：
ByteBuf bb = ((BinaryWebSocketFrame) frame).content();
Message message = decodeBinaryWebsocketFrame(bb);
使用完，并没有进行释放，因为我发现无论怎么释放，最后都会报错，不释放反倒没有报错。而且，《Netty权威指南》也并没有释放这个ByteBuf啊

最后，消息处理完成之后，使用：
ByteBuf bb = ByteBufAllocator.DEFAULT.buffer();
//然后对bb写入数据
BinaryWebSocketFrame b = new BinaryWebSocketFrame(bb);
s.getCtx().writeAndFlush(b); //发送
这里写出之后，也没有对bytebuf进行释放。
我尝试用try finally的方式进行释放，但是线程会卡死。

正常的运行报错（内存溢出）如下，求问各位大神，是我对bytebuf的使用方式有问题吗？

22:03:54,386 [ERROR] [nioEventLoopGroup-3-8] ResourceLeakDetector.error - LEAK: ByteBuf.release() was not called before it's garbage-collected. See http://netty.io/wiki/reference-counted-objects.html for more information.
Recent access records: 2
#2:
	io.netty.buffer.AdvancedLeakAwareByteBuf.getBytes(AdvancedLeakAwareByteBuf.java:249)
	com.xiuxian.core.net.WebSocketServerHandler.decodeBinaryWebsocketFrame(WebSocketServerHandler.java:188)
	com.xiuxian.core.net.WebSocketServerHandler.handlerWebSocketFrame(WebSocketServerHandler.java:97)
	com.xiuxian.core.net.WebSocketServerHandler.channelRead0(WebSocketServerHandler.java:258)
	io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:372)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:358)
	io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:350)
	io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:372)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:358)
	io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:350)
	io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:293)
	io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:267)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:372)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:358)
	io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:350)
	io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:372)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:358)
	io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926)
	io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:129)
	io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:610)
	io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:551)
	io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:465)
	io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:437)
	io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873)
	io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
	java.lang.Thread.run(Thread.java:745)
#1:
	io.netty.buffer.AdvancedLeakAwareByteBuf.writeBytes(AdvancedLeakAwareByteBuf.java:603)
	io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:829)
	com.xiuxian.core.net.WebSocketServerHandler.decodeBinaryWebsocketFrame(WebSocketServerHandler.java:176)
	com.xiuxian.core.net.WebSocketServerHandler.handlerWebSocketFrame(WebSocketServerHandler.java:97)
	com.xiuxian.core.net.WebSocketServerHandler.channelRead0(WebSocketServerHandler.java:258)
	io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:372)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:358)
	io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:350)
	io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:372)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:358)
	io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:350)
	io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:293)
	io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:267)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:372)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:358)
	io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:350)
	io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:372)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:358)
	io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926)
	io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:129)
	io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:610)
	io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:551)
	io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:465)
	io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:437)
	io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873)
	io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
	java.lang.Thread.run(Thread.java:745)
Created at:
	io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:69)
	io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:179)
	io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:115)
	io.netty.buffer.AbstractByteBuf.readBytes(AbstractByteBuf.java:828)
	com.xiuxian.core.net.WebSocketServerHandler.decodeBinaryWebsocketFrame(WebSocketServerHandler.java:176)
	com.xiuxian.core.net.WebSocketServerHandler.handlerWebSocketFrame(WebSocketServerHandler.java:97)
	com.xiuxian.core.net.WebSocketServerHandler.channelRead0(WebSocketServerHandler.java:258)
	io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:372)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:358)
	io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:350)
	io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:372)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:358)
	io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:350)
	io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:293)
	io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:267)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:372)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:358)
	io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:350)
	io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:372)
	io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:358)
	io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926)
	io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:129)
	io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:610)
	io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:551)
	io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:465)
	io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:437)
	io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873)
	io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
	java.lang.Thread.run(Thread.java:745)
13 回复

把这个语句的方法贴一下

ByteBuf bb = ByteBufAllocator.DEFAULT.buffer();

申请了肯定要回收的

非常感谢大哥你一直回复我。这边finally里面的bb.release()，可以运行，但是会造成整个线程卡死。
我怀疑是不是执行到finally的时候，上一句，s.getCtx().writeAndFlush(b)，还没有执行完毕，所以，意味着，bytebuf不能进行释放。
我想知道，应该怎样正确返回bytebuf呢？

public void sendMsg(Session s, Message message) {

        ByteBuf bb = ByteBufAllocator.DEFAULT.buffer();
        try {
            int status = message.getStatus();
            int protoId = message.getProtoId();

            if (status != 2) {
                writeHeader(bb, 0, status, protoId);// 写头
            } else {
                if (!(message.getMessage() instanceof MessageLite)) {
                    log.error("发送消息类型错误,ProtoId：" + protoId);
                    return;
                }
                MessageLite mes = (MessageLite) message.getMessage();
                byte[] body = mes.toByteArray();
                writeHeader(bb, body.length, status, protoId);// 写头
                bb.writeBytes(body);// body
            }
            BinaryWebSocketFrame b = new BinaryWebSocketFrame(bb);
            s.getCtx().writeAndFlush(b);
        } finally {
            // bb.release(); // 释放内存
        }
    }

改成这样试试?

ByteBuf byteBuf = Unpooled.buffer(10240);

改成ByteBuf byteBuf = Unpooled.buffer(10240);之后，然后打开bb.release();的注释，运行的时候，就会出现下面的报错。

22:48:27,300 [ERROR] [nioEventLoopGroup-3-1] WebSocketServerHandler.handlerWebSocketFrame - [Ljava.lang.StackTraceElement;@2dfe29d9
io.netty.util.IllegalReferenceCountException: refCnt: 0, decrement: 1
	at io.netty.buffer.AbstractReferenceCountedByteBuf.release(AbstractReferenceCountedByteBuf.java:111)
	at com.xiuxian.core.msg.SessionManager.sendMsg(SessionManager.java:193)
	at com.xiuxian.core.net.WebSocketServerHandler.handlerWebSocketFrame(WebSocketServerHandler.java:125)
	at com.xiuxian.core.net.WebSocketServerHandler.channelRead0(WebSocketServerHandler.java:258)
	at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:372)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:358)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:350)
	at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:372)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:358)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:350)
	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:293)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:267)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:372)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:358)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:350)
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:372)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:358)
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926)
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:129)
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:610)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:551)
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:465)
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:437)
	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873)
	at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
	at java.lang.Thread.run(Thread.java:745)

refCnt: 0, 貌似不需要释放了? 试试不release...

不release就没问题了，但是频繁访问，还是会出现那个内存泄漏的问题，我用的是netty4.1.5的包，这个版本是不是bytebuf就不需要release啊？还是说我的release方式不对呢

话说, 是不是找错地方了?? 报的提示是 decodeBinaryWebsocketFrame 呢

com.xiuxian.core.net.WebSocketServerHandler.decodeBinaryWebsocketFrame(WebSocketServerHandler.java:188)

因为这个提示“ResourceLeakDetector.error - LEAK: ByteBuf.release() was not called before it's garbage-collected.”，是典型的netty内存泄漏的提示。
而且，decodeBinaryWebsocketFrame这个方法是我写的，解析ByteBuf的方法。在小规模访问的时候不会出问题，但是频率稍微高一点，就会报这个错。
下面是decodeBinaryWebsocketFrame的源码：

private Message decodeBinaryWebsocketFrame(ByteBuf in) {
        // log.info("开始解码,字节数为：" + in.readableBytes() + " ......");
        // 如果没有接收完Header部分（5字节），直接退出该方法
        while (in.readableBytes() >= HEAD_LENGTH) {

            // 标记开始位置，如果一条消息没传输完成则返回到这个位置
            in.markReaderIndex();
            // 消息体长度
            byte[] bytes = new byte[2];
            bytes[0] = in.readByte();
            bytes[1] = in.readByte();
            int bodyLength = NetUtil.getLittleEndianInt(bytes);
            // log.info("消息体长度：" + bodyLength);

            // 保留字节
            in.readShort();
            // 方法ID
            byte[] bytes2 = new byte[2];
            bytes2[0] = in.readByte();
            bytes2[1] = in.readByte();
            int protoId = NetUtil.getLittleEndianInt(bytes2);
            // log.info("收到消息，协议号：" + protoId);
            // 如果可读长度小于body长度，恢复读指针，退出。
            if (in.readableBytes() < bodyLength) {
                in.resetReaderIndex();
                // log.info("如果可读长度小于body长度，恢复读指针，退出");
                return null;
            }

            // 读取body
            ByteBuf bodyByteBuf = in.readBytes(bodyLength);
            int readableLen = bodyByteBuf.readableBytes();
            byte[] array = new byte[readableLen];
            boolean derect = false;
            if (bodyByteBuf.hasArray()) { // 在堆空间（堆内存字节缓冲区(HEAP BUFFERS)）
                byte[] tempByte = bodyByteBuf.array();
                int offset = bodyByteBuf.arrayOffset() + bodyByteBuf.readerIndex();
                for (int i = 0; i < readableLen; i++) {
                    array[i] = tempByte[offset + i];
                }
            } else { // 不在堆空间（直接内存字节缓冲区(DIRECT BUFFERS)），就把可读的读到array中
//                log.info("处理直接内存");
                bodyByteBuf.getBytes(bodyByteBuf.readerIndex(), array, 0, readableLen);
                derect = true;
            }
            Object o = ProtoManager.getInstance().parseReq(protoId, array);
            // 反序列化
            Message message = new Message(o, protoId, this.sessionId);
            message.setDerectCache(derect);
            return message;
        }
        log.error("没有接收完Header部分（3字节），直接退出解码： " + this.sessionId);
        return null;
    }

另外，报错的188行，就是这个：
bodyByteBuf.getBytes(bodyByteBuf.readerIndex(), array, 0, readableLen);

            // 读取body
            ByteBuf bodyByteBuf = in.readBytes(bodyLength);

这个得释放掉吧?

wendal大哥麻烦你了。我找到bug了，是这样的，我的某个系统有个while死循环，每次跑到这里，就会制造一个死循环.......
应该是卡着死循环，才导致，本来会释放的Bytebuf没有及时释放。所以.......
我还有一个问题，就是，我的这个netty版本，不需要手动释放Bytebuf的么.....

。。。 不知道呢，官方文档查查吧

我好好学英文.......非常感谢！！！

添加回复
请先登陆
回到顶部