Boost.Asio and resource deallocation

So:

Boost.Asio is a cross-platform C++ library for network and low-level I/O programming that provides developers with a consistent asynchronous model using a modern C++ approach.

Ok. Also (“Threads and Boost.Asio“):

io_service provide a stronger guarantee that it is safe to use a single object concurrently

Multiple threads may call io_service::run() to set up a pool of threads from which completion handlers may be invoked.

Oh… so it’s thread-safe, and can dispatch events on multiple threads? That’s great! Let’s write a web server. Let’s see, we can set up a basic_stream_socket to handle an incoming connection:

    boost::asio::io_service io;

    // start some threads which call io.run() (implementation not shown)
    setup_service_threads();

    // supposed we've accepted a connection and have native handle in fd
    auto bss = new boost::asio::ip::tcp::socket(io, ..., fd);

Then we get a request and we set up an asynchronous write to the client:

    WriteHandler handler = ...; // (not an actual type, example only)
    bss->async_write_some(boost::asio::buffer(data, size), handler);

Ok; now the handler gets notified when the write completes (or fails) and can proceed to write the next chunk as appropriate.

Oh, but then we detect overload (or get shutdown, or …) and we need to drop the connection:

    bss->close();

According to documentation for close:

This function is used to close the socket. Any asynchronous send, receive or connect operations will be cancelled immediately, and will complete with the boost::asio::error::operation_aborted error.

Great, now we can delete the socket:

    delete bss;

Aaannnnnd segfault (in another thread).

What happened? Well, it turned out that the write handler had already been called in another thread, just before we called close() on the socket. That handler is now running with the expectation that the socket still exists, and when it tries to access the now-deleted socket (to write the next chunk of data for instance) everything goes brown.

Even if close() were to wait for the handler to finish executing before it returns, that would mean that our current thread is now blocked on an (unbounded) operation in another thread. What we really want, of course, is a way to close the socket and have an asynchronous callback that tells me when there are no more pending asynchronous operations on it, i.e. that it is safe to delete any associated data (including the socket object itself).

Of the course the above is a somewhat oversimplified example, but it demonstrates the essence of the problem; an event source has associated data; at some point, you don’t need the event source any more, and you want to free the data, but you need to be sure there are no pending events before you do. Boost.Asio doesn’t make this as easy as it needs to be.

To be clear: this is an API design issue, not really a bug. I realise that the problem isn’t technically in the Boost code, but the burden of managing this issue without support from the library is significant, and furthermore is a fundamental problem in multi-threaded event loop handling.  (One workaround is to maintain shared_ptr references to the relevant data; the write handler itself needs to maintain one reference in order to prevent the data being swept out from underneath it, as it were. But this would most definitely be a kludge).

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s