msg/Accepter: fix race in accepter shutdown

We want to avoid a race like:

- entry() starts, populates pfd with listen_sd, gets past !done check
- stop() does shutdown + close on listen_sd
- someone else opens a new fd
- entry() thread calls poll(2) on wrong sd
- stop() calls join, waits forever for entry thread

Signed-off-by: Sage Weil <sage@inktank.com>

1 files changed, 7 insertions, 1 deletions

diff --git a/src/msg/Accepter.cc b/src/msg/Accepter.cc
index 03e2d65211e..28a475079c6 100644
--- a/src/msg/Accepter.cc
+++ b/src/msg/Accepter.cc
@@ -247,10 +247,16 @@ void Accepter::stop()
   ldout(msgr->cct,10) << "stop accepter" << dendl;
   if (listen_sd >= 0) {
     ::shutdown(listen_sd, SHUT_RDWR);
+  }
+
+  // wait for thread to stop before closing the socket, to avoid
+  // racing against fd re-use.
+  join();
+
+  if (listen_sd >= 0) {
     ::close(listen_sd);
     listen_sd = -1;
   }
-  join();
   done = false;
 }