A server with 10,000 open connections spends most of its time waiting. At any instant only a handful of sockets have data, and the question is how the process finds those few without asking each of the 10,000 in turn. That’s what readiness APIs do: you hand the kernel a set of file descriptors and it tells you which ones can be read or written without blocking.
Unix has accumulated three generations of this. select dates from 4.2BSD, poll fixed some of its limits, and epoll on Linux (with kqueue as the BSD and macOS equivalent) changed the model. Interviewers ask about the differences because they explain why event-loop servers such as nginx, Node.js and Redis scale to many idle connections.
select and poll: the whole set, every time
select takes three bitmaps (fd_set): descriptors to watch for reading, for writing, and for exceptional conditions, plus a timeout. The kernel scans every descriptor up to the highest one you passed, sets the bits for those that are ready, and returns. Two problems follow from that design.
First, the bitmaps are a fixed size, FD_SETSIZE, which is 1024 on Linux and macOS by default. A descriptor numbered 1024 or higher can’t be represented at all. The select(2) manual page is blunt about it and recommends poll or epoll instead. Python refuses outright:
select: filedescriptor out of range in select()Second, the bitmaps are modified in place, so you rebuild them before every call, and the kernel scans them every call. The work is proportional to the number of descriptors you watch, even if only one is ready.
poll fixes the first problem with an array of struct pollfd entries, so there’s no fixed limit. It doesn’t fix the second: you still pass the full array on each call, and the kernel still checks each entry.
Measuring the cost of idle descriptors
On an M1 Mac, with Python 3.14, I created N socket pairs, made exactly one of them readable, and timed a non-blocking readiness check over the N read ends, averaged over 200 calls. macOS doesn’t have epoll, so the third column is kqueue, which follows the same model:
kq = select.kqueue()
kq.control([select.kevent(s, select.KQ_FILTER_READ, select.KQ_EV_ADD) for s in reads], 0, 0) # register once
kq.control(None, 64, 0) # each wait 100 fds, 1 ready: select 9.5 us, poll 21.8 us, kqueue 0.5 us
500 fds, 1 ready: select 47.2 us, poll 101.5 us, kqueue 0.4 us
1000 fds, 1 ready: select: filedescriptor out of range in select(), poll 201.6 us, kqueue 0.4 us
5000 fds, 1 ready: select: filedescriptor out of range in select(), poll 1093.3 us, kqueue 0.5 us“fds” here counts the watched sockets; each pair uses two descriptors, which is why select already fails at 1,000 pairs. The absolute numbers include Python’s own overhead for building and parsing arguments, but the shape is what matters. select and poll grow linearly with the number of watched sockets: about 0.2 µs per socket for poll. At 5,000 idle sockets, each poll call costs over a millisecond to find one ready socket, and a busy event loop makes thousands of those calls per second. kqueue stays flat at about half a microsecond, because the interest set is registered once and the wait only returns what’s ready.
epoll: an interest list inside the kernel
epoll splits the API into setup and waiting, as described in the epoll(7) manual page:
epoll_create1()creates an epoll instance, itself a file descriptor.epoll_ctl()adds, modifies or removes descriptors from the instance’s interest list.epoll_wait()blocks until at least one descriptor on the interest list is ready and returns only the ready ones.
The kernel keeps a ready list. When data arrives on a socket, the network stack’s wakeup callback puts that socket on the ready list of each epoll instance watching it, so epoll_wait doesn’t scan anything. The cost of a wait is proportional to the number of ready descriptors, and the cost of an idle connection is a bit of kernel memory.
Here’s the shape of an epoll loop using Python’s select.epoll. It’s Linux-only and I didn’t run it on this Mac; the kqueue measurements above show the same model in action.
import select, socket
srv = socket.socket(); srv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
srv.bind(("0.0.0.0", 8080)); srv.listen(1024); srv.setblocking(False)
ep = select.epoll()
ep.register(srv.fileno(), select.EPOLLIN)
conns = {}
while True:
for fd, events in ep.poll(timeout=1.0):
if fd == srv.fileno():
c, _ = srv.accept(); c.setblocking(False)
conns[c.fileno()] = c
ep.register(c.fileno(), select.EPOLLIN)
elif events & (select.EPOLLHUP | select.EPOLLERR):
ep.unregister(fd); conns.pop(fd).close()
else:
data = conns[fd].recv(65536)
if not data:
ep.unregister(fd); conns.pop(fd).close()
else:
conns[fd].send(data) # echo; real code buffers and waits for EPOLLOUT if send is partialNote
Readiness doesn’t mean the operation will transfer everything. A socket reported writable may accept only part of your buffer. Keep unsent bytes, register for
EPOLLOUT, and finish the write when the kernel says there’s room.
Level-triggered versus edge-triggered
By default epoll is level-triggered: epoll_wait reports a descriptor as long as it’s ready. With EPOLLET it becomes edge-triggered: it reports a descriptor when its state changes, such as new data arriving, and then not again until another change. kqueue offers the same choice through the EV_CLEAR flag, which let me show the difference on macOS. I wrote 100 bytes into a socket, then waited three times, reading only 10 bytes after each wakeup:
level wait1: ready, 100 bytes | wait2: ready, 90 bytes | wait3: ready, 80 bytes
edge (EV_CLEAR) wait1: ready, 100 bytes | wait2: nothing | wait3: nothingLevel-triggered kept reporting the socket while data remained. Edge-triggered reported it once, and because no new data arrived, it never reported it again, even though 90 bytes were still waiting. In a real server that’s a connection that hangs until the client sends something else, which it may never do because it’s waiting for your reply.
The rule for edge-triggered mode follows directly, and epoll(7) states it: use non-blocking descriptors and keep reading (or writing) until the call fails with EAGAIN. Only then go back to waiting. The payoff is fewer wakeups for busy descriptors and the ability to avoid re-arming. The risk is the stall above, plus starvation if one very busy connection is drained in a loop while others wait; production loops cap how much they read per connection per iteration.
EPOLLONESHOT is a related flag: after one event, the descriptor is disabled until you re-arm it with epoll_ctl(EPOLL_CTL_MOD). It’s useful when a thread pool handles events, so two threads never process the same connection at once.
Mistakes worth mentioning
- Mixing edge-triggered mode with blocking sockets. The drain-until-
EAGAINloop blocks forever on the last read. - Forgetting that closing a descriptor doesn’t always remove it. epoll tracks the underlying open file description; if a duplicate descriptor (from
duporfork) still refers to it, events can still arrive for a descriptor number you’ve closed. Unregister before closing. - Thinking epoll makes I/O asynchronous. It tells you when a read won’t block; the read itself is still a synchronous system call that copies data. Completion-based models such as io_uring are a different design.
- Assuming epoll helps regular files. select and poll always report regular files as ready, and
epoll_ctlrefuses to register them withEPERM. Disk I/O needs threads or io_uring.
Interview practice: the chat server at 1,100 users
A chat server uses select() and works fine in testing. In production it starts failing once about 1,000 users are connected: new connections are accepted, but those users never receive messages, and CPU use climbs as more people join. Explain both symptoms and your fix.
Answer
Both symptoms come from select’s design. Each client is a descriptor, and once descriptor numbers reach 1024 they can’t be placed in an fd_set, which is limited to FD_SETSIZE. Depending on the language, that either raises an error or silently writes past the bitmap in C, so the newest connections are never watched and their users never get messages. The CPU growth is because select scans every watched descriptor on every call and the server rebuilds the sets each time, so each loop iteration costs time proportional to all connected users even when only a few are active. The fix is to move to epoll on Linux (or kqueue on BSD and macOS), usually through a library such as libuv, libevent or the language’s own event loop rather than by hand: register each connection once, and let each wait return only ready sockets. I’d use level-triggered mode unless there’s a measured need for edge-triggered, check that the process’s open-file limit (ulimit -n) is well above the expected connection count, and load test with several thousand mostly idle connections to confirm CPU stays flat.