A Service is a stable name and virtual IP in front of a changing set of Pods. The interesting part is how it decides which Pods are in that set, because there’s no reference from a Service to a Deployment. It’s a label query, re-evaluated continuously. Most “my Service doesn’t work” problems are that query matching nothing, matching the wrong Pods, or matching the right Pods on the wrong port.
Manifests here target Kubernetes 1.30+ and were checked for valid YAML. Treat the kubectl output as representative of what you’d see, not a capture from a running cluster.
From selector to EndpointSlices
Here’s a Deployment and a Service for an inventory API:
apiVersion: apps/v1
kind: Deployment
metadata:
name: inventory
namespace: shop
spec:
replicas: 3
selector:
matchLabels:
app: inventory
template:
metadata:
labels:
app: inventory
track: stable
spec:
containers:
- name: api
image: registry.example.com/inventory:2.4.0
ports:
- name: http
containerPort: 8080
readinessProbe:
httpGet:
path: /readyz
port: http
---
apiVersion: v1
kind: Service
metadata:
name: inventory
namespace: shop
spec:
selector:
app: inventory
ports:
- name: http
port: 80
targetPort: httpWhen this is applied, the EndpointSlice controller in the control plane watches Pods in shop labelled app: inventory. For each one it records the Pod’s IP, the resolved target port (the container port named http, so 8080), the node, and whether the Pod is ready. It writes those into one or more EndpointSlice objects labelled with the Service’s name:
$ kubectl get endpointslices -n shop -l kubernetes.io/service-name=inventory
NAME ADDRESSTYPE PORTS ENDPOINTS AGE
inventory-x7k2p IPv4 8080 10.244.1.21,10.244.2.14,10.244.3.9 5mkube-proxy on every node (or your CNI’s replacement for it, such as Cilium) watches EndpointSlices and programs rules so that a connection to the Service’s ClusterIP on port 80 is redirected to one of those Pod IPs on 8080. CoreDNS answers inventory.shop.svc.cluster.local with the ClusterIP.
You’ll still see the older Endpoints object (kubectl get endpoints inventory). It’s kept for compatibility and is officially deprecated as of 1.33 in favour of EndpointSlices, which split large Services into chunks of up to 100 endpoints so a single Pod change doesn’t rewrite a 5,000-entry object.
Readiness decides who gets traffic
A Pod that matches the selector but isn’t Ready is still listed in the slice, with ready: false. kube-proxy doesn’t route new connections to it. That’s how readiness probes keep traffic away during warm-up and shutdown:
$ kubectl get endpointslice inventory-x7k2p -n shop \
-o jsonpath='{range .endpoints[*]}{.addresses[0]}{" ready="}{.conditions.ready}{"\n"}{end}'
10.244.1.21 ready=true
10.244.2.14 ready=true
10.244.3.9 ready=falseIf every matching Pod is unready, the Service has no usable endpoints. Connections to its ClusterIP are then rejected: with the iptables mode, clients typically get Connection refused immediately, because kube-proxy installs a reject rule for Services without endpoints. That’s a useful clue: a refusal from a Service IP usually means “no ready backends”, not “the app is down” (the app might be up and failing readiness).
The exception is publishNotReadyAddresses: true, used on headless Services for StatefulSets that need peers to find each other before they’re ready.
Three ways the match goes wrong
The selector matches nothing. Labels are exact, case-sensitive strings. app: Inventory in the Service and app: inventory on the Pods means zero endpoints. So does a Service in default selecting Pods in shop: selectors only look in the Service’s own namespace. Check both sides directly:
$ kubectl get svc inventory -n shop -o jsonpath='{.spec.selector}'
{"app":"inventory","tier":"backend"}
$ kubectl get pods -n shop -l app=inventory,tier=backend
No resources found in shop namespace.
$ kubectl get pods -n shop -l app=inventory --show-labels
NAME READY LABELS
inventory-5c9f8d7b6-2mxqz 1/1 app=inventory,pod-template-hash=5c9f8d7b6,track=stableThe Service requires tier: backend and the Pods don’t have it. All selector terms must match.
The selector matches too much. Suppose someone deploys inventory-canary with labels app: inventory, track: canary to test version 2.5.0. The Service selects only app: inventory, so the canary’s Pods join the Service immediately and take a share of production traffic proportional to their count. That might be intended (it’s a cheap canary) or a surprise. The Deployment’s own selector is separate and immutable after creation; it decides which Pods the Deployment manages, while the Service’s selector decides which receive traffic. Keeping them aligned is your job.
The port is wrong. The selector is fine, endpoints exist, but connections fail or time out. Usually targetPort names a port the container doesn’t listen on: the Service says targetPort: 8000 and the app listens on 8080. Note that containerPort in the Pod spec is informational; the app listens wherever its code says. Using a named port (targetPort: http) helps, because if the Pod template changes to 9090 the Service follows. If a named port doesn’t exist on a Pod, that Pod gets no port in the slice.
A debugging walkthrough
A developer reports that curl http://inventory.shop/items from another Pod hangs, then fails. Work outward from the client:
- DNS.
nslookup inventory.shopfrom the client Pod returns10.96.41.7. Name resolution works. - Endpoints.
kubectl get endpointslices -n shop -l kubernetes.io/service-name=inventoryshows three addresses, all ready. Selector and readiness are fine. - Direct to a Pod.
curl -m 3 http://10.244.1.21:8080/itemsfrom the client Pod works. The app is fine on 8080. - Service port mapping.
kubectl get svc inventory -n shop -o yamlshowsport: 80, targetPort: 8000. That’s it: traffic goes to port 8000, where nothing listens.
The fix is targetPort: http (or 8080). If step 3 had also failed, the next suspects would be a NetworkPolicy dropping the traffic or the app binding only to 127.0.0.1 instead of 0.0.0.0, which accepts loopback connections but nothing from the Pod network.
Services without selectors
A Service with no selector gets no automatic EndpointSlices. You create them yourself, pointing at anything: a database VM, a service in another cluster. Clients use a normal Service name, and you can repoint it later without changing them.
apiVersion: v1
kind: Service
metadata:
name: legacy-erp
namespace: shop
spec:
ports:
- name: https
port: 443
---
apiVersion: discovery.k8s.io/v1
kind: EndpointSlice
metadata:
name: legacy-erp-1
namespace: shop
labels:
kubernetes.io/service-name: legacy-erp
addressType: IPv4
ports:
- name: https
port: 443
endpoints:
- addresses: ["10.30.4.18"]The kubernetes.io/service-name label is what ties the slice to the Service, and the port name must match.
Interview practice
“After a refactor, the orders team switched their labels from app: orders to app.kubernetes.io/name: orders. Because a Deployment’s selector is immutable, their pipeline deleted and recreated the Deployment with the new selector and template. All new Pods are Running and Ready, but the orders Service now returns connection refused. Explain, and describe a safe way to make this kind of change.”
Answer
The Service still selects app: orders. Deleting the old Deployment removed every Pod carrying that label, and the new Pods carry only app.kubernetes.io/name: orders, so the EndpointSlice controller dropped the old addresses and had nothing to add. The Service went from three endpoints to zero. With no endpoints, kube-proxy rejects connections to the ClusterIP, hence the immediate refusal. Running and Ready Pods don’t help if nothing selects them, and the Deployment itself looks perfectly healthy because it’s happy with its own Pods. (Deleting and recreating also caused a brief full outage on its own, even before the label mismatch.)
The safe sequence is additive. Add the new label to the existing Deployment’s Pod template while keeping the old one (the template may carry extra labels beyond the selector) and roll that out. Then change the Service selector to the new label, which every serving Pod now has, and confirm endpoints with kubectl get endpointslices. If the Deployment’s selector itself must change, create a second Deployment with the new selector alongside the old one, let it become Ready (the Service, selecting the new label, covers both sets only if the old Pods have it too), then scale the old one to zero. At every step, the Service matches at least the Pods that are serving.
Further reading
- Kubernetes interview questions
- Kubernetes MCQs
- Service types
- Network Policies and traffic direction
- Official docs: Service