5. Java Concurrency — Architect-Level Deep Dive

5. Java Concurrency — Architect-Level Deep Dive

5.1 Thread Lifecycle and Creation (Runnable, Thread, Callable)

Thread Lifecycle (states from Thread.State enum):

NEW → RUNNABLE → [BLOCKED / WAITING / TIMED_WAITING] → TERMINATED
State Meaning
NEW Thread object created, start() not yet called
RUNNABLE Eligible to run — may be actually executing or waiting for CPU scheduling (JVM doesn’t distinguish these as separate states; that’s the OS scheduler’s concern)
BLOCKED Waiting to acquire a monitor lock (entering a synchronized block held by another thread)
WAITING Waiting indefinitely for another thread’s action — Object.wait() (no timeout), Thread.join() (no timeout), LockSupport.park()
TIMED_WAITING Same as WAITING but bounded — Thread.sleep(ms), wait(timeout), join(timeout)
TERMINATED run() completed (normally or via uncaught exception)

Three ways to define a task:

// 1. Extending Thread — limits you to single inheritance, generally discouraged
class MyThread extends Thread {
    public void run() { /* work */ }
}

// 2. Implementing Runnable — preferred; no return value, can't throw checked exceptions
Runnable task = () -> System.out.println("running");
new Thread(task).start();

// 3. Implementing Callable<V> — returns a value, can throw checked exceptions, used with ExecutorService
Callable<Integer> task = () -> { return computeSomething(); };
Future<Integer> future = executorService.submit(task);

Why Runnable over extending Thread: Java has single inheritance — extending Thread burns your one superclass slot and conflates “being a thread” with “having a task,” which is a poor separation of concerns. Runnable is a pure functional task abstraction (and as of Java 8, a lambda target) that can be handed to Thread, any ExecutorService, or scheduled — it’s the architecturally correct default.

start() vs run(): Calling run() directly executes the task synchronously on the current thread — no new thread is created. Only start() triggers JVM/OS thread creation and asynchronous execution. This is a very common junior-developer bug (and a fair interview screening question) — calling run() “still works” but defeats the entire purpose.

Thread cannot be start()ed twice — throws IllegalThreadStateException; once TERMINATED, a thread object is single-use.

5.2 Synchronization, Locks, and Deadlocks

Intrinsic locking (synchronized):

synchronized void increment() { count++; }          // instance monitor lock (this)
synchronized static void staticMethod() { }          // class-level lock (MyClass.class)
void method() {
    synchronized(lockObject) { /* critical section */ }  // explicit monitor object
}
  • Every Java object has an associated intrinsic monitor. synchronized acquires it on entry, releases on exit (even via exception) — this exception-safety is synchronized‘s biggest practical advantage over explicit locks if you forget a finally.
  • Guarantees both mutual exclusion (one thread in the critical section at a time) and visibility (changes made inside a synchronized block by one thread are visible to the next thread that synchronizes on the same monitor — a happens-before relationship per the Java Memory Model).
  • Reentrant — a thread already holding a monitor can re-enter another synchronized block guarded by the same monitor without deadlocking itself (tracked via a recursion counter on the monitor).

Explicit Locks — java.util.concurrent.locks.Lock / ReentrantLock:

Lock lock = new ReentrantLock();
lock.lock();
try {
    // critical section
} finally {
    lock.unlock();   // MUST be in finally — no automatic release
}

Advantages over synchronized:

  • tryLock() — non-blocking attempt, with optional timeout — enables deadlock-avoidance strategies impossible with synchronized.
  • lockInterruptibly() — allows a blocked thread to respond to interruption.
  • ReentrantReadWriteLock — separate read/write lock pair; multiple readers can hold the read lock concurrently, writers get exclusive access — significant throughput win for read-heavy/write-rare shared state versus a single exclusive lock.
  • Fairness option (new ReentrantLock(true)) — FIFO ordering for waiting threads, at some throughput cost (default is non-fair, which is faster but can starve long-waiting threads under contention).

Deadlock — classic conditions (all four must hold, per Coffman):

  1. Mutual exclusion (locks are exclusive)
  2. Hold and wait (thread holds one lock while waiting for another)
  3. No preemption (locks can’t be forcibly taken away)
  4. Circular wait (Thread A waits for lock held by B, B waits for lock held by A)

Classic deadlock pattern:

// Thread 1: synchronized(lockA) { synchronized(lockB) { ... } }
// Thread 2: synchronized(lockB) { synchronized(lockA) { ... } }
// → circular wait → deadlock

Prevention strategies an architect should mandate:

  • Lock ordering — always acquire locks in a globally consistent order (e.g., by object hash code or an assigned ID) across all code paths — eliminates circular wait by construction.
  • tryLock() with timeout — back off and retry rather than block indefinitely, breaking the “hold and wait” condition.
  • Minimize lock scope/nesting — avoid holding a lock while calling into code you don’t control (which might acquire other locks).
  • Single-lock-at-a-time discipline where feasible — eliminates nested-lock deadlocks entirely.

Diagnosis in production: jstack <pid> (or thread dump via JFR/VisualVM) explicitly reports Found one Java-level deadlock with the full thread/lock cycle — this is usually the fastest path to root-causing a hung service before reaching for heavier tooling.

Livelock and starvation (related but distinct failure modes worth distinguishing in interviews):

  • Livelock — threads actively keep responding to each other (e.g., both repeatedly back off and retry in lockstep) but make no actual progress — unlike deadlock, threads aren’t blocked, just unproductively busy.
  • Starvation — a thread never gets CPU time or lock access because other threads are greedily favored (e.g., unfair lock policies, or a thread perpetually low-priority relative to others).

5.3 volatile, Atomic Classes, and Memory Visibility

The core problem volatile solves — the Java Memory Model (JMM):

Without synchronization, each thread may keep a cached copy of a variable (in CPU registers or per-core caches) and isn’t guaranteed to see writes made by other threads in any particular timeframe — or even ever, if the JIT decides to hoist a loop-invariant read of that variable out of a loop entirely. This is not a hardware quirk alone — it’s a defined consequence of the JMM allowing certain reorderings for performance, absent explicit synchronization.

volatile guarantees:

  1. Visibility — every write to a volatile variable is immediately flushed to main memory; every read fetches the current value from main memory, not a cached/stale copy.
  2. Ordering — establishes a happens-before relationship: a write to a volatile field happens-before every subsequent read of that same field by any thread. This also prevents the compiler/JIT from reordering other instructions across the volatile read/write boundary.

What volatile does NOT guarantee:

  • Atomicity of compound operations. volatile int counter; counter++; is still a read-modify-write — two threads can both read the same value, both increment, and one update is lost. volatile only makes single reads/writes of that variable atomic and visible — it does nothing for multi-step operations.
private volatile boolean running = true;   // correct use: simple flag, single writer/many readers
public void stop() { running = false; }
public void run() { while (running) { /* work */ } }

This is the textbook correct volatile use case — a flag with one writer thread and one-or-more reader threads, no compound operations.

Atomic classes (java.util.concurrent.atomic):

AtomicInteger counter = new AtomicInteger(0);
counter.incrementAndGet();              // atomic read-modify-write
counter.compareAndSet(expected, update); // CAS — the foundational primitive
  • Built on CAS (Compare-And-Swap), a hardware-level atomic instruction (cmpxchg on x86) exposed via sun.misc.Unsafe / VarHandle in modern JDKs — no locking required, making these significantly faster than synchronized under low-to-moderate contention.
  • Under high contention, CAS-based classes can suffer from retry storms (many threads repeatedly failing CAS and retrying) — this is what LongAdder/DoubleAdder address, by striping the counter across multiple internal cells and summing them only when the total is read, dramatically reducing contention for write-heavy, read-rare counters (metrics/statistics counters being the classic use case).
  • AtomicReference<V> — same CAS pattern for object references, commonly used to implement lock-free data structures (e.g., a lock-free stack via CAS-looping on a head pointer).

Decision guide:

Need Tool
Simple visibility flag, single writer volatile
Atomic counter/accumulator, low-moderate contention AtomicInteger/AtomicLong
High-contention counter (metrics, request counts under load) LongAdder
Atomic compound logic, mutual exclusion needed synchronized / ReentrantLock
Lock-free reference swap (e.g., immutable config hot-swap) AtomicReference

5.4 Executor Framework and Thread Pools

Why not manually manage Thread objects: Uncontrolled thread creation per task is expensive (OS thread creation/teardown overhead, unbounded resource consumption under load) and offers no lifecycle management. The Executor framework decouples task submission from task execution policy.

Core abstraction:

ExecutorService executor = Executors.newFixedThreadPool(10);
executor.submit(() -> doWork());           // Runnable, returns Future<?>
Future<Integer> f = executor.submit(callable);  // Callable, returns Future<V>
executor.shutdown();                        // graceful — finishes queued tasks, rejects new ones
executor.shutdownNow();                      // attempts to interrupt running tasks

Executors factory methods — and why architects should generally avoid them in production:

Factory Behavior Risk
newFixedThreadPool(n) Fixed pool size, unbounded LinkedBlockingQueue Queue can grow unbounded under sustained overload → OOM
newCachedThreadPool() Unbounded pool size, threads reused/expire after idle Can spawn unbounded threads under burst load → resource exhaustion
newSingleThreadExecutor() One thread, unbounded queue Same unbounded-queue risk as fixed pool
newScheduledThreadPool(n) For delayed/periodic tasks Same unbounded queue concern

The common thread: most Executors factories use unbounded queues, meaning backpressure is invisible until you hit an OOM rather than failing fast. Production-grade code generally constructs ThreadPoolExecutor directly with explicit bounds:

ThreadPoolExecutor executor = new ThreadPoolExecutor(
    10,                                   // corePoolSize
    20,                                   // maximumPoolSize
    60L, TimeUnit.SECONDS,                // keepAliveTime for excess threads above core
    new ArrayBlockingQueue<>(100),        // BOUNDED queue — forces explicit overload handling
    new ThreadFactoryBuilder().setNameFormat("worker-%d").build(),  // named threads (critical for thread-dump diagnosis)
    new ThreadPoolExecutor.CallerRunsPolicy()  // rejection policy when queue + max threads both exhausted
);

ThreadPoolExecutor sizing logic (the part architects get asked to whiteboard):

  • New task arrives → if running threads < corePoolSize, start a new core thread (even if other core threads are idle).
  • Else → if queue isn’t full, enqueue the task.
  • Else → if running threads < maximumPoolSize, start a new (non-core) thread.
  • Else → apply the rejection policy (AbortPolicy throws, CallerRunsPolicy runs the task on the submitting thread as backpressure, DiscardPolicy/DiscardOldestPolicy drop tasks).

Pool sizing heuristics:

  • CPU-bound tasks: pool size ≈ number of available cores (Runtime.availableProcessors()), since more threads than cores just adds context-switch overhead.
  • I/O-bound tasks: pool size can exceed core count substantially, since threads spend most time blocked waiting on I/O rather than consuming CPU — common formula: threads = cores * (1 + waitTime/computeTime).

Future<V> limitations that motivate CompletableFuture (5.5): Future.get() blocks the calling thread; no way to chain dependent async operations, combine multiple futures, or register a callback without blocking — purely a handle to retrieve a result later, not a composition primitive.

5.5 CompletableFuture and Async Programming

CompletableFuture<T> (Java 8+) implements both Future<T> and CompletionStage<T> — adding a fluent, non-blocking composition API for chaining async work.

Creation:

CompletableFuture<String> cf = CompletableFuture.supplyAsync(() -> fetchData());
CompletableFuture<Void> cf2 = CompletableFuture.runAsync(() -> doSideEffect());
  • Without an explicit Executor argument, tasks run on the JVM-wide common fork-join pool (ForkJoinPool.commonPool()) — a frequently-overlooked production risk, since this pool is shared by everything using default async/parallel APIs (including parallel streams), so a slow/blocking task submitted here can starve unrelated concurrent work elsewhere in the same JVM. Architects should pass a dedicated Executor explicitly for any non-trivial workload.

Chaining (sequential composition):

CompletableFuture<String> result = CompletableFuture
    .supplyAsync(() -> fetchUser(id))
    .thenApply(user -> user.getName())          // transform result, sync on completing thread
    .thenApplyAsync(name -> enrich(name), executor)  // transform, explicitly async on given pool
    .thenAccept(name -> log.info(name))          // consume, no return value
    .exceptionally(ex -> { log.error(ex); return "fallback"; });  // error handling
  • thenApply/thenAccept/thenRun — sync variants run on whichever thread completed the prior stage (could be the calling thread if already complete, or the async worker thread).
  • thenApplyAsync/etc. — explicitly hop to the common pool or a supplied executor, useful when the prior stage’s thread shouldn’t be tied up by subsequent work.

Combining multiple futures:

CompletableFuture<Combined> combined = cf1.thenCombine(cf2, (r1, r2) -> merge(r1, r2));

CompletableFuture<Void> allDone = CompletableFuture.allOf(cf1, cf2, cf3);  // waits for all
CompletableFuture<Object> anyDone = CompletableFuture.anyOf(cf1, cf2, cf3); // first to complete
  • allOf() returns CompletableFuture<Void> — doesn’t aggregate individual results; you retrieve them via the original future handles after allOf completes (a common point of confusion).

Error handling:

cf.handle((result, ex) -> ex != null ? fallback : result);  // always runs, sees both outcomes
cf.exceptionally(ex -> fallback);                             // only runs on exception
cf.whenComplete((result, ex) -> { /* side effect, doesn't transform */ });

Production patterns architects should enforce:

  • Always supply a dedicated, bounded Executor for non-trivial async work — never let real workloads silently run on the common pool.
  • Always terminate a chain with .exceptionally()/.handle() — an unhandled exception in a CompletableFuture chain is silently swallowed into the future’s completed-exceptionally state until someone calls .get()/.join(), which is a frequent source of “the async task just disappeared” bugs.
  • Use timeouts (orTimeout()/completeOnTimeout(), Java 9+) on external-call-backed futures to avoid indefinite hangs propagating upstream.

5.6 java.util.concurrent Package (CountDownLatch, Semaphore, CyclicBarrier)

CountDownLatch — one-time gate; N threads must complete before waiting thread(s) proceed.

CountDownLatch latch = new CountDownLatch(3);
// 3 worker threads each call: latch.countDown();
latch.await();   // blocks until count reaches 0
  • Not reusable — once it hits zero, it stays at zero forever. Common use: wait for N parallel initialization tasks to finish before starting main processing, or coordinate a simultaneous “start” signal (initialize latch to 1, have all worker threads await(), then the controller calls countDown() once to release them all at once).

Semaphore — controls concurrent access to a limited resource pool via permits.

Semaphore semaphore = new Semaphore(5);  // 5 permits
semaphore.acquire();
try { /* access limited resource, e.g., connection pool slot */ }
finally { semaphore.release(); }
  • Generalizes mutual exclusion (new Semaphore(1) ≈ a lock, though without ownership/reentrancy semantics) to N-way concurrent access — classic use: rate-limiting concurrent calls to an external API/database connection pool to N simultaneous in-flight requests.
  • Supports fairness option and tryAcquire() with timeout, same pattern as Lock.

CyclicBarrier — N threads must all reach a common barrier point before any proceeds; reusable across cycles (unlike CountDownLatch).

CyclicBarrier barrier = new CyclicBarrier(4, () -> System.out.println("All arrived, proceeding"));
// each of 4 threads calls:
barrier.await();   // blocks until all 4 have called await()
  • The optional constructor Runnable runs once per barrier trip, on whichever thread arrives last — useful for a “merge results from this round, then let everyone proceed to the next round” pattern (e.g., synchronized phases in a parallel simulation).
  • Key distinction from CountDownLatch: the barrier resets automatically after release, supporting repeated synchronization points (round-based/iterative parallel algorithms), whereas a latch is consumed once.

Quick comparison for interview framing:

Primitive Reusable Use case
CountDownLatch No Wait for N events before proceeding (one-time)
Semaphore Yes (permits cycle) Bound concurrent access to a resource pool
CyclicBarrier Yes Synchronize N threads at repeated checkpoints

Also worth knowing: Exchanger<V> — pairs of threads swap objects at a synchronization point (less commonly asked, but rounds out the j.u.c synchronizer family); Phaser — a more flexible generalization of CyclicBarrier/CountDownLatch supporting dynamic party registration and multi-phase coordination.

5.7 Fork/Join Framework and Parallel Streams

Fork/Join Framework (java.util.concurrent.ForkJoinPool) — Java 7+, designed for divide-and-conquer parallelism: recursively split a task into smaller subtasks, execute them in parallel, then combine results.

class SumTask extends RecursiveTask<Long> {
    private final int[] arr; private final int start, end;
    static final int THRESHOLD = 1000;

    protected Long compute() {
        if (end - start <= THRESHOLD) {
            long sum = 0;
            for (int i = start; i < end; i++) sum += arr[i];
            return sum;                                   // base case — direct computation
        }
        int mid = (start + end) / 2;
        SumTask left = new SumTask(arr, start, mid);
        SumTask right = new SumTask(arr, mid, end);
        left.fork();                                       // asynchronously schedule left
        long rightResult = right.compute();                 // compute right on current thread
        long leftResult = left.join();                      // wait for left's result
        return leftResult + rightResult;
    }
}
  • RecursiveTask<V> (returns a value) / RecursiveAction (no return value, e.g., parallel array mutation) are the two base classes.
  • Work-stealing: each worker thread in the pool maintains its own double-ended task queue. A thread pushes/pops from the head of its own queue (LIFO, cache-friendly for recently-created subtasks); when a thread’s queue is empty, it steals from the tail of another thread’s queue (FIFO from the stealer’s perspective) — this specific design (steal from the opposite end) minimizes contention between the owning thread and the stealing thread, since they’re operating on opposite ends of the same deque.
  • This is why Fork/Join dramatically outperforms a naive fixed-size ThreadPoolExecutor for unevenly-sized recursive workloads — idle threads automatically rebalance work without explicit coordination.

Parallel Streams — built directly on the common ForkJoinPool:

list.parallelStream().map(this::process).collect(Collectors.toList());
  • Internally, the stream’s source is split via a Spliterator, and processing follows the same fork/join divide-and-conquer model under the hood.
  • Critical production gotcha: parallel streams use ForkJoinPool.commonPool() by default — the same shared pool used by default CompletableFuture.supplyAsync() calls and any other JDK code relying on the common pool. A long-running or blocking operation inside a parallelStream() pipeline can starve unrelated parallel work elsewhere in the same JVM. There’s no built-in way to specify a custom executor for parallelStream() directly (the documented workaround is submitting the entire parallel stream operation as a task into a custom ForkJoinPool via customPool.submit(() -> list.parallelStream()...).get()).

When parallel streams help vs hurt:

Favorable Unfavorable
Large datasets (typically >10,000 elements — exact threshold is workload-dependent, benchmark don’t assume) Small collections — fork/join overhead exceeds savings
CPU-bound, stateless, side-effect-free operations I/O-bound or blocking operations (starves shared pool)
Splittable source (ArrayList, arrays — good Spliterator support) Poorly splittable source (LinkedList — bad locality, expensive splitting)
Independent element processing Operations requiring ordering guarantees (ordering adds overhead to reassemble)

Architect’s rule of thumb: Default to sequential streams; reach for parallelStream() only after profiling shows a genuine CPU-bound bottleneck on a sufficiently large, easily-splittable dataset — and even then, benchmark on realistic data sizes, since the fork/join overhead and common-pool contention risk can make parallel streams slower than sequential for many real-world cases, a fact that surprises engineers who treat .parallelStream() as a free performance lever.