A Deep Dive into Virtual Threads: The Evolution of Multithreading in Java

In modern software development, managing concurrency is a critical challenge. For years, developers have used threads to achieve parallelism, but as applications scale, the traditional threading model becomes inefficient. Enter virtual threads, a feature in Java designed to revolutionize how we handle concurrency.

In this blog, we’ll dive into virtual threads, compare them with normal threads, when they should (and shouldn’t) be used, and the architectural principles that make them so powerful. We’ll also explore their impact on CPU usage and thread count. To wrap it up, we’ll walk through some sample code to give you a hands-on understanding of how virtual threads work in practice.

What are Virtual Threads?

Virtual threads are lightweight threads that are managed by the Java Virtual Machine (JVM) rather than by the operating system (OS). They are part of Java’s Project Loom, which aims to make concurrency easier to work with by allowing you to create thousands or even millions of threads without incurring significant performance overhead.

Traditional threads are directly managed by the OS, and they can be resource-intensive. Each thread requires its stack and other system resources. Virtual threads, on the other hand, share the same OS-level thread and are scheduled by the JVM, making them much lighter in comparison.

Why are Virtual Threads Better Than Traditional Threads?

1. Lower Overhead

Virtual threads are much lighter than normal threads because they don’t require a separate stack or OS-level context switching. This means you can have a much higher number of threads running concurrently without worrying about consuming excessive memory.

2. Scalability

Because virtual threads are so lightweight, you can create thousands, or even millions, of them in your application. Traditional threads struggle with this level of concurrency, as the operating system is limited in how many threads it can manage effectively.

3. Better CPU Utilization

Virtual threads allow better CPU utilization since the JVM can schedule virtual threads more efficiently, and it can pause threads that are waiting for I/O operations without blocking OS resources.

4. Better Resource Utilization

Virtual threads eliminate the need for complex thread pooling and allow for dynamic scaling of concurrent tasks, improving overall resource utilization compared to traditional threads.

When to Use Virtual Threads

• I/O-bound Tasks: Virtual threads are great for applications that involve many I/O operations, such as web servers, database queries, or any tasks where a thread often waits for data or a response from an external service.
• High-concurrency Systems: If your system needs to handle thousands or millions of tasks simultaneously—think of a server handling requests from multiple clients—virtual threads provide an efficient way to handle this without overwhelming the JVM or the OS.

When Not to Use Virtual Threads

• CPU-bound Tasks: Virtual threads are ideal for I/O-bound tasks, but if your application is CPU-bound (i.e., performing heavy computations), virtual threads may not offer a performance advantage. Traditional threads may be more efficient in these cases since they don’t rely on the JVM for scheduling, and the overhead of managing many virtual threads can outweigh the benefits.
• Heavy Blocking on Synchronization: Virtual threads are most efficient when performing non-blocking I/O operations. Suppose your application spends a lot of time blocking synchronization (e.g., locks or semaphores). In that case, the performance benefits of virtual threads might be diminished, as the blocking operations can still lead to contention and resource inefficiency.

Behind the Scenes: Virtual Threads Architecture

Traditional Threads (Platform Threads)

• Managed by OS: Platform threads are created and scheduled by the operating system. Each thread gets a dedicated OS-level stack, and the OS handles the context switching between threads.
• Heavy: Each thread can consume significant memory (typically around 1MB per thread), and creating a large number of threads can cause the JVM to run out of resources.

Virtual Threads (Managed by JVM)

• Managed by JVM: Virtual threads are managed by the JVM, not the OS. The JVM schedules virtual threads onto platform threads. These threads share OS-level resources, reducing overhead.
• Lightweight: Virtual threads share memory and stack space efficiently. The JVM can handle hundreds of thousands or even millions of virtual threads.

Thread Count in JConsole for Platform Threads vs Virtual Threads

When using JConsole (Java’s monitoring tool), you’ll notice a difference in how platform threads and virtual threads are counted:

• Platform Threads: These threads are visible as standard Java threads, and they are listed under the “Threads” tab in JConsole. You’ll see them in the “Thread” count, and these threads are the ones that the JVM interacts with at the OS level.
• Virtual Threads: Virtual threads are also listed under the same “Threads” tab but are labeled as part of a special virtual thread group. The number of virtual threads can often be much higher than platform threads, as virtual threads are lightweight and can be created in large numbers.

Code Example to Demonstrate Both Platform and Virtual Threads

Let’s write a simple example to demonstrate the difference between virtual threads and normal threads.

public class ThreadDemo {
  public static void main(String[] args) {
      int numThreads = 1000;

      System.out.println("Starting Platform Thread Example...");
      long platformStartTime = System.nanoTime();
      for (int i = 0; i < numThreads; i++) {
           Thread platformThread = new Thread(() -> {
              try {
                  Thread.sleep(5000);
              } catch (InterruptedException e) {
                  e.printStackTrace();
              }
         });
         platformThread.start();
    }
    long platformEndTime = System.nanoTime();
    System.out.println("Platform Threads created and started in: " + (platformEndTime - platformStartTime) / 1_000_000 + " ms");


    System.out.println("\nStarting Virtual Thread Example...");
    long virtualStartTime = System.nanoTime();
    for (int i = 0; i < numThreads; i++) {
        Thread.ofVirtual().start(() -> {
           try {
               Thread.sleep(1000);
           } catch (InterruptedException e) {
               e.printStackTrace();
           }
       });
    }
    long virtualEndTime = System.nanoTime();
    System.out.println("Virtual Threads created and started in: " + (virtualEndTime - virtualStartTime) / 1_000_000 + " ms");


   }
}

Starting Platform Thread Example...
Platform Threads created and started in: 42 ms

Starting Virtual Thread Example...
Virtual Threads created and started in: 15 ms

Exception in thread "main" java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
at java.base/java.lang.Thread.start0(Native Method)
at java.base/java.lang.Thread.start(Thread.java:1526)
at com.trestleiq.portal.ThreadDemo.main(ThreadDemo.java:17)