Hiring Apache Pulsar Engineers: The Complete Guide

What Pulsar Engineers Build

Apache Pulsar is a distributed messaging and streaming platform. Before requiring "Pulsar experience," understand what it offers and when it makes sense.

Why Companies Choose Pulsar

Splunk uses Pulsar for their cloud data platform, processing petabytes of machine data with built-in multi-tenancy for customer isolation.

Verizon leverages Pulsar's geo-replication for their global messaging infrastructure, reducing latency across regions.

Tencent runs one of the largest Pulsar deployments, handling billions of messages daily for gaming and social platforms.

Pulsar vs Kafka: A Realistic Comparison

Both are excellent choices for different scenarios. Understanding when to use each helps you evaluate candidates.

Bottom line: Kafka has a much larger talent pool. Choose Pulsar only if its specific features (multi-tenancy, geo-replication, tiered storage) meaningfully benefit your use case.

Skills That Transfer from Kafka

Kafka engineers can learn Pulsar quickly. These concepts are nearly identical:

Direct Transfers

• Topics and partitions: Same concept, similar configuration
• Producers and consumers: Near-identical patterns
• Consumer groups: Similar coordination (subscriptions in Pulsar)
• Offset management: Cursor management in Pulsar
• Exactly-once semantics: Available in both

Pulsar-Specific Concepts (Learnable)

• Subscriptions (more flexible than consumer groups)
• Pulsar Functions (lightweight stream processing)
• Tiered storage (automatic hot/cold data management)
• BookKeeper (storage layer details)

A strong Kafka engineer becomes productive with Pulsar in 2-3 weeks.

What to Look For in Interviews

Essential Skills (Evaluate These)

Distributed Systems Understanding:

• Partitioning strategies and load balancing
• Replication and consistency trade-offs
• Failure handling and recovery patterns
• Back-pressure and flow control

Messaging Concepts:

• At-least-once vs exactly-once delivery
• Message ordering guarantees
• Consumer group coordination
• Dead letter queues and retry patterns

Operational Experience:

• Monitoring and alerting for message lag
• Capacity planning and scaling
• Performance tuning and optimization
• Incident response for messaging issues

Interview Questions

"Your message processing latency is spiking. Walk me through debugging it."

Good answer signals:

• Checks consumer lag first
• Considers partition imbalance
• Looks at producer throughput
• Examines consumer processing time
• Investigates network and I/O

"How do you design message schemas for forward/backward compatibility?"

Good answer signals:

• Understands schema evolution challenges
• Mentions schema registry or similar
• Considers consumer versioning
• Plans for breaking changes

"A consumer falls behind and can't catch up. What do you do?"

Good answer signals:

• Analyzes why it's falling behind
• Considers horizontal scaling
• Evaluates batch processing changes
• Knows when to reset offsets (carefully)

Common Hiring Mistakes

Mistake 1: Requiring Pulsar-Specific Experience

Why it's a mistake: Pulsar's talent pool is much smaller than Kafka's. You'll eliminate excellent messaging engineers who can learn Pulsar quickly.

Better approach: Require "distributed messaging experience (Kafka, Pulsar, or similar)" and accept Kafka expertise.

Mistake 2: Over-Testing Pulsar Configuration

Why it's a mistake: Asking "How do you configure BookKeeper replication?" tests documentation reading.

Better approach: Ask about messaging patterns, failure handling, and scaling strategies that apply to any system.

Mistake 3: Choosing Pulsar Without Clear Need

Why it's a mistake: Pulsar's smaller ecosystem means fewer integrations, less tooling, and harder hiring.

Better approach: Choose Pulsar only if multi-tenancy, geo-replication, or tiered storage are genuine requirements.

Building Pulsar Expertise

The Messaging Mindset

Distributed messaging requires understanding patterns that apply across all platforms:

Message Delivery Semantics:
Understanding at-least-once, at-most-once, and exactly-once delivery is fundamental. Engineers should know the trade-offs and when each is appropriate.

Partitioning Strategies:
How you partition data affects throughput, ordering, and consumer parallelism. Good engineers think about partition keys and their implications.

Consumer Coordination:
Managing multiple consumers reading from the same topic requires understanding offset management, rebalancing, and failure handling.

Back-Pressure Handling:
When producers outpace consumers, systems need strategies for handling the pressure. This includes buffering, rate limiting, and consumer scaling.

Developing Messaging Skills

From Kafka to Pulsar:
Kafka engineers transition to Pulsar smoothly:

• Topics and partitions work similarly
• Producer/consumer patterns are nearly identical
• Consumer groups map to subscriptions
• Offset management maps to cursor management

Pulsar-Specific Learning:
After mastering fundamentals, Pulsar-specific features include:

• Subscription types (exclusive, shared, failover, key-shared)
• Pulsar Functions for lightweight processing
• Tiered storage configuration
• BookKeeper internals for troubleshooting

Pulsar Team Composition

Who You Need

Messaging Platform Engineers:
Engineers who understand distributed messaging deeply. They may have Kafka, RabbitMQ, or other messaging experience—the concepts transfer.

Operations Capability:
Pulsar requires operating ZooKeeper and BookKeeper alongside brokers. Someone needs to understand the full stack for troubleshooting.

Application Integration:
Engineers who can help application teams use messaging effectively. This includes schema design, consumer patterns, and error handling.

Hiring Approach

Accept Kafka Experience:
Kafka engineers have all the fundamental skills. Pulsar-specific knowledge can be learned. Don't artificially limit your candidate pool.

Evaluate Distributed Systems Thinking:
Ask about partitioning, replication, failure handling, and scaling. These skills matter more than specific tool experience.

Consider the Ecosystem:
Pulsar's smaller ecosystem means fewer integrations and community resources. Ensure candidates are comfortable working with less mature tooling.