Symfony Messenger serializes a message, hands the bytes to a transport and forgets about them. Two things routinely go wrong with those bytes:
messenger_messages, in a Redis RDB dump, on a RabbitMQ disk and in the dead-letter queue nobody has looked at since March.mattleads/messenger-compress-bundle fixes both at the transport-serializer layer. You opt in per message with a stamp or enforce a policy per transport, and your handlers receive the same plain PHP objects they always did. We'll walk through the whole thing on a small Doctrine (SQLite) transport, because then the queue is a table you can SELECT from and see what changed.
ext-zlib and ext-sodium ship with PHP, so there is nothing to install.symfony/messenger. The outputs below come from Symfony 8.1.mattleads/messenger-compress-bundle v0.1.1.composer require mattleads/messenger-compress-bundle
Symfony Flex registers the bundle. Without Flex, add it yourself:
// config/bundles.php
return [
// ...
MattLeads\MessengerCompressBundle\MattLeadsMessengerCompressBundle::class => ['all' => true],
];
Nothing happens yet. The bundle registers a serializer service, mattleads_messenger_compress.serializer, that wraps Messenger's default serializer. Until a transport uses it, your messages are untouched.
You have two options. Start with one transport:
# config/packages/messenger.yaml
framework:
messenger:
failure_transport: failed
transports:
async:
dsn: '%env(MESSENGER_TRANSPORT_DSN)%'
serializer: mattleads_messenger_compress.serializer
failed: 'doctrine://default?queue_name=failed'
routing:
'App\Message\GenerateInvoiceBatch': async
'App\Message\ExportCustomerData': async
Or wrap the default serializer so every transport gets it:
# config/packages/mattleads_messenger_compress.yaml
mattleads_messenger_compress:
decorate_default_serializer: true
One tip for the walkthrough only: Messenger's default serializer is PHP serialize(). If you switch the default to the JSON one, plain bodies become human-readable in the database, which makes the "before" picture much clearer:
framework:
messenger:
serializer:
default_serializer: messenger.transport.symfony_serializer
The bundle doesn't care which one you wrap. It compresses and encrypts bytes.
Here is a message you would not want sitting in a queue table:
final readonly class ExportCustomerData
{
public function __construct(
public string $customerId,
public string $fullName,
public string $email,
public string $iban,
) {}
}
Dispatch it as usual and look at the row. In the demo, app:queue:inspect --grep <string> does this. By hand it's just SQL:
SELECT id, length(body), substr(body, 1, 60), headers FROM messenger_messages;
id message X-Compression X-Encryption key encoding body bytes body preview
1 ExportCustomerData - - - - 119 {"customerId":"cust-4711","fullName":"Nino Beri…
[ERROR] "GE29NB0000000101904917" is readable in 1 stored message(s).
The IBAN is right there. Every backup of that database has it too.
bin/console messenger-compress:generate-key -v
k3Jm…(44 characters of base64)…=
Add to .env.local (never commit it):
MESSENGER_ENCRYPTION_KEY_V1=k3Jm…
and to config/packages/mattleads_messenger_compress.yaml:
encryption: { enabled: true, keys: { v1: '%env(MESSENGER_ENCRYPTION_KEY_V1)%' } }
That's 32 random bytes, base64-encoded. Put it in .env.local for development and in your secrets store for production. Symfony's own vault works fine:
bin/console secrets:set MESSENGER_ENCRYPTION_KEY_V1
# config/packages/mattleads_messenger_compress.yaml
mattleads_messenger_compress:
encryption:
enabled: true
keys:
v1: '%env(MESSENGER_ENCRYPTION_KEY_V1)%'
use MattLeads\MessengerCompressBundle\Stamp\EncryptStamp;
$bus->dispatch(new ExportCustomerData(...), [new EncryptStamp()]);
Same row, after:
2 ExportCustomerData - xchacha20poly1305 v1 base64 212 jUa0PNnAtVBUT5+uTPOOxsbUrOTsIg9cYKnD1WAKsnRQEyN…
The body is XChaCha20-Poly1305 ciphertext (libsodium, fresh 192-bit nonce per message, authenticated). The headers are still readable: the message type, the scheme, the key id. That's deliberate: the consumer needs them to know what to do and Messenger needs the type to route the message. Which brings up the one rule worth memorizing, call it the header rule: stamps travel as headers and are not encrypted. Put secrets in the message body, never in a stamp.
Notice the size went up: 119 bytes became 212. A 24-byte nonce, a 16-byte tag and base64's 33 % overhead cost more than a tiny body saves. That's fine. Encryption is a decision about sensitivity, not about size.
If a transport only ever carries sensitive work, stop thinking about stamps:
mattleads_messenger_compress:
encryption:
enabled: true
always: true
keys: { v1: '%env(MESSENGER_ENCRYPTION_KEY_V1)%' }
A single message can opt out with PlainStamp. An explicit EncryptStamp always wins over policy.
Now the other problem. GenerateInvoiceBatch with 5,000 order lines serializes to 759 KB of JSON. Three ways to shrink it:
Explicitly, per message:
use MattLeads\MessengerCompressBundle\Stamp\CompressStamp;
$bus->dispatch($batch, [new CompressStamp()]); // configured default: deflate
$bus->dispatch($batch, [new CompressStamp('gzip')]);
$bus->dispatch($batch, [new CompressStamp('zstd', 3)]); // algorithm + level; zstd needs ext-zstd
Automatically above a size. This is the default to pick, because nobody remembers to add a stamp to the one message that grows:
mattleads_messenger_compress:
compression:
default_algorithm: deflate # deflate | gzip | zstd
level: null # null = the algorithm's default
auto_threshold: 4096 # any body >= 4 KiB is compressed, no stamp needed
Always:
mattleads_messenger_compress:
compression:
always: true
Two details of the automatic policies: they only keep the compressed body when it is actually smaller than the original (random blobs and already-compressed data pass through untouched) and PlainStamp opts a message out. An explicit CompressStamp is always honored even if the result is larger.
What it looks like with auto_threshold: 4096:
|
Message |
Stamps |
Raw |
On the wire |
Ratio |
|---|---|---|---|---|
|
50 invoice lines |
none (auto) |
7.6 KB |
1,416 B |
18.6 % |
|
5,000 invoice lines |
none (auto) |
758,571 B |
98,060 B |
12.9 % |
|
5,000 invoice lines |
|
758,571 B |
98,136 B |
12.9 % |
|
Customer export |
|
119 B |
212 B |
178 % |
The third row is the common production combination: compress and encrypt. The bundle always compresses first and encrypts second regardless of the order of your stamps, because ciphertext does not compress. The 76 bytes between rows two and three are the gzip header plus the nonce and tag.
One caveat from the CRIME/BREACH family: compressed-then-encrypted length can leak information about the plaintext. If a single message mixes attacker-controlled input with a secret and the length is observable to that attacker, do not compress it. For almost every queue job this isn't a concern, but it's worth knowing.
The bundle adds up to four headers to the encoded envelope. Everything else (the type header, the stamp headers) is whatever the inner serializer produced.
|
Header |
Value |
Meaning |
|---|---|---|
|
|
|
body was compressed with this algorithm |
|
|
|
body was encrypted with this scheme |
|
|
e.g. |
which key, so consumers can rotate |
|
|
|
body is base64 text |
Base64 is the default because the Doctrine, Redis and SQS transports all want text bodies. On AMQP you can drop it and save the 33 %:
mattleads_messenger_compress:
encoding: raw
Because everything is in headers and the body is self-describing, a consumer with the bundle installed decodes any mix of plain, compressed, encrypted and compressed-and-encrypted messages from the same queue. You can roll the bundle out to consumers first and start stamping messages later.
Nothing changes in the handler:
#[AsMessageHandler]
final readonly class ExportCustomerDataHandler
{
public function __invoke(ExportCustomerData $message): void
{
// $message is the plain object. Decryption happened in the transport serializer.
}
}
But the envelope your handler is wrapped in now carries a TransformedStamp describing what the serializer undid: compression, encryption, keyId, wireSize, rawSize and a ratio() helper. It implements NonSendableStampInterface, so it is never sent back out if the message is re-dispatched. A tiny listener turns it into logs or metrics:
use MattLeads\MessengerCompressBundle\Stamp\TransformedStamp;
use Symfony\Component\EventDispatcher\Attribute\AsEventListener;
use Symfony\Component\Messenger\Event\WorkerMessageReceivedEvent;
final readonly class TransformedStampLogger
{
public function __construct(private LoggerInterface $logger) {}
#[AsEventListener]
public function __invoke(WorkerMessageReceivedEvent $event): void
{
$stamp = $event->getEnvelope()->last(TransformedStamp::class);
if (null === $stamp) {
return; // message was neither compressed nor encrypted
}
$this->logger->info('compression={c} encryption={e} key={k} wire={w} raw={r} ratio={ratio}', [
'c' => $stamp->compression ?? '-',
'e' => $stamp->encryption ?? '-',
'k' => $stamp->keyId ?? '-',
'w' => $stamp->wireSize,
'r' => $stamp->rawSize,
'ratio' => sprintf('%.1f%%', 100 * $stamp->ratio()),
]);
}
}
$ bin/console messenger:consume async --limit=5
Received ExportCustomerData: compression=- encryption=xchacha20poly1305 key=v1 wire=212 B raw=119 B ratio=178.2%
Handled export for cust-4711 (Nino Beridze, iban ends 4917)
Received GenerateInvoiceBatch: compression=deflate encryption=- key=- wire=98,060 B raw=758,571 B ratio=12.9%
Handled batch-5000: 5000 lines, total 15,537,656.65 EUR
Push ratio() and wireSize into your metrics backend tagged by message class and you've got a dashboard of what each job actually costs your broker.
Keys leak, people leave, policies say "every 90 days". Rotation is the part most home-grown encryption layers get wrong, because there's always a message in flight encrypted with the old key.
The bundle's answer is a key ring. Every encrypted message carries its key id in X-Encryption-Key-Id; consumers decode with whichever key the message names and active_key says which key new messages use.
mattleads_messenger_compress:
encryption:
enabled: true
active_key: '%env(MESSENGER_ACTIVE_KEY)%'
keys:
v1: '%env(MESSENGER_ENCRYPTION_KEY_V1)%'
v2: '%env(MESSENGER_ENCRYPTION_KEY_V2)%'
The procedure, in order:
v2 (messenger-compress:generate-key --id v2 -v) and deploy it to every consumer alongside v1. Nothing changes yet; active_key is still v1.active_key to v2 on producers. New messages are v2; consumers still decode the v1 backlog.v1.You can watch it happen in the demo by running the producer with one key and the consumer with the other:
php bin/console app:dispatch --export --encrypt # active_key=v1
MESSENGER_ACTIVE_KEY=v2 php bin/console app:dispatch --export --encrypt
MESSENGER_ACTIVE_KEY=v2 php bin/console messenger:consume async --limit=2
Received ExportCustomerData: … encryption=xchacha20poly1305 key=v1 …
Received ExportCustomerData: … encryption=xchacha20poly1305 key=v2 …
If active_key is omitted, the last key in the map is used.
This is the part to understand before production. Decryption is authenticated, so a message encrypted with a key the consumer doesn't have, or modified in the broker, is refused rather than handed to your handler as garbage. Let's corrupt a stored body by hand:
php bin/console app:dispatch --export --encrypt
sqlite3 var/data.db "UPDATE messenger_messages SET body = substr(body,1,40) || 'X' || substr(body,42) WHERE delivered_at IS NULL;"
php bin/console messenger:consume async --limit=3
php bin/console messenger:failed:show
Error thrown while handling message MessageDecodingFailedException. Sending for retry #1 using 93 ms delay. Error: "Decryption failed: authentication tag mismatch (wrong key or tampered payload)."
Error thrown while handling message MessageDecodingFailedException. Sending for retry #2 using 110 ms delay. …
Error thrown while handling message MessageDecodingFailedException. Removing from transport after 2 retries. …
There is 1 message pending in the failure transport.
Id Class Failed at Error
4 Symfony\Component\Messenger\Exception\MessageDecodingFailedException 2026-11-01 15:12:53 Decryption failed: authentication tag mismatch (wrong key or tampered payload).
So: the worker doesn't crash, the handler never runs, the retry strategy you configured applies and the message is parked on the failure transport with the original bytes intact. That last point matters for the "wrong key" case. Suppose a consumer was deployed without v2 and some v2 messages failed. Add the key and replay:
php bin/console messenger:failed:retry --force
Handled export for cust-4711 (Nino Beridze, iban ends 4917)
The failed message is re-sent through the serializer untouched (it isn't compressed or encrypted a second time), decoded with the now-present key and handled normally.
A note on Symfony 8.1. Messenger 8.1 changed how a transport serializer must report a message it cannot decode: it has to return an envelope wrapping MessageDecodingFailedException instead of throwing, so the stamps that count retries survive. Bundle v0.1.0 threw and on 8.1 that meant "retry #1" forever with an empty failure transport. The scenario above is how I found it; v0.1.1 returns the wrapped failure. The official custom-serializer example in the Symfony docs now shows the same pattern (symfony-docs#23459) and the throwing path itself is tracked in symfony/symfony#67791, with a one-file reproducer.
The errors you can see and what they mean:
|
Message |
Cause |
|---|---|
|
|
The key with that id exists on the consumer but is different from the producer's or the body was modified. |
|
|
The consumer has not received the new key yet. Rotation step 1 was skipped. |
|
|
|
|
|
Producer side: you dispatched with |
|
|
|
Your handler tests do not change; they never saw the serializer. For an integration test that exercises the real round trip, the in-memory transport can run the serializer like a broker would:
# config/packages/test/messenger.yaml
framework:
messenger:
transports:
async:
dsn: 'in-memory://?serialize=true'
serializer: mattleads_messenger_compress.serializer
Dispatch with stamps, consume, assert on the TransformedStamp of the received envelope. This is how the bundle's own functional tests work.
secrets:set, Vault, SSM, whatever you already use.active_key.auto_threshold: 4096 catches the message that grows six months from now. Reserve explicit stamps for zstd/level tuning.sensitive transport with encryption.always: true is easier to audit than stamps scattered through services.ratio(). If a message class is consistently above 100 %, it's small and doesn't need compression; if it is consistently at 100 %, the data is already compressed (images, archives) and PlainStamp saves CPU.The bundle is on Packagist and GitHub (MIT, Symfony 6.4–8.x, PHP 8.2+). The runnable walkthrough with the inspect command, the listener and the tamper scenario is at github.com/mattleads/messenger-compress-demo.
Custom compressors (Brotli, LZ4) plug in by implementing CompressorInterface and are picked up by the mattleads_messenger_compress.compressor tag. Custom encryptors implement EncryptorInterface. If you run this against SQS, Redis or AMQP at volume, open an issue with the ratios you see and anything that surprised you. That's what the issue tracker is for.
Source Code: You can find the full implementation and follow the project's progress on GitHub: https://github.com/mattleads/messenger-compress-bundle
If you found this helpful or have questions about the implementation, I'd love to hear from you. Let's stay in touch and keep the conversation going across these platforms: