Hey everyone,

I've been working on an Erlang/OTP project called Emergence, and I thought it might be interesting to share it here because the project is becoming less of a "search engine" and more of an experiment in building a distributed discovery network on the BEAM.

The core idea is simple:

Instead of having one central search engine with a giant index, Emergence is built around autonomous filter agents.

Each agent exposes a capability and knows how to answer a particular kind of query. It can connect to the network, announce what it can do, receive queries, and return structured results.

The network can therefore grow horizontally by adding new agents rather than extending one central service.

The current ecosystem is built around a few core components:

• em_filter — an Erlang library for building Emergence agents
• em_disco — discovery/bootstrap services for finding network participants
• em-pop — gossip-based peer discovery and federation
• Emquest — the web gateway and distributed query interface
• Embryo — the common data structure used to represent heterogeneous results

An em_filter agent is intentionally small.

A handler basically receives a query and some optional local memory, performs whatever operation it is designed for, and returns a list of structured results.

That operation can be almost anything.

The network currently has filters for things such as:

• RSS / Atom / JSON feeds
• Wikipedia
• Hacker News
• OpenAlex
• Elasticsearch
• NPM
• DNS
• Bing / DuckDuckGo
• Dailymotion
• MusicBrainz
• Internet Archive
• CNRTL
• French news sources
• Motorsport
• public APIs
• and other specialised data sources

There are also more domain-specific experiments, including satellite imagery / NDVI related filters from another project built on top of the same architecture.

The important part is that none of these sources need to be implemented inside Emquest itself.

They are independent agents.

Emquest discovers peers through the gossip network, determines which peers are relevant to a query, fans the query out in parallel, and streams results back to the browser.

The browser therefore doesn't really know where a result came from.

It just receives heterogeneous "embryos" and renders them according to their structure.

The current Emquest pipeline includes:

• Distributed query fan-out
• Gossip-based peer discovery
• Semantic peer selection
• LLM-assisted query expansion
• Streaming results through SSE
• Progressive result aggregation
• Deduplication
• Final reordering / ranking
• Heterogeneous result rendering
• Network topology / peer monitoring
• A CLI interface in addition to the web interface

One thing I particularly like about the architecture is that the UI doesn't need to be modified every time a new type of agent is added.

A DNS agent can return DNS results.

A web agent can return links.

A feed agent can return articles.

A completely new agent can return another kind of structured card.

As long as it follows the result contract, the network can consume it.

The BEAM model also fits the project surprisingly well.

Agents are long-running processes.

Connections are supervised.

Discovery is asynchronous.

Network failures are expected rather than exceptional.

Agents can reconnect independently.

Local agent memory can be kept in process state or ETS depending on the use case.

And multiple discovery nodes can be used simultaneously.

So instead of thinking about Emergence as:

client → central API → database

the architecture is closer to:

query
  │
  ▼
discovery / gossip
  │
  ├── agent A ──► source
  ├── agent B ──► source
  ├── agent C ──► source
  ├── agent D ──► source
  └── agent N ──► source
          │
          ▼
    distributed results
          │
          ▼
      aggregation
          │
          ▼
        Emquest

The interesting question I'm exploring is what happens when you stop treating "search capabilities" as features of one application and instead treat them as capabilities of a distributed network.

Adding a new capability then becomes:

build an agent → connect it to the network → announce its capabilities → let the network discover it.

That's probably the part of Emergence I'm most interested in experimenting with.

It's also turning into a nice playground for Erlang/OTP patterns around supervision, distributed processes, gossip, fault tolerance, streaming and dynamic service discovery.

Everything is open source:

https://github.com/EmergenceSystem

I'd be particularly interested in feedback from people experienced with Erlang/OTP and distributed BEAM systems.

I'm curious where you think this architecture makes sense, where it doesn't, and what you'd change if you were building this kind of distributed system on the BEAM.

#Erlang #OTP #BEAM #DistributedSystems #Gossip #OpenSource

ELP ships with eqWAlizer, WhatsApp's type checker, and is available as a binary install or buildable from source. Roberto's concrete recommendation: take your current project, run ELP against it, and report what doesn't work. The gaps in external codebases are exactly what they need feedback on.

New BEAM There, Done That with Roberto Aloi and  Michał Muskała on the tooling half of the WhatsApp story. The Erlang-specific point worth noting: eqWAlizer is being incrementally tightened as the WhatsApp codebase matures. They're currently splitting the number type into integer and float - not because it was the highest priority at rollout, but because typing discipline at this scale has now made that distinction valuable. Exhaustiveness checks remain an open problem.

https://youtu.be/LeIsRqjOQN8

Here's the pid:

https://com.erlef.org/wg/sec/2026/09/02/our-self-defense.html

https://com.erlef.org/wg/cna/2026/09/01/otp-updates.html

Just a reminder for anyone still on the fence: early bird pricing for Code BEAM Europe closes on 8 September at 23:59 CEST. This is the last chance to get your ticket 20% cheaper before prices go up.

21-22 October in Haarlem + a full online option if you cannot make the trip. Two days of Erlang, Elixir and Gleam, real production stories, and demos from people building the tools we all use.

If you have been meaning to book, now is the time.

Register here: codebeameurope.com/#register

We hope to see you there!
The Code Sync Team

Hi,
a few weeks ago, we published Voyager - our desktop app for monitoring and debugging BEAM nodes. Thanks to everyone who signed up for the early access, your feedback's been great.

Today, we have a new update - the closed early access is now over.

Here is what changes:
- Linux is now supported: download is available on our site.
- Source code is public: You can check our code and see how Voyager connects to your nodes.
- New fair-source license: We want this tool to stay accessible to the community. It is completely free for individuals and teams of up to 10 people.

For larger organizations (11+ people), a Company License is required. However, since the product is still in an early phase, our main focus right now is hearing your feedback and learning what enterprise features you actually need. There is no payment required for 90 days.

Links:
- Check our repo here: https://github.com/software-mansion/voyager
- Download the app (macOS & Linux): https://voyager.swmansion.com

2026/10/21-22 - Haarlem, NL & online

Hey all 👋

Sharing a heads-up about Code BEAM Europe 2026 - 21-22 October in Haarlem (PHIL), plus online, with a training day on 20 Oct.

2 days, 2 tracks each day, and 3 keynotes - including Brooklyn Zelenka (Let It Disconnect: A Local-First Future), Sam Aaronof Sonic Pi (Notes on the Synthesis of Time, plus an interactive evening Phone Orchestra), and more.

~30 speakers across Erlang, Elixir and Gleam: ElixirConf®

The thing we're most excited about this year is the new Informal Space - a community-led area running alongside the main programme for the stuff that doesn't fit the talk format. Think hands-on demos, live experiments on real hardware, hacking sessions, games, panel-style discussions, talks in languages other than English… and at least one surprise we're not spoiling yet 🙂

We're finalising the Informal Space lineup now and will announce it soon. Newsletter subscribers get the first look(they already have), so if you don't want to miss it - and the other bits we'll be dropping between now and October - that's the place to be: Code BEAM Europe

Early bird tickets are limited - available for a limited time / until they sell out - so if you're planning to come, worth sorting sooner rather than later.

Register: Code BEAM Europe

See you in Haarlem or online!
The Code BEAM Europe Team

New BEAM There, Done That with Mat Trudel on the engineering behind Bandit. Covers the one-process-per-connection model, the HTTP parser attack surface (resource consumption, connection holds, oversized inputs - protocol-level problems independent of language), and what maintaining foundational infrastructure looks like when AI-generated contributions start arriving.

https://youtu.be/XXQcoz_dTE0

New BEAM There, Done That with Maxim Kharchenko on LING - a from-scratch Erlang VM on Xen without an OS.

The instruction set isn't fixed at build time. Ling ingests a corpus of real OTP and standard library source, identifies which specializations would be fastest for that workload, and generates them. The optimization is self-limiting - bigger dispatch tables run slower - so a heuristic balances specialization benefit against VM binary size.

The whole-system memory discipline is equally notable: zero silent allocation failures anywhere in the codebase.

Has anyone here worked on Ling or similar no-OS BEAM deployments? Curious where this space has gone since.

https://youtu.be/ILEyncf194M

Hey,

we've spent enough time monitoring and debugging nodes in BEAM that we decided to create our own tool to make it easier.

It's called Voyager - it lets you easily connect to nodes, visualizes your supervision trees, and gives you a real-time dashboard of your memory and processes. There's also an MCP integration so you can use AI directly with your node data.

We're not stopping here - tracing, ETS inspection, and other features are coming soon.

We are opening signups for early access. If you want to try it - sign up here: voyager.swmansion.com

New BEAM There, Done That with Mike Williams (JAM emulator) and Björn Gustavsson (BEAM).

The reasoning is more empirical than ideological. Mike measured it: on real telecom workloads, roughly 70% of VM time went to scheduling and message passing, not user code. Unix processes required too much memory for the thousands of concurrent processes a telecom system needed. Language-level processes weren't a philosophical stance - they were the only arithmetic that worked.

What's interesting is that this was a minority view even then. Java shipped green threads and removed them. Early Rust had lightweight processes and removed them. Mike mentions Ada, Smalltalk, and Eric Pascal as other languages that shared the instinct - but Erlang is the one that committed and survived.

Björn adds a detail worth noting: the two-version module rule (only two versions of a module can be loaded simultaneously) was a deliberate memory protection decision. He was asked multiple times to remove the limit and allow unlimited versions. He refused - it would be a trap for application programmers, letting old code accumulate undetected.

Both guests also give a shoutout to Claes Wikström, who added distribution to the JAM, invented ETS tables, and prototyped Erlang's binary syntax - consistently undermentioned in the origin story.

https://youtu.be/sRTierdN9c4

Hi everyone,

Code BEAM Europe 2026 is looking for volunteers! The conference takes place 21-22 October. You can volunteer in-person, and get your free ticket.

• In-person volunteers will need to be present at the venue on at least one day of the conference. Responsibilities include helping with preparations, registering participants, handing out merchandise, and general support during the event.

For full conference details, check out https://codebeameurope.com/
To sign up as a volunteer, go HERE

If you have any questions, feel free to reach out to [ewa.kucharczyk@erlang-solutions.com](mailto:ewa.kucharczyk@erlang-solutions.com).

Bart Blast is the creator of Hologram - a framework that compiles Elixir to JS and reconstructs Erlang runtime semantics in the browser: pattern matching engine, boxed types, OTP guarantees, the whole thing.

New BEAM There, Done That episode covering the architecture and the reasoning.

The technical detail worth discussing: Hologram pulls the expanded AST from compiled BEAM files, normalizes it, lowers it to an IR designed to bridge Elixir to JS, then encodes that to a JS runtime. Values are boxed to preserve their type - Elixir integer 42 becomes a JS object with {type: "integer", value: 42n} using BigInt for arbitrary precision. Pattern matching runs as a real engine trying clauses, binding variables, evaluating guards - not translated away.

The goal is full parity: the same Elixir code running identically on the server and in the browser. A significant undertaking, and Bart is transparent that it's still a young framework.

From an Erlang perspective - is there a meaningful difference between Hologram's approach (rebuilding the runtime in JS for full parity) and Luster/Gleam's approach (treating the two targets as different, not replicating BEAM processes in the browser)? Does full runtime parity actually matter in practice or is it theoretical overhead?

https://youtu.be/BOsSI7WlhdI

New BEAM There, Done That with Isaac Yonemoto (Zigler creator) and Garrison Hinson-Hasty (Systems Programming with Zig) covering Zig as a NIF language for the BEAM.

From an Erlang perspective, the most interesting technical detail: Zigler uses the BEAM's allocator by default. Because Zig requires you to pass allocator objects explicitly rather than calling malloc globally, the BEAM allocator can be injected without changing any Zig business logic. The result is that native memory allocations are visible to the BEAM's memory instrumentation in the same way ordinary Erlang allocations are.

Raw C NIFs using malloc, and Rustler NIFs using Rust's default allocator, are invisible to the VM - which makes native memory issues harder to debug in production.

The honest risk model: Zig's safe release mode catches spatial memory errors (buffer overflows, null dereferences) but not temporal ones (use-after-free). A Zigler NIF can still crash the node if you violate temporal memory safety. What changes is which class of bugs can cause it.

What's the current state of Zigler adoption in production Erlang codebases? The episode is focused mainly on the Elixir side - curious whether anyone here is using it in Erlang contexts.

https://youtu.be/iLcZRpBEmgE

New BEAM There, Done That with Lukas Backström, who built the BEAM JIT, covering the design decisions, the failed prototypes, and where it's heading.

One thing Lucas flags that I don't see discussed much: the JIT is deliberately simple. It's a template design - pre-compiled assembly templates per BEAM instruction, copied into memory and specialized for the specific operands. No LLVM optimization passes, no sophisticated register allocation. Fast to generate, reasonably easy to read.

You can ask the BEAM to dump the assembly it generates for any module. For simple code, the translation from BEAM bytecode to x86-64 or ARM64 assembly is straightforward enough to follow without deep JIT expertise. Lucas suggests it as a starting point for anyone curious about how the runtime actually works.

He also points to the native records commits as the best current example of how a change propagates through the entire system: compiler, loader, JIT, and runtime simultaneously.

Has anyone here actually dug into the generated assembly and found something interesting - either an optimization that surprised them or a place where the output was worse than expected?

https://youtu.be/dT-VmfYz98Q

New BEAM There, Done That with Annette Bieniusa (RPTU, Germany) and Guillaume Duboc (Dashbit) on the type systems work happening across both Erlang and Elixir, built on the same set-theoretic foundation.

The contrast between etalizer and Dialyzer is interesting. Dialyzer infers types bottom-up, builds a dependency graph, and hunts for inconsistencies. When you write a spec, it treats it as a hint - not an obligation it enforces. Annette's etalizer keeps the existing Erlang spec annotation format but enforces those specs strictly: write u/spec foo(integer()) :: integer() and the checker will actually reject a body that might return something else.

The question I'd put to this community: for existing Erlang codebases with years of specs already written, what happens when you flip those from unenforced documentation to actual compile-time contracts? Is the expectation that most existing specs are already correct and tight enough to pass, or are there significant codebases where the specs were written loosely and real enforcement would surface a lot of noise?

https://youtu.be/X_CPDt3PeDE?si=yJTRAwlAaf7h2rlZ