Skip to content

Four sessions from Code BEAM V Europe 2021

Code BEAM V Europe ran online from 19 to 21 May 2021. Notes on four sessions: the Phoenix LiveView fireside chat, Hygeia's contact tracing on the BEAM, a code design chat with Ulf Wiger and Eric Meadows-Jönsson, and the machine learning panel.

September 27, 2021
7 min read

Code BEAM V Europe ran online from 19 to 21 May 2021. These are my notes on four sessions, for anyone who works with Elixir or Erlang and missed them; each section ends with the recording.

The BEAM

BEAM is the virtual machine of the Erlang Open Telecom Platform (OTP for short). It’s part of the Erlang Run-Time System (ERTS for short), which compiles Erlang source code into bytecode. The virtual machine then executes these bytecode files (extension .beam).

Besides Erlang and Elixir, many other languages run on the BEAM. The Code BEAM conference series brings them all together. This year’s European edition ran online over three half days.

My highlights

Fireside chat: Phoenix LiveView

Sophie DeBenedetto and Bruce Tate (who were writing Programming Phoenix LiveView at the time) talked about why Phoenix LiveView works. They also talked about how to keep a LiveView from growing into the equivalent of a fat controller.

Bruce’s case for it is the layering: OTP, then request and response, then Channels, then LiveView on top. It lets you put your brain in one place, on the server. The infrastructure handles the back-and-forth of autocomplete or infinite scroll, so your head isn’t split across client and server.

Sophie’s pattern for structure was CRC: construct, reduce, convert. Plug already works this way (the conn is constructed, reduced by plugs and converted by render), and so does Ecto (you build a query, add predicates and execute it). In LiveView, mount constructs, each handle_event is a reducer that takes a socket and returns a socket, and render converts. Her example was a single-purpose reducer, assign_age_group_filter(socket), reused in a component’s update/2 and in an event handler.

Bruce added the split between a functional core and a boundary. The core is schemas and queries, composed with pipes. The boundary is a Phoenix context, handling uncertainty with tagged tuples and changesets. The LiveView is just another client of that boundary.

From the Q&A:

  • JavaScript hooks are the documented escape hatch, in both directions.
  • An audience member described a payment provider’s merchant portal and checkout form running on LiveView, with thousands of users a day.
  • The parts of Surface (a component library for LiveView) that belong in LiveView were being moved into LiveView.
  • For native mobile apps, a persistent connection clashes with bandwidth economics, so LiveView isn’t aimed at that.

COVID-19 contact tracing on the BEAM

Jonatan Männchen and Jeremy Zahner of JOSHMARTIN (a seven-person company in St. Gallen) presented Hygeia, the contact-tracing system they built for Swiss health authorities. The stack is Elixir, Ecto, Phoenix, LiveView and Surface. It ships as a mix release in Docker and is deployed with Helm on a dedicated Kubernetes cluster in Switzerland, because health data must not leave the country. It was their first project with LiveView.

The first release went live one month after the start. Two backend developers, one frontend developer and a project manager built it, and it replaced a Trello-based process within three days. At the time of the talk the system had handled more than 60,000 cases with more than 185 tracers. Tracing one case took about 15 minutes instead of about two hours.

Most of my notes are on the technical lessons:

  • An Elixir versioning library for the audit trail couldn’t see Repo.update_all or changes the database makes itself, such as ON DELETE SET NULL. They replaced it with Postgres triggers and pass the author in a session variable.
  • Grouping entities as JSON embeds meant no foreign keys, so dangling references and 404s. They were normalising again.
  • The tenants had no usable SMTP and no Swiss SaaS was allowed. So they send mail directly with gen_smtp and a home-grown spool in Postgres with row locking. SPF, reverse DNS and DKIM built up reputation quickly.
  • They started without an error tracker, and bugs were hard to reproduce. Adding Sentry turned up many they hadn’t known about.
  • Background jobs with external dependencies sat under the main supervisor, and repeated crashes took the whole application down. The fix was dedicated supervisors, tuned restart settings and handling the expected failures (such as skipping a sync and logging when there is no token).

Hygeia was published as open source. The team said they aimed to decouple it from COVID-19 so it could serve other purposes. The repository has since been archived.

Fireside chat: Code design

Ulf Wiger and Eric Meadows-Jönsson talked about code design. Ulf goes back to the Ericsson days of Erlang, Eric is on the Elixir core team.

For Ulf, design problems at Ericsson were social before they were technical. Erlang’s mental model matched how the telecom domain experts thought. So product managers and customers could follow a discussion about the implementation, while C++ and UML projects failed on that disconnect.

His story about accidental complexity: the AXD 301 (an Ericsson ATM switch) ran on 32 MHz microSPARCs with a VM limit of 32,000 processes, against a requirement of 30,000 calls. That forced process-conserving designs. When multicore arrived, the fastest code was textbook Erlang, and they had to undo the single-core tricks.

The mother of all anti-patterns, in his words, is trying to make it object-oriented. Model processes on the problem’s natural concurrency, and don’t weld modules to processes. He also gave the figure I remembered from the session: 15% of the effort is writing the code and 85% is maintaining it, usually by someone else. (He called it a saying rather than a measurement.)

Eric talked about the language and its tooling. Pipes list operations in execution order, where nested calls read backwards. Good tooling gets out of your way; with the formatter, you just don’t have to think about formatting anymore.

On metaprogramming, his one tip was not to use macros for code reuse. It’s easy to build a huge macro that injects the common code, but behaviours and higher-order functions were there before macros and do the job. His closing line: don’t try to be too clever, because very short code usually becomes more complex.

For metaprogramming itself, Adi Iyengar’s talk Metaprogramming + DSL Design in Elixir went into the details. His main advice was the same: use it when you have exhausted the other options, and keep the macro a thin interface over plain, testable functions.

Panel: Machine learning on the BEAM

José Valim, Dave Lucia and Svetlana Levitan, moderated by Garrett Smith, talked about the state of machine learning on the BEAM a few months after Nx was announced.

Garrett opened with a primer: data preparation is 80 or 90% of the work, then training, then inference in production. José walked through the ecosystem as it was:

  • Nx, announced in February 2021, with tensors compiled to run on CPU or GPU
  • Axon for neural networks on top of it, training first, with serialization of large models still to do
  • Livebook as the collaborative notebook
  • Scidata for standard datasets
  • a machine learning working group in the Erlang Ecosystem Foundation

The gaps he named were data preparation, model serialization and models that aren’t neural networks.

His goal was that a company on the BEAM can do machine learning without switching stacks. Dirty NIFs (native C functions that run on separate schedulers, so long computations don’t block the VM) had removed the last blocker for that. He was open about one weakness: Python frees GPU memory through reference counting the moment a tensor goes away, while the BEAM waits for garbage collection.

Dave Lucia’s experience from production was that tree models such as LightGBM and XGBoost see far more use than neural networks, because they iterate faster. He also said the ecosystem needs a DataFrame equivalent. Svetlana Levitan’s closing advice was to separate building models from deploying them through standards such as ONNX (an open file format for trained models). In her view, deployment may be the BEAM’s best bet.

—

Back to Blog

Next: OTP 26 changes SSL defaults and your Postgres connection might break

Phil-Bastian Berndt

Phil-Bastian Berndt
Tech Lead at Naymspace