It builds on top of Nx, Numerical Elixir, which allows for models to be compiled for cpu or gpu. Done this way, the model does not have to run by executing BEAM bytecode. Additionally, Liveview and Livebook enables the equivalent of Jupyter notebooks. All of this has been happening for the past couple years.
The advantage here is data being fed into the model can use frameworks like Broadway (streaming ingestion) or Membrane (realtime media streaming), or Nerves (BEAM at edge / IoT), or as a controller for unreliable agents.
Digging into the source of its dependencies a bit, it looks like it uses NIFs (inlined functions) to bind into your choice of a couple of different potential C backends, the big ones being Torch and Google's XLA compiler.
So performance shouldn't be an issue at all, when compared to similar "high-level language calling low-level language for heavy tasks" situations, like Tensorflow or PyTorch. It does, however, come at the cost of sacrificing the extreme stability BEAM applications typically enjoy: misbehavior by the backend can cause hangs or even crashes of the BEAM node they're running in.
Yup. And it can be mitigated somewhat by isolating workloads that might crash BEAM, while using dist erl so nodes can still communicate with each other.
Rust is an exciting option for NIF, but those backends are not written in it.
I’m not expert here but I think most of that risk is mitigated these days since BEAM introduced Dirty Schedulers/Dirty NIFs for any NIFs that take over 1ms to execute… I think this isolates (or can isolate) their issues to only effecting other work on the same Dirty scheduler, aka other Dirty NIFs…
Dirty schedulers isolate the potential scheduling issues, but there's no memory isolation between NIFs and the rest of the BEAM state, so there's danger there.
Best practice for doing this with the BEAM (the Erlang VM) is to just run two VMs, one for your app code and a “dirty” one that is allowed to crash and be restarted without bringing down your app VM.
If you're considering running a second (or multiple) BEAM for your nifs, you should also consider running your native code as a c-node or a port program.
A compelling use case is where your website and numerical engine are running on different machines. In that circumstance, your main application is already isolated from issues on the numerical engine.
