Thinking about how a distributed computing framework could be written in

Like... Execution units which are compiled to where data transfer is done with serde, tokio within the network and the network is a DAG.

Executions should be able to run with worker=thread, worker=container or even worker-host

Show thread

@fin_ger I guess I rather thought about stream processing, but there's which looks like a nice thing.

@musicmatze I've done this in the past with HTTP/s-based RPC, and a work stealer on the daemon which keeps tracks of which WUs have been sent to what client(s). Each WU containing the command(s) to execute on the client, and a deadline for completion. Then each client having its own work-stealer which distributes WUs across threads.

Sign in to participate in the conversation
Mastodon for Tech Folks

The social network of the future: No ads, no corporate surveillance, ethical design, and decentralization! Own your data with Mastodon!