Follow

Does someone in here have extreme skill with async , especially with streams?

I want to build a type that can be used to iterate over a DAG with a stream. Fetching a node is async. Each node can have 0..n parents. That is a nontrivial problem in async Rust!

@musicmatze "[...]extreme skill with async Rust[...]", hm...I don't even have "skill with async Rust", sorry.😧 😅

But this sounds very interesting, so I'd really appreciate it, if you could keep me updated on this.

It seems you are trying to combine two of the hardest problems in Rust (IMO): cyclic data structures and async 😱

@janriemer
Not cyclic! A DAG has no cycles. But yes, the problem is hard, especially because my future is not Send and I don't know how i can make it Send...

@musicmatze Oh, stupid me!🤦
The "A" in DAG stands for "acyclic" of course.
Thanks for the correction.

Although from 2019, this might be helpful:

"In other words, a future is safe to send across threads if all of the types that are held across .await points implement Send." -
from the Rust blog:
blog.rust-lang.org/inside-rust

I hope this helps in some way!?

@musicmatze Are you using async traits from third-party crates? Maybe they have opted out of `Send` (with this: `#[async_trait::async_trait(?Send)]`)?

See also this excellent article by fasterthanlime:
fasterthanli.me/articles/getti

@musicmatze Can you iterate over the DAG synchronously, and pipe it into an async channel?

Also, try lib.rs/async-stream

@kornel no I can not, because the fetching of the node is async itself ...

The problem is even a bit harder because if I get a Node that has more than one parent, I need to fetch both parents and return them one by one from the stream, having two streams essentially now 😄

@janriemer

Sign in to participate in the conversation
Mastodon for Tech Folks

This Mastodon instance is for people interested in technology. Discussions aren't limited to technology, because tech folks shouldn't be limited to technology either!