this post was submitted on 01 Dec 2023
5 points (100.0% liked)

Rust Programming

211 readers
1 users here now

founded 5 years ago
MODERATORS
 

Ed: solved with the help of the async_stream crate.

I'm struggling with the borrow checker!

My problem: I'm using actix-web and rusqlite. I want to return an unlimited number of records from an rusqlite query, and actix provides a Stream trait for that kind of thing. You just impl the trait and return your records from a poll_next() fn.

On the rusqlite side, there's this query_map that returns an iterator of records from a query. All I have to do is smush these two features together.

So the plan is to put the iterator returned by query_map into a struct that impls Stream. Problem is the lifetime of a var used by query_map. How to make the var have the same lifetime as the iterator??

So here's the code:

pub struct ZkNoteStream<'a, T> {
  rec_iter: Box<dyn Iterator<Item = T> + 'a>,
}

// impl of Stream just calls next() on the iterator.  This compiles fine.
impl<'a> Stream for ZkNoteStream<'a, serde_json::Value> {
  type Item = serde_json::Value;

  fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<Self::Item>> {
    Poll::Ready(self.rec_iter.next())
  }
}

// init function to set up the ZkNoteStream.
impl<'a> ZkNoteStream<'a, Result<ZkListNote, rusqlite::Error>> {
  pub fn init(
    conn: &'a Connection,
    user: i64,
    search: &ZkNoteSearch,
  ) -> Result<Self, Box<dyn Error>> {
    let (sql, args) = build_sql(&conn, user, search.clone())?;

    let sysid = user_id(&conn, "system")?;
    let mut pstmt = conn.prepare(sql.as_str())?;

    // Here's the problem!  Borrowing pstmt.
    let rec_iter = pstmt.query_map(rusqlite::params_from_iter(args.iter()), move |row| {
      let id = row.get(0)?;
      let sysids = get_sysids(&conn, sysid, id)?;
      Ok(ZkListNote {
        id: id,
        title: row.get(1)?,
        is_file: {
          let wat: Option<i64> = row.get(2)?;
          wat.is_some()
        },
        user: row.get(3)?,
        createdate: row.get(4)?,
        changeddate: row.get(5)?,
        sysids: sysids,
      })
    })?;

    Ok(ZkNoteStream::<Result<ZkListNote, rusqlite::Error>> {
      rec_iter: Box::new(rec_iter),
    })
  }
}

And here's the error:

error[E0515]: cannot return value referencing local variable `pstmt`
   --> server-lib/src/search.rs:170:5
    |
153 |       let rec_iter = pstmt.query_map(rusqlite::params_from_iter(args.iter()), move |row| {
    |                      ----- `pstmt` is borrowed here
...
170 | /     Ok(ZkNoteStream::<Result<ZkListNote, rusqlite::Error>> {
171 | |       rec_iter: Box::new(rec_iter),
172 | |     })
    | |______^ returns a value referencing data owned by the current function

So basically it boils down to pstmt getting borrowed in the query_map call. It needs to have the same lifetime as the closure. How do I ensure that?

you are viewing a single comment's thread
view the rest of the comments
[–] pr06lefs@lemmy.ml 2 points 11 months ago* (last edited 11 months ago) (4 children)

I'm not quite ready to give up, but its not looking good. One of the rusqlite maintainers issued this haiku-like missive:

yeah you just have to collect
you can't return that as an iterator
it needs to borrow from the statement

Got to wondering how Vec does this interator-with-internal-state thing, and its with unsafe.

update from the maintainer on discord:

fundamentally you're asking for a self-referential type. e.g. one field borrows from another field of the same struct. cant  be done without  unsafe
very easy to have soundness holes even if you use unsafe
[–] hallettj 2 points 11 months ago (3 children)

Well if you want to try another avenue, I've read about implementing self-referential structs using Pin.

There's a discussion here, https://blog.cloudflare.com/pin-and-unpin-in-rust/

There's a deep dive on pins here, but I don't remember if it addresses self-referential types: https://fasterthanli.me/articles/pin-and-suffering

[–] pr06lefs@lemmy.ml 2 points 11 months ago* (last edited 11 months ago) (1 children)

Got this solved over on the rust-lang discourse. The solution was to use the async_stream crate. Ends up being a small amount of code too.

[–] hallettj 1 points 11 months ago
load more comments (1 replies)
load more comments (1 replies)