Cookbook

File upload

The core reads the file through Fs, sends it in parts through Http and shows progress as a signal. The offline queue does the retrying, so no part of the recipe asks whether the network is up.

The recipe#

• A part is a mutation. put_part is idempotent, so while the device is offline it waits in the persisted queue and is replayed when the network returns, with the same Idempotency-Key. The upload just awaits it.
• Progress is a patch. uploads is a keyed list; every acknowledged part is a one-row update of sent and total, which is a progress bar.
• One part in the queue at a time. The queue stores a part's bytes, so memory and disk cost one part, not the file.
• Retry resumes. A part the server refuses fails the upload with the reason in its row. retry reads the file again and continues from the first part the server has not acknowledged.
• The server's name for the upload comes from the Rng port: random in an app, seeded in a test.

/// Sends part `index` of `total`: `PUT /uploads/{upload}/parts/{index}`. Idempotent, so while the
/// device is offline it waits in the persisted queue and is replayed when the network returns.
#[undra::mutation(key = "upload:{upload}", idempotent)]
pub async fn put_part(
    ctx: &Ctx,
    upload: String,
    index: u32,
    data: Bytes,
) -> Result<u32, UploadError> {
    let url = net::url(ctx, &format!("/uploads/{upload}/parts/{index}"))?;
    let mut request = HttpRequest::new(HttpMethod::Put, url).with_body(data.0.clone());
    if let Some(key) = undra::query::idempotency_key() {
        request = request.with_header("Idempotency-Key", key.to_string());
    }
    net::ok(net::send(ctx, request).await?)?;
    Ok(u32::try_from(data.0.len()).unwrap_or(u32::MAX))
}

async fn send_parts(&self, ctx: &Ctx, id: u32, data: &[u8]) -> Result<(), UploadError> {
    let row = self.row(id).ok_or(UploadError::Unknown)?;
    let first = row.sent as usize / PART;
    for (index, part) in data.chunks(PART).enumerate().skip(first) {
        // Waits here while the device is offline, and carries on when the queue replays it.
        let sent = ctx
            .mutate::<PutPartMutation>((
                row.remote.clone(),
                u32::try_from(index).unwrap_or(u32::MAX),
                Bytes(part.to_vec()),
            ))
            .await?;
        // One row, one `Update` patch: this is the progress the screen draws.
        self.set_state(id, |row| row.sent = (index * PART) as u32 + sent);
    }
    let parts = u32::try_from(data.chunks(PART).count()).unwrap_or(u32::MAX);
    ctx.mutate::<CompleteUploadMutation>((row.remote, parts))
        .await?;
    Ok(())
}

On the platform#

A row per upload; progress and a retry button are plain reads and one call.

struct UploadsView: View {
    let uploads: Uploads

    var body: some View {
        ForEach(uploads.uploads, id: \.id) { row in
            ProgressView(value: Double(row.sent), total: Double(max(row.total, 1)))
            if case .failed(let reason) = row.state {
                Button("Retry (\(reason))") { Task { try? await uploads.retry(id: row.id) } }
            }
        }
    }
}

class UploadsScreen(private val uploads: Uploads) {
    val rows: StateFlow<List<Upload>> = uploads.uploads

    fun progress(row: Upload): Float = row.sent.toFloat() / maxOf(row.total, 1u).toFloat()

    suspend fun retry(row: Upload) = uploads.retry(row.id)
}

export function UploadsView({ uploads }: { uploads: Uploads }) {
  const rows = useSignal(uploads.uploads);
  return rows.map((row) => (
    <progress key={row.id} value={row.sent} max={Math.max(row.total, 1)} />
  ));
}

What the tests prove#

• A 150,000-byte file goes up as three parts and the screen sees 0, 64 KiB, 128 KiB, then all of it.
• Offline, the upload waits with one part in the queue; when the network returns it finishes, and the queue is empty.
• A refused second part fails the upload at 64 KiB; retry sends parts 1 and 2 and never part 0.

Limits#

• The Http port has no streaming body and Fs reads a file whole, so the recipe suits photos and documents, not a multi-hundred-megabyte video.
• The queue replays while the app runs, and in the windows the OS gives a background run (BGTaskScheduler, WorkManager). Undra does not hand the transfer itself to the OS, so a part is sent only while the app or one of those windows is alive.
• Progress is per acknowledged part, not per byte on the wire.

Source: examples/cookbook/core/src/upload.rs.

Edit this page on GitHub