OnOff: Bridging Online and Offline Handwriting via Differentiable Physical Rendering

Seonmi Park1, Seunghyun Shin1, Vihaan Misra2, Dongmin Shin3, Ukcheol Shin4*, Jean Oh2, Hae-Gon Jeon3*
1GIST   2CMU   3Yonsei University   4KENTECH
*Corresponding authors
ECCV 2026
arXiv Code Data
Teaser: unified online-offline handwriting framework

OnOff unifies online and offline handwriting generation via a differentiable physical brush renderer — enabling trajectory estimation, image rendering, image refinement, and direct robot execution within a single framework.

Framework Overview

Given style references Isty and target text T, our framework predicts brush parameters θ, generates a stroke sequence S, renders an image Irend, and refines it into the final offline image Ioff.

Overview of the proposed joint online-offline handwriting framework.

Fig. 1. Overview of the proposed joint online-offline handwriting framework.

Differentiable Brush Renderer

Six physically-interpretable parameters θ = { wbase, kspread, ρink, σsharp, pmin, pmax } control brush footprint geometry, ink transfer, and stroke composition — all differentiable.

Conceptual overview of the proposed brush model.

Fig. 2. Conceptual overview of the proposed brush model.

Effect of brush parameters.

Fig. 3. Effect of brush parameters.

Online-Offline Paired Dataset

Using our renderer, we build a synthetic paired dataset Dsyn from existing online corpora (IAM-OnDB, CASIA-OLHWDB) — composited onto diverse real-world backgrounds.

Online-Offline paired dataset (example 1). Online-Offline paired dataset (example 2).

Fig. 4. Paired samples generated by our differentiable renderer — diverse backgrounds, styles, and word lengths.

Qualitative Comparison

Combining our differentiable renderer with existing offline baselines (DiffPen, One-DM, Emuru, VATr++) improves both structural consistency and visual realism.

Qualitative comparison of offline handwriting generation results.

Fig. 5. Qualitative comparison of offline handwriting generation results.

Application: Robotic Calligraphy

Predicted stroke trajectories are directly executed on an Lite 6 robot — the (x, y) path drives horizontal motion, and the pressure proxy modulates the vertical position z

Results across writing tools

Robot writing execution across writing instruments.

Fig. 6. Robot writing execution results

Fig. 7. Robot writing demonstrations

Sentence-Level Application

Word-level trajectories compose into sentences via fixed spatial offsets — scalable to both offline generation and robotic execution.

Sentence-level application: rendered image, offline image, and robot execution.

Fig. 8. Sentence-level application. (A) rendered image, (B) offline image, (C) robot execution results.

BibTeX

@inproceedings{park2026onoff,
  title     = {OnOff: Bridging Online and Offline Handwriting via Differentiable Physical Rendering},
  author    = {Park, Seonmi and Shin, Seunghyun and Misra, Vihaan and Shin, Dongmin
               and Shin, Ukcheol and Oh, Jean and Jeon, Hae-Gon},
  booktitle = {European Conference on Computer Vision (ECCV)},
  year      = {2026}
}