Zayden-Jung/endorphin-tessera-gui
1
1
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

feat: compose interactive world ui across nested 3d viewports

Browse Source
This commit is contained in:
Codex Bot
2026-11-11 00:08:23 +08:00
parent ad2c01bc29
commit 15fc01542c
27 changed files with 1412 additions and 84 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/scene-architecture.md
+14 -1
View File
@@ -20,7 +20,8 @@
| 基线审查 | 完成 | 保留共享 Block / 独立 Mount、源协议和已合并工作区示例 |
| 嵌套布局与盒模型 | 完成 | Row / Column / Overlay、双轴对齐、子项覆盖、外边距、非对称内边距和边框、尺寸策略、裁剪；见下面接口 |
| 独立 2D 场景与视口 | 完成 | 独立场景/摄像机、多视口及嵌套、覆盖 UI、投影/命中、原生输入路由、真实 HTTP 与像素回归；旧 Page.mode 仅保留兼容 |
| 3D 场景、视口和摄像机 | 实现中 | 软件深度缓冲、网格/变换/灯光、透视/正交相机、可替换控制器、射线拾取已接通；世界空间 UI 面板仍待完成 |
| 3D 场景、视口和摄像机 | 完成 | 软件深度缓冲、网格/变换/灯光、透视/正交相机、可替换控制器、射线拾取、世界空间 UI 及 2D/3D 嵌套 |
| 可见性与资源防护 | 验证并补齐中 | 已有场景/绘制/像素/递归预算；继续补空间索引与输入队列回归 |
| 原生综合验收 | 待执行 | Rust / VM / HTTP 回归不能代替鼠标、触控板和窗口外观验收 |
| 完成交付 | 待完成 | 完成实现、检查和文档后，更新可评审 PR，等待用户手动合并 |
@@ -78,3 +79,15 @@
示例见 [scene3d.endor](../examples/scene3d.endor)。测试覆盖真正的 Endor 构造/修改、最终渲染像素、深度顺序、透视/正交独立视图、近面裁剪、射线回调、可替换控制器的真实 HTTP 路由以及循环/非法几何拒绝。原生硬件交互尚未验收。
3D 基础阶段检查：176 项本地回归全部通过，all-target Clippy 拒绝 warnings 通过。后续世界空间 UI 与嵌套资源/可见性工作尚未完成，因此整体目标及 PR 仍处于进行中。
## 世界空间 UI 与组合
`Panel3D(worldWidth, worldHeight, content)` 把普通 Endor Block 放进 3D 场景。面板有独立的 uiWidth/uiHeight（默认 512×320，整数 1–2048）和不透明 background；本次 UI 表面不实现透明材质。世界尺寸控制平面几何，UI 尺寸提供有限排版约束，两者互不代替。面板与组/网格一样支持位置、旋转、缩放及 translate/rotate。
内容仍是同一个托管对象树，正常执行 measure/arrange/paint/hitTest；每次出现有自己的 Mount、有限尺寸、滚动状态和纹理。一个组件可以放进多个不同尺寸的面板，不复制其业务状态。3D 摄像机移动不改变 UI 的排版尺寸。UI 绘制指令按面板收集，透视裁剪时保留 UV，并按深度进行透视校正插值；普通网格会正确遮挡面板。
指针先沿视口摄像机发射射线，再沿可见面板的逆世界变换映射到 UI 像素，然后继续通过内部 CanvasViewport 的逆变换。被其他网格挡住的 UI 无法收到指针命中。拖动捕获使用平面的连续逆投影，因此指针移出面板后不会把同一手势交给另一个摄像机。Mount.toView(x,y) 将组件本地坐标转到页面视图坐标；toLocal(x,y) 执行反向转换，无法命中可见面板时返回空值。pick3D 另提供实际 node/mesh 和实例编号。
面板 UI 可以继续嵌入 CanvasViewport 或 Viewport3D，后者也可以再包含 Panel3D；普通 Overlay 的兄弟 UI 仍固定在其 UI 空间。引用环在派生布局前拒绝；世界 UI 最多嵌套 8 层，光栅/纹理像素采用统一总预算。滚动视口与普通画布的世界内容不会撑大外层有限视口或外层滚动范围。
示例见 [world-ui.endor](../examples/world-ui.endor)。世界 UI 阶段的 181 项完整本地回归和 all-target Clippy 均通过；新增回归覆盖真正的像素、托管回调、材质遮挡、多面板独立尺寸、再次嵌入 3D、真实 HTTP 射线/UV 摄像机输入、循环和递归预算。空间索引、运行中的原生窗口及最终交付检查仍在继续。
examples/README.md
+1
View File
@@ -2,6 +2,7 @@
| File | Purpose |
| --- | --- |
| [world-ui.endor](world-ui.endor) | Interactive Endor UI on a transformed 3D plane, with shared counter state, nested canvas, programmable camera and fixed UI controls. |
| [scene3d.endor](scene3d.endor) | Shared 3D meshes, transforms and light with independent perspective/orthographic cameras, orbit controls and Endor buttons. |
| [scene-viewports.endor](scene-viewports.endor) | One Scene2D in two CanvasViewports with independent Camera2D, a nested viewport, shared state and fixed overlay UI. |
| [box-layout.endor](box-layout.endor) | Nested Row/Column/Overlay, shared text, finite border-box sizing, margins, borders, alignment and clipping. |
examples/world-ui.endor
+56
View File
@@ -0,0 +1,56 @@
// A normal Endor UI tree rendered on a transformed plane in the 3D world.
class WorldActions {
view: Viewport3D
panel: Panel3D
canvas: CanvasViewport
function reset(self: WorldActions): Unit {
self.view.camera.x = 0
self.view.camera.y = 0.5
self.view.camera.z = 7
self.view.camera.targetX = 0
self.view.camera.targetY = 0
self.view.camera.targetZ = 0
}
function turn(self: WorldActions): Unit { self.panel.rotate(0, 15, 0) }
function fit(self: WorldActions): Unit { self.canvas.mounts().at(0).fitView() }
}
count = ui.state(0)
increment = Button("点击：世界里的真实按钮")
increment.onClick(count.increment)
card = Column(10, 6, TextBlock("CanvasViewport", 1), RectBlock(180, 26, 6, "#9CB8A8"))
card.boxWidth = 240
card.x = 10
card.y = 4
canvas = CanvasViewport(280, 80, Scene2D(card))
ui_root = Column(8, 8, TextBlock("世界空间 UI", 2), ui.textOf(count, "value", 2), increment, canvas)
ui_root.crossAlign = "stretch"
panel = Panel3D(4, 3, ui_root)
panel.uiWidth = 320
panel.uiHeight = 240
panel.x = -1.2
panel.rotationY = 15
cube = Box3D(1.4, 1.4, 1.4, "#7B9CB7")
cube.x = 2
cube.rotationY = 25
floor = Box3D(10, 0.08, 5, "#D8DEE6")
floor.y = -1.8
scene = Scene3D(panel, cube, floor, DirectionalLight(-1, -2, -3, "#FFFFFF", 0.9))
scene.ambient = 0.35
view = Viewport3D(960, 520, scene)
view.flex = 1
view.camera.y = 0.5
view.camera.z = 7
actions = WorldActions(view, panel, canvas)
reset = Button("重置摄像机")
reset.onClick(actions.reset)
turn = Button("转动 UI 面板")
turn.onClick(actions.turn)
fit = Button("适配面板内的画布")
fit.onClick(actions.fit)
body = Column(24, 16, TextBlock("UI → 3D 世界 → UI → Canvas", 2), view, Row(0, 12, reset, turn, fit), TextBlock("空白处左键拖动旋转 3D · 中键/空格平移 · 滚轮缩放\n面板里的按钮、文本、布局和画布都由同一份 Endor 对象驱动。"))
body.flex = 1
body.crossAlign = "stretch"
page = Page()
page.title = "世界空间 UI"
page.crossAlign = "stretch"
page.render(body)
src/bridge/ui_bridge.rs
+4 -2
View File
@@ -49,6 +49,7 @@ pub const TYPE_NAMES: &[&str] = &[
"CanvasViewport",
"Camera2D",
"RayHit3D",
"Panel3D",
"CameraGesture",
"Vec3",
"Triangle3D",
@@ -1314,8 +1315,9 @@ impl UiBridge {
let delta = input
.delta
.ok_or_else(|| anyhow::anyhow!("scroll has no delta"))?;
let zoom = data.geometry.view.unwrap_or_default().zoom;
let delta = [delta[0] / zoom, delta[1] / zoom];
let Some(delta) = data.geometry.logical_delta(input.point, delta) else {
return Ok(());
};
let method = vm.bind_attr(mount, "scrollBy")?;
let result = match method {
Value::Object(function) => vm.call_with_limits(
src/bridge/ui_bridge/bootstrap.rs
+3
View File
@@ -107,6 +107,7 @@ pub fn install(vm: &mut Vm, roots: &mut Vec<ObjectRef>) -> Result<UiBridge> {
("OrthographicCamera", "Object", Some("orthographicCamera")),
("OrbitController", "Object", Some("orbitController")),
("Viewport3D", "Block", Some("viewport3D")),
("Panel3D", "Object", Some("panel3D")),
("RayHit3D", "Object", None),
("Button", "ContainerBlock", Some("button")),
("Page", "ContainerBlock", Some("page")),
@@ -377,10 +378,12 @@ pub fn install(vm: &mut Vm, roots: &mut Vec<ObjectRef>) -> Result<UiBridge> {
("OrbitController", "update", Operation::CameraControl),
("Mesh3D", "translate", Operation::Translate3D),
("Group3D", "translate", Operation::Translate3D),
("Panel3D", "translate", Operation::Translate3D),
("PerspectiveCamera", "translate", Operation::Translate3D),
("OrthographicCamera", "translate", Operation::Translate3D),
("Mesh3D", "rotate", Operation::Rotate3D),
("Group3D", "rotate", Operation::Rotate3D),
("Panel3D", "rotate", Operation::Rotate3D),
("Block", "mounts", Operation::Mounts),
("Block", "invalidate", Operation::Invalidate),
("Block", "onClick", Operation::OnClick),
src/bridge/ui_bridge/page_input.rs
+20 -7
View File
@@ -51,9 +51,18 @@ impl UiBridge {
let delta = input
.delta
.ok_or_else(|| anyhow::anyhow!("3D camera delta required"))?;
let local_delta = if input.kind == "scene-zoom" {
[0., 0.]
} else {
let Some(delta) = geometry.logical_delta(input.point, delta) else {
return Ok(());
};
delta
};
let point = input
.point
.map(|p| outer.inverse(crate::geometry::Point::new(p[0], p[1])))
.and_then(|p| geometry.view_point(crate::geometry::Point::new(p[0], p[1]), false))
.map(|p| outer.inverse(p))
.unwrap_or(crate::geometry::Point::new(
geometry.x + inset.left,
geometry.y + inset.top,
@@ -66,8 +75,8 @@ impl UiBridge {
)?;
vm.set_attr(gesture, "kind", Value::Text(input.kind.clone()))?;
for (name, value) in [
("dx", delta[0] / outer.zoom),
("dy", delta[1] / outer.zoom),
("dx", local_delta[0]),
("dy", local_delta[1]),
("factor", delta[0]),
("width", (geometry.width - inset.horizontal()).max(0.)),
("height", (geometry.height - inset.vertical()).max(0.)),
@@ -141,7 +150,6 @@ impl UiBridge {
let delta = input
.delta
.ok_or_else(|| anyhow::anyhow!("viewport input has no delta"))?;
let outer = geometry.view.unwrap_or_default();
let (mut x, mut y, mut zoom) = (old.x, old.y, old.zoom);
if input.kind == "viewport-zoom" {
let pointer = input
@@ -152,7 +160,9 @@ impl UiBridge {
geometry.contains(pointer),
"zoom anchor is outside viewport"
);
let pointer = outer.inverse(pointer);
let pointer = geometry
.logical_point(pointer)
.ok_or_else(|| anyhow::anyhow!("zoom anchor cannot be projected"))?;
let node = block::snapshot(vm.object_store(), block)?;
let inset = crate::layout::insets(&node);
let local = [
@@ -170,8 +180,11 @@ impl UiBridge {
x += local[0] / old.zoom - local[0] / zoom;
y += local[1] / old.zoom - local[1] / zoom;
} else {
x += delta[0] / outer.zoom / zoom;
y += delta[1] / outer.zoom / zoom;
let Some(delta) = geometry.logical_delta(input.point, delta) else {
return Ok(());
};
x += delta[0] / zoom;
y += delta[1] / zoom;
}
x = x.clamp(-MAX_DIMENSION as f32, MAX_DIMENSION as f32);
y = y.clamp(-MAX_DIMENSION as f32, MAX_DIMENSION as f32);
src/bridge/ui_bridge/tree.rs
+8 -1
View File
@@ -30,6 +30,7 @@ impl UiNode {
| UiKind::Canvas { children, .. }
| UiKind::Scene2D { children, .. }
| UiKind::CanvasViewport { children, .. }
| UiKind::Viewport3D { children, .. }
| UiKind::Row { children, .. }
| UiKind::Button { children, .. }
| UiKind::Scroll { children, .. } => children,
@@ -87,14 +88,18 @@ impl UiNode {
height: b,
view: c,
controls: g,
panels: i,
..
},
UiKind::Viewport3D {
width: d,
height: e,
view: f,
controls: h,
panels: j,
..
},
) => a == d && b == e && c == f && g == h,
) => a == d && b == e && c == f && g == h && i == j && same_children(),
(
UiKind::CanvasViewport {
width: a,
@@ -199,6 +204,8 @@ pub enum UiKind {
height: f32,
view: Arc<crate::scene3d::View>,
controls: bool,
panels: Vec<crate::layout::Size>,
children: Vec<Arc<UiNode>>,
},
Scene2D {
positions: Vec<[f32; 2]>,
src/model/document.rs
+33 -2
View File
@@ -263,6 +263,7 @@ impl RenderDocument {
if let Some(focus) = frame
.focus
.and_then(|id| frame.mounts.iter().find(|m| m.id == id))
.filter(|mount| mount.panel.is_none())
{
if let Some(c) = focus.clip {
list.push(DrawCommand::PushClip(scaled_rect(
@@ -419,6 +420,8 @@ impl Default for Frame {
/// A position in a published frame. Identifiers are opaque data, never VM handles.
#[derive(Clone, Debug, Serialize, Deserialize, PartialEq)]
pub struct MountGeometry {
#[serde(default, skip_serializing_if = "Option::is_none")]
pub panel: Option<Arc<crate::scene3d::panel::Mapping>>,
#[serde(default, skip_serializing_if = "std::ops::Not::not")]
pub canvas_viewport: bool,
#[serde(default, skip_serializing_if = "std::ops::Not::not")]
@@ -446,7 +449,33 @@ pub struct MountGeometry {
}
impl MountGeometry {
pub fn view_point(&self, point: Point, clipped: bool) -> Option<Point> {
self.panel
.as_ref()
.map_or(Some(point), |panel| panel.texture_point(point, clipped))
}
pub fn logical_point(&self, point: Point) -> Option<Point> {
let point = self.view_point(point, true)?;
Some(self.view.unwrap_or_default().inverse(point))
}
pub fn logical_delta(&self, point: Option<[f32; 2]>, delta: [f32; 2]) -> Option<[f32; 2]> {
let zoom = self.view.unwrap_or_default().zoom;
if self.panel.is_none() {
return Some([delta[0] / zoom, delta[1] / zoom]);
}
let point = point?;
let current = self.view_point(Point::new(point[0], point[1]), false)?;
let previous =
self.view_point(Point::new(point[0] + delta[0], point[1] + delta[1]), false)?;
Some([
((previous.x - current.x) / zoom).clamp(-1_000_000., 1_000_000.),
((previous.y - current.y) / zoom).clamp(-1_000_000., 1_000_000.),
])
}
pub fn contains(&self, point: Point) -> bool {
let Some(point) = self.view_point(point, true) else {
return false;
};
let local = self.view.map_or(point, |view| view.inverse(point));
Rect::new(self.x, self.y, self.width, self.height).contains(local)
&& self.clip.is_none_or(|c| {
@@ -461,7 +490,9 @@ impl Frame {
(mount.canvas_viewport || mount.scene3d_viewport && mount.camera_controls)
&& mount.contains(point)
&& mount.viewport_clip.is_some_and(|clip| {
Rect::new(clip[0], clip[1], clip[2], clip[3]).contains(point)
mount.view_point(point, true).is_some_and(|point| {
Rect::new(clip[0], clip[1], clip[2], clip[3]).contains(point)
})
})
})
}
@@ -484,7 +515,7 @@ fn default_background() -> String {
"#F7F7F5".to_owned()
}
#[derive(Clone, Debug, Serialize, Deserialize)]
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
#[serde(tag = "type", rename_all = "snake_case")]
pub enum RenderCommand {
Scene3D {
src/model/mount.rs
+69 -3
View File
@@ -57,6 +57,8 @@ enum Query {
ScrollIntoView,
FitView,
Pick3D,
ToView,
ToLocal,
Parent,
FirstChild,
NextSibling,
@@ -81,7 +83,12 @@ impl NativeCallable for Method {
Some(endor_core::CallArity::exact(
if matches!(
self.query,
Query::HitTest | Query::Pick3D | Query::ScrollTo | Query::ScrollBy
Query::HitTest
| Query::Pick3D
| Query::ToView
| Query::ToLocal
| Query::ScrollTo
| Query::ScrollBy
) {
2
} else {
@@ -103,6 +110,8 @@ impl NativeCallable for Method {
| Query::ScrollIntoView
| Query::FitView
| Query::Pick3D
| Query::ToView
| Query::ToLocal
| Query::ScrollPosition
)
{
@@ -110,7 +119,12 @@ impl NativeCallable for Method {
}
let arity = if matches!(
self.query,
Query::HitTest | Query::Pick3D | Query::ScrollTo | Query::ScrollBy
Query::HitTest
| Query::Pick3D
| Query::ToView
| Query::ToLocal
| Query::ScrollTo
| Query::ScrollBy
) {
2
} else {
@@ -120,6 +134,50 @@ impl NativeCallable for Method {
return Err(VmError::InvalidArgument("incorrect Mount method arity"));
}
let value = match self.query {
Query::ToView | Query::ToLocal => {
if mount.layout.is_some() {
return Err(VmError::InvalidArgument(
"coordinate conversion requires committed geometry",
));
}
let point = crate::geometry::Point::new(
crate::scroll::coordinate(&args[0])?,
crate::scroll::coordinate(&args[1])?,
);
let point =
if matches!(self.query, Query::ToView) {
let point = mount.geometry.view.unwrap_or_default().point(
crate::geometry::Point::new(
point.x + mount.geometry.x,
point.y + mount.geometry.y,
),
);
mount
.geometry
.panel
.as_ref()
.map_or(Some(point), |panel| panel.project_point(point))
} else {
mount.geometry.logical_point(point).map(|point| {
crate::geometry::Point::new(
point.x - mount.geometry.x,
point.y - mount.geometry.y,
)
})
};
if let Some(point) = point {
let result = cx.new_object()?;
let desktop = cx.upgrade(self.desktop)?.ok_or(VmError::InvalidArgument(
"coordinate conversion desktop expired",
))?;
crate::ui_bridge::attach_desktop_type(cx, result, "Point", desktop)?;
cx.set_attr(result, "x", crate::native_visual::value(point.x))?;
cx.set_attr(result, "y", crate::native_visual::value(point.y))?;
Value::Object(result)
} else {
Value::Nil
}
}
Query::ScrollTo | Query::ScrollBy => {
crate::scroll::set(
cx,
@@ -281,6 +339,7 @@ fn mount_signature(
Query::Page => CallType::nullable(nominal("Page")?),
Query::HitTest => T::Bool.into(),
Query::Pick3D => CallType::nullable(nominal("RayHit3D")?),
Query::ToView | Query::ToLocal => CallType::nullable(nominal("Point")?),
Query::ScrollPosition => T::Any.into(),
_ => T::Unit.into(),
};
@@ -294,7 +353,12 @@ fn mount_signature(
};
let parameters = if matches!(
query,
Query::HitTest | Query::Pick3D | Query::ScrollTo | Query::ScrollBy
Query::HitTest
| Query::Pick3D
| Query::ToView
| Query::ToLocal
| Query::ScrollTo
| Query::ScrollBy
) {
vec![T::Number, T::Number]
} else {
@@ -433,6 +497,8 @@ impl NativeCallable for Sync {
("scrollIntoView", Query::ScrollIntoView),
("fitView", Query::FitView),
("pick3D", Query::Pick3D),
("toView", Query::ToView),
("toLocal", Query::ToLocal),
("parent", Query::Parent),
("firstChild", Query::FirstChild),
("nextSibling", Query::NextSibling),
src/model/scene3d.rs
+113 -13
View File
@@ -23,6 +23,7 @@ pub enum Constructor {
Perspective,
Orthographic,
Orbit,
Panel,
}
impl Constructor {
pub fn name(self) -> &'static str {
@@ -38,6 +39,7 @@ impl Constructor {
Self::Perspective => "PerspectiveCamera",
Self::Orthographic => "OrthographicCamera",
Self::Orbit => "OrbitController",
Self::Panel => "Panel3D",
}
}
pub fn function(self) -> &'static str {
@@ -53,6 +55,7 @@ impl Constructor {
Self::Perspective => "perspectiveCamera",
Self::Orthographic => "orthographicCamera",
Self::Orbit => "orbitController",
Self::Panel => "panel3D",
}
}
pub fn from_function(name: &str) -> Option<Self> {
@@ -68,6 +71,7 @@ impl Constructor {
Self::Perspective,
Self::Orthographic,
Self::Orbit,
Self::Panel,
]
.into_iter()
.find(|kind| kind.function() == name)
@@ -79,6 +83,7 @@ impl Constructor {
Self::Box => (4, 4),
Self::Group | Self::Scene => (0, 4096),
Self::Light => (5, 5),
Self::Panel => (3, 3),
_ => (0, 0),
}
}
@@ -89,6 +94,7 @@ impl Constructor {
Self::Mesh => &["Geometry3D", "String"],
Self::Box => &["Number", "Number", "Number", "String"],
Self::Light => &["Number", "Number", "Number", "String", "Number"],
Self::Panel => &["Number", "Number", "Block"],
_ => &[],
}
}
@@ -108,6 +114,7 @@ pub enum NodeKind {
Perspective,
Orthographic,
Orbit,
Panel,
}
#[derive(Debug)]
pub struct Node(pub NodeKind);
@@ -198,6 +205,7 @@ pub fn allocate(
Constructor::Perspective => NodeKind::Perspective,
Constructor::Orthographic => NodeKind::Orthographic,
Constructor::Orbit => NodeKind::Orbit,
Constructor::Panel => NodeKind::Panel,
Constructor::Geometry => {
let vertices = object_list::items(cx.store(), object_ref(&args[0])?)?
.iter()
@@ -233,6 +241,7 @@ pub fn allocate(
let geometry = scene3d::Geometry {
vertices,
triangles,
uv: vec![],
};
if !geometry.valid() {
return Err(invalid(
@@ -282,7 +291,7 @@ pub fn allocate(
for child in &children {
if !matches!(
cx.store().payload::<Node>(*child, "3D node")?.0,
NodeKind::Mesh | NodeKind::Group | NodeKind::Light
NodeKind::Mesh | NodeKind::Group | NodeKind::Light | NodeKind::Panel
) {
return Err(invalid(
"Scene3D/Group3D children require meshes, groups or lights",
@@ -307,6 +316,16 @@ pub fn allocate(
cx.set_attr(instance, "color", args[3].clone())?;
set_number(cx, instance, "intensity", number(&args[4])?)?;
}
NodeKind::Panel => {
let content = object_ref(&args[2])?;
crate::block::validate(cx.store(), content, None)?;
set_number(cx, instance, "width", number(&args[0])?)?;
set_number(cx, instance, "height", number(&args[1])?)?;
set_number(cx, instance, "uiWidth", 512.)?;
set_number(cx, instance, "uiHeight", 320.)?;
cx.set_attr(instance, "content", Value::Object(content))?;
cx.set_attr(instance, "background", Value::Text("#FFFFFF".into()))?;
}
NodeKind::Perspective | NodeKind::Orthographic => {
for (name, value) in [
("x", 0.),
@@ -331,7 +350,7 @@ pub fn allocate(
set_number(cx, instance, "panSpeed", 1.)?;
}
}
if matches!(kind, NodeKind::Mesh | NodeKind::Group) {
if matches!(kind, NodeKind::Mesh | NodeKind::Group | NodeKind::Panel) {
for (name, value) in [
("x", 0.),
("y", 0.),
@@ -397,7 +416,16 @@ impl Reader<'_> {
}
}
pub type Snapshot = (Arc<scene3d::View>, Vec<(ObjectRef, String)>);
pub struct PanelMount {
pub key: u64,
pub content: ObjectRef,
pub size: crate::layout::Size,
}
pub type Snapshot = (
Arc<scene3d::View>,
Vec<(ObjectRef, String)>,
Vec<PanelMount>,
);
pub fn snapshot(
store: &ObjectStore,
scene: ObjectRef,
@@ -422,11 +450,12 @@ pub fn snapshot(
ambient,
background,
};
let mut work = vec![(scene, Matrix::default(), false)];
let mut work = vec![(scene, Matrix::default(), false, 0u64)];
let mut panels = Vec::new();
let mut path = HashSet::new();
let mut visits = 0;
let mut triangles = 0;
while let Some((object, parent, finish)) = work.pop() {
while let Some((object, parent, finish, key)) = work.pop() {
if finish {
path.remove(&object);
continue;
@@ -438,9 +467,9 @@ pub fn snapshot(
if !path.insert(object) {
return Err(invalid("cyclic 3D scene graph"));
}
work.push((object, parent, true));
work.push((object, parent, true, key));
let kind = store.payload::<Node>(object, "3D node")?.0;
let world = if matches!(kind, NodeKind::Mesh | NodeKind::Group) {
let world = if matches!(kind, NodeKind::Mesh | NodeKind::Group | NodeKind::Panel) {
parent.compose(Matrix::trs(
reader.vector(object, ["x", "y", "z"])?,
reader.vector(object, ["rotationX", "rotationY", "rotationZ"])?,
@@ -452,8 +481,13 @@ pub fn snapshot(
match kind {
NodeKind::Scene | NodeKind::Group => {
let list = object_ref(&reader.value(object, "children")?)?;
for child in object_list::items(store, list)?.iter().rev() {
work.push((*child, world, false));
for (index, child) in object_list::items(store, list)?.iter().enumerate().rev() {
use std::hash::{Hash, Hasher};
let mut hasher = std::collections::hash_map::DefaultHasher::new();
key.hash(&mut hasher);
index.hash(&mut hasher);
child.hash(&mut hasher);
work.push((*child, world, false, hasher.finish()));
}
}
NodeKind::Mesh => {
@@ -469,6 +503,60 @@ pub fn snapshot(
geometry,
world,
color: material,
panel: None,
});
}
NodeKind::Panel => {
if panels.len() >= 64 {
return Err(VmError::ExecutionLimit("3D UI panels"));
}
let width = reader.number(object, "width")?;
let height = reader.number(object, "height")?;
let ui_width = reader.number(object, "uiWidth")?;
let ui_height = reader.number(object, "uiHeight")?;
if width <= 0.
|| height <= 0.
|| ![ui_width, ui_height]
.iter()
.all(|size| (1.0..=2048.0).contains(size) && size.fract() == 0.)
{
return Err(invalid(
"Panel3D requires positive world dimensions and integer UI dimensions from 1 through 2048",
));
}
let content = object_ref(&reader.value(object, "content")?)?;
crate::block::resolve(store, content)?;
let background = color(&text(&reader.value(object, "background")?)?)?;
let geometry = Arc::new(scene3d::Geometry {
vertices: vec![
Vec3::new(-width / 2., height / 2., 0.),
Vec3::new(width / 2., height / 2., 0.),
Vec3::new(width / 2., -height / 2., 0.),
Vec3::new(-width / 2., -height / 2., 0.),
],
triangles: vec![[0, 1, 2], [0, 2, 3]],
uv: vec![[0., 0.], [1., 0.], [1., 1.], [0., 1.]],
});
output.meshes.push(scene3d::Mesh {
identity: object.to_string(),
geometry,
world,
color: [1., 1., 1.],
panel: Some(scene3d::Panel {
key,
width: ui_width as u32,
height: ui_height as u32,
background,
surface: None,
}),
});
panels.push(PanelMount {
key,
content,
size: crate::layout::Size {
width: ui_width,
height: ui_height,
},
});
}
NodeKind::Light => {
@@ -501,7 +589,7 @@ pub fn snapshot(
if !view.valid() {
return Err(invalid("invalid 3D scene snapshot"));
}
Ok((view, reader.reads.into_iter().collect()))
Ok((view, reader.reads.into_iter().collect(), panels))
}
pub fn pick(
@@ -523,7 +611,9 @@ pub fn pick(
}
let node = crate::block::snapshot(cx.store(), mount.block)?;
let inset = crate::layout::insets(&node);
let point = mount.geometry.view.unwrap_or_default().inverse(point);
let Some(point) = mount.geometry.logical_point(point) else {
return Ok(Value::Nil);
};
let (width, height) = (
(mount.geometry.width - inset.horizontal()).max(0.),
(mount.geometry.height - inset.vertical()).max(0.),
@@ -531,7 +621,7 @@ pub fn pick(
if width <= 0. || height <= 0. {
return Ok(Value::Nil);
}
let (view, reads) = snapshot(
let (view, reads, _) = snapshot(
cx.store(),
data.scene3d.ok_or(invalid("missing Scene3D"))?,
data.camera.ok_or(invalid("missing 3D camera"))?,
@@ -551,6 +641,12 @@ pub fn pick(
let result = cx.new_object()?;
crate::ui_bridge::attach_desktop_type(cx, result, "RayHit3D", desktop)?;
cx.set_attr(result, "mesh", Value::Object(mesh))?;
cx.set_attr(result, "node", Value::Object(mesh))?;
cx.set_attr(
result,
"instance",
Value::Number((hit.instance as i64).into()),
)?;
cx.set_attr(
result,
"distance",
@@ -700,7 +796,11 @@ pub fn transform(
let camera = matches!(kind, NodeKind::Perspective | NodeKind::Orthographic);
if !matches!(
kind,
NodeKind::Mesh | NodeKind::Group | NodeKind::Perspective | NodeKind::Orthographic
NodeKind::Mesh
| NodeKind::Group
| NodeKind::Panel
| NodeKind::Perspective
| NodeKind::Orthographic
) || rotate && camera
{
return Err(invalid("unsupported 3D transform target"));
src/model/snapshot.rs
+30 -17
View File
@@ -128,7 +128,7 @@ impl RenderCache {
}
store.check_typed_instance_storage(block)?;
let (data_ref, raw_data, raw_dependencies) = block::resolve_tracked(store, block)?;
let data = block::visual_data(store, raw_data)?;
let mut data = block::visual_data(store, raw_data)?;
let mut dependencies: Vec<_> = raw_dependencies
.into_iter()
.map(|object| Dependency {
@@ -164,6 +164,33 @@ impl RenderCache {
});
}
}
let mut panel_sizes = Vec::new();
let view3d = if matches!(data.kind, Kind::Viewport3D { .. }) {
let (view, reads, panels) = crate::scene3d_model::snapshot(
store,
data.scene3d
.ok_or(VmError::InvalidArgument("missing Scene3D"))?,
data.camera
.ok_or(VmError::InvalidArgument("missing 3D camera"))?,
)?;
dependencies.extend(reads.into_iter().map(|(object, name)| Dependency {
object,
property: Some(name),
}));
data.children = panels
.into_iter()
.map(|panel| {
panel_sizes.push(panel.size);
crate::block::Child {
slot: panel.key,
block: panel.content,
}
})
.collect();
Some(view)
} else {
None
};
if !finish {
if !path.insert(data_ref) {
return Err(VmError::InvalidArgument("cyclic Block composition"));
@@ -175,22 +202,6 @@ impl RenderCache {
continue;
}
path.remove(&data_ref);
let view3d = if matches!(data.kind, Kind::Viewport3D { .. }) {
let (view, reads) = crate::scene3d_model::snapshot(
store,
data.scene3d
.ok_or(VmError::InvalidArgument("missing Scene3D"))?,
data.camera
.ok_or(VmError::InvalidArgument("missing 3D camera"))?,
)?;
dependencies.extend(reads.into_iter().map(|(object, name)| Dependency {
object,
property: Some(name),
}));
Some(view)
} else {
None
};
let mut protocol_key = String::new();
let mut custom_event = false;
if let Some(definition) = store.type_of(block)? {
@@ -290,6 +301,8 @@ impl RenderCache {
height: *height,
view: view3d.ok_or(VmError::InvalidArgument("missing 3D view"))?,
controls: data.controller.is_some(),
panels: panel_sizes,
children,
},
Kind::CanvasViewport { width, height } => UiKind::CanvasViewport {
width: *width,
src/render/layout.rs
+15 -2
View File
@@ -259,7 +259,12 @@ pub fn occurrence_constraints(
child: &UiNode,
mut base: [f32; 4],
offer: Size,
child_index: usize,
) -> [f32; 4] {
if let UiKind::Viewport3D { panels, .. } = &parent.kind {
let size = panels[child_index];
return [size.width, size.height, size.width, size.height];
}
let inset = insets(parent);
for (axis, dimension, available, padding) in [
(0, child.sizing.width, offer.width, inset.horizontal()),
@@ -280,6 +285,7 @@ pub fn initial_child_constraints(
parent: &UiNode,
child: &UiNode,
constraints: [f32; 4],
child_index: usize,
) -> [f32; 4] {
let offer = parent.sizing.available(constraints);
occurrence_constraints(
@@ -290,6 +296,7 @@ pub fn initial_child_constraints(
width: offer[2],
height: offer[3],
},
child_index,
)
}
/// Scroll cross-axis constraints come from this occurrence's actual viewport,
@@ -752,6 +759,9 @@ pub fn arrange_native(
bounds: Size,
children: &[Size],
) -> Result<Vec<(f32, f32)>> {
if matches!(node.kind, UiKind::Viewport3D { .. }) {
return Ok(node.children().iter().map(|_| (0., 0.)).collect());
}
if let UiKind::Overlay { .. } = node.kind {
let padding = insets(node);
let horizontal = match node.main_align {
@@ -977,12 +987,14 @@ impl LayoutRuntime {
id,
constraints,
});
for (child, slot) in node.children().iter().zip(&node.slots).rev() {
for (child_index, (child, slot)) in
node.children().iter().zip(&node.slots).enumerate().rev()
{
work.push((
child.as_ref(),
Some(index),
*slot,
initial_child_constraints(node, child, constraints),
initial_child_constraints(node, child, constraints, child_index),
));
}
}
@@ -1017,6 +1029,7 @@ impl LayoutRuntime {
coordinate(x + size.width)?;
coordinate(y + size.height)?;
mounts.push(MountGeometry {
panel: None,
canvas_viewport: matches!(item.node.kind, UiKind::CanvasViewport { .. }),
scene3d_viewport: matches!(item.node.kind, UiKind::Viewport3D { .. }),
camera_controls: matches!(
src/render/native_visual.rs
+3 -1
View File
@@ -214,7 +214,8 @@ impl NativeCallable for Method {
} else {
node.children()
.iter()
.map(|child| {
.enumerate()
.map(|(child_index, child)| {
size(
child,
layout::occurrence_constraints(
@@ -226,6 +227,7 @@ impl NativeCallable for Method {
Size { width, height },
),
Size { width, height },
child_index,
),
)
})
src/render/scene3d.rs
+127 -11
View File
@@ -1,6 +1,7 @@
//! Software 3D surfaces share the application's existing CPU compositor. Mesh
//! visibility and picking use the same camera and transformed world triangles.
pub mod math;
pub mod panel;
use math::{Camera, Matrix, Projection, Vec3};
use serde::{Deserialize, Serialize};
use std::sync::Arc;
@@ -12,10 +13,15 @@ pub const MAX_VERTICES: usize = 32_768;
pub struct Geometry {
pub vertices: Vec<Vec3>,
pub triangles: Vec<[u32; 3]>,
#[serde(default)]
pub uv: Vec<[f32; 2]>,
}
impl Geometry {
pub fn valid(&self) -> bool {
self.vertices.len() <= MAX_VERTICES
&& (self.uv.is_empty()
|| self.uv.len() == self.vertices.len()
&& self.uv.iter().flatten().all(|value| value.is_finite()))
&& self.triangles.len() <= MAX_TRIANGLES
&& self.vertices.iter().all(|point| point.valid())
&& self
@@ -27,6 +33,7 @@ impl Geometry {
pub fn cuboid(width: f32, height: f32, depth: f32) -> Self {
let (x, y, z) = (width / 2., height / 2., depth / 2.);
Self {
uv: vec![],
vertices: vec![
Vec3::new(-x, -y, -z),
Vec3::new(x, -y, -z),
@@ -60,6 +67,22 @@ pub struct Mesh {
pub geometry: Arc<Geometry>,
pub world: Matrix,
pub color: [f32; 3],
pub panel: Option<Panel>,
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct UiSurface {
pub commands: Vec<crate::document::RenderCommand>,
pub clips: Vec<Option<[f32; 4]>>,
pub views: Vec<Option<crate::viewport::Transform>>,
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct Panel {
pub key: u64,
pub width: u32,
pub height: u32,
pub background: [f32; 3],
pub surface: Option<Arc<UiSurface>>,
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct Light {
@@ -91,6 +114,7 @@ impl endor_core::ObjectPayload for PaintView {
}
#[derive(Debug, Clone, PartialEq)]
pub struct Hit {
pub instance: usize,
pub identity: String,
pub point: Vec3,
pub normal: Vec3,
@@ -101,6 +125,24 @@ pub struct Hit {
impl View {
pub fn valid(&self) -> bool {
thread_local! { static DEPTH:std::cell::Cell<usize>=const { std::cell::Cell::new(0) }; }
struct Guard;
impl Drop for Guard {
fn drop(&mut self) {
DEPTH.with(|depth| depth.set(depth.get() - 1));
}
}
if !DEPTH.with(|depth| {
if depth.get() >= 8 {
false
} else {
depth.set(depth.get() + 1);
true
}
}) {
return false;
}
let _guard = Guard;
let color_valid = |color: [f32; 3]| {
color
.iter()
@@ -121,6 +163,22 @@ impl View {
.vertices
.iter()
.all(|point| mesh.world.point(*point).valid())
&& mesh.panel.as_ref().is_none_or(|panel| {
panel.width > 0
&& panel.height > 0
&& panel.width <= 2048
&& panel.height <= 2048
&& color_valid(panel.background)
&& panel.surface.as_ref().is_none_or(|surface| {
surface.commands.len() <= 10001
&& surface.commands.len() == surface.views.len()
&& surface.commands.len() == surface.clips.len()
&& surface
.commands
.iter()
.all(|command| command.validate().is_ok())
})
})
})
&& self.scene.lights.iter().all(|light| {
color_valid(light.color)
@@ -156,7 +214,7 @@ impl View {
let ray = self.camera.ray(u, v, aspect)?;
let basis = self.camera.basis()?;
let mut hit: Option<Hit> = None;
for mesh in &self.scene.meshes {
for (instance, mesh) in self.scene.meshes.iter().enumerate() {
for (triangle, indices) in mesh.geometry.triangles.iter().enumerate() {
let [a, b, c] =
indices.map(|i| mesh.world.point(mesh.geometry.vertices[i as usize]));
@@ -175,6 +233,7 @@ impl View {
continue;
};
hit = Some(Hit {
instance,
identity: mesh.identity.clone(),
point,
normal,
@@ -240,7 +299,23 @@ pub fn surface(view: &Arc<View>, width: u32, height: u32) -> Result<Arc<[u32]>,
/// Homogeneous frustum planes expressed in view space. Clipping before division
/// handles triangles crossing the near plane without huge screen coordinates.
fn clip(mut polygon: Vec<Vec3>, camera: Camera, aspect: f32) -> Vec<Vec3> {
#[derive(Clone, Copy)]
struct Vertex {
position: Vec3,
uv: [f32; 2],
}
impl Vertex {
fn lerp(self, other: Self, t: f32) -> Self {
Self {
position: self.position + (other.position - self.position) * t,
uv: [
self.uv[0] + (other.uv[0] - self.uv[0]) * t,
self.uv[1] + (other.uv[1] - self.uv[1]) * t,
],
}
}
}
fn clip(mut polygon: Vec<Vertex>, camera: Camera, aspect: f32) -> Vec<Vertex> {
let (x, y, slope) = match camera.projection {
Projection::Perspective { fov } => {
let y = (fov.to_radians() / 2.).tan();
@@ -260,14 +335,17 @@ fn clip(mut polygon: Vec<Vec3>, camera: Camera, aspect: f32) -> Vec<Vec3> {
if polygon.is_empty() {
break;
}
let distance = |p: Vec3| a * p.x + b * p.y + c * p.z + d;
let distance = |vertex: Vertex| {
let p = vertex.position;
a * p.x + b * p.y + c * p.z + d
};
let mut output = Vec::new();
let mut previous = *polygon.last().unwrap();
let mut before = distance(previous);
for current in polygon {
let after = distance(current);
if (before >= 0.) != (after >= 0.) {
output.push(previous + (current - previous) * (before / (before - after)));
output.push(previous.lerp(current, before / (before - after)));
}
if after >= 0. {
output.push(current);
@@ -281,11 +359,13 @@ fn clip(mut polygon: Vec<Vec3>, camera: Camera, aspect: f32) -> Vec<Vec3> {
}
pub fn render(view: &View, width: u32, height: u32) -> Result<Vec<u32>, &'static str> {
let _scope = crate::software_renderer::RenderScope::enter()?;
let area = width as usize * height as usize;
// Offscreen targets have a hard per-surface resource bound.
if area > 4_194_304 || width == 0 || height == 0 || !view.valid() {
return Err("invalid 3D surface or resource budget exceeded");
}
crate::software_renderer::reserve_pixels(area.saturating_mul(2))?;
let mut pixels = vec![rgb(view.scene.background); area];
let mut depths = vec![f32::INFINITY; area];
let camera = view.camera;
@@ -305,6 +385,7 @@ pub fn render(view: &View, width: u32, height: u32) -> Result<Vec<u32>, &'static
};
let mut work = 0usize;
for mesh in &view.scene.meshes {
let texture = mesh.panel.as_ref().map(panel::texture).transpose()?;
for indices in &mesh.geometry.triangles {
let [a, b, c] = indices.map(|i| mesh.world.point(mesh.geometry.vertices[i as usize]));
let Some(normal) = (b - a).cross(c - a).normalized() else {
@@ -319,11 +400,20 @@ pub fn render(view: &View, width: u32, height: u32) -> Result<Vec<u32>, &'static
}
let color = rgb(std::array::from_fn(|i| mesh.color[i] * light[i]));
let polygon = clip(
vec![
camera.view(basis, a),
camera.view(basis, b),
camera.view(basis, c),
],
indices
.map(|index| Vertex {
position: camera.view(
basis,
mesh.world.point(mesh.geometry.vertices[index as usize]),
),
uv: mesh
.geometry
.uv
.get(index as usize)
.copied()
.unwrap_or([0., 0.]),
})
.to_vec(),
camera,
aspect,
);
@@ -331,7 +421,8 @@ pub fn render(view: &View, width: u32, height: u32) -> Result<Vec<u32>, &'static
continue;
}
for index in 1..polygon.len() - 1 {
let [a, b, c] = [polygon[0], polygon[index], polygon[index + 1]].map(project);
let vertices = [polygon[0], polygon[index], polygon[index + 1]];
let [a, b, c] = vertices.map(|vertex| project(vertex.position));
let min_x = a.x.min(b.x).min(c.x).floor().max(0.) as u32;
let max_x = a.x.max(b.x).max(c.x).ceil().min(width as f32) as u32;
let min_y = a.y.min(b.y).min(c.y).floor().max(0.) as u32;
@@ -367,7 +458,30 @@ pub fn render(view: &View, width: u32, height: u32) -> Result<Vec<u32>, &'static
let index = y as usize * width as usize + x as usize;
if depth < depths[index] {
depths[index] = depth;
pixels[index] = color;
pixels[index] =
if let (Some(texture), Some(panel)) = (&texture, &mesh.panel) {
let interpolated = |axis| {
if perspective {
depth
* (weights[0] * vertices[0].uv[axis] / a.z
+ weights[1] * vertices[1].uv[axis] / b.z
+ weights[2] * vertices[2].uv[axis] / c.z)
} else {
weights[0] * vertices[0].uv[axis]
+ weights[1] * vertices[1].uv[axis]
+ weights[2] * vertices[2].uv[axis]
}
};
let u = (interpolated(0).clamp(0., 1.) * panel.width as f32)
.min((panel.width - 1) as f32)
as usize;
let v = (interpolated(1).clamp(0., 1.) * panel.height as f32)
.min((panel.height - 1) as f32)
as usize;
texture[v * panel.width as usize + u]
} else {
color
};
}
}
}
@@ -385,6 +499,7 @@ mod tests {
camera: Camera::default(),
scene: Arc::new(Scene {
meshes: vec![Mesh {
panel: None,
identity: "mesh".into(),
geometry: Arc::new(geometry),
world: Matrix::default(),
@@ -399,6 +514,7 @@ mod tests {
#[test]
fn triangles_crossing_near_plane_are_clipped_and_pick_uses_the_same_range() {
let mut view = view(Geometry {
uv: vec![],
vertices: vec![
Vec3::new(-1., -1., 0.5),
Vec3::new(1., -1., 2.),
src/render/scene3d/math.rs
+32
View File
@@ -63,6 +63,38 @@ impl Default for Matrix {
}
}
impl Matrix {
pub fn inverse(self) -> Option<Self> {
let m = self.0;
let determinant = m[0] * (m[5] * m[10] - m[6] * m[9]) - m[1] * (m[4] * m[10] - m[6] * m[8])
+ m[2] * (m[4] * m[9] - m[5] * m[8]);
if !determinant.is_finite() || determinant.abs() < 1e-9 {
return None;
}
let d = 1. / determinant;
let mut out = Self([
(m[5] * m[10] - m[6] * m[9]) * d,
(m[2] * m[9] - m[1] * m[10]) * d,
(m[1] * m[6] - m[2] * m[5]) * d,
0.,
(m[6] * m[8] - m[4] * m[10]) * d,
(m[0] * m[10] - m[2] * m[8]) * d,
(m[2] * m[4] - m[0] * m[6]) * d,
0.,
(m[4] * m[9] - m[5] * m[8]) * d,
(m[1] * m[8] - m[0] * m[9]) * d,
(m[0] * m[5] - m[1] * m[4]) * d,
0.,
0.,
0.,
0.,
1.,
]);
let translation = out.point(Vec3::new(m[3], m[7], m[11])) * -1.;
out.0[3] = translation.x;
out.0[7] = translation.y;
out.0[11] = translation.z;
Some(out)
}
pub fn compose(self, local: Self) -> Self {
let mut out = [0.; 16];
for row in 0..4 {
src/render/scene3d/panel.rs
+255
View File
@@ -0,0 +1,255 @@
//! Perspective UI mapping. The same world plane supplies texture coordinates
//! during rendering and inverse pointer coordinates during event dispatch.
use super::{
View,
math::{Projection, Vec3},
};
use crate::{
geometry::{Point, Rect},
viewport::Transform,
};
use std::sync::Arc;
#[derive(Debug, Clone, PartialEq, serde::Serialize, serde::Deserialize)]
pub struct Mapping {
pub owner: u64,
pub key: u64,
pub view: Arc<View>,
pub rect: [f32; 4],
pub outer: Transform,
pub clip: Option<[f32; 4]>,
pub parent: Option<Arc<Mapping>>,
}
impl Mapping {
pub fn texture_point(&self, point: Point, clipped: bool) -> Option<Point> {
let point = if let Some(parent) = &self.parent {
parent.texture_point(point, clipped)?
} else {
point
};
if clipped
&& self
.clip
.is_some_and(|clip| !Rect::new(clip[0], clip[1], clip[2], clip[3]).contains(point))
{
return None;
}
let point = self.outer.inverse(point);
let [x, y, width, height] = self.rect;
if width <= 0. || height <= 0. {
return None;
}
let (u, v) = ((point.x - x) / width, (point.y - y) / height);
let mesh = self.view.scene.meshes.iter().find(|mesh| {
mesh.panel
.as_ref()
.is_some_and(|panel| panel.key == self.key)
})?;
let panel = mesh.panel.as_ref()?;
if clipped {
let hit = self.view.pick(u, v, width / height)?;
if self.view.scene.meshes[hit.instance]
.panel
.as_ref()
.is_none_or(|panel| panel.key != self.key)
{
return None;
}
}
let ray = self.view.camera.ray(u, v, width / height)?;
let center = mesh.world.point(Vec3::default());
let normal = (mesh.world.point(Vec3::new(1., 0., 0.)) - center)
.cross(mesh.world.point(Vec3::new(0., 1., 0.)) - center);
let denominator = normal.dot(ray.direction);
if denominator.abs() < 1e-7 {
return None;
}
let distance = (center - ray.origin).dot(normal) / denominator;
if !distance.is_finite() || distance <= 0. {
return None;
}
let local = mesh
.world
.inverse()?
.point(ray.origin + ray.direction * distance);
let world_width = mesh.geometry.vertices[1].x - mesh.geometry.vertices[0].x;
let world_height = mesh.geometry.vertices[0].y - mesh.geometry.vertices[2].y;
let point = Point::new(
(local.x / world_width + 0.5) * panel.width as f32,
(0.5 - local.y / world_height) * panel.height as f32,
);
(point.x.is_finite() && point.y.is_finite()).then_some(point)
}
pub fn project_point(&self, point: Point) -> Option<Point> {
let mesh = self.view.scene.meshes.iter().find(|mesh| {
mesh.panel
.as_ref()
.is_some_and(|panel| panel.key == self.key)
})?;
let panel = mesh.panel.as_ref()?;
let width = mesh.geometry.vertices[1].x - mesh.geometry.vertices[0].x;
let height = mesh.geometry.vertices[0].y - mesh.geometry.vertices[2].y;
let world = mesh.world.point(Vec3::new(
(point.x / panel.width as f32 - 0.5) * width,
(0.5 - point.y / panel.height as f32) * height,
0.,
));
let camera = self.view.camera;
let local = camera.view(camera.basis()?, world);
if local.z <= camera.near || local.z >= camera.far {
return None;
}
let [x, y, w, h] = self.rect;
let half = match camera.projection {
Projection::Perspective { fov } => local.z * (fov.to_radians() / 2.).tan(),
Projection::Orthographic { height } => height / 2.,
};
let point = self.outer.point(Point::new(
x + (local.x / (half * w / h) + 1.) * w / 2.,
y + (1. - local.y / half) * h / 2.,
));
if let Some(parent) = &self.parent {
parent.project_point(point)
} else {
Some(point)
}
}
}
#[derive(Clone)]
struct Entry {
command: crate::document::RenderCommand,
clip: Option<[f32; 4]>,
view: Option<Transform>,
owner: u64,
}
/// Collect a panel's ordinary paint subtree into its texture. Nested 3D
/// commands are assembled recursively before their containing texture.
pub fn compose(frame: &mut crate::document::Frame) -> anyhow::Result<()> {
use crate::document::RenderCommand;
use std::collections::{HashMap, HashSet};
let mounts: HashMap<_, _> = frame.mounts.iter().map(|mount| (mount.id, mount)).collect();
let mut groups = HashMap::<(u64, u64), Vec<Entry>>::new();
let mut roots = Vec::new();
for (index, command) in frame.commands.iter().enumerate() {
let owner = frame.command_mounts[index];
let mount = mounts[&owner];
let entry = Entry {
command: command.clone(),
clip: frame.command_clips[index],
view: frame.command_views[index],
owner,
};
if let Some(panel) = &mount.panel {
groups
.entry((panel.owner, panel.key))
.or_default()
.push(entry);
} else {
roots.push(entry);
}
}
if let Some(mount) = frame.focus.and_then(|id| mounts.get(&id))
&& let Some(panel) = &mount.panel
{
groups
.entry((panel.owner, panel.key))
.or_default()
.push(Entry {
command: RenderCommand::StrokeRoundedRect {
x: mount.x,
y: mount.y,
width: mount.width,
height: mount.height,
radius: 7.,
stroke_width: 2.,
color: "#4F6FDB".into(),
},
clip: mount.clip,
view: mount.view,
owner: mount.id,
});
}
fn assemble(
entry: Entry,
groups: &HashMap<(u64, u64), Vec<Entry>>,
path: &mut HashSet<u64>,
depth: usize,
) -> anyhow::Result<Entry> {
let mut entry = entry;
if let RenderCommand::Scene3D { view, .. } = &mut entry.command
&& view.scene.meshes.iter().any(|mesh| mesh.panel.is_some())
{
anyhow::ensure!(
depth < 8 && path.insert(entry.owner),
"3D UI paint nesting cycle or depth budget exceeded"
);
let mut scene = (*view.scene).clone();
for mesh in &mut scene.meshes {
if let Some(panel) = &mut mesh.panel {
let mut surface = super::UiSurface {
commands: Vec::new(),
clips: Vec::new(),
views: Vec::new(),
};
for child in groups.get(&(entry.owner, panel.key)).into_iter().flatten() {
let child = assemble(child.clone(), groups, path, depth + 1)?;
surface.commands.push(child.command);
surface.clips.push(child.clip);
surface.views.push(child.view);
}
panel.surface = Some(Arc::new(surface));
}
}
*view = Arc::new(View {
scene: Arc::new(scene),
camera: view.camera,
});
path.remove(&entry.owner);
}
Ok(entry)
}
let mut path = HashSet::new();
frame.commands.clear();
frame.command_clips.clear();
frame.command_views.clear();
frame.command_mounts.clear();
for entry in roots {
let entry = assemble(entry, &groups, &mut path, 0)?;
frame.commands.push(entry.command);
frame.command_clips.push(entry.clip);
frame.command_views.push(entry.view);
frame.command_mounts.push(entry.owner);
}
Ok(())
}
pub fn texture(panel: &super::Panel) -> Result<Vec<u32>, &'static str> {
crate::software_renderer::reserve_pixels(panel.width as usize * panel.height as usize)?;
let mut pixels =
vec![super::rgb(panel.background); panel.width as usize * panel.height as usize];
if let Some(surface) = &panel.surface {
let document = crate::document::RenderDocument {
frame: crate::document::Frame {
background: format!("#{:06X}", super::rgb(panel.background)),
commands: surface.commands.clone(),
command_views: surface.views.clone(),
command_clips: surface.clips.clone(),
..Default::default()
},
..Default::default()
};
let list = document
.display_list(1.)
.map_err(|_| "invalid 3D UI texture commands")?;
crate::software_renderer::SoftwareRenderer::render(
&list,
&mut pixels,
panel.width,
panel.height,
)?;
}
Ok(pixels)
}
src/render/scroll.rs
+16 -2
View File
@@ -76,7 +76,12 @@ pub fn project(vm: &mut Vm, table: &MountTable, root: &UiNode, frame: &mut Frame
.mounts
.iter()
.map(|m| {
let inset = layout::insets(nodes[m.block_id.as_str()]);
let node = nodes[m.block_id.as_str()];
let inset = if matches!(node.kind, UiKind::Scroll { .. }) {
layout::insets(node)
} else {
layout::Insets::default()
};
(m.x + m.width - inset.right, m.y + m.height - inset.bottom)
})
.collect();
@@ -84,6 +89,12 @@ pub fn project(vm: &mut Vm, table: &MountTable, root: &UiNode, frame: &mut Frame
let m = &frame.mounts[i];
if let Some(parent) = m.parent {
let parent = indices[&parent];
if matches!(
nodes[frame.mounts[parent].block_id.as_str()].kind,
UiKind::CanvasViewport { .. } | UiKind::Viewport3D { .. }
) {
continue;
}
let e = if scrolling[i] {
(m.x + m.width, m.y + m.height)
} else {
@@ -166,6 +177,9 @@ pub fn project(vm: &mut Vm, table: &MountTable, root: &UiNode, frame: &mut Frame
),
);
}
if matches!(node.kind, UiKind::Viewport3D { .. }) {
child = (0., 0., None);
}
contexts.insert(m.id, child);
for command in layout::paint_commands(
nodes[m.block_id.as_str()],
@@ -259,7 +273,7 @@ pub fn into_view(
};
let mount = cx.store().payload::<Mount>(owner, "Mount")?.clone();
parent = mount.parent;
if mount.geometry.canvas_viewport {
if mount.geometry.canvas_viewport || mount.geometry.scene3d_viewport {
rect = [
mount.geometry.x,
mount.geometry.y,
src/render/software_renderer.rs
+37
View File
@@ -6,6 +6,42 @@ use crate::{
pub struct SoftwareRenderer;
thread_local! {
static RENDER_DEPTH:std::cell::Cell<usize>=const { std::cell::Cell::new(0) };
static PIXEL_UNITS:std::cell::Cell<usize>=const { std::cell::Cell::new(0) };
}
pub(crate) struct RenderScope;
impl RenderScope {
pub fn enter() -> Result<Self, &'static str> {
RENDER_DEPTH.with(|depth| {
let current = depth.get();
if current >= 18 {
return Err("render nesting budget exceeded");
}
if current == 0 {
PIXEL_UNITS.with(|pixels| pixels.set(0));
}
depth.set(current + 1);
Ok(Self)
})
}
}
impl Drop for RenderScope {
fn drop(&mut self) {
RENDER_DEPTH.with(|depth| depth.set(depth.get() - 1));
}
}
pub(crate) fn reserve_pixels(units: usize) -> Result<(), &'static str> {
PIXEL_UNITS.with(|pixels| {
let next = pixels.get().saturating_add(units);
if next > 16_777_216 {
return Err("nested render pixel budget exceeded");
}
pixels.set(next);
Ok(())
})
}
impl SoftwareRenderer {
pub fn render(
list: &DisplayList,
@@ -13,6 +49,7 @@ impl SoftwareRenderer {
width: u32,
height: u32,
) -> Result<(), &'static str> {
let _scope = RenderScope::enter()?;
let mut canvas = Canvas {
pixels: target,
width,
src/render/source_hit.rs
+3 -1
View File
@@ -80,7 +80,9 @@ fn selector(
}
let geometry = &target.geometry;
let point = crate::geometry::Point::new(point[0], point[1]);
let point = geometry.view.map_or(point, |view| view.inverse(point));
let Some(point) = geometry.logical_point(point) else {
continue;
};
let x = point.x - geometry.x;
let y = point.y - geometry.y;
if x < 0.0 || y < 0.0 || x > geometry.width || y > geometry.height {
src/render/source_layout.rs
+4 -2
View File
@@ -534,12 +534,13 @@ impl NativeCallable for Settle {
available,
Size { width, height },
);
for child in children {
for (child_index, child) in children.into_iter().enumerate() {
let natural = layout::occurrence_constraints(
&plan.nodes[index],
&plan.nodes[child],
base,
Size { width, height },
child_index,
);
let mut constraints = natural;
if let Some(width) = plan.allocations[child][0] {
@@ -887,11 +888,12 @@ pub fn start(
}
let mut constraints = vec![runtime.root_constraints(); nodes.len()];
for index in 0..nodes.len() {
for child in &children[index] {
for (child_index, child) in children[index].iter().enumerate() {
constraints[*child] = layout::initial_child_constraints(
&nodes[index],
&nodes[*child],
constraints[index],
child_index,
);
}
}
src/render/source_paint.rs
+5 -2
View File
@@ -416,7 +416,8 @@ fn finish(vm: &mut Vm, mut work: Work, reads: Vec<endor_core::ReadDependency>) -
.output
.lock()
.map_err(|_| anyhow::anyhow!("paint output poisoned"))?;
if output.commands.is_empty() {
let native_only = output.commands.is_empty();
if work.root.is_none() {
return Ok(Progress::Complete(Box::new(Draft {
frame: work.frame,
reads: Vec::new(),
@@ -465,12 +466,14 @@ fn finish(vm: &mut Vm, mut work: Work, reads: Vec<endor_core::ReadDependency>) -
frame.command_clips = clips;
frame.command_views = views;
frame.command_mounts = owners;
crate::scene3d::panel::compose(frame)?;
let registry = object(vm.get_attr(desktop, "types")?)?;
let reads = reads
.into_iter()
.filter(|dependency| {
let object = dependency.object();
object != desktop
!native_only
&& object != desktop
&& object != registry
&& vm
.object(object)
src/render/viewport.rs
+62 -6
View File
@@ -194,24 +194,46 @@ pub fn project(
) -> anyhow::Result<()> {
use crate::ui_bridge::UiKind;
use std::collections::HashMap;
use std::sync::Arc;
let mut nodes = HashMap::new();
let mut work = vec![root];
while let Some(node) = work.pop() {
nodes.insert(node.block_id.as_str(), node);
work.extend(node.children().iter().map(|node| node.as_ref()));
}
let mut contexts = HashMap::<u64, (Transform, Option<[f32; 4]>)>::new();
type Scope = (
Transform,
Option<[f32; 4]>,
Option<Arc<crate::scene3d::panel::Mapping>>,
);
let mut contexts = HashMap::<u64, Scope>::new();
let next: HashMap<_, _> = frame
.mounts
.iter()
.map(|mount| (mount.id, mount.next_sibling))
.collect();
let mut panels =
HashMap::<u64, (Arc<crate::scene3d::panel::Mapping>, crate::layout::Size)>::new();
for mount in &mut frame.mounts {
let node = nodes[mount.block_id.as_str()];
let (view, clip) = mount
.parent
.and_then(|parent| contexts.get(&parent).copied())
.unwrap_or_default();
let (view, clip, panel) = if let Some((panel, size)) = panels.get(&mount.id) {
(
Transform::default(),
Some([0., 0., size.width, size.height]),
Some(panel.clone()),
)
} else {
mount
.parent
.and_then(|parent| contexts.get(&parent).cloned())
.unwrap_or_default()
};
anyhow::ensure!(
view.valid(),
"nested viewport transform exceeds numeric budget"
);
mount.view = (view != Transform::default()).then_some(view);
mount.panel = panel.clone();
mount.clip = clip;
mount.viewport_clip = None;
let mut child_view = view;
@@ -260,7 +282,41 @@ pub fn project(
mount.viewport_clip = child_clip;
}
}
contexts.insert(mount.id, (child_view, child_clip));
if let UiKind::Viewport3D {
view: scene,
panels: sizes,
..
} = &node.kind
{
let mut depth = 0;
let mut ancestor = panel.clone();
while let Some(mapping) = ancestor {
depth += 1;
ancestor = mapping.parent.clone();
}
anyhow::ensure!(depth < 8, "3D UI viewport nesting exceeds 8 surfaces");
let mut child = mount.first_child;
for (key, size) in node.slots.iter().zip(sizes) {
let id = child.ok_or_else(|| anyhow::anyhow!("missing 3D UI root occurrence"))?;
let mapping = Arc::new(crate::scene3d::panel::Mapping {
owner: mount.id,
key: *key,
view: scene.clone(),
rect: [
mount.x + inset.left,
mount.y + inset.top,
(mount.width - inset.horizontal()).max(0.),
(mount.height - inset.vertical()).max(0.),
],
outer: view,
clip: mount.viewport_clip,
parent: panel.clone(),
});
panels.insert(id, (mapping, *size));
child = next[&id];
}
}
contexts.insert(mount.id, (child_view, child_clip, panel));
}
let mounts: HashMap<_, _> = frame.mounts.iter().map(|mount| (mount.id, mount)).collect();
frame.command_views.clear();
src/runtime/api/guide.rs
+3 -1
View File
@@ -146,7 +146,9 @@ Finite box layout: Overlay(padding, gap, children...) stacks children (gap is un
Independent 2D scenes are now implemented: Scene2D(children...) stores world content at child x/y; CanvasViewport(width,height,scene) is a finite layout box with its own Camera2D(). Camera2D(x=0,y=0,zoom=1) fields are ordinary managed attributes; zoom is 0.01–100. Sharing a scene does not share its camera; assigning second.camera=first.camera does. Scene content may contain layout containers and nested finite viewports; Overlay siblings stay fixed. Mount.fitView() fits finite direct scene-child bounds using that occurrence's allocation. Native wheel/pinch/space or middle drag targets the innermost viewport; drag retains its initial target. viewport-pan/viewport-zoom carry the viewport Mount and current frame identities, with signed logical page coordinates. Pan delta is measured before the viewport projection; zoom delta is [factor,0] with an in-content pointer anchor. Worker validation rejects stale/invalid input before invoking the VM. Draw and hit transforms compose consistently while layout geometry and measurement caches remain in world coordinates. ScrollView is a nominal ScrollBlock subtype. The default page uses these APIs; old Page.mode/panX/panY/zoom remain for compatibility. 3D support is described below. See examples/scene-viewports.endor.
3D basics are now implemented: Scene3D/Group3D, Box3D, immutable Geometry3D made from Vec3/Triangle3D ObjectLists, Mesh3D, DirectionalLight, and finite Viewport3D. PerspectiveCamera and OrthographicCamera expose x/y/z, targetX/Y/Z, upX/Y/Z, near/far, fov or height; viewport aspect is derived separately. New viewports have independent cameras and OrbitController; explicit camera assignment shares a camera. A controller can be replaced with an Endor object defining update(camera,gesture), or disabled by assigning "none". Native scene-orbit/scene-pan/scene-zoom events carry viewport Mount/frame identities. Mesh/group translate and rotate use native arithmetic; camera.translate moves position and target. This avoids the fixed Aster version's unsupported numeric arithmetic expressions. Mount.pick3D(x,y) returns nullable RayHit3D with managed mesh and world intersection data; pointer click Event.hit carries the same result. Rendering uses frustum clipping and a depth buffer in a bounded cached surface. World-space interactive UI panels and complete nesting budgets remain pending. See examples/scene3d.endor and docs/scene-architecture.md.
3D basics are now implemented: Scene3D/Group3D, Box3D, immutable Geometry3D made from Vec3/Triangle3D ObjectLists, Mesh3D, DirectionalLight, and finite Viewport3D. PerspectiveCamera and OrthographicCamera expose x/y/z, targetX/Y/Z, upX/Y/Z, near/far, fov or height; viewport aspect is derived separately. New viewports have independent cameras and OrbitController; explicit camera assignment shares a camera. A controller can be replaced with an Endor object defining update(camera,gesture), or disabled by assigning "none". Native scene-orbit/scene-pan/scene-zoom events carry viewport Mount/frame identities. Mesh/group translate and rotate use native arithmetic; camera.translate moves position and target. This avoids the fixed Aster version's unsupported numeric arithmetic expressions. Mount.pick3D(x,y) returns nullable RayHit3D with managed mesh and world intersection data; pointer click Event.hit carries the same result. Rendering uses frustum clipping and a depth buffer in a bounded cached surface. World-space UI composition is described below; visibility and resource work is continuing. See examples/scene3d.endor and docs/scene-architecture.md.
World UI is implemented with Panel3D(worldWidth,worldHeight,content). uiWidth/uiHeight provide an independent finite UI layout resolution (integer 1–2048); background is opaque. Ordinary Endor measure/arrange/paint/hitTest, shared objects, distinct Mounts and per-occurrence scroll state remain active. Panel transforms place a perspective-correct UI texture in the world. A visible ray hit maps through the plane to UI coordinates, then through any nested 2D camera; opaque geometry blocks UI input. Panel UI may contain another Viewport3D and further panels (maximum 8 world UI levels). Mount.toView(localX,localY) converts component-local coordinates to the page view; toLocal(viewX,viewY) performs the reverse when visible. Camera transport from a world UI viewport requires a pointer so pan/scroll increments can use the plane's projective mapping. Its scroll and camera gestures retain one target instead of driving both parent and child. Cyclic UI/scene composition and nested pixel allocations are bounded. See examples/world-ui.endor.
Block.flex is a nonnegative weight (default 0). Native containers reserve fixed measured children, padding and gaps, then share remaining allocated main-axis space between positive weights with zero basis. Actual child allocation feeds remeasurement. It does not request parent expansion. Further controls are described below.
src/runtime/api/tests.rs
+86
View File
@@ -756,3 +756,89 @@ fn scene3d_controllers_cross_http_and_can_be_replaced_by_endor_code() {
assert_eq!(response.0, 409);
assert!(namespace.read().unwrap().document.frame.fault.is_none());
}
#[test]
fn canvas_in_world_ui_uses_ray_uv_coordinates_for_real_http_camera_events() {
let server = Arc::new(Server::http("127.0.0.1:0").unwrap());
let namespace = Arc::new(RwLock::new(NamespaceState {
viewport: Some(crate::layout::Size {
width: 400.,
height: 300.,
}),
..NamespaceState::default()
}));
let api = start_server(server.clone(), namespace.clone(), Arc::new(|| {})).unwrap();
let client = TestServer {
address: server.server_addr().to_ip().unwrap(),
server,
api: Some(api),
};
let source = r##"
canvas=CanvasViewport(100,100,Scene2D(Button("UI in a world")))
panel=Panel3D(2,2,canvas)
panel.uiWidth=100
panel.uiHeight=100
panel.rotationY=25
view=Viewport3D(200,200,Scene3D(panel))
page=Page()
page.render(view)
"##;
let response = client.request("PUT", "/v1/namespaces/app.main/code", source);
assert_eq!(response.0, 200, "{}", response.2);
let gesture = |kind: &str, delta: [f32; 2], point: Option<[f32; 2]>| {
let current = namespace.read().unwrap();
let frame = &current.document.frame;
serde_json::to_string(&crate::events::Input {
window: 1,
session: frame.session,
layout_version: frame.layout_version,
mount: frame
.mounts
.iter()
.find(|mount| mount.canvas_viewport && mount.panel.is_some())
.unwrap()
.id,
kind: kind.into(),
point,
press_point: None,
delta: Some(delta),
tab_drop: None,
screen_position: None,
})
.unwrap()
};
let path = "/v1/namespaces/app.main/events";
assert_eq!(
client
.request("POST", path, &gesture("viewport-pan", [10., 0.], None))
.0,
400
);
let response = client.request(
"POST",
path,
&gesture("viewport-pan", [10., 0.], Some([100., 100.])),
);
assert_eq!(response.0, 200, "{}", response.2);
let value = |expression: &str| {
let response = client.request("POST", "/v1/namespaces/app.main/repl", expression);
assert_eq!(response.0, 200, "{}", response.2);
serde_json::from_str::<serde_json::Value>(&response.2).unwrap()["value"]
.as_str()
.unwrap()
.parse::<f32>()
.unwrap()
};
let pan = value("canvas.camera.x");
assert!(pan > 10.);
assert_eq!(value("view.camera.x"), 0.);
let response = client.request(
"POST",
path,
&gesture("viewport-zoom", [2., 0.], Some([100., 100.])),
);
assert_eq!(response.0, 200, "{}", response.2);
assert!((value("canvas.camera.x") - pan - 25.).abs() < 0.001);
assert_eq!(value("canvas.camera.zoom"), 2.);
assert!(namespace.read().unwrap().document.frame.fault.is_none());
}
src/runtime/api/worker.rs
+18 -4
View File
@@ -479,10 +479,12 @@ impl Worker {
let point = crate::geometry::Point::new(point[0], point[1]);
mount.contains(point)
&& mount.viewport_clip.is_some_and(|clip| {
point.x >= clip[0]
&& point.y >= clip[1]
&& point.x < clip[0] + clip[2]
&& point.y < clip[1] + clip[3]
mount.view_point(point, true).is_some_and(|point| {
point.x >= clip[0]
&& point.y >= clip[1]
&& point.x < clip[0] + clip[2]
&& point.y < clip[1] + clip[3]
})
})
})
})
@@ -492,6 +494,18 @@ impl Worker {
"zoom anchor is outside content viewport",
));
}
if (input.kind.starts_with("viewport-") || input.kind.starts_with("scene-"))
&& input.point.is_none()
&& frame
.mounts
.iter()
.any(|mount| mount.id == input.mount && mount.panel.is_some())
{
return RequestProgress::Complete(error_response(
StatusCode(400),
"world-space UI camera input requires a pointer",
));
}
Some(input)
} else {
None
src/runtime/endor_host/tests.rs
+392
View File
@@ -4924,6 +4924,398 @@ fn scene3d_example_custom_geometry_and_cycle_diagnostics_use_real_source() {
);
}
#[test]
fn world_ui_panel_renders_live_endor_and_ray_hits_respect_mesh_occlusion() {
let (mut session, initial) = ReplSession::load(
r##"
class Tile: Button {
tint: String
function paint(self: Tile, mount: Mount, geometry: Bounds): DisplayList {
fill=PaintCommand()
fill.kind="roundedRect"
fill.x=geometry.x
fill.y=geometry.y
fill.width=geometry.width
fill.height=geometry.height
fill.radius=0
fill.color=self.tint
return DisplayList(ObjectList(fill))
}
}
count=ui.state(0)
button=Tile("world UI","#FF0000")
button.onClick(count.increment)
panel=Panel3D(2,2,button)
panel.uiWidth=100
panel.uiHeight=100
panel.rotationY=25
scene=Scene3D(panel)
scene.ambient=1
view=Viewport3D(200,200,scene)
page=Page()
page.render(view)
"##,
"app.main",
314,
1,
)
.unwrap();
initial.validate().unwrap();
let button = initial
.frame
.mounts
.iter()
.find(|mount| mount.interactive && mount.panel.is_some())
.unwrap();
assert_eq!((button.width, button.height), (100., 100.));
let point = button
.panel
.as_ref()
.unwrap()
.project_point(crate::geometry::Point::new(50., 50.))
.unwrap();
assert!((point.x - 100.).abs() < 0.001 && (point.y - 100.).abs() < 0.001);
assert_eq!(initial.frame.hit_test(point).unwrap().id, button.id);
assert_eq!(
session
.execute("button.mounts().at(0).toView(50,50).x", "app.main")
.unwrap()
.value,
"100"
);
let local = session
.execute("button.mounts().at(0).toLocal(100,100).x", "app.main")
.unwrap()
.value
.parse::<f32>()
.unwrap();
assert!((local - 50.).abs() < 0.001);
let mut pixels = vec![0; 200 * 200];
crate::software_renderer::SoftwareRenderer::render(
&initial.display_list(1.).unwrap(),
&mut pixels,
200,
200,
)
.unwrap();
assert_eq!(pixels[100 * 200 + 100], 0xFF0000);
let latest = session
.execute("button.invalidate(\"paint\")", "app.main")
.unwrap()
.document
.unwrap();
let input = crate::events::Input {
window: 1,
session: latest.frame.session,
layout_version: latest.frame.layout_version,
mount: button.id,
kind: "click".into(),
point: Some([point.x, point.y]),
press_point: Some([point.x, point.y]),
delta: None,
tab_drop: None,
screen_position: None,
};
session.dispatch(&input).unwrap();
assert_eq!(
session.execute("count.value", "app.main").unwrap().value,
"1"
);
let updated = session
.execute("button.tint=\"#00FF00\"", "app.main")
.unwrap()
.document
.unwrap();
updated.validate().unwrap();
crate::software_renderer::SoftwareRenderer::render(
&updated.display_list(1.).unwrap(),
&mut pixels,
200,
200,
)
.unwrap();
assert_eq!(pixels[100 * 200 + 100], 0x00FF00);
let blocked = session
.execute(
r##"
blocker=Box3D(1.2,1.2,1.2,"#0000FF")
blocker.z=1.5
scene.children=ObjectList(panel,blocker)
"##,
"app.main",
)
.unwrap()
.document
.unwrap();
blocked.validate().unwrap();
let button = blocked
.frame
.mounts
.iter()
.find(|mount| mount.panel.is_some() && mount.interactive)
.unwrap();
assert!(!button.contains(point));
assert!(blocked.frame.hit_test(point).unwrap().scene3d_viewport);
crate::software_renderer::SoftwareRenderer::render(
&blocked.display_list(1.).unwrap(),
&mut pixels,
200,
200,
)
.unwrap();
assert_eq!(pixels[100 * 200 + 100], 0x0000FF);
}
#[test]
fn nested_world_ui_can_embed_another_3d_viewport_and_preserve_ray_dispatch() {
let (mut session, initial) = ReplSession::load(
r##"
class Actions {
function select(self: Actions,event: Event): Unit { event.hit.mesh.color="#00FF00" }
}
cube=Box3D(2,2,2,"#0000FF")
inner_scene=Scene3D(cube)
inner_scene.ambient=1
inner=Viewport3D(100,100,inner_scene)
actions=Actions()
inner.onClick(actions.select)
panel=Panel3D(2,2,inner)
panel.uiWidth=100
panel.uiHeight=100
panel.rotationY=20
outer=Viewport3D(200,200,Scene3D(panel))
page=Page()
page.render(outer)
"##,
"app.main",
315,
1,
)
.unwrap();
initial.validate().unwrap();
let inner = initial
.frame
.mounts
.iter()
.find(|mount| mount.scene3d_viewport && mount.panel.is_some())
.unwrap();
let point = crate::geometry::Point::new(100., 100.);
assert_eq!(initial.frame.camera_at(point).unwrap().id, inner.id);
assert_eq!(initial.frame.hit_test(point).unwrap().id, inner.id);
let mut pixels = vec![0; 200 * 200];
crate::software_renderer::SoftwareRenderer::render(
&initial.display_list(1.).unwrap(),
&mut pixels,
200,
200,
)
.unwrap();
assert_eq!(pixels[100 * 200 + 100], 0x0000FF);
let updated = session
.dispatch(&crate::events::Input {
window: 1,
session: initial.frame.session,
layout_version: initial.frame.layout_version,
mount: inner.id,
kind: "click".into(),
point: Some([100., 100.]),
press_point: Some([100., 100.]),
delta: None,
tab_drop: None,
screen_position: None,
})
.unwrap()
.document
.unwrap();
updated.validate().unwrap();
crate::software_renderer::SoftwareRenderer::render(
&updated.display_list(1.).unwrap(),
&mut pixels,
200,
200,
)
.unwrap();
assert_eq!(pixels[100 * 200 + 100], 0x00FF00);
}
#[test]
fn world_ui_cycles_and_surface_nesting_are_rejected_before_publication() {
let error = ReplSession::load(
r##"
panel=Panel3D(2,2,Button("cycle"))
view=Viewport3D(200,200,Scene3D(panel))
panel.content=view
page=Page()
page.render(view)
"##,
"app.main",
316,
1,
)
.err()
.unwrap();
assert!(
format!("{error:#}").contains("cyclic Block composition"),
"{error:#}"
);
let mut source = String::from("seed=Button(\"deep\")\n");
for index in 0..9 {
let child = if index == 0 {
"seed".to_string()
} else {
format!("view{}", index - 1)
};
source.push_str(&format!("panel{index}=Panel3D(2,2,{child})\npanel{index}.uiWidth=100\npanel{index}.uiHeight=100\nview{index}=Viewport3D(100,100,Scene3D(panel{index}))\n"));
}
source.push_str("page=Page()\npage.render(view8)\n");
let error = ReplSession::load(&source, "app.main", 317, 1)
.err()
.unwrap();
assert!(
format!("{error:#}").contains("3D UI viewport nesting exceeds 8"),
"{error:#}"
);
}
#[test]
fn one_ui_object_has_independent_mounts_and_texture_sizes_in_two_world_panels() {
let (mut session, initial) = ReplSession::load(
r##"
shared=Button("shared UI")
left=Panel3D(2,2,shared)
left.x=-1.2
left.uiWidth=100
left.uiHeight=80
right=Panel3D(2,2,shared)
right.x=1.2
right.uiWidth=200
right.uiHeight=120
view=Viewport3D(400,240,Scene3D(left,right))
view.camera.z=6
page=Page()
page.render(view)
"##,
"app.main",
318,
1,
)
.unwrap();
initial.validate().unwrap();
assert_eq!(
session
.execute("shared.mounts().count", "app.main")
.unwrap()
.value,
"2"
);
assert_eq!(
session
.execute("shared.mounts().at(0).bounds().width", "app.main")
.unwrap()
.value,
"100"
);
assert_eq!(
session
.execute("shared.mounts().at(1).bounds().width", "app.main")
.unwrap()
.value,
"200"
);
assert_eq!(
session
.execute("shared.mounts().at(0).bounds().height", "app.main")
.unwrap()
.value,
"80"
);
assert_eq!(
session
.execute("shared.mounts().at(1).bounds().height", "app.main")
.unwrap()
.value,
"120"
);
let mounts: Vec<_> = initial
.frame
.mounts
.iter()
.filter(|mount| mount.panel.is_some() && mount.interactive)
.collect();
let first = mounts[0]
.panel
.as_ref()
.unwrap()
.project_point(crate::geometry::Point::new(50., 40.))
.unwrap();
let second = mounts[1]
.panel
.as_ref()
.unwrap()
.project_point(crate::geometry::Point::new(100., 60.))
.unwrap();
assert!(first.x < second.x);
assert_eq!(initial.frame.hit_test(first).unwrap().id, mounts[0].id);
assert_eq!(initial.frame.hit_test(second).unwrap().id, mounts[1].id);
let mut pixels = vec![0; 400 * 240];
crate::software_renderer::SoftwareRenderer::render(
&initial.display_list(1.).unwrap(),
&mut pixels,
400,
240,
)
.unwrap();
}
#[test]
fn world_ui_example_runs_with_window_constraints_and_programmable_actions() {
let (mut session, document) = ReplSession::load_with_viewport(
include_str!("../../../examples/world-ui.endor"),
"app.main",
319,
1,
Some(crate::layout::Size {
width: 1024.,
height: 800.,
}),
)
.unwrap();
document.validate().unwrap();
assert!(
document
.frame
.mounts
.iter()
.any(|mount| mount.panel.is_some() && mount.canvas_viewport)
);
session
.execute("actions.turn()\nactions.fit()\nactions.reset()", "app.main")
.unwrap()
.document
.unwrap()
.validate()
.unwrap();
assert_eq!(
session
.execute("panel.rotationY", "app.main")
.unwrap()
.value,
"30"
);
assert_eq!(
session.execute("view.camera.z", "app.main").unwrap().value,
"7"
);
let mut pixels = vec![0; 1024 * 800];
crate::software_renderer::SoftwareRenderer::render(
&document.display_list(1.).unwrap(),
&mut pixels,
1024,
800,
)
.unwrap();
}
#[test]
fn scroll_axes_wrap_shared_text_and_clamp_only_the_enabled_offsets() {
let (mut session, initial) = ReplSession::load(
src/shell/app.rs
+3 -6
View File
@@ -675,7 +675,7 @@ impl TesseraApp {
"viewport-pan"
}
.into(),
point: (kind == "page-zoom").then_some([world.x, world.y]),
point: Some([world.x, world.y]),
press_point: None,
delta: Some([
delta[0]
@@ -1254,11 +1254,6 @@ impl ApplicationHandler<AppEvent> for TesseraApp {
.and_then(|point| current.document.frame.camera_at(point))
.is_some_and(|mount| mount.scene3d_viewport)
});
if scene3d {
self.page_view("page-zoom", [(dy * 0.005).exp().clamp(0.1, 10.), 0.]);
window.request_redraw();
return;
}
let point = self.guest_point();
let input = point.and_then(|point| {
self.namespace.read().ok().and_then(|namespace| {
@@ -1284,6 +1279,8 @@ impl ApplicationHandler<AppEvent> for TesseraApp {
});
if let Some(input) = input {
self.send_guest(input);
} else if scene3d {
self.page_view("page-zoom", [(dy * 0.005).exp().clamp(0.1, 10.), 0.]);
} else {
let horizontal = self.namespace.read().is_ok_and(|n| {
n.document.frame.surface.mode == crate::page::Mode::Horizontal