feat: compose interactive world ui across nested 3d viewports

This commit is contained in:
2026-11-11 00:08:23 +08:00
parent ad2c01bc29
commit 15fc01542c
27 changed files with 1412 additions and 84 deletions
+3 -1
View File
@@ -146,7 +146,9 @@ Finite box layout: Overlay(padding, gap, children...) stacks children (gap is un
Independent 2D scenes are now implemented: Scene2D(children...) stores world content at child x/y; CanvasViewport(width,height,scene) is a finite layout box with its own Camera2D(). Camera2D(x=0,y=0,zoom=1) fields are ordinary managed attributes; zoom is 0.01–100. Sharing a scene does not share its camera; assigning second.camera=first.camera does. Scene content may contain layout containers and nested finite viewports; Overlay siblings stay fixed. Mount.fitView() fits finite direct scene-child bounds using that occurrence's allocation. Native wheel/pinch/space or middle drag targets the innermost viewport; drag retains its initial target. viewport-pan/viewport-zoom carry the viewport Mount and current frame identities, with signed logical page coordinates. Pan delta is measured before the viewport projection; zoom delta is [factor,0] with an in-content pointer anchor. Worker validation rejects stale/invalid input before invoking the VM. Draw and hit transforms compose consistently while layout geometry and measurement caches remain in world coordinates. ScrollView is a nominal ScrollBlock subtype. The default page uses these APIs; old Page.mode/panX/panY/zoom remain for compatibility. 3D support is described below. See examples/scene-viewports.endor.
3D basics are now implemented: Scene3D/Group3D, Box3D, immutable Geometry3D made from Vec3/Triangle3D ObjectLists, Mesh3D, DirectionalLight, and finite Viewport3D. PerspectiveCamera and OrthographicCamera expose x/y/z, targetX/Y/Z, upX/Y/Z, near/far, fov or height; viewport aspect is derived separately. New viewports have independent cameras and OrbitController; explicit camera assignment shares a camera. A controller can be replaced with an Endor object defining update(camera,gesture), or disabled by assigning "none". Native scene-orbit/scene-pan/scene-zoom events carry viewport Mount/frame identities. Mesh/group translate and rotate use native arithmetic; camera.translate moves position and target. This avoids the fixed Aster version's unsupported numeric arithmetic expressions. Mount.pick3D(x,y) returns nullable RayHit3D with managed mesh and world intersection data; pointer click Event.hit carries the same result. Rendering uses frustum clipping and a depth buffer in a bounded cached surface. World-space interactive UI panels and complete nesting budgets remain pending. See examples/scene3d.endor and docs/scene-architecture.md.
3D basics are now implemented: Scene3D/Group3D, Box3D, immutable Geometry3D made from Vec3/Triangle3D ObjectLists, Mesh3D, DirectionalLight, and finite Viewport3D. PerspectiveCamera and OrthographicCamera expose x/y/z, targetX/Y/Z, upX/Y/Z, near/far, fov or height; viewport aspect is derived separately. New viewports have independent cameras and OrbitController; explicit camera assignment shares a camera. A controller can be replaced with an Endor object defining update(camera,gesture), or disabled by assigning "none". Native scene-orbit/scene-pan/scene-zoom events carry viewport Mount/frame identities. Mesh/group translate and rotate use native arithmetic; camera.translate moves position and target. This avoids the fixed Aster version's unsupported numeric arithmetic expressions. Mount.pick3D(x,y) returns nullable RayHit3D with managed mesh and world intersection data; pointer click Event.hit carries the same result. Rendering uses frustum clipping and a depth buffer in a bounded cached surface. World-space UI composition is described below; visibility and resource work is continuing. See examples/scene3d.endor and docs/scene-architecture.md.
World UI is implemented with Panel3D(worldWidth,worldHeight,content). uiWidth/uiHeight provide an independent finite UI layout resolution (integer 1–2048); background is opaque. Ordinary Endor measure/arrange/paint/hitTest, shared objects, distinct Mounts and per-occurrence scroll state remain active. Panel transforms place a perspective-correct UI texture in the world. A visible ray hit maps through the plane to UI coordinates, then through any nested 2D camera; opaque geometry blocks UI input. Panel UI may contain another Viewport3D and further panels (maximum 8 world UI levels). Mount.toView(localX,localY) converts component-local coordinates to the page view; toLocal(viewX,viewY) performs the reverse when visible. Camera transport from a world UI viewport requires a pointer so pan/scroll increments can use the plane's projective mapping. Its scroll and camera gestures retain one target instead of driving both parent and child. Cyclic UI/scene composition and nested pixel allocations are bounded. See examples/world-ui.endor.
Block.flex is a nonnegative weight (default 0). Native containers reserve fixed measured children, padding and gaps, then share remaining allocated main-axis space between positive weights with zero basis. Actual child allocation feeds remeasurement. It does not request parent expansion. Further controls are described below.
+86
View File
@@ -756,3 +756,89 @@ fn scene3d_controllers_cross_http_and_can_be_replaced_by_endor_code() {
assert_eq!(response.0, 409);
assert!(namespace.read().unwrap().document.frame.fault.is_none());
}
#[test]
fn canvas_in_world_ui_uses_ray_uv_coordinates_for_real_http_camera_events() {
let server = Arc::new(Server::http("127.0.0.1:0").unwrap());
let namespace = Arc::new(RwLock::new(NamespaceState {
viewport: Some(crate::layout::Size {
width: 400.,
height: 300.,
}),
..NamespaceState::default()
}));
let api = start_server(server.clone(), namespace.clone(), Arc::new(|| {})).unwrap();
let client = TestServer {
address: server.server_addr().to_ip().unwrap(),
server,
api: Some(api),
};
let source = r##"
canvas=CanvasViewport(100,100,Scene2D(Button("UI in a world")))
panel=Panel3D(2,2,canvas)
panel.uiWidth=100
panel.uiHeight=100
panel.rotationY=25
view=Viewport3D(200,200,Scene3D(panel))
page=Page()
page.render(view)
"##;
let response = client.request("PUT", "/v1/namespaces/app.main/code", source);
assert_eq!(response.0, 200, "{}", response.2);
let gesture = |kind: &str, delta: [f32; 2], point: Option<[f32; 2]>| {
let current = namespace.read().unwrap();
let frame = &current.document.frame;
serde_json::to_string(&crate::events::Input {
window: 1,
session: frame.session,
layout_version: frame.layout_version,
mount: frame
.mounts
.iter()
.find(|mount| mount.canvas_viewport && mount.panel.is_some())
.unwrap()
.id,
kind: kind.into(),
point,
press_point: None,
delta: Some(delta),
tab_drop: None,
screen_position: None,
})
.unwrap()
};
let path = "/v1/namespaces/app.main/events";
assert_eq!(
client
.request("POST", path, &gesture("viewport-pan", [10., 0.], None))
.0,
400
);
let response = client.request(
"POST",
path,
&gesture("viewport-pan", [10., 0.], Some([100., 100.])),
);
assert_eq!(response.0, 200, "{}", response.2);
let value = |expression: &str| {
let response = client.request("POST", "/v1/namespaces/app.main/repl", expression);
assert_eq!(response.0, 200, "{}", response.2);
serde_json::from_str::<serde_json::Value>(&response.2).unwrap()["value"]
.as_str()
.unwrap()
.parse::<f32>()
.unwrap()
};
let pan = value("canvas.camera.x");
assert!(pan > 10.);
assert_eq!(value("view.camera.x"), 0.);
let response = client.request(
"POST",
path,
&gesture("viewport-zoom", [2., 0.], Some([100., 100.])),
);
assert_eq!(response.0, 200, "{}", response.2);
assert!((value("canvas.camera.x") - pan - 25.).abs() < 0.001);
assert_eq!(value("canvas.camera.zoom"), 2.);
assert!(namespace.read().unwrap().document.frame.fault.is_none());
}
+18 -4
View File
@@ -479,10 +479,12 @@ impl Worker {
let point = crate::geometry::Point::new(point[0], point[1]);
mount.contains(point)
&& mount.viewport_clip.is_some_and(|clip| {
point.x >= clip[0]
&& point.y >= clip[1]
&& point.x < clip[0] + clip[2]
&& point.y < clip[1] + clip[3]
mount.view_point(point, true).is_some_and(|point| {
point.x >= clip[0]
&& point.y >= clip[1]
&& point.x < clip[0] + clip[2]
&& point.y < clip[1] + clip[3]
})
})
})
})
@@ -492,6 +494,18 @@ impl Worker {
"zoom anchor is outside content viewport",
));
}
if (input.kind.starts_with("viewport-") || input.kind.starts_with("scene-"))
&& input.point.is_none()
&& frame
.mounts
.iter()
.any(|mount| mount.id == input.mount && mount.panel.is_some())
{
return RequestProgress::Complete(error_response(
StatusCode(400),
"world-space UI camera input requires a pointer",
));
}
Some(input)
} else {
None
+392
View File
@@ -4924,6 +4924,398 @@ fn scene3d_example_custom_geometry_and_cycle_diagnostics_use_real_source() {
);
}
#[test]
fn world_ui_panel_renders_live_endor_and_ray_hits_respect_mesh_occlusion() {
let (mut session, initial) = ReplSession::load(
r##"
class Tile: Button {
tint: String
function paint(self: Tile, mount: Mount, geometry: Bounds): DisplayList {
fill=PaintCommand()
fill.kind="roundedRect"
fill.x=geometry.x
fill.y=geometry.y
fill.width=geometry.width
fill.height=geometry.height
fill.radius=0
fill.color=self.tint
return DisplayList(ObjectList(fill))
}
}
count=ui.state(0)
button=Tile("world UI","#FF0000")
button.onClick(count.increment)
panel=Panel3D(2,2,button)
panel.uiWidth=100
panel.uiHeight=100
panel.rotationY=25
scene=Scene3D(panel)
scene.ambient=1
view=Viewport3D(200,200,scene)
page=Page()
page.render(view)
"##,
"app.main",
314,
1,
)
.unwrap();
initial.validate().unwrap();
let button = initial
.frame
.mounts
.iter()
.find(|mount| mount.interactive && mount.panel.is_some())
.unwrap();
assert_eq!((button.width, button.height), (100., 100.));
let point = button
.panel
.as_ref()
.unwrap()
.project_point(crate::geometry::Point::new(50., 50.))
.unwrap();
assert!((point.x - 100.).abs() < 0.001 && (point.y - 100.).abs() < 0.001);
assert_eq!(initial.frame.hit_test(point).unwrap().id, button.id);
assert_eq!(
session
.execute("button.mounts().at(0).toView(50,50).x", "app.main")
.unwrap()
.value,
"100"
);
let local = session
.execute("button.mounts().at(0).toLocal(100,100).x", "app.main")
.unwrap()
.value
.parse::<f32>()
.unwrap();
assert!((local - 50.).abs() < 0.001);
let mut pixels = vec![0; 200 * 200];
crate::software_renderer::SoftwareRenderer::render(
&initial.display_list(1.).unwrap(),
&mut pixels,
200,
200,
)
.unwrap();
assert_eq!(pixels[100 * 200 + 100], 0xFF0000);
let latest = session
.execute("button.invalidate(\"paint\")", "app.main")
.unwrap()
.document
.unwrap();
let input = crate::events::Input {
window: 1,
session: latest.frame.session,
layout_version: latest.frame.layout_version,
mount: button.id,
kind: "click".into(),
point: Some([point.x, point.y]),
press_point: Some([point.x, point.y]),
delta: None,
tab_drop: None,
screen_position: None,
};
session.dispatch(&input).unwrap();
assert_eq!(
session.execute("count.value", "app.main").unwrap().value,
"1"
);
let updated = session
.execute("button.tint=\"#00FF00\"", "app.main")
.unwrap()
.document
.unwrap();
updated.validate().unwrap();
crate::software_renderer::SoftwareRenderer::render(
&updated.display_list(1.).unwrap(),
&mut pixels,
200,
200,
)
.unwrap();
assert_eq!(pixels[100 * 200 + 100], 0x00FF00);
let blocked = session
.execute(
r##"
blocker=Box3D(1.2,1.2,1.2,"#0000FF")
blocker.z=1.5
scene.children=ObjectList(panel,blocker)
"##,
"app.main",
)
.unwrap()
.document
.unwrap();
blocked.validate().unwrap();
let button = blocked
.frame
.mounts
.iter()
.find(|mount| mount.panel.is_some() && mount.interactive)
.unwrap();
assert!(!button.contains(point));
assert!(blocked.frame.hit_test(point).unwrap().scene3d_viewport);
crate::software_renderer::SoftwareRenderer::render(
&blocked.display_list(1.).unwrap(),
&mut pixels,
200,
200,
)
.unwrap();
assert_eq!(pixels[100 * 200 + 100], 0x0000FF);
}
#[test]
fn nested_world_ui_can_embed_another_3d_viewport_and_preserve_ray_dispatch() {
let (mut session, initial) = ReplSession::load(
r##"
class Actions {
function select(self: Actions,event: Event): Unit { event.hit.mesh.color="#00FF00" }
}
cube=Box3D(2,2,2,"#0000FF")
inner_scene=Scene3D(cube)
inner_scene.ambient=1
inner=Viewport3D(100,100,inner_scene)
actions=Actions()
inner.onClick(actions.select)
panel=Panel3D(2,2,inner)
panel.uiWidth=100
panel.uiHeight=100
panel.rotationY=20
outer=Viewport3D(200,200,Scene3D(panel))
page=Page()
page.render(outer)
"##,
"app.main",
315,
1,
)
.unwrap();
initial.validate().unwrap();
let inner = initial
.frame
.mounts
.iter()
.find(|mount| mount.scene3d_viewport && mount.panel.is_some())
.unwrap();
let point = crate::geometry::Point::new(100., 100.);
assert_eq!(initial.frame.camera_at(point).unwrap().id, inner.id);
assert_eq!(initial.frame.hit_test(point).unwrap().id, inner.id);
let mut pixels = vec![0; 200 * 200];
crate::software_renderer::SoftwareRenderer::render(
&initial.display_list(1.).unwrap(),
&mut pixels,
200,
200,
)
.unwrap();
assert_eq!(pixels[100 * 200 + 100], 0x0000FF);
let updated = session
.dispatch(&crate::events::Input {
window: 1,
session: initial.frame.session,
layout_version: initial.frame.layout_version,
mount: inner.id,
kind: "click".into(),
point: Some([100., 100.]),
press_point: Some([100., 100.]),
delta: None,
tab_drop: None,
screen_position: None,
})
.unwrap()
.document
.unwrap();
updated.validate().unwrap();
crate::software_renderer::SoftwareRenderer::render(
&updated.display_list(1.).unwrap(),
&mut pixels,
200,
200,
)
.unwrap();
assert_eq!(pixels[100 * 200 + 100], 0x00FF00);
}
#[test]
fn world_ui_cycles_and_surface_nesting_are_rejected_before_publication() {
let error = ReplSession::load(
r##"
panel=Panel3D(2,2,Button("cycle"))
view=Viewport3D(200,200,Scene3D(panel))
panel.content=view
page=Page()
page.render(view)
"##,
"app.main",
316,
1,
)
.err()
.unwrap();
assert!(
format!("{error:#}").contains("cyclic Block composition"),
"{error:#}"
);
let mut source = String::from("seed=Button(\"deep\")\n");
for index in 0..9 {
let child = if index == 0 {
"seed".to_string()
} else {
format!("view{}", index - 1)
};
source.push_str(&format!("panel{index}=Panel3D(2,2,{child})\npanel{index}.uiWidth=100\npanel{index}.uiHeight=100\nview{index}=Viewport3D(100,100,Scene3D(panel{index}))\n"));
}
source.push_str("page=Page()\npage.render(view8)\n");
let error = ReplSession::load(&source, "app.main", 317, 1)
.err()
.unwrap();
assert!(
format!("{error:#}").contains("3D UI viewport nesting exceeds 8"),
"{error:#}"
);
}
#[test]
fn one_ui_object_has_independent_mounts_and_texture_sizes_in_two_world_panels() {
let (mut session, initial) = ReplSession::load(
r##"
shared=Button("shared UI")
left=Panel3D(2,2,shared)
left.x=-1.2
left.uiWidth=100
left.uiHeight=80
right=Panel3D(2,2,shared)
right.x=1.2
right.uiWidth=200
right.uiHeight=120
view=Viewport3D(400,240,Scene3D(left,right))
view.camera.z=6
page=Page()
page.render(view)
"##,
"app.main",
318,
1,
)
.unwrap();
initial.validate().unwrap();
assert_eq!(
session
.execute("shared.mounts().count", "app.main")
.unwrap()
.value,
"2"
);
assert_eq!(
session
.execute("shared.mounts().at(0).bounds().width", "app.main")
.unwrap()
.value,
"100"
);
assert_eq!(
session
.execute("shared.mounts().at(1).bounds().width", "app.main")
.unwrap()
.value,
"200"
);
assert_eq!(
session
.execute("shared.mounts().at(0).bounds().height", "app.main")
.unwrap()
.value,
"80"
);
assert_eq!(
session
.execute("shared.mounts().at(1).bounds().height", "app.main")
.unwrap()
.value,
"120"
);
let mounts: Vec<_> = initial
.frame
.mounts
.iter()
.filter(|mount| mount.panel.is_some() && mount.interactive)
.collect();
let first = mounts[0]
.panel
.as_ref()
.unwrap()
.project_point(crate::geometry::Point::new(50., 40.))
.unwrap();
let second = mounts[1]
.panel
.as_ref()
.unwrap()
.project_point(crate::geometry::Point::new(100., 60.))
.unwrap();
assert!(first.x < second.x);
assert_eq!(initial.frame.hit_test(first).unwrap().id, mounts[0].id);
assert_eq!(initial.frame.hit_test(second).unwrap().id, mounts[1].id);
let mut pixels = vec![0; 400 * 240];
crate::software_renderer::SoftwareRenderer::render(
&initial.display_list(1.).unwrap(),
&mut pixels,
400,
240,
)
.unwrap();
}
#[test]
fn world_ui_example_runs_with_window_constraints_and_programmable_actions() {
let (mut session, document) = ReplSession::load_with_viewport(
include_str!("../../../examples/world-ui.endor"),
"app.main",
319,
1,
Some(crate::layout::Size {
width: 1024.,
height: 800.,
}),
)
.unwrap();
document.validate().unwrap();
assert!(
document
.frame
.mounts
.iter()
.any(|mount| mount.panel.is_some() && mount.canvas_viewport)
);
session
.execute("actions.turn()\nactions.fit()\nactions.reset()", "app.main")
.unwrap()
.document
.unwrap()
.validate()
.unwrap();
assert_eq!(
session
.execute("panel.rotationY", "app.main")
.unwrap()
.value,
"30"
);
assert_eq!(
session.execute("view.camera.z", "app.main").unwrap().value,
"7"
);
let mut pixels = vec![0; 1024 * 800];
crate::software_renderer::SoftwareRenderer::render(
&document.display_list(1.).unwrap(),
&mut pixels,
1024,
800,
)
.unwrap();
}
#[test]
fn scroll_axes_wrap_shared_text_and_clamp_only_the_enabled_offsets() {
let (mut session, initial) = ReplSession::load(