Attach and detach geometries at runtime

After a grasp, the object becomes part of the robot: it rides the gripper, it can collide with obstacles, and the planner needs to account for it on every subsequent motion. WorldState.transforms is how you tell the planner that. You attach the object’s geometry to the gripper’s frame, pass the world state on every Move while the object is held, and stop passing it after release.

This is the runtime complement to the passive objects pattern, which covers objects that are always attached (camera mounts, adapter plates). Use this page’s pattern for objects the robot grasps and releases dynamically.

Before you start

• A working motion service call (motion.Move for arms, a grasp-capable gripper configured on the machine).
• The grasped object’s rough dimensions. You can approximate with a box, sphere, or capsule.
• The offset between the gripper frame origin and the object’s center (usually close to the gripper’s tool center point when the object is held centered).

The Transform message

WorldState.transforms is a list of Transform entries. Each entry defines a new frame and optionally attaches a geometry to it:

Set pose_in_observer_frame.reference_frame to the parent frame name (for example, my-gripper). The pose inside pose_in_observer_frame is the new frame’s position and orientation relative to that parent.

Pattern: attach on grasp, detach on release

1. After a successful grasp, build a Transform whose parent is the gripper, with a geometry approximating the object.
2. On every subsequent motion.Move call while the object is held, include the transform in WorldState.transforms.
3. After release, stop including the transform. No explicit detach call is needed: WorldState is per-call, so the next Move without the transform leaves the object out of the planner’s world.

Example: attach a grasped box

from viam.proto.common import (
    Pose, PoseInFrame, Vector3, RectangularPrism,
    Geometry, GeometriesInFrame, Transform, WorldState,
)

# Object: a 80mm x 80mm x 100mm box centered 50mm below the gripper origin.
grasped_object = Transform(
    reference_frame="grasped-object",
    pose_in_observer_frame=PoseInFrame(
        reference_frame="my-gripper",
        pose=Pose(x=0, y=0, z=50, o_x=0, o_y=0, o_z=1, theta=0),
    ),
    physical_object=Geometry(
        center=Pose(x=0, y=0, z=0),
        box=RectangularPrism(dims_mm=Vector3(x=80, y=80, z=100)),
        label="grasped-box",
    ),
)

# Static obstacles already known (for example, the table the arm is mounted on).
table = Geometry(
    center=Pose(x=0, y=0, z=-20),
    box=RectangularPrism(dims_mm=Vector3(x=800, y=600, z=40)),
    label="table",
)
obstacles = GeometriesInFrame(
    reference_frame="world",
    geometries=[table],
)

world_state = WorldState(
    obstacles=[obstacles],
    transforms=[grasped_object],
)

# Use world_state on every Move call while the object is held.
await motion_service.move(
    component_name="my-arm",
    destination=destination,
    world_state=world_state,
)

import (
    "github.com/golang/geo/r3"
    commonpb "go.viam.com/api/common/v1"
    "go.viam.com/rdk/referenceframe"
    "go.viam.com/rdk/spatialmath"
    "go.viam.com/rdk/services/motion"
)

// Object: a 80mm x 80mm x 100mm box centered 50mm below the gripper origin.
grasped, err := spatialmath.NewBox(
    spatialmath.NewZeroPose(),
    r3.Vector{X: 80, Y: 80, Z: 100},
    "grasped-box",
)
if err != nil {
    logger.Fatal(err)
}

graspedPose := spatialmath.NewPoseFromPoint(r3.Vector{X: 0, Y: 0, Z: 50})

graspedTransform := referenceframe.NewLinkInFrame(
    "my-gripper",
    graspedPose,
    "grasped-object",
    grasped,
)

// Static obstacles already known.
tableOrigin := spatialmath.NewPose(
    r3.Vector{X: 0, Y: 0, Z: -20},
    &spatialmath.OrientationVectorDegrees{OZ: 1},
)
table, _ := spatialmath.NewBox(tableOrigin, r3.Vector{X: 800, Y: 600, Z: 40}, "table")

obstaclesInFrame := referenceframe.NewGeometriesInFrame(
    referenceframe.World,
    []spatialmath.Geometry{table},
)

worldState, err := referenceframe.NewWorldState(
    []*referenceframe.GeometriesInFrame{obstaclesInFrame},
    []*referenceframe.LinkInFrame{graspedTransform},
)
if err != nil {
    logger.Fatal(err)
}

_, err = motionService.Move(ctx, motion.MoveReq{
    ComponentName: "my-arm",
    Destination:   destination,
    WorldState:    worldState,
})

Detach: omit the transform from the next call

The API has no “detach” call because WorldState is per-request. After release, build the next WorldState without the grasped-object transform. The planner sees an empty world for that frame from that call forward.

world_state = WorldState(obstacles=[obstacles])  # no transforms

await motion_service.move(
    component_name="my-arm",
    destination=after_release_pose,
    world_state=world_state,
)

The planner’s view of the world for this and subsequent calls does not include the grasped object.

Allow contact while grasping

During the grasp motion itself, the gripper must come into contact with the object. The planner rejects paths with any collision by default, so allow the gripper-to-object pair:

from viam.proto.service.motion import Constraints, CollisionSpecification

constraints = Constraints(
    collision_specification=[
        CollisionSpecification(
            allows=[
                CollisionSpecification.AllowedFrameCollisions(
                    frame1="my-gripper",
                    frame2="target-object",
                ),
            ],
        ),
    ],
)

For the full pattern, see Allow specific frames to collide.

Compared to passive attachment

Troubleshooting

What’s next

• Define obstacles: the static obstacles you pass alongside grasped-object transforms.
• Allow specific frames to collide: permit the contact between gripper and object that the grasp requires.
• Pick an object: end-to-end grasp flow that uses this pattern.
• Frame system API: the TransformPose method, which uses the same supplemental transforms mechanism for one-off pose conversions.