Configure a color_detector

The color_detector vision service is a heuristic detector that draws boxes around connected regions of a specified hue. It runs entirely on the machine with no ML model. Use it for any task where the target stands out by color: red objects on a conveyor, green plants against soil, a blue marker against a wall.

The detector cannot detect black, white, or perfect grays (pixels whose red, green, and blue values are equal). It only detects hues on the color wheel.

Configure

• Builder
• JSON Template
• JSON Example

1. Navigate to the CONFIGURE tab of your machine’s page.
2. Click the + icon next to your machine part and select Configuration block.
3. In the search field, type color detector and select the vision/color_detector result.
4. Click Add component, enter a name, and click Add component again to confirm.
5. Choose a color and a hue tolerance, then set a segment size in pixels.
6. Select a default camera.

"services": [
  {
    "name": "<service_name>",
    "api": "rdk:service:vision",
    "model": "color_detector",
    "attributes": {
      "detect_color": "#RRGGBB",
      "hue_tolerance_pct": <number>,
      "segment_size_px": <integer>,
      "saturation_cutoff_pct": <number>,
      "value_cutoff_pct": <number>,
      "label": "<label>",
      "camera_name": "<camera-name>"
    }
  }
]

"services": [
  {
    "name": "blue_square",
    "api": "rdk:service:vision",
    "model": "color_detector",
    "attributes": {
      "detect_color": "#1C4599",
      "hue_tolerance_pct": 0.07,
      "segment_size_px": 100,
      "value_cutoff_pct": 0.15,
      "label": "blue",
      "camera_name": "camera-1"
    }
  },
  {
    "name": "green_triangle",
    "api": "rdk:service:vision",
    "model": "color_detector",
    "attributes": {
      "detect_color": "#62963F",
      "hue_tolerance_pct": 0.05,
      "segment_size_px": 200,
      "value_cutoff_pct": 0.20,
      "label": "green",
      "camera_name": "camera-1"
    }
  }
]

Attributes

Test your detector

Live camera footage

1. Open your machine in the Viam app and either click the vision service’s Test area or navigate to the CONTROL tab and select the vision service.
2. In the Camera dropdown, select the camera whose feed you want the detector to run on. Detections appear as bounding boxes on the live camera feed and refresh automatically.

For a continuous overlay, configure a transform camera:

{
  "pipeline": [
    {
      "type": "detections",
      "attributes": {
        "confidence_threshold": 0.5,
        "detector_name": "<vision-service-name>",
        "valid_labels": ["<label>"]
      }
    }
  ],
  "source": "<camera-name>"
}

Code

from viam.components.camera import Camera
from viam.services.vision import VisionClient

robot = await connect()
camera_name = "camera-1"

cam = Camera.from_robot(robot, camera_name)
my_detector = VisionClient.from_robot(robot, "blue_square")

# Get detections from the camera in one call
detections = await my_detector.get_detections_from_camera(camera_name)

# Or capture an image first, then run detections on it
images, _ = await cam.get_images()
img = images[0]
detections_from_image = await my_detector.get_detections(img)

await robot.close()

import (
  "go.viam.com/rdk/components/camera"
  "go.viam.com/rdk/services/vision"
)

cameraName := "camera-1"
myCam, err := camera.FromProvider(machine, cameraName)
if err != nil {
  logger.Fatalf("cannot get camera: %v", err)
}

myDetector, err := vision.FromProvider(machine, "blue_square")
if err != nil {
  logger.Fatalf("cannot get vision service: %v", err)
}

// Get detections from the camera in one call
detections, err := myDetector.DetectionsFromCamera(context.Background(), cameraName, nil)
if err != nil {
  logger.Fatalf("could not get detections: %v", err)
}
if len(detections) > 0 {
  logger.Info(detections[0])
}

// Or capture an image first, then run detections on it
img, err := camera.DecodeImageFromCamera(context.Background(), myCam, nil, nil)
if err != nil {
  logger.Fatalf("could not decode image: %v", err)
}
detectionsFromImage, err := myDetector.Detections(context.Background(), img, nil)
if err != nil {
  logger.Fatalf("could not get detections: %v", err)
}
if len(detectionsFromImage) > 0 {
  logger.Info(detectionsFromImage[0])
}

Troubleshooting

Next steps

Detect objects

Retrieve 2D bounding-box detections from a vision service, filter by confidence and class, and run detections in a loop.

Configure a vision pipeline

Wire up an ML model service and a vision service so your machine's camera can produce detections, classifications, or 3D point cloud objects.

mlmodel

Configure the mlmodel vision service to turn a deployed ML model into a detector, classifier, or 3D segmenter.