KWIVER Processes in ACT Detector

ACT [KWFS17]

class act_detector.ACTDetector(conf)

Forward pass for ACT

• Input Ports:

  • rgb_image RGB image (Required)
  • flow_image Flow image (Required)
  • timestamp Timestamp associated with the images (Required)
  • file_name Name of the input source (Required)

• Output Ports:

  • object_track_set Tracks produced by forward pass of RC3D

• Configuration:

  • exp Experiment configuration used by ACT (Eg. exp.yml)
  • model_itr Model number associated with with the weight file (default=60000)
  • img_width Original image width (default=1920)
  • img_height Original image height (default=1080)
  • gpu GPU index used by ACT (default=0)

class act_json_writer.ACTJsonWriter(conf)

Write object_track_set from ACT and detected_object_set from an object detector in NIST specified JSON format

• Input Ports:

  • object_track_set Tracks obtained from ACT (Required)
  • timestamp Timestamp associated with the input from which tracks were obtained (Required)
  • file_name Name of the input source (Required)
  • detected_object_set Detections obtained from object detector

• Output Ports:

  • None

• Configuration:

  • exp Experiment configuration for ACT (Eg. exp.yml)
  • is_aod Flag to specify the task for which ACT is used (Default=False)
  • confidence_threshold Lower bound for confidence associated with an activity (Default=0.2)
  • json_path Path to json file produced by the writer (default=sysfile.json)

class act_visualizer.ACTVisualizer(conf)

Render track_object_set from ACT and detected_object_set from object detector on an image

Note

The tracks visualized are intermediate tracks and not the final track.

• Input Ports:

  • image Input RGB image used by ACT (Required)
  • timestamp Timestamp associated with the image (Required)
  • object_track_set Tracks obtained from ACT (Required)
  • detected_object_set Detections obtained from object detector (Optional)

• Output Ports:

  • image Output image with bounding box representing spatial localization of an activity

• Configuration:

  • exp Experiment configuration for ACT (Eg. exp.yml)
  • is_aod Flag to specify the task for which ACT is used (Default=False)

class modify_bbox_resolution.ModifyBboxResolution(conf)

Modify the bounding box based on the difference between input and output image resolution

• Input Ports:

  • detected_object_set Set of input detections (Required)

• Output Ports:

  • detected_object_set Set of scaled detections (Optional)

• Configuration:

  • input_image_width Width of the image from which input detections were generated (default=512)
  • input_image_height Height of the image from which input detections were generated (default=512)
  • output_image_width Width of the image that output detections would be scaled to (default=1920)
  • output_image_height Height of the image that output detections would be scaled to (default=1080)

class merge_tubes.MergeTubes(conf)

Merge tubelets produced by ACT to create larger tubes.

• Input Ports:

  • object_track_set Intermediate detections from ACT forward pass (Required)
  • file_name Input source (Required)
  • timestamp Timestamp assocaited with the input from which intermediated tracks were computed (Required)

• Output Ports:

  • object_track_set Finished tubes
  • current_object_track_set Incomplete tubes

• Configuration

  • exp Experiment configuration for ACT (Eg. exp.yml)
  • num_classes Number of classes that ACT was trained on (default=20)

[KWFS17]

Vicky Kalogeiton, Philippe Weinzaepfel, Vittorio Ferrari, and Cordelia Schmid. Action Tubelet Detector for Spatio-Temporal Action Localization. In ICCV 2017 - IEEE International Conference on Computer Vision. Venice, Italy, October 2017.