Pen Stealing Robot
Computer Vision, OpenCV, Python, PincherX 100, Intel Realsense
As part of Northwestern’s orientation hackathon, I used an Intel Realsense D435i camera to detect the location of a purple Northwestern pen and grab it with a Trossen PincherX 100 robot arm. To detect the pen, I converted the RGB image from the Realsense to an HSV image and used the HSV values to find all purple pixels. Next, I created a binary map where the detected pixels were shown as white and all other pixels black. Using this and OpenCv’s contour detection I found the pixel location of the centroid of pen. This allowed me to find the location of the pen relative to the camera using the aligned depth image generated from the Realsense. I then convert the pen position from the camera’s frame to the robot’s frame and control the robot to move to the pen and grasp it.