Abhinav
Golas
S. Arun Nair
Under the supervision of Prof. Subhashis
Banerjee
Problem statement:
We wish to segment a human from a video clip and determine his/her pose, and perform these 2 processes as one step.
Approach:
We follow the model proposed in the paper “PoseCut: Simultaneous Segmentation and 3D Pose Estimation of Humans using Dynamic Graph-Cuts” by Matthieu Bray, Pushmeet Kohli, and Philip H.S. Torr.
The Algorithm:
The algorithm works by modeling the segmentation and pose detection processes as one step. This is a departure from most previous attempts, and allows the process to use all of the information available in the image for the pose detection step. Also, segmentation is improved as we use a human model as a prior for the segmentation process, which allows us to get better segmentations.
The problem is modelled as a Bayesian labelling problem on MRF i.e. a Markov Random Fields. The cost function that is used depends on the background-foreground model and the pose-specific prior that we use. The pose-specific prior is got from the stickman model used. This model has 26 degrees of freedom that is used to completely describe the human body. The energy function used in this case is as follows:
where we have
and finally the pose-specific term can be expressed as:
where
The MRF is then solved using Dynamic Graph Cut. Dynamic Graph Cut is used to successively detect pose of the human being segmented out from the video.
Result:
The result of the
project can be found here.
A
complete presentation for our work can be found here
(PPT).
The videos showing segmentation can be found here:
Result1 : This video shows segmentation in a video containing a single person. (Source video)
Result2 : This video shows segmentation for cases where more than one person is in view. (Source video)