In this paper, we have proposed probabilistic parser for identifying the face in a given scene image. Many object detection techniques use pattern statistical methods for feature extraction which is resource intensive and time consuming. We proposed a novel certainty factor based geometrical formulation for facial feature extraction. The proposed method accurately detects the facial components like eyes, nose and mouth in the presence of complex background. In the next stage, the AND/OR graph based recursive Top-down/Bottom-up image parser is used to detect the face in the input image by using the detected facial components. The image parser grammar represents both the decomposition of the scene image and the context for spatial relation by horizontal link between the vertices of the graph. The AND/OR graph is used to represent compositional structure of the image. The AND node represents the decomposition of the visual object into number of components and OR node represents the alternative sub-configuration /component. The experimental result confirms that our method outperforms some of the existing face detection methods.