Text/Caption Augmentation for Experimental Setup Diagrams

About: In this project, we are working on a template based approach for augmenting textual description / caption provided in the textbook for experimental setup diagram, which is on of the profound category of diagrams found in STEM textbooks. This approach requires image processing to detect objects and their positionings, which can help in understanding the interaction among the objects. Further, Natural language processing techniques will be used to generate the precise description of the diagram.

Students: Akshansh Chahal (BTP 2017)

Description of Geometrical Diagrams

About: In this project, we are working on generating textual description for the geometry diagrams, which is one of the most commonly found diagram in mathematics textbooks. The objective involves to come up with a precise description which can lead to exact replication of the diagram. Further analysis will be done to make the description contextual dependent and least verbose. Image processing techniques are used for finding primitive shapes along with its properties and associated labels. Then the natural language processing techniques will be explored to generate the desired textual description.

Students: Sruti Goyal, Tuhina Verma (MTP 2019 & MiniP 2018), Jyoti (MiniP 2018)

Image Classifier

About: In this project, we are building a model to categorize images into predefined classes. Images dataset consist of diagrams from high school textbooks. Each category will have its own module which will process the diagram to make it accessible by providing an automatic description or any other mode of representation of the diagram. The classifier will be built on the neural network model.

Students: Revanth Babu Balla (BTP 2018)

Table Accessibility

About: In this project, we are working on making the table accessible by processing table images with image processing techniques to understand the structure of the table and by OCR (optical character recognition) tool to extract text present inside table structure. This content of the table will be resented into HTML format which will help screen readers for navigation purpose.

Students: Yash Malviya (NDN 2018), Sanket Hire (NDN 2018), Hadik Khichi(NDN 2018)