COL864: Advanced Topics in AI (Information Extraction, Question Answering & Dialog) - Spring 2017
Monday, Thursday 2-3:20 pm in SIT 113

Instructor: Mausam
(mausam at cse dot iitd dot ac dot in)
Office hours: by appointment, SIT Building Room 402
TA: Ankit Anand
(ankit.anand at cse dot iitd dot ac dot in)
Office hours: by appointment

Course Contents

Information Extraction: bootstrapping, NELL, Open IE, relation kernels, distant supervision, deep learning for IE, reinforcement learning for IE.

Inference over KBs: Horn clause inference, random walks over graphs, neural models.

Question Answering: open question answering, semantic parsing, neural models.

Dialog: goal-oriented dialog, chatbots.


  1. Administrivia.

  2. Information Extraction

  3. Topic: Bootstrapping
    Reading: Snowball: Extracting Relations from Large Plain-Text Collections.
    [Slides] Concepts discussed: bootstrapping, semantic draft, zipf distribution.
  4. Topic: NELL
    Reading: Never-Ending Learning.
    Additional Reading: Toward an Architecture for Never-Ending Language Learning.
    [Slides] Concepts discussed: co-training, multi-task learning, coupled semi-supervised learning, macro reading vs micro reading.
  5. Topic: Open IE
    Reading: Open Information Extraction: the Second Generation.
    Additional Reading: Open Language Learning for Information Extraction.
    Additional Reading (section 2): Open Information Extraction Systems and Downstream Applications.
    [Slides] Concepts discussed: lexicalized vs unlexicalized; types of semantics: distributional semantics, frame semantics, full semantic parsing; Davidsonian semantics.
  6. Topic: Tree Kernels
    Reading: A Shortest Path Dependency Kernel for Relation Extraction.
    Additional Reading: Kernel Methods for Relation Extraction.
    Background Viewing: Michael Collins's videos on dependency parsing.
    [Slides] Concepts discussed: dependency parsing, combinatory categorial grammar, supervised vs. semi-supervised vs. unsupervised IE, tree kernels.
  7. Topic: CRFs
    Reading: Semi-Markov Conditional Random Fields for Information Extraction.
    [Slides] Concepts discussed: CRFs, Semi-CRFs, Higher-order CRFs, Viterbi, BIO encodings, Newton's method for optimization.
  8. Topic: Distant supervision
    Reading: Knowledge-Based Weak Supervision for Information Extraction of Overlapping Relations.
    Additional Reading: Modeling Missing Data in Distant Supervision for Information Extraction.
    Background Reading: Distant supervision for relation extraction without labeled data.
    [Slides] Concepts discussed: macro vs micro reading, voted perceptron, stochastic gradient descent, multi-instance learning, self learning.
  9. Topic: CNNs
    Reading: Distant Supervision for Relation Extraction via Piecewise Convolutional Neural Networks.
    [Slides] Concepts discussed: subgradient, non-linearity (sigmoid, tanh, relu, leaky relu, maxout), intialization, residual networks, regularization.
  10. Topic: Topic Models & Noisy Texts
    Reading: Open Domain Event Extraction from Twitter.
    Background Reading: Named Entity Recognition in Tweets: An Experimental Study.
    [Slides] Concepts discussed: LDA, Bayesian non-parametrics (Chinese restaurant process), Dirichlet distribution, conjugate distribution.
  11. Topic: Reinforcement Learning
    Reading: Improving Information Extraction by Acquiring External Evidence with Reinforcement Learning.
    [Slides] Concepts discussed: Q-learning.

  12. KB Inference

  13. Topic: Symbolic methods for KB Inference
    Reading: Random Walk Inference and Learning in A Large Scale Knowledge Base .
    [Slides] Concepts discussed: Pagerank, random walks.
  14. Topic: Neural models for KB Inference
    Reading: Joint Matrix-Tensor Factorization for Knowledge Base Inference.
    [Slides] Concepts discussed: .
  15. Topic: Text and Open IE for KB Inference
    Guest Lecture by Prof. Partha Pratim Talukdar, Indian Institute of Science.
    Reading: PIDGIN: ontology alignment using web text as interlingua
    Reading: Improving Learning and Inference in a Large Knowledge-base using Latent Syntactic Cues
    Reading: Acquiring temporal constraints between relations
    [Slides] Concepts discussed: Graph-based semi-supervised learning, text as interlingua, text for inference, temporal scoping.

  16. Question Answering

  17. Topic: Open Question Answering
    Reading: Open Question Answering Over Curated and Extracted Knowledge Bases.
    Background Reading: Paraphrase-Driven Learning for Open Question Answering.
    [Slides] Concepts discussed: .
  18. Topic: Semantic Parsing for Question Answering
    Reading: Towards Answering Multi-Sentence Recommendation Questions
    [Slides] Concepts discussed: .
  19. Topic: LSTMs for Question Answering
    Reading: Teaching Machines to Read and Comprehend.
    [Slides] Concepts discussed: .
  20. Topic: Attention in Neural models
    Reading: Dynamic Co-attention Networks for Question Answering.
    [Slides] Concepts discussed: Highway networks, Maxout networks, additive vs. multiplicative models, attention over attention.
  21. Topic: Research Trends in Question Answering
    Guest Lecture by Dr. Manoj Kumar Chinnakotla, Sr. Applied Scientist, Microsoft.
    Reading: A Neural Network for Factoid QA over Paragraphs
    Reading: Open Question Answering with Weakly Supervised Embedding Models
    Reading: Hand in Glove: Deep Feature Fusion Network Architectures for Answer Quality Prediction in Community Question Answering
    [Slides] Concepts discussed: dependency tree RNNs, weighted rank loss, deep feature fusion networks
  22. Topic: Question Answering and Dialog in Real World
    Guest Lecture by Dr. Gautam Shroff, VP, Tata Consultancy Services.
    Concepts discussed: Squaring probabilities, synthetic oversampling, Bi-LSTMs, Siamese networks vs. LSTM.

  23. Dialog Systems

  24. Topic: An Overview of Conversational Agents
    Guest Lecture by Dr. Manoj Kumar Chinnakotla, Sr. Applied Scientist, Microsoft.
    Reading: Filter, Rank, and Transfer the Knowledge: Learning to Chat
    Reading: Detecting Inappropriate Query Suggestions
    [Slides] Concepts discussed: .
  25. Topic: Generative Hierarchical Neural Networks
    Reading: Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models.
    Additional Reading: A Hierarchical Latent Variable Encoder-Decoder Model for Generating Dialogues.
    Additional Reading: Application to Dialogue Response Generation .
    [Slides] Concepts discussed: .
  26. Topic: Memory Networks
    Reading: End-to-end dialog system based on Memory Networks
    [Slides] Concepts discussed: .
  27. Topic: Generative Adversarial Networks
    Reading: Adversarial Learning for Neural Dialogue Generation
    Background Reading: Generative Adversarial Nets.
    [Slides] Concepts discussed: .

  28. WrapUp.


Project: 40%; Midterm Survey: 10%; Final: 20%; Reviews: 20%, Presentation: 10%.