Speaker: Dr. Vikram Vij, Senior Vice President of Samsung Research

Date and Venue: Friday 8th September 2017, 12:00 - 13:00, SIT Building Seminar Room (Room 001).

Title: Bixby – Intelligent Assistant for Smartphones – From Concept to Launch

Abstract: I will go into the challenges we faced in the Development of Bixby NLU and how we solved them.
Data Balancing for Domain Classification (DC)– DC is a critical function of the Bixby NLU as it determines which Domain should process the utterance. DC is very sensitive to the data from the domains. As we can give only limited data from each domain (in order to control DC model build time and model size), we have to be very careful with what data we provide to DC from each domain (we have to sample this data but we can control what/how much we sample). We had to make sure that most commonly used path rules and most commonly used utterances for them are in DC data. As most users use shorter utterances, we gave higher weightage to those. We needed to ensure that non-relevant words for a domain don’t outnumber the relevant words. Data from 3rd party Application Domains is very varied so it required special treatment. This went through several iterations to achieve current quality levels.

Grammar Restructuring– This pertains to the grammar for generating the Intent Classification Model (ICM) and Slot Tagging Model (STM) of the Bixby NLU. Initially the grammar was not generic enough and also we did not have global concepts, phrases and criteria. To ensure re-usability across Domains, to lower the training time and to ensure that most variations are covered, it was important to restructure the grammar for ICM and STM for each Domain. We also had to handle grammar conflicts (where more than one intent could match an utterance) and grammar variations (for example, pluralization, optional words, criteria combination, politeness and colloquial). This went through several iterations to achieve current quality levels.

Challenges in realizing True Multi-Modal Handling - Realization of User intent through depth breaking in UX flow is major challenge in Bixby. User any time can ask Bixby to start for full command completion or partial command with follow-up. System expected to be designed intelligent enough in understand where to start and where to end in particular application flow. We have to ensure close commands are realized well enough with Machine Learning algorithms in order to provide best experience to end user. Second challenge is enabling user to start with voice to some UX flow, handle next few things through touch and again regain with voice (and vice versa). Next level challenge is handling of 3rd party applications whose UX flow is not directly with Samsung control. Team has gone through numerous brainstorming sessions in finalizing design. Tried with multiple architecture approaches like rejection models, in-domain session, capability based approach in determining true user intent.

Speaker Bio: Dr. Vikram Vij completed his Ph.D. and Master’s degree from University of California Berkeley in the field of Parallel Computing, after graduating from IIT Kanpur in Electronics and Communication, followed by an M.B.A. degree from Santa Clara University. Dr. Vij has over 26 years of industrial experience in multiple technical domains from Parallel Databases, Storage & File Systems, Embedded systems, Intelligent Services and IoT. He has worked at Samsung since 2004 and is currently working as Sr. Vice President and Division Head for Intelligence & IoT Division at Samsung R&D Institute in Bangalore. Current focus is on Smarter, Extensible and Connected Core Apps & Services, Connectivity/Convergence in Car, Wearables & Home, World’s Best Voice Intelligence Experience, World’s most preferred Fitness Companion & Intelligent Cloud based Services.