Projects
-
Current Projects
-
- Knowledge Infused Dialogue Generation and Summarization
- Dialogue Generation Loss Function and Evaluation Metrics
-
Towards Building Multi-modal Diseaese Diagnosis Virtual Assistant
-
- Problem: When we consult with doctors, we often report and describe our health conditions with visual aids. Moreover, many people are unacquainted with several symptoms and medical terms, such as mouth ulcer and skin growth. Therefore, visual form of symptom reporting is a necessity. Motivated by the efficacy of visual form of symptom reporting, we propose and build a novel end-to-end Multi-modal Disease Diagnosis Virtual Assistant (MDD-VA) using reinforcement learning technique. In conversation, users' responses are heavily influenced by the ongoing dialogue context, and multi-modal responses appear to be of no difference. We also propose and incorporate a Context-aware Symptom Image Identification module that leverages discourse context in addition to the symptom image for identifying symptoms effectively.
Outcome- VDr. Dialogue Assistant v1
- Findings are published in LREC-COLING 2024, CIKM 2023, CIKM 2022, and ECAI 2023
- Patent - System and Method for a Knowledge-Infused Multi-Modal Symptom Investigation and Disease Diagnosis Assistant (Filed & Published)
- Problem: When we consult with doctors, we often report and describe our health conditions with visual aids. Moreover, many people are unacquainted with several symptoms and medical terms, such as mouth ulcer and skin growth. Therefore, visual form of symptom reporting is a necessity. Motivated by the efficacy of visual form of symptom reporting, we propose and build a novel end-to-end Multi-modal Disease Diagnosis Virtual Assistant (MDD-VA) using reinforcement learning technique. In conversation, users' responses are heavily influenced by the ongoing dialogue context, and multi-modal responses appear to be of no difference. We also propose and incorporate a Context-aware Symptom Image Identification module that leverages discourse context in addition to the symptom image for identifying symptoms effectively.
-
Knowledge Infused Context Driven Disease Diagnosis Virtual Assistant
-
Problem: In real life, doctors generally go in reverse order, i.e., they first hypothesize a set of diseases based on patients' chief complaints \& other confirmed symptoms. They investigate potential signs of the hypothesized condition, allowing them to diagnose patients with greater confidence in fewer conversation turns. We build a novel knowledge-infused context-driven (KI-CD) hierarchical reinforcement learning (HRL) based diagnosis dialogue system, which leverages a bayesian learning-inspired symptom investigation module called potential candidate module (PCM) for aiding context-aware, knowledge grounded symptom investigation.
Outcome- VDr. Dialogue Assistant v0
- Patent: System and Method for Automatic Disease Diagnosis, Published (Filed & Published)
- Findings are published in IEEE Transaction on AI, BMC and Knowledge Based System (KBS)
-
Personalized Persuasive Multi-modal Virtual Sales Assistant (Sponsored by Accenture Labs)
-
Problem: Task-oriented conversational agents are gaining immense popularity and success in a wide range of tasks, from flight ticket booking to online shopping. However, the existing systems presume that end-users will always have a pre-determined and servable task goal, which results in dialogue failure in hostile scenarios, such as goal unavailability. On the other hand, human agents accomplish users’ tasks even in a large number of goal unavailability scenarios by persuading them towards a very similar and servable goal. Motivated by the limitation, we propose and build a novel end-to-end multi-modal persuasive dialogue system incorporated with a personalized persuasive module aided goal controller and goal persuader. We also present a novel automatic evaluation metric called Persuasiveness Measurement Rate (PMeR) for quantifying the persuasive capability of a conversational agent.
Outcomes:- PeaRL v1
- Findings are published in IJCNLP 2022, ICPR 2022 and Expert System with Applications
-
Dynamic Goal Orineted Virtual Assistant (Sponsored by Accenture Labs)
-
Problem: Developing an adequate and human-like virtual agent has been one of the primary applications of artificial intelligence. Task-oriented virtual agents are rapidly becoming our companions in completing various tasks, including online shopping and early disease diagnosis. Existing virtual assistants presume that end-users will always have a pre-determined and servable task goal, which results in dialogue failure in hostile scenarios such as goal unavailability. On the other hand, human agents accomplish users' tasks even in a large number of goal unavailability scenarios by persuading them towards a very similar and servable goal. Motivated by the limitations of existing VAs, we build an end-to-end persuasive dialogue system incorporated with a personalized persuasive module aided goal controller and goal persuader. We formulated a unique Markov decision process with a cumulative reward model (task-based, sentiment-based, and persuasion-based) for simultaneously reinforcing task-specific, user-adaptive, and personalized persuasive behavior and optimized dialogue policy using reinforcement learning techniques.
Outcomes:- PeaRL v0
- US Patent: Dynamic Goal Orineted Dialogue Assistant (Filed & Published)
- Findings are published in IJCNN 2021, Expert System with Applications and Plos ONE