Gen AI based Social Media based Surveillance
for Capital Markets Ecosystem

Background:

A leading commodity exchange in India plays a pivotal role in facilitating the trading of various commodities, including metals, energy, and agricultural products. As a key platform in the financial ecosystem, it enables buyers and sellers to trade standardized contracts with efficiency and transparency. The exchange is instrumental in price discovery, risk management, and providing a secure and regulated environment for market participants.

Challenges Faced:

Broking house platforms, crucial for online commodity and stock trading, face several operational challenges such as login issues and API glitches. These technical problems can lead to significant monetary losses for users and generate widespread negative sentiment on social media platforms like X (formerly Twitter).

Objectives:

• Real-Time Issue Detection: Implement AI to monitor social media platforms for real-time detection of complaints and issues related to broking house platforms and exchanges, enabling prompt identification of operational problems.

• Insightful Analysis: Utilize Generative AI to analyze and categorize social media posts, extracting insights on issue types, severity, sentiment, and affected entities to understand the impact and nature of user grievances.

• Proactive Response Management: Develop a system to automatically generate and display actionable notifications on a dashboard, allowing stakeholders to address issues swiftly and manage user sentiment effectively.

• Enhanced Operational Transparency: Provide comprehensive reports and trend analysis to enhance transparency, helping broking houses and exchanges improve their services and address recurring issues to prevent future complaints.

Solution Summary:

Our solution addresses the need for effective monitoring and management of user sentiment in the trading ecosystem through a three-pronged approach:

• Interactive Dashboard: Provides a comprehensive view of frequent issues reported by trade users, highlighting problem areas and the specific broking houses involved. This dashboard offers real-time insights and visualizations to help stakeholders quickly identify and address operational challenges.
• Email Notifications: Sends alerts regarding the severity of user sentiment, ensuring that relevant parties are promptly informed about critical issues and can take necessary actions to mitigate any negative impact.
• Conversational Interface: Features an “Ask Me a Question” style interface that allows users to query and obtain information about collective user sentiment across multiple broking houses, facilitating better understanding and response to customer concerns.

Leveraging state-of-the-art foundational models, this solution supports a diverse range of end-users including:

• Exchange for monitoring issues across different broking houses.
• Market Regulator for tracking customer grievances and their potential impact on exchanges.
• Broking houses for surveilling customer sentiment and addressing growing mistrust.

Data Processing using Generative AI:

• Sentiment Classification: Data points are analyzed for sentiment using the Roberta model, which classifies text as positive, negative, or neutral. This model is trained on approximately 124 million tweets and fine-tuned for accurate sentiment analysis.
• Issue Labelling: Negative sentiment data is further processed to identify and label the cause of the issue using the Mistral InstructLLM, a 7 billion parameter Large Language Model (LLM). This model labels issues with concise, two-word descriptions.
• Label Similarity: Labels are merged based on semantic similarity to ensure consistency. For instance, “Login issue” and “Login problem” are recognized as the same issue. This is achieved using the “all-MiniLM-L6-v2” model from the Sentence-Transformers framework.

Tech Stack:

• Development: Python

• Sentiment analyzer: Deep Learning model - Twitter Roberta

• Issue labeller: LLM - Mistral AI

• Label similarity: all-MiniLM-L6-v2

• AWS Services: AWS SageMaker

• Dashboard Interface: Gradio