Satya Mallick

Agentic is not a novel concept emerging from LLMs ! The roots of Agentic systems run much deeper, tracing back to decades of research efforts in behavior and coordination in multi-agent systems. Two such agent-oriented frameworks in Java for developing multi-agent systems are : 1. JADE JADE complies with the FIPA standards (Foundation for Intelligent Physical Agents) and provides tools for agent communication, mobility, and lifecycle management. It allows agents to interact through an ACL (Agent Communication Language) and includes built-in tools for debugging and monitoring agent behaviors. JADE is widely used in academic and industrial applications where distributed, autonomous systems are required, such as simulations and robotics​. 2. JADEX This framework builds on top of JADE but introduces a belief-desire-intention (BDI) model for agents. In JADEX, agents are developed using XML-based plans and goal structures, making it more suitable for complex reasoning and behavior modeling. It extends JADE's functionality by focusing on higher-level cognitive agent concepts​. While building Agentic Frameworks using Java, it would be useful to leverage the strengths of these frameworks and provide additional context using a LLM. ------------------------------------------------------------------------------- To learn more about these systems, check the link in the comments. #Java #LLM #Agentic

4

1 Comment

I am happy to announce our latest publication, "Deep Learning-Based Road Sign Detection: The Ultimate Road Technology," co-authored with esteemed colleagues A. P. Singh, P. Rahi, B. Sahu, N. Basu, and V. Yadav, presented at the 1st International Conference on Advanced Computing and Emerging Technologies (ACET) 2024 in Ghaziabad, India. This work delves into cutting-edge deep learning techniques, specifically using Convolutional Neural Networks (CNN) and the YOLO (You Only Look Once) framework, to create a powerful, efficient system for road sign detection. Our approach simplifies object detection into a single-unit regression model, mapping image pixels directly to bounding box coordinates with class probabilities—enabling instantaneous recognition and categorization of road signs. Such advancements are crucial for enhancing road safety, contributing to the evolution of autonomous driving, and ensuring faster, more reliable navigation systems. 📖 Read the full paper: IEEE Xplore

3

The next guest speaker in my LLM course is Ram Sriharsha, the CTO of Pinecone, who will talk about Retrieval Augmented Generation (RAG) on Monday October 28. ➡️ Register here: https://lu.ma/r7yokyvc Title: Knowledgeable Models Combine Retrieval with Generation: An Introduction to RAG Date: Monday October 28 from 1:45 PM to 3:15 PM ET Abstract: Large Language Models have emerged into mainstream over the last few years. While they have demonstrated some huge successes already, they have also had challenges with hallucinations, difficulty with dealing with new knowledge and in retrieving relevant, accurate knowledge for downstream tasks. In this talk, we will provide a brief overview of the problem, and discuss a promising approach to solving these problems through what is known as Retrieval Augmented Generation (RAG). Alongside, we will also introduce vector databases, what makes them different from traditional databases and how they fit into RAG. Bio: Dr. Ram Sriharsha is the CTO at Pinecone. At Pinecone, Ram is responsible for overseeing the company’s technical direction in both engineering and research. Prior to joining Pinecone, Ram held numerous senior positions at some of the most well-known companies in the tech industry. Most recently, he was Vice President of Engineering at Splunk. Before that, he was a product and engineering lead at Databricks for their unified analytics platform for genomics. Some years after receiving a PhD in Physics from the University of Maryland, Ram was a Principal Software Engineer and later a Principal Research Scientist at Yahoo. Supplementary Readings: ➡️ The original paper on RAG: "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks" (https://lnkd.in/e_e56Y54) ➡️ Long context considerations: "Lost in the Middle: How Language Models Use Long Contexts" (https://lnkd.in/ehED53Fe) ➡️ Article on contextual RAG from Anthropic AI (https://lnkd.in/ehK4GwMc)

69

Let's have a look at the Best Paper Honorable Mention of ACM SIGGRAPH 2024! High quality asset generation "CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assets" is a great paper published by DeemosTech, Huazhong University of Science and Technology and ShanghaiTech University. They introduce CLAY, a 3D geometry and material generator designed to effortlessly transform human imagination into intricate 3D digital structures. CLAY supports classic text or image inputs as well as 3D-aware controls from diverse primitives (multi-view images, voxels, bounding boxes, point clouds, implicit representations, etc). At its core is a large-scale generative model composed of a multi-resolution Variational Autoencoder (VAE) and a minimalistic latent Diffusion Transformer (DiT), to extract rich 3D priors directly from a diverse range of 3D geometries. Specifically, it adopts neural fields to represent continuous and complete surfaces and uses a geometry generative module with pure transformer blocks in latent space. They present a progressive training scheme to train CLAY on an ultra large 3D model dataset obtained through a carefully designed processing pipeline, resulting in a 3D native geometry generator with 1.5 billion parameters. For appearance generation, CLAY sets out to produce physically-based rendering (PBR) textures by employing a multi-view material diffusion model that can generate 2K resolution textures with diffuse, roughness, and metallic modalities. They demonstrate using CLAY for a range of controllable 3D asset creations, from sketchy conceptual designs to production ready assets with intricate details. Even first time users can easily use CLAY to bring their vivid 3D imaginations to life, unleashing unlimited creativity. You can test it with Rodin-Gen-1 already! https://lnkd.in/entaZjtR All links in comments below #generativeai #mesh #cad #assets #vfx #assetsgeneration #diffusionmodels #neuralradiance #machinelearning #deeplearning #futurism #computervision #graphics #graphicsengine

26

4 Comments

Our paper on #XR and #haptic enabled #hri (human #robot interaction) framework is available for download now... https://lnkd.in/gGYrkpVZ

26

Uncover the future of Indic languages with our guest speakers as they discuss the potential of generative AI. Tune in for insightful discussions! Orgnanized by: Odia Generative AI #OdiaGenAI #IndicLangEmpowerment #NLP #GuestSpeakerSeries

8

Day 2 of ROScon India 2024 was a treasure trove of knowledge and insights! I attended keynote addresses and talks by industry experts, including Angelo Corsaro , CEO/CTO of ZettaScale Technology, who shared his expertise on Zenoh and its potential in ROS and beyond. Other notable talks included: - Robotics Application Stack by Ajay Sethi and Prateek Nagras - Simulation-based IDE for ROS Development by Sanjana - Integrating Physics-Based Neural Networks with Gazebo and ROS by Nidhi Choudhary - Roboverse: Developing a Virtual ROS2 Robotics Ecosystem by Sanjuna Mariam Mathews Mariam Mathews The panel discussion on ROS, AMRs, and Industrial Productivity was also enlightening, with insights from industry experts. The day concluded with a fireside chat on the Future of ROS, featuring Geoffrey Biggs, Yadunund Vijay , and Angelo Corsaro. Thanks to the organizers and speakers for making Day 2 of ROScon India 2024 an unforgettable experience! ROS India Open Robotics RigBetel Labs Acceleration Robotics NVIDIA #ROSconIndia2024 #Robotics #ROS #Zenoh #Gazebo #Simulation #ArtificialIntelligence #MachineLearning #IndustrialProductivity #AMRs #Nvidia

12

📢 2 𝐩𝐚𝐩𝐞𝐫𝐬 𝐚𝐜𝐜𝐞𝐩𝐭𝐞𝐝 𝐚𝐭 𝐭𝐡𝐞 𝐩𝐫𝐞𝐬𝐭𝐢𝐠𝐢𝐨𝐮𝐬 𝐈𝐧𝐭𝐞𝐫𝐬𝐩𝐞𝐞𝐜𝐡 2024 𝐜𝐨𝐧𝐟𝐞𝐫𝐞𝐧𝐜𝐞! 🎉 ⛳ 𝐅𝐚𝐬𝐭𝐀𝐒𝐓: 𝐀𝐜𝐜𝐞𝐥𝐞𝐫𝐚𝐭𝐢𝐧𝐠 𝐀𝐮𝐝𝐢𝐨 𝐒𝐩𝐞𝐜𝐭𝐫𝐨𝐠𝐫𝐚𝐦 𝐓𝐫𝐚𝐧𝐬𝐟𝐨𝐫𝐦𝐞𝐫 𝐯𝐢𝐚 𝐓𝐨𝐤𝐞𝐧 𝐌𝐞𝐫𝐠𝐢𝐧𝐠 𝐚𝐧𝐝 𝐂𝐫𝐨𝐬𝐬-𝐌𝐨𝐝𝐞𝐥 𝐊𝐧𝐨𝐰𝐥𝐞𝐝𝐠𝐞 𝐃𝐢𝐬𝐭𝐢𝐥𝐥𝐚𝐭𝐢𝐨𝐧 𝐀𝐮𝐭𝐡𝐨𝐫𝐬: Swarup Ranjan Behera, PhD, Abhishek Dhiman, Karthik Gowda, Aalekhya Satya Narayani. 𝐀𝐛𝐬𝐭𝐫𝐚𝐜𝐭: Audio classification models, particularly the Audio Spectrogram Transformer (AST), play a crucial role in efficient audio analysis. However, optimizing their efficiency without compromising accuracy remains a challenge. In this paper, we introduce FastAST, a framework that integrates Token Merging (ToMe) into the AST framework. FastAST enhances inference speed without requiring extensive retraining by merging similar tokens in audio spectrograms. Furthermore, during training, FastAST brings about significant speed improvements. The experiments indicate that FastAST can increase audio classification throughput with minimal impact on accuracy. To mitigate the accuracy impact, we integrate Cross-Model Knowledge Distillation (CMKD) into the FastAST framework. Integrating ToMe and CMKD into AST results in improved accuracy compared to AST while maintaining faster inference speeds. FastAST represents a step towards real-time, resource-efficient audio analysis. ⛳ 𝐓𝐨𝐰𝐚𝐫𝐝𝐬 𝐌𝐮𝐥𝐭𝐢𝐥𝐢𝐧𝐠𝐮𝐚𝐥 𝐀𝐮𝐝𝐢𝐨-𝐕𝐢𝐬𝐮𝐚𝐥 𝐐𝐮𝐞𝐬𝐭𝐢𝐨𝐧 𝐀𝐧𝐬𝐰𝐞𝐫𝐢𝐧𝐠 𝐀𝐮𝐭𝐡𝐨𝐫𝐬: Swarup Ranjan Behera, PhD, Orchid Chetia Phukan, Priyabrata Mallick, Aalekhya Satya Narayani, Arun Balaji Buduru, Rajesh Sharma. 𝐀𝐛𝐬𝐭𝐫𝐚𝐜𝐭: In this paper, we work towards extending Audio-Visual Question Answering (AVQA) to multilingual settings. Existing AVQA research has predominantly revolved around English and replicating it for addressing AVQA in other languages requires a substantial allocation of resources. As a scalable solution, we leverage machine translation and present two multilingual AVQA datasets for eight languages created from existing benchmark AVQA datasets. This prevents extra human annotation efforts of collecting questions and answers manually. To this end, we propose, MERA framework, by leveraging state-of-the-art (SOTA) video, audio, and textual foundational models for AVQA in multiple languages. We introduce a suite of models namely MERA-L, MERA-C, MERA-T with varied model architectures to benchmark the proposed datasets. We believe our work will open new research directions and act as a reference benchmark for future works in multilingual AVQA. Stay tuned for detailed posts about each paper in the coming days. Excited to share our work with you all! #Interspeech2024 #FastAST #AudioSpectrogramTransformer #TokenMerging #CrossModelKnowledgeDistillation #MultilingualAVQA #AudioVisualQuestionAnswering #MultilingualAI

117

25 Comments

🌟 Exciting News! 🌟 Join us for our new webinar series, "Understanding Reversibility: Edward Fredkin Memorial Lectures," organized by Cellular Automata India (CAI). This series pays tribute to the pioneer in reversible computing, Edward Fredkin, whom we lost on June 13, 2023. Our focus spans across various scientific disciplines, exploring the theme of Reversibility. Here's a glimpse of what to expect: 1. Logic Synthesis methods for reversible and quantum circuits by Susmita Sur-Kolay, ISI Kolkata 2. Time-reversal Symmetries in reversible Cellular Automata by Kenichi Morita, Hiroshima University, Japan 3. Exploring irreversibility in the macro-world by Pradeep Kumar Mohanty, IISER Kolkata 4. Asynchronism and Reversibility by Sukanta Das, IIEST Shibpur 5. Reversibility and Irreversibility in Cellular Automata and Physics, with insights from Stephen Wolfram, Wolfram Research 6. The gift of invertibility across math, physics, and cellular automata by Tommaso Toffoli, Boston University, USA 7. Negation, Absence, and Reversibility by Smita Sirker, JNU India Mark your calendars! 📅 🗓️ Every Saturday from April 27 to May 18, 2024 ⏰ 2:00 – 6:00 pm IST Join us on Google Meet: (https://lnkd.in/gF-4-VPJ) For detailed schedule, click here: (https://lnkd.in/ge3aUWvn) Don't miss out on this unique opportunity to engage in lively discussions with eminent speakers from diverse domains. Let's make this event a resounding success together! #CellularAutomataIndia #CAI #Reversibility #webinar #research #toffoligate #quantum #cellularautomata #wolfram #reversiblecomputing

11

Special Issue: Designing Intelligent Man-Machine Interfaces for Enhanced User Interaction ( JISIoT) Designing Intelligent Man-Machine Interfaces for Enhanced User Interaction Intelligent Man-Machine Interfaces (IMMI) is to revolutionize human interaction with technology by utilizing advanced technologies such as artificial intelligence, machine learning, and natural language processing to create intuitive and spontaneous user experiences. The scope of IMMI includes enhancing the efficiency and authenticity of these interactions, improving customer satisfaction through individualized solutions based on user behavior and preferences, and making technology more accessible to a wider audience, including individuals with disabilities. IMMI also aims to boost productivity and efficiency in professional settings by streamlining workflows, automating routine tasks, and enabling quicker decision-making processes. As AI and machine learning systems learn from user interactions, they progressively enhance the relevance and accuracy of their responses, while multimodal interactions—combining speech, touch, gesture, and facial recognition—offer a versatile and robust user experience. The development and testing of these interfaces rely on tools like Sketch, Figma, Adobe XD, and AI frameworks such as TensorFlow, PyTorch, and Keras. IMMI also integrates speech recognition and NLP capabilities through tools like Google Cloud Speech-to-Text, Amazon Lex, and IBM Watson, and leverages AR and VR technologies to create immersive experiences in education, gaming, and healthcare. As AI continues to advance, IMMI is expected to become more prevalent in everyday electronics, enabling sophisticated customization and improved accessibility for all. However, the increasing use of AI and data collection within IMMI presents significant privacy concerns, requiring robust data security measures and transparent privacy policies. Technological constraints and user acceptance challenges also highlight the need for ongoing research, development, and effective communication of IMMI's benefits. Despite these challenges, IMMI remains focused on enhancing user experiences, increasing accessibility, and driving productivity across various domains. List of topics include but not limited to: • Role of Intelligent Man-Machine Interfaces for enhanced user interaction. • Utilization of AI and ML for enhanced user experiences in IMMI. • Analysis of IMMI and Accessibility for disabled users. • Optimization of privacy and security considerations in Intelligent Man-Machine Interfaces. • Future trends and emerging technologies of IMMI. • Contribution of emotion recognition in enhancing IMMI interactions. Submission deadline: January 25, 2025 Author notification: March 25, 2025 Revised papers due: May 30, 2025 Final notification: July 30, 2025 Full details found here: https://lnkd.in/dMRRdNEr

4

1 Comment

Mixtral 8x22B comes reasonably close to Llama3 70B on MMLU eval at little more than 50% of the Active parameters! More the active parameters, more the memory & compute needed and hence more the cost. So in the current Open Source model matrix, if you are looking for close to GPT4 performance at the lowest cost, it's time to checkout Mixtral 8x22B Source: Sebastian Reschka's newsletter, where he added the Llama3 datapoint on top of the Mixtral chart published earlier by Mistral.

3

🤖 OpenAI o1 series launched in mid september, marks a significant advancement in Generative pretrained transformers, focusing on enhanced reasoning capabilities. ♧ Optimized for complex reasoning specially in STEM fields (science, technology, engineering, and mathematics) They are designed to "think" more before answering. Hence better problem-solving and high accuracy. ♧ Chain-of-Thought reasoning: This is a new approach that involves long chain of thoughts before reaching to a conclusion. ♧ Reinforcement learning with human feedback: o1 uses RLHF, which means the model receives rewards for generating desired steps. for developers who need advanced reasoning capabilities without paying more, you can go for mini version that starts at $0.03 per 1k tokens, it's cost effective! #openAI #ArtificialIntelligence #MachineLearning #AIinDevelopment #ChatGPT #o1series #SoftwareDevelopment

13

1 Comment

“Transformer oversimplified for all” Poloclub has released a beautiful transformer visualization. They use GPT-2 to explain how transformers (the heart of LLMs) work. This is one of the best visualizations of transformers I have come across among a few others. Please check out: https://lnkd.in/guWi_af7 This visualization inspired me to create a lecture on how transformers work. I have released the lecture on Vizuara’s YouTube channel. Here is the link to the lecture: https://lnkd.in/gFWE7XbJ I title this lecture “Transformer Oversimplified” because I have eliminated the difficult aspects of transformers and made them as beginner-friendly as possible to the extent of oversimplification. Check out this lecture. I am sure you will enjoy it and understand a lot about transformers, LLMs, and GPT: https://lnkd.in/gFWE7XbJ If you are interested in learning about LLMs, we are releasing a lecture series on LLMs from scratch soon on our channel. The first lecture will be out in 2 days. Stay tuned.

50

Hello Connections My research article entitled "Synchronization and implementation of a real-time traffic signal optimization regulator" has been published in the MTAP Springer Journal with IF-3.6 #ESCI #Scopus #springer In this article, the proposed system most cosmopolitan cities like Bengaluru, the major problems of congestion, such as increased vehicular density, travel time, stress, and vehicular carbon emissions. These issues have led to quantifiable and non-quantifiable losses, such as high vehicle operation and maintenance costs, reduced human productivity and mobility, increased health hazards, etc. Sustainable transportation, coupled with Technological interventions, could alleviate these issues by efficiently managing traffic flow. The major aims of this work are to reduce the overall commute time, traffic congestion, and air pollution, making the commutation of public stress-free and happy. So, we propose a comprehensive Real-time Traffic Signal Optimizer and Regulator (RTSOR) that identifies the right High-Density Corridors (HDC) and achieves these aims by successfully attaining coordination among the Adaptive Traffic Signal Control System (ATSCS) fixed on the successive junctions of HDC considered in this work as a pilot initiative by upgrading the existing traffic signal in Bengaluru City to the latest ATSCS technology. Implementing this proposed RTSOR has effectively lessened the waiting time in traffic, travel time, and fuel consumption and improved air quality by reducing pollution and lowering stress levels, leading to improved quality of health and life of the public. In Bengaluru city, the current innovation would be implemented for the first time, and no prior model exists that could help optimize the travel time. The method that the study envisaged through RTSOR focuses on collecting real-time data on an HDC to elucidate our claim of originality in reducing the delay time on roads by signal synchronization. https://lnkd.in/gnDQEAQg

43

2 Comments