Understanding Agentic AI through MCP (Model Context Protocol)

                                                 art by: J. Sridharan, Dubai

Agentic AI Orchestrator Protocols in Simple Terms

Earlier in Nov 2024, Anthropic open sourced MCP (Model Context Protocol) - an open standard that enables developers to build secure, two-way connections between their GenAI applications and the tools and data sources. MCP is an open-source protocol that simplifies connections between AI systems and various data sources to help deliver faster innovation in context-aware Agentic AI applications. 

What are Agentic AI Orchestrators

Agentic AI is seen as the second big evolution of GenAI, and Agentic AI orchestrators are seen to be the enablers of this evolution. As LLMs continue to evolve providing multi-modal and extended built-in capabilities - in bare terms, LLMs are good in the predictability of the next word, which makes them great poem reciters and essay writers or converting text to visuals or translating languages and many more standalone tasks.  But there is a fundamental limitation – LLMs cannot take any intelligent action unless they are integrated with tools and data sources – for example: if you ask an LLM any information that it was not trained on (for example: current stock market trends) - unless it is connected to a web search engine – it cannot provide you with an accurate answer.

 In the GenAI application space, in terms of standardizing the communication protocols we are still in an era that can be compared to the pre-Rest API era - Just like how RESTful APIs accomplished simplified, standardized communication between client and server applications, there is a significant opportunity to standardize communication protocol between LLMs, tools and data sources. Consider a simple GenAI application built with a single model (a single Agentic AI application) – let’s say - a personal travel assistant – helping you to not only plan but do the bookings for a holiday - this agent ideally must fetch details from multiple sources to fulfil the task – Google maps to determine the place of interest, an OTA such as Expedia and other providers such as Booking.com, execute your credit card etc. Without a standardized way of connecting to the tools, building GenAI applications though not impossible, is very engineering intensive.

 In simple terms, building a GenAI Agentic application has 4 components (in short TATA)

• Task to accomplish,
• The model/s or Agent/s,
• Tools it needs to accomplish the task
• Answer the agent provides.

Without standardized protocols, the following are few of the key challenges to accomplish TATA.

• Custom built implementations required significant engineering effort to plumb tools and data sources. In addition, consider the re-engineering efforts when sources change.
• When connecting multiple agents, inconsistent prompt logic with different methods for accessing and federating tools and data will provide inefficient answers.
• The Scale problem - "n times m problem" - where a a large number of client applications interacting with a mesh of servers and tools will result in a complex web of integrations, duplicity, each requiring specific integration efforts.

MCP allows AI Agents to use tooling, resources and even prompt libraries in a standardized manner, thus extending the Agentic AI capabilities significantly to build more meaningful GenAI applications.

Just to keep the MCP architecture simple, MCP uses a client-server architecture, primarily at a high level, the key components being an MCP client, the MCP server and the MCP communication protocol. Developers expose their data through lightweight MCP servers. For example, Anthropic has released a few popular MCP server codes already such as for Google maps, or Slack. By connecting to these MCP servers, you can easily build an Agentic AI MCP client following the MCP protocols.

MCP Architecture

MCP uses a client-server architecture that contains the following components and is shown in the following figure:

• Host: An MCP host is a program or AI tool that requires access to data through the MCP protocol, such as Claude Desktop, an integrated development environment (IDE), or any other AI application.
• Client: Protocol clients that maintain one-to-one connections with servers.
• Server: Lightweight programs that expose capabilities through standardized MCP, allows access to data sources tools and even prompt libraries.
•  Local data sources: Your databases, local data sources, and services that MCP servers can securely access.
• Remote services: External systems available over the internet through APIs that MCP servers can connect to.

MCP, thus by providing an open-source protocol and a universal standard that simplifies connections between AI systems and various data sources - will deliver agility in building efficient and context-aware AI applications. Consequently, this will enable AI agents to autonomously perform complex tasks.

The success and widespread adoption of protocols like MCP depends upon industry participation and standardization efforts on interoperability and portability, and adherence to common standards, allowing AI applications to operate across different platforms and jurisdictions, crucial for global companies and responsible AI. 

MCP will help build trust by ensuring AI systems are transparent, reliable and secure. The clarity provided by the MCP protocol guidelines will reduce compliance complexity, will lower barriers to innovation and will foster faster development of AI products. 

 

From a Software Engineer to a CTO

How to break the glass ceiling?

Key differences between a Transformer Architecture and a State space model Architecture for Building LLMs

 

A transformer architecture primarily focuses on capturing local relationships within a sequence by using attention mechanisms, while a state space model architecture is designed to model the evolution of a system over time by maintaining a fixed-size "state" that represents the current system status, making it more efficient for handling long sequences but potentially limiting its ability to capture fine-grained details within the data; essentially, transformers excel at short-range dependencies while state space models prioritize long-range dependencies and overall system dynamics. 

Key differences: 

Attention mechanism:

Transformers heavily rely on attention mechanisms to weigh the importance of different parts of an input sequence when generating the output, allowing for flexible context understanding. State space models typically do not use attention mechanisms in the same way. 

State representation:

In a transformer, the "state" is essentially the current hidden representation at each layer, which can dynamically change with the sequence length. In a state space model, the "state" is a fixed-size vector representing the system's current status, which is updated based on input and system dynamics. 

Handling long sequences:

Transformers can struggle with very long sequences due to quadratic computational complexity, while state space models are generally better suited for handling long sequences because of their fixed-size state representation. 

Applications:

Transformers are widely used in natural language processing tasks like machine translation, text summarization, and question answering due to their ability to capture complex relationships between words. State space models are often applied in areas like time series forecasting, control systems, and scenarios where tracking the evolution of a system over time is crucial. 

Recent developments: 

Mamba Model: Researchers have developed architectures like "Mamba" which attempt to combine the strengths of transformers and state space models, leveraging attention mechanisms while still maintaining a fixed-size state to handle long sequences more efficiently. 

The Impact of GenAI on DevSecOps

 

The Impact of GenAI on DevSecOPs

DevSecOps, is inevitably impacted in this age of GenAI. As AI transforms the way we work, here are some areas where GenAI can be used in DevSecOps.

Integration with DevSecOps tools: GenAI can integrate with DevSecOps tools, enabling real-time feedback and continuous monitoring of security posture throughout the software development lifecycle. This can help ensure that security is integrated into the development process, reducing the risk of security breaches.

GenAI based coding assistants: Many organizations have already embraced AI-powered coding assistants such as Amazon Q and GitHub Copilot to improve the developer experience and speed time to deployment of software. A recent McKinsey study found developers can complete coding tasks up to twice as fast with generative AI. Coding assistants can perform at various levels of capabilities the following tasks:

• Diagnose common errors.
• Turn comments into code
• Completing your next line or function in context
• Bring knowledge to you, such as finding a useful library or API call for an application
• Transform Legacy code into later versions of software
• Add comments
• Rewriting code for efficiency
• Write Software based on  prompts
• Chat about Code.
• Provide inline code suggestions.
• Scan code for security vulnerabilities.

Automated Security Testing: GenAI can significantly enhance automated security testing by analyzing code, identifying vulnerabilities, and providing real-time feedback to developers. This can lead to faster and more accurate detection of security flaws, reducing the risk of security breaches.

Intelligent Anomaly Detection: Incorporating Generative AI into DevSecOps enables intelligent anomaly detection in real-time. AI models can continuously monitor system behavior, user activity, and network traffic, promptly identifying suspicious patterns and potential security breaches. This enhances proactive threat mitigation and incident response.

Intelligent Threat Detection: GenAI-powered threat detection systems can analyze vast amounts of data, identifying patterns and anomalies that may indicate potential security threats. This can help security teams respond more effectively to emerging threats and reduce the risk of attacks.

Enhanced Incident Response: GenAI can help streamline incident response by analyzing large amounts of data, identifying the root cause of incidents, and providing actionable insights to security teams. This can lead to faster and more effective incident response, reducing the impact of security breaches.

Improved Compliance: GenAI can help organizations comply with regulatory requirements by automating compliance checks, identifying potential non-compliance issues, and providing recommendations for remediation.

Predictive Maintenance: GenAI can predict potential security threats and vulnerabilities, enabling proactive measures to mitigate risks. This can lead to reduced downtime, improved system reliability, and enhanced overall security.

Enhanced Collaboration: GenAI can facilitate collaboration between security teams, developers, and other stakeholders by providing a shared understanding of security risks and vulnerabilities. This can lead to more effective communication, reduced miscommunication, and improved overall security.

Continuous Monitoring: GenAI can continuously monitor systems, networks, and applications, identifying potential security threats and vulnerabilities in real-time. This can help security teams respond quickly to emerging threats and reduce the risk of security breaches.

Reduced False Positives: GenAI can help reduce false positives in security systems, reducing the risk of false alarms and improving the overall effectiveness of security measures.

Improved Security Orchestration: GenAI can help orchestrate security tools and systems, enabling more effective incident response, threat hunting, and security operations.

Automating Security Patching: GenAI can expedite security patching by automating the analysis and application of patches. AI models can scan codebases, identify vulnerabilities, and suggest appropriate patches, accelerating the patching process and reducing the window of exposure to potential threats

As GenAI continues to evolve, its applications in DevSecOps will help organizations stay ahead of emerging threats and vulnerabilities. The integration of generative AI into DevSecOps promises a future of faster, more secure, and more efficient software development. By automating tasks, enhancing security, and improving software quality, generative AI empowers developers build faster, cheaper and better.

The risks of over reliance remains, developers and security teams must remember that AI coding tools  are not a substitute for human oversight and testing. For example, a code generated by an AI assistant cannot be merged to the master blindly without proper validation by a human. Being aware of the limitations enables DevSecOps teams gain efficiencies.

The optimistic view is that by training AI tools on libraries of clean and secure code, teaching it best practices, and fine-tuning it to a company’s internal policies and frameworks, the contributions of the AI assistant would be secure from the start. Plus, its usage for finding security problems in existing code, for debugging, for generating tests, for writing documentation, and many other tasks related to DevSecOps far outweighs the risks of overreliance. Overall, pairing generative AI DevSecOps tools with a programmer creates a happier, more productive developer experience. This experience will streamline software development, and help companies ship applications and innovations faster to get better business outcomes.

Airport Metaverse Mundane Benefits

 Here are some potential benefits of using metaverse technologies for airports:

- Improved passenger experience. The metaverse could allow passengers to virtually navigate airports before their trip. This could help reduce confusion and stress upon arrival. Passengers could find gates, shops, restaurants, etc. in a realistic 3D environment.

- Enhanced wayfinding. Detailed 3D maps and guides in the metaverse could make it easier for passengers to find their way through large, complex airports. Real-time directions, notifications, and maps could minimize getting lost. 

- New advertising and retail opportunities. Airports could showcase stores, products, and services in immersive 3D spaces. Passengers may be more inclined to shop or browse offers in a fun, engaging virtual environment. Retailers gain new ways to promote their brands.

- Remote assistance solutions. Passengers could access live virtual assistants, information booths, or customer service representatives within metaverse airports no matter their physical location. This could help address questions or issues without having to search the actual airport.

- Environmental impact reduction. The metaverse may allow some passenger interactions, simulations, or information sharing to occur remotely rather than requiring physical presence. This could potentially reduce congestion, energy use, emissions from travel to/from airports in some situations.

- Training and education benefits. Airports could use metaverse platforms to provide virtual training to employees, demonstrate new procedures before implementation, or educate passengers on airport policies and processes in an immersive way.

- Future testing ground. The metaverse may give airports an environment to experiment with and test potential future technologies, designs, or operational changes before physical implementation. This can inform long-term strategic planning and capital investment decisions.

Define Reliability in a minute

 I was asked to define Reliability in a minute at a recent conference. This was my reply. 

Thinking beyond software, hardware and networks, resilience is about how we deisign, build and operate systems, who does this. what processes we use and how consistently do we do this?  It is about having a wholistic mental model and removing barriers from all aspects, always keeping the end user business outcomes in mind.

Reliability engineering is about anticipating failures, building emergency responses, building guardrails and mechanisms such as quick-heal and self-heal into the ecosystems. Eventually when failures do happen (they will always happen), how can we quickly recover and go back to normalcy, how do we retrospect the failure to derive learnings, and how do we apply the learnings from a people-process-technoogy perspective back into the ecosystem, and build improvements in a continuous manner.

It is also about having a frugal mindset, and building cost-effectiveness throughout the conceptaulisation to operational phases. Its not about over-sizing and over engineering to achieve outcomes, rather how intelligently can we achieve goals with minimum costs. 

This is Reliability Engineering in a nutshell. Not a ground breaking answer, but I believe this simple ground-truth is what organisations struggle to implement in spirit. #devsecops #reliabilityengineering