Using Agentic Frameworks to Build New AI Services

The original promise of AI was that it would write most of the code for us. In reality, we’re not there yet. So where can AI meaningfully improve developer productivity today?

In this post, we look at how AI powers development productivity across the SDLC, practical tools to use and frameworks for overcoming AI operationalization bottlenecks.

This blog post is based on the webinar “Using Agentic Frameworks to Build New AI Services” with Rory Richardson, Next Gen Dev Experience, Gen AI GTM, from AWS and Yaron Haviv, co-founder and CTO, Iguazio (a McKinsey company).

Watch the full recording here.

Top Limitations of Current AI-Code Generation Tools

Practitioners who work with code generation tools face challenges such as:

• Hallucinations & incorrect assumptions about APIs, frameworks, or systems - Models infer nonexistent methods, wrong configuration, outdated libraries, or misunderstand how a system actually runs.
• Weak cross-file and cross-layer coherence - Tools struggle to propagate changes across modules, maintain invariants, or update all references consistently in complex architectures.
• Limited architectural competence - No reliable understanding of architectural patterns, domain boundaries, layering, clean architecture, or future-proof design principles.
• Poor modularity and maintainability - AI generates tightly coupled code, hard-coded dependencies, duplicated logic, minimal abstraction, and fragile extension points.
• Shallow reasoning about concurrency, state, and distributed workflows - Async patterns, transactions, state machines, event-driven flows, or cross-service semantics.
• Low-quality testing and verification - Unit tests are shallow, overly mocked, or incomplete and lack integration tests; do not validate failures, boundaries, or performance characteristics.
• Inconsistent adherence to engineering best practices - AI breaks  conventions around naming, structure, logging, error handling, linters, formatting, CI/CD structure, and observability.
• Non-deterministic and unpredictable outputs - Same prompt can yield different plans, agents may loop or change unrelated files, and long tasks drift or lose context.

For example:

• Al calls an SDK function that doesn't exist or assumes a blocking API is async.
• Renames a model but misses serializer updates, route handlers, or integration tests.
• Places business logic directly in a controller, creates god- objects, or mixes concerns.

Currently, code generation tools can speed up typing, boilerplate code, generate modules, explain code, understand repos, create PRs, run tests, and more. However, they cannot conduct deep reasoning, long-range code coherence, or multi-file refactoring. They can also not maintain consistency across large repos or operate workflows at scale.

Increasing Productivity Across the SDLC with AI

Examining AI for developers requires a change of mindset, from developer productivity to development productivity. Rather than measuring how fast developers code, autonomous AI agents can be used across the SDLC: for testing, building, deploying, operations and monitoring.

When we look at where AI can most effectively boost productivity across the SDLC, three main buckets emerge.

• Build - The need to increase development velocity, while taking into account that developers spend the majority of time on undifferentiated tasks instead of coding
• Operate - Finding an efficient way to manage and optimize cloud environments, resources, configurations, and costs across multiple accounts and service
• Transform - Modernizing legacy workloads faster and reliably to retire tech debt, save costs, improve performance, and drive innovation.

Example tools that can be used for building include:

• Kiro, an agentic IDE, which combines the flow vibe coding with the clarity of creating specs. With Kiro, spec-driven development turns vibe coding prompts into clear requirements, system designs and discrete tasks. Agent hooks automate workflows based on pre-defined triggers. See a short demo of how Kiro works in the webinar. 

• For non-developers, an agentic CLI can help create applications, run tests, debug, access APIs, deploy code, connect to MCP servers, and more. See a demo in the webinar. 

5 Key Considerations When Building Agents

1. Development tooling - IDE or CLI 

2. Framework

3. Platform for runtime, memory, integrations, AI gateway, identity management and observability

4. Inference 

5. Data - the secret sauce of AI. Choosing tools that can unlock data and value.

In the future, AWS predicts that fully autonomous agents will be directing, coordinating, evaluating and verifying the work of other agents across environments. This is especially valuable when tools are siloed, e.g when using different tools for monitoring, and data management. Agents can provide a single pane of glass across the environments.

According to McKinsey:

• GenAI tools and incorporating AI outputs in tasks helps achieve 1.2X productivity.
• Agentic workflows and agents who create outputs and perform a task end-to-end can achieve 2X productivity.
• The next frontier will be 20X productivity, as practitioners build and supervise a virtual organization of 15-20 agents, with humans finalizing outputs as needed. 

5 Key Agentic AI Challenges

1. Limited scaling beyond pilots - Despite significant promise, most projects remain trapped in proof-of-concept phase and don't get scaled up to the full potential

2 - Lack of end-to-end impact - Even "successful" pilots are point- solutions and rarely demonstrate measurable impact across the entire domain or function

3 - Slow shift to production - Scaling use-cases in production is hard, especially in regulated industries, causing use-cases to stall before launch to customers

4 - Fragmented tech usage - Tech vendor offerings are siloed and prevent building out integrated and reusable solutions that can drive enterprise-wide adoption

5 - Insufficient change management - Few organizations have adopted the structural and cultural transformation approach needed to drive adoption at scale

This requires a different approach to build & scale Agentic AI to realize the full potential.

The Solution: Agentic AI Architecture and Components

To address this, integrated, production-grade agentic architectures are needed.

At the infrastructure layer, this includes MLRun and Kubernetes-based microservices that provision LLMs, MCP servers, A2A services, and shared data assets.

On top of this, McKinsey is introducing a higher-level abstraction called “Agents Scale.” Agents at Scale allows users to create agents, tools, groups (or “squads”), transformers, and workflows using natural language descriptions instead of code. Users define what each group or agent is trying to achieve (e.g., planner, critic, data interface), and the system translates those descriptions into executable agentic workflows.

These workflows can include multiple steps, pass data between agents, access files, generate artifacts, and log execution.

The goal is to let teams build business and operational workflows,  like product code modernization or customer service automation, with minimal technical friction, while still running on a resilient, microservices-based, production-ready foundation.

To see the full details, including demos, examples and a fascinating Q&A about the future of development and AI, watch the full webinar.