Artificial Intelligence and Architecture

Artificial intelligence studies were initially developed based on symbolic logic systems, not brain-like ones. The Dartmouth Conference held in 1956 was a critical turning point that first defined the concept of artificial intelligence and brought together pioneers in this field. Pioneers such as John McCarthy, Marvin Minsky, Claude Shannon, and Allen Newell argued at the time that machines could conduct artificial intelligence with networks formed by logical symbols.

This approach was based on mathematical inference systems rather than modeling the nervous system. The theoretical foundations of deep learning-based systems such as GPT, DALL-E, and Stable Diffusion, which shape our lives today, were laid by Geoffrey Hinton, Yoshua Bengio, and Yann LeCun in the 1980s. However, due to hardware limitations in those years, these studies did not receive enough attention and were limited to academic studies only.

The development of artificial intelligence is not limited to text and visual production. Complex digital games such as AlphaGo, Dota 2, and StarCraft II have become experimental environments for testing the decision-making abilities of artificial intelligence models under environmental conditions. These environments have made it possible to observe how competent artificial intelligence systems are in practice, as they include parameters such as uncertainty, time pressure, and strategic planning.

Today, models such as GPT-3 contain approximately 175 billion parameters, while it is estimated that the human brain has 100 trillion synapses, which is approximately 100 times that number. Although this comparison is not one-to-one, it shows that artificial intelligence systems have not yet reached the complexity of biological intelligence. Although this means that artificial intelligence is still much “smaller” than biological intelligence, the results it produces are at a level that will affect society. This shows that the full potential of artificial intelligence models has not yet been revealed, but they can still exhibit human-like production and decision-making capabilities.

Artificial intelligence is a set of algorithmic systems that can imitate human-like cognitive functions; learn, make inferences, and make decisions. These systems can imitate mental skills such as learning, reasoning, problem solving, understanding, and even creative thinking. For my article on big data, you can visit: https://emrecicek.net/en/big-data-and-machine-learning/

Artificial intelligence learns by analyzing a large amount of data and extracting patterns from this data. Its basic working logic is to perform statistical calculations, but the results are impressive. At this stage, learning is not limited to just interpreting, but can also involve imitating certain events. There are 3 basic factors that artificial intelligence must have.

Data: The “raw material” that artificial intelligence needs to learn.
Algorithms: Mathematical structures that analyze data and try to find patterns.
Computational Power: The technological infrastructure required to perform these operations.

In this case, the point that distinguishes the artificial intelligence used today is that it can only interpret or imitate in the light of the data it has at the point of productivity. Artificial Intelligence systems are generally trained with the machine learning method and enable the system to learn by extracting patterns from the data given to it or collected.

Artificial Intelligence is divided into 3 groups depending on the fields of study. All of the systems we use today are in the narrow artificial intelligence category.

• Narrow & Weak Artificial Intelligence Models

Many artificial intelligence models that are used today and that we believe make our lives easier are included in this group. Applications with automatic translation feature, applications similar to Siri, Alexa, Google Assistant, applications such as Youtube, Spotify, Netflix that offer their recommendation systems for a specific service, applications that combine the data they collect from sources such as ChatGPT, DeepSeek, Copilot, Gemini, Midjourney, Soundraw, Suno and present them directly.

• Broad & Strong Artificial Intelligence Models

It includes artificial intelligence models that are equivalent to human intelligence, can learn and think in every subject. These are systems that are still being worked on at the theoretical level today.

• Super Artificial Intelligence Models

It includes models that are expected to surpass human intelligence, have awareness, emotion, perception, and decision-making ability, and that we have not seen examples of yet.

It is necessary to use some special tools to carefully examine complex processes, defective or unknown situations. Using artificial intelligence tools to inform machines about certain issues and to project multiple results, probabilities, and margins of error in line with the information they have.

Artificial Intelligence Working Principle

Artificial intelligence models process natural language not as “words” or “sentences” but by breaking them down into smaller pieces, or syllable-like structures called tokens. The system that does this is called a tokenizer. For example, the phrase “Emre Cicek is an engineer” might look like this for the model:

["Em", "re", " ", "Ci", "cek", " ", "is", " ", "an", " ", "en", "gi", "ne", "er"]

If the tokenizer does not work correctly, the model will misunderstand the entire context. It is possible to get very different outputs from the same model with two different tokenizers. Techniques such as Byte Pair Encoding (BPE) and Unigram are tokenization algorithms frequently used in today’s language models. In particular, OpenAI has optimized the tokenizer architecture, both speeding up the model and increasing the accuracy of the response. In other words, the understanding of the commands given to them by artificial intelligence systems depends on this fragmentation system.

The “Transformer” architecture is at the core of modern artificial intelligence models. Introduced in 2017, this structure is a multi-layered modeling system that can understand the context of each word or concept using the attention mechanism. Systems known as LLM (Large Language Model) are built on this structure and learn with billions of parameters. Unlike traditional sequential structures, Transformer processes all inputs simultaneously and can analyze the context much more accurately.

The New Architecture of Artificial Intelligence

Artificial intelligence systems are no longer just models; they have begun to become agents that can make their own decisions, interact with their environment, use tools, and continuously learn. The surface-level part of artificial intelligence systems, that is, the model output that users directly interact with, is only the end product of the system. The quality, meaning, and task-appropriateness of this output depend on the numerous invisible but critical infrastructures and decision-making mechanisms operating behind the model. In order to understand this development, it is necessary to talk about the eight-layer architecture known as “Agentic AI Layer 8 Architect”.

Understanding the working principle and architectural structure helps organizations implement these systems effectively. Agentic AI’s 8-layer architecture represents an advanced framework that enables autonomous agents to perceive, reason, learn, and act independently in complex environments. This comprehensive structure provides the necessary components to create AI systems that can operate with minimal human intervention while maintaining reliability, security, and ethical compliance.

1. Infrastructure Layer

The Infrastructure Layer includes the hardware and cloud systems required for any agent AI system to function properly. It covers the basic computing resources and connectivity methods. This foundation layer consists of several critical components that ensure robust system performance. Without a robust and scalable infrastructure, AI agents cannot operate efficiently. Agents need memory, CPU/GPU power, and low-latency network access to process large data and models at optimized cost.

It forms the foundation as the primary processing backbone, including GPUs for intensive machine learning computations, CPUs for general processing tasks, and specialized hardware for specific AI workloads. It plays a key role in ensuring uninterrupted communication between distributed system components. High-speed network capabilities enable fast data exchange between different layers and external systems, while robust connectivity protocols ensure reliable agent-to-agent communication in multi-agent environments.

Storage Systems provide both persistent data storage and high-speed memory access. These systems must process a variety of data types, from structured databases to unstructured multimedia content, and ensure data integrity and availability for autonomous operations.

2. Agent Internet Layer

The Agent Internet Layer is the digital interaction layer that allows an AI agent to connect with the outside world. This layer provides access to online resources, data services, and communication environments with other agents that an agent needs to perform tasks. For modern AI applications, structuring and using this information in real time is as critical as accessing information.

The basic components of the layer include web browser interfaces, API clients, subscription mechanisms that provide data streams (e.g., RSS, webhooks), and service-based connection protocols. With this layer, the agent can pull information from search engines, integrate with SaaS systems, or directly exchange data with other software (e.g, CRM, ERP, e-mail servers).

For an agent with real-time data needs, the Agent Internet layer acts as a neural network that captures dynamic data sources such as access to news feeds, live financial data, user movements, and location services. Thanks to external connections, the agent’s Layer 5 process is directly fed with information, and its decision-making capacity increases.

In addition, this layer enables simultaneous operation in multi-agent systems. Different agents can connect to each other, delegate tasks to other agents, or produce solutions together with different systems or artificial intelligence products. In this context, the Agent Internet layer provides the infrastructure for not only external data access but also interoperability within an “agent ecosystem”.

In terms of security and access management, this layer should also support security mechanisms such as session management, data authentication, and data encryption. Otherwise, the external data flow to which the agent is exposed may become open to manipulation, and in the event of evaluation of possible vulnerabilities, there is a risk for personal data or confidential information.

Agent-based artificial intelligence systems do not have to work alone. In scenarios where multiple agents work together, strategies such as task sharing, synchronous messaging, and task delegation in case of errors are used. These agents evaluate the situation by sharing message sequences and reach the determined goal more efficiently. In environments where multiple agents operate simultaneously, protocols and shared memory spaces that standardize this communication are used.

3. Protocol Layer

The Protocol Layer represents a set of communication rules and data formats that enable secure, consistent, and structured data exchange with external systems in agent-based AI systems. This layer regulates the agent’s access to the Internet or other service providers at a technical level, while ensuring that systems can work together and correctly.

Modern AI applications must work with many different protocols to process data streams from different sources. In this context, the Protocol layer supports communication protocols such as HTTP(S), WebSocket, gRPC, GraphQL, MQTT, RESTful API, and SOAP, allowing agents to interact with both synchronous and asynchronous data sources. At the data format level, standards such as JSON, XML, YAML, CSV, and Protobuf come into play in this layer.

Agents use these formats correctly with serialization and parsing processes to make the data they receive from external systems meaningful and usable. For example, the main task of this layer is for an agent to process a JSON response coming from a REST API or to generate content by interpreting queries coming from GraphQL.

Authentication and authorization mechanisms are also critical components of the protocol layer. Using methods such as JSON Web Token and API key-based security policies, agents are controlled to provide authenticated and restricted access to certain services. Especially in multi-user environments, different agents need to have different access policies, and this should be resolved at the protocol level.

In addition, this layer should also include TLS/SSL encryption, HMAC verification, nonce usage, and retransmission or redirection-based attack prevention strategies for data integrity and transfer security. Otherwise, malicious actions can be triggered by changing the commands of the agents, or sensitive data can be leaked.

The protocol layer includes not only the interaction between the agent and the outside world, but also the protocol adaptation between the agents. For example, while an agent sends data to a microservice working with WebSocket, the other agent may be receiving information from an HTTP-based interface. In such heterogeneous communication scenarios, interface mapping is performed through protocol converters to ensure that the correct outputs are presented.

As a result, the protocol layer is the communication nervous system of the agent-based architecture. An incorrectly configured protocol infrastructure can cause data loss, security vulnerabilities, and incompatibility between systems. Therefore, this layer must be carefully designed in terms of both engineering and cybersecurity.

4. Tooling Enrichment Layer

The Tooling Enrichment Layer is the component that enables an AI system to transform from just producing information to a functional structure that can perform real tasks. This layer enables the system to perform complex operations using external digital tools, produce goal-oriented actions, and interact more actively with its environment.

It enables the ability to receive, process, transform, or export data from external systems by working integrated manner with various software systems, web services, or information processing modules. Thus, they become a system that can not only produce text-based responses but also implement tasks. At the same time, it is a process that includes logical decision structures and strategic usage flows that determine when and how the agent will use which tool. Thus, decisions about which tool is suitable for which scenario are modeled for the agent.

Technical details such as usability, functional capacity, and data type compatibility are defined in advance for each tool. The AI ​​system can choose which tool to use in line with this information and shape the task flow accordingly. The evaluation of the outputs received from the vehicles and the creation of new action plans according to the task success status are also carried out in this layer. The agent can analyze the results obtained after using a vehicle and plan the next step in the process of reaching the goal.

This structure provides the artificial intelligence system with the ability to not only produce information, but also to make decisions and implement them. Thus, the system not only understands the user’s request and produces a response, but also implements the necessary steps to fulfill this request with the help of external tools. In this way, it becomes an active task performer, not a passive information system.

Task sharing, vehicle selection, and data routing processes work in an integrated manner with the cognitive layer of artificial intelligence. The artificial intelligence system can make a vehicle preference by considering previous usage data in line with the task goal. Thus, the process of learning from past experiences is also included in this layer. Since it is a structure that works with external tools, both internal and external data flows must be authorized and controllable.

Some architectural approaches developed in recent years form the basis of this layer. Artificial intelligence works more naturally and effectively, especially thanks to systems that combine tool use and reasoning. In this way, decision processes that are not only based on information but also action-oriented emerge.

5. Cognition & Reasoning Layer

The Cognitive and Reasoning Layer is the center where the mental processes that enable any agent AI system to make logical, goal-oriented decisions by processing the information it receives from its environment are carried out. It represents the thinking part of AI systems. This layer includes high-level capabilities such as situation assessment, task planning, contextual analysis, and problem solving. The basic components are as follows:

• Situational Awareness: The process of the agent analyzing the current context, covering the time spent understanding task priorities or user intentions.
• Plan Generation: The process of constructing multi-step and comprehensive process flows to achieve a specific goal.
• Decision Tree: The process of making decisions with conditional rules, predefined policies, or learned behavior models.
• Error Management: The process of detecting and replanning failed tasks and activating alternative paths.

The agent not only makes logical inferences here; it can also dynamically update its behavior when faced with uncertainty, missing data, or contradictory inputs. It can benefit from past patterns to reach the desired result or add new flows to the alternative paths in the plan. In order for this layer to work properly, a strong data exchange is required with Layer 6 and Layer 4. While the results of why the decision was made are revealed on this layer, how the decisions made in other layers will be implemented is determined.

In order for these cognitive structures to work, agents should not be limited to the data they were trained on. In real-world tasks, dynamic and variable information that the model does not see during training is often needed. This is where RAG (Retrieval-Augmented Generation) comes into play. This structure is of great importance for the agent to evaluate the current context more accurately and to base its decision processes on real-time data.

As part of the reasoning process, RAG allows the model to access external information sources (such as documents, databases, APIs, or current internet content). In this way, the agent can make decisions supported by situation-specific, up-to-date, and contextual data, not just memorized information. In this way, the model gains the ability to think and reason outside of the box and produces more logical and contextual results with up-to-date data.

Note: AI models can sometimes produce answers that are unrelated to reality but convincing, called “hallucinations.” This situation occurs when the system draws incorrect conclusions from the data it was trained on or produces without sufficient context. RAG (Retrieval-Augmented Generation) systems allow the model to draw information from external sources to reduce this problem. However, to obtain reliable results, model outputs must be verified and, if necessary, tested with human supervision.

6. Memory & Personalization Layer

The Memory and Personalization Layer is the structure that allows an AI system to produce smarter, more consistent, and personalized outputs by remembering past experiences, user interaction history, and contextual information. Thus, the AI ​​agent improves itself with the feedback it receives as a result of correct or incorrect outputs presented against the requested data.

While correct results provide positive feedback for the flow charts, it will create in the next stage, incorrect results provide insight into how to plan over different flow charts in the next stage. In this case, Layer 6 includes both episodic memory (individual events and user interactions) and semantic memory (general knowledge and world knowledge) components. The memory systems in this layer can be classified into the following types:

• Short-Term Memory (Working Memory): Keeps the instantaneous task context, temporary variables, and decision traces in that session.
• Long-Term Memory (Persistent Memory): Contains permanent information such as the agent’s task history, user preferences, and previous successes/failures.
• Retrieval-Augmented Memory (RAM): Contains vector search-based retrieval systems to speed up access to information (e.g., FAISS, Pinecone, Weaviate). Vector search is a technique used in information retrieval and machine learning to quickly find elements in a large dataset.
• Personalization Profiles: Contains parameters such as user-specific behavioral models, language preferences, tone settings, and content length.

At the data processing level, this layer works with embedding-based query matching, vector similarity, memory addressing mechanisms, and time-stamped record arrays. Embedding is a mathematical transformation method that represents data as lower-dimensional, continuous vectors.

In addition, in inter-system or system-human interactions, the ability to “remember the past” plays a critical role in ensuring contextual consistency in multi-step tasks. Thanks to this layer, AI systems can remember the answer a user previously gave, choose the optimal path by learning from previous tasks, or provide user-based consistency. An example of this is a content production service producing text in accordance with a writing style that the user has previously liked.

The effective operation of these personalized memory structures is possible not only by storing the data, but also by the system being able to flexibly adapt its behavior according to this data. This is where techniques such as LoRA (Low-Rank Adaptation) come into play. This method allows large language models to adapt to tasks or users only at certain sub-layers without requiring full retraining. Thus, the AI ​​system can update its behavior based on past interactions or user preference profiles.

For example, it can respond to the same question with different tone and content strategies for different users. LoRA enables the personalized outputs in the memory layer to be directly reflected in the model behavior, transforming this layer into an active structure that not only stores information but also determines behavior.

In terms of security and privacy, this layer should also include encrypting and storing private data, controlling personal data access permissions, and data management policies by regulations such as GDPR/KVKK.

It transforms into an entity that develops over time and learns from user interactions, and thus can establish correct communication with the user. The success of this layer directly affects the adaptive intelligence level of the entire system.

Personal Note: During the times when artificial intelligence models were popular, the situation that a large majority wanted to portray or describe themselves as they knew was realized exactly, thanks to this layer. Information such as previous chat histories, location services, or application tracking, if any, is recorded in personalized profiles to reach faster results. Whether this situation will create a security gap is up to you. 🙂

7. Application Layer

The Application Layer is the interface layer where the final outputs of the AI ​​architecture meet the user and the functional capabilities of the system are presented in the form of tangible products or services. This layer enables the decision-making (Layer 5), tool usage (Layer 4), and personalization (Layer 6) skills of AI systems to be transformed into a user experience. Applications in this layer can be classified into two basic levels:

• Task-Oriented Applications: Systems that perform a specific function, such as drafting emails, analyzing software errors, answering customer questions, or summarizing documents.
• Continuously Interactive Agents: Personal artificial assistant systems that work integrated into the user’s life, offer proactive suggestions or autonomously execute multi-step processes.

The Application layer includes the front-end components (web interfaces, mobile applications, voice interfaces, messaging bots) through which any AI system interacts with the user, as well as background agents running in the background, cron-job-based triggers, and webhook/event-driven tasks. There are some critical technical requirements for this layer. These are:

• UI/UX Integration: Dynamic interfaces that optimize user experience, provide meaningful feedback, and offer interactive responses. UI (User Interface) and UX (User Experience) refer to two basic components in the user-centered design of digital products. UI covers the visual and functional interface elements that a user encounters when interacting with the system; elements such as buttons, menus, color palettes, typography, and iconography form the basis of UI design. This layer aims to present the product in an aesthetically consistent, accessible, and intuitive way for the user. UX addresses the overall experience dimension of the product; analyzes user behaviors, goals, and interaction processes, and offers solutions to increase the functionality, usability, and user satisfaction of the product.
  
  Note: The ability of artificial intelligence systems to produce correct outputs depends not only on the capacity of the model, but also on the correct design of the input (prompt) from the user. Prompt engineering is the process of structuring commands according to the purpose, clarity, and correct context. Thanks to techniques such as Few-shot, zero-shot, chain-of-thought, and system prompt, very different outputs can be obtained from the same model. Especially in task-based applications, this structure directly affects the behavior of the model.

• Orchestrator: It is a control mechanism that manages and coordinates the operation of artificial intelligence systems that can execute more than one task at the same time. It separates the operations to be performed step by step for a high-level request from the user and forwards them to the relevant services for implementation, and collects and combines the obtained outputs. The orchestrator stores context information during the task and keeps a record of the relevant stages.
  
  Modern Agent systems include much more than the operation of a single model. Frameworks such as CrewAI, LangChain, AutoGPT, and OpenAgents provide specialized orchestration structures for the separation of tasks, synchronized use of resources, and execution of multi-step processes. These structures create smarter and more autonomous agents by integrating components such as tool usage, memory management, decision flow, and task sharing.

• Session Continuity: It is the technical unit where session-aware structures that maintain the context by remembering the user’s previous interactions are established. It works integrated with the memory unit. It allows communication to continue uninterrupted from the first transaction to the result, or even at different stages. It recognizes the user and maintains the context over past communication.

• Performance and Reliability: Performance shows how quickly, accurately, and efficiently the system responds to user requests. Reliability ensures that the system always produces accurate, consistent, and error-free results. Because the response time, success rate, and error tolerance of the system are directly reflected to the user at this layer. System tests, log analytics, and error reporting systems (Sentry, DataDog) are important at this stage. In this way, the user needs to reach the correct information as quickly as possible.

In summary, the application layer determines how an AI system looks and feels to the user. No matter how powerful the system is, it can be perceived as inadequate or dysfunctional due to a weak application layer. Therefore, in addition to technical power, design, accessibility, seamless communication, and user habits are also important.

8. Ops & Governance Layer

The Operation and Management Layer covers the operational and management mechanisms that ensure that AI systems operate in a sustainable, secure, transparent, and ethical manner. The main purpose of this layer is that systems are not only functional, but also auditable, controllable, and compliant with social norms.

AI systems apply control lists in different profiles and rules for users. These profiles cover multiple authorizations such as purchasing, account upgrades, or admin access. Authentication, authorization, and access control lists (ACLs) operating on systems define which users can access which resources at what level.

Content filtering, role-based access control (RBAC), and zero-trust security models that prevent the leakage of confidential information are also included in this level. Logging systems are available to record all activities on the system in detail. Performance, security, and user experience are optimized by analyzing metrics such as system behavior, error reports, and resource usage. Infrastructure management strategies such as system uptime information, resource consumption, scheduled task processes, load balancing, disaster recovery, data backup, and fault tolerance are also evaluated in this layer.

Artificial intelligence systems should be evaluated not only in terms of technical success but also in terms of ethical compliance, social responsibility, and compliance with legal regulations. Certain policies are also put into effect at this layer to ensure that artificial intelligence agents act in a fair, impartial, non-discriminatory, and ethical manner. In addition, compliance with regulations such as GDPR, KVKK, and HIPAA requires respect for principles such as data privacy, the right to be forgotten, and user consent.

Especially in architectures where multiple artificial intelligence systems work together, the distribution of tasks, areas of responsibility, and their relationships with each other are defined here. In addition, situations such as the model producing biased results and causing discrimination should be evaluated within the ethical framework. Therefore, ethical control mechanisms, explainability principles, and transparency should be taken as a basis in system design.

LLM systems should not only be trained, but also constantly monitored, tested, and updated in the production environment when necessary. This process is called LLMOps. Processes such as version management, log analysis, performance monitoring, A/B testing, prompt versioning, and user feedback analysis are carried out under this heading. Thus, the sustainability, reliability, and adaptability of the system are increased.

As a result, the Operations and Governance Layer is essential not only for technical stability but also for ethical, legal, and operational sustainability. Without this layer, a powerful AI agent can quickly become a security risk, a data breach, or a source of social discord.

LoRA (Low-Rank Adaptation): It is an efficient fine-tuning technique that allows task-based customization with fewer parameters without retraining an existing model. It is used in corporate or domain-specific training.

• RAG (Retrieval-Augmented Generation): It allows the model to access information sources other than the training data. The model does not have up-to-date data; RAG closes this gap by pulling information from external systems and using it in response generation. These systems stretch the fixed information limits of LLM.

• Vector Database + Embedding Systems: Vector databases perform semantic similarity searches using this representation format. Embedding systems convert words or concepts into numerical vectors. Thus, LLM can query information not only according to letter or word matches but also according to contextual proximity.

These components allow the model to go beyond the classical question-answer boundaries and transform into dynamic, task-based systems. It also enables the scalability, updateability, and modularity of the model.

However, this fascinating development has a significant cost. Training large language models like GPT-3 requires months of GPU computations, tons of cooling energy, and hundreds of thousands of kilowatt-hours of electricity. According to calculations, training GPT-3 alone is estimated to have caused emissions equivalent to about 500 tons of carbon dioxide. This value is close to the total emissions of a person who lives without emitting carbon for decades on an individual scale. The more promising the future of artificial intelligence

The training and operation processes of large AI systems cause significant energy consumption. Green AI encourages the development of smaller, more efficient, and environmentally friendly models to balance this situation. The future of AI will be evaluated not only by its ability to produce intelligence, but also by its ability to ensure sustainability.