The disclosure by METR of two security incidents during 2026 is particularly important because it demonstrates how quickly experimental AI infrastructure can become part of an organization’s serious attack surface. The most striking incident occurred in March, when attackers discovered a publicly reachable agent-orchestration dashboard running on a researcher’s personal EC2 instance. The application was intended to sit behind Google authentication, but a fail-open vulnerability in the “vibe-coded” application silently disabled that protection. The exposed system contained an API key associated with METR’s general-access account for public AI models. Once the attackers found the application, they reportedly interacted directly with the agent and prompted it to reveal the model-provider API key, then installed an SSH key for persistence and used the stolen credential for approximately three weeks to consume a substantial amount of inference capacity. METR estimates that the corresponding API usage would have been worth approximately $600,000 if the credits had not been provided to the organization for free by the model developer.
This incident is a useful reminder that AI agents do not need to be “hacked” in some exotic science-fiction sense to create serious security problems. The agent appears to have done something much more ordinary and therefore much more useful to an attacker: it revealed a secret available within its operating environment when asked. The fundamental security failure was not artificial intelligence spontaneously defeating security controls. It was that an internet-exposed application had access to a reusable credential, authentication failed open, and the agent itself could access information that should never have been disclosed to an untrusted user. The AI component effectively became another interface through which existing trust could be abused.
The fail-open behavior is particularly important. Secure authentication systems should fail closed. If identity verification cannot be completed correctly, access should be denied. A fail-open system does the opposite: when something goes wrong, the application silently allows access. That is one of the most dangerous possible defaults for an internet-facing service because security protection can disappear without creating an obvious outage. From the researcher’s perspective, the application may continue operating normally. From the attacker’s perspective, the authentication boundary has ceased to exist.
This is exactly why authentication should not be implemented casually around experimental applications. A researcher building a temporary dashboard may view authentication as peripheral plumbing around the “real” AI functionality, but once the dashboard contains credentials, access to compute or internal data, authentication becomes one of the most security-critical components in the system. Vibe coding can accelerate prototyping dramatically, but generated application code still needs the same threat modelling, access-control testing and failure-mode analysis as traditionally developed software. Faster software generation unfortunately does not repeal several decades of application-security mistakes.
The use of a personal EC2 instance also adds an important governance dimension. METR specifically noted that the researcher had no access to its highly sensitive information categories, which substantially limited the eventual impact. Nevertheless, the personal instance contained an organizational API key and was not operating under the same controls that would ordinarily apply to centrally managed infrastructure. This illustrates why organizations need clear policies governing which credentials and data can be placed on personally managed cloud resources or devices. Shadow infrastructure becomes dangerous not merely because security teams cannot see it, but because it often inherits valuable organizational credentials without inheriting equivalent monitoring, configuration management or incident-response controls.
The attacker’s suspected discovery method is equally noteworthy. METR believes the instance may have been identified by monitoring recently registered websites or certificate transparency information and searching for high-signal terms related to AI, LLMs or agents. Certificate Transparency exists for a legitimate security purpose by making TLS certificate issuance publicly visible, but that same transparency allows attackers to discover newly created internet services quickly. A developer registering a hostname such as an experimental agent dashboard may unintentionally advertise the existence of that service to automated reconnaissance systems almost immediately.
This reinforces an increasingly important principle: putting something on the public internet effectively means assuming it will be discovered. Security cannot depend on obscurity, unusual hostnames or the belief that an experimental system is too small to attract attention. Automated reconnaissance does not care whether the application was intended for ten researchers or ten million users. If DNS, TLS, cloud infrastructure or response fingerprints make the service discoverable, attackers can identify and test it at machine speed.
The attacker reportedly interacted with the agent itself to obtain the API key. This raises another important AI-security question: what information should an agent be capable of revealing even when directly instructed to do so? Agents often need access to API credentials, system prompts, filesystem information and tool configurations in order to perform useful work. If those secrets are available in the same execution environment and are readable by the agent, prompt restrictions alone provide weak protection. Attackers may simply ask for the secret directly, indirectly, through tool invocation, through debugging prompts or through another workflow that causes the information to appear in output.
Secrets should therefore be architecturally separated from model context wherever possible. An AI agent should not receive the raw provider credential merely because it needs to invoke the provider. A broker or service layer can hold the credential and expose only the limited operation the agent requires. This follows the same principle used in traditional software: give the application the capability it needs without necessarily giving it direct possession of the underlying master secret.
Short-lived credentials would also have reduced the incident substantially. The stolen API key reportedly remained useful for approximately three weeks. Long-lived static keys are operationally convenient because they require little maintenance, but that same longevity benefits attackers after theft. Temporary credentials, scoped tokens and automatic rotation can reduce the usable lifetime of compromised secrets dramatically. If a token expires after an hour, credential theft is still serious, but the attacker’s window is fundamentally different from stealing a key valid indefinitely.
Credential scope matters equally. In this case, METR says the API key granted access only to public models rather than hidden chain-of-thought, non-public models or other sensitive model access. That separation appears to have prevented the incident from escalating into something considerably more serious. This is an excellent example of least privilege working as intended even though another security control failed. The API key was stolen, but the attacker inherited only the permissions assigned to that key.
Organizations working with frontier AI systems should take this lesson seriously. Evaluation platforms may need credentials for public models, pre-release models, models without normal safeguards and systems exposing additional internal outputs. Those different levels of access should never be represented by one universal credential. Separate accounts, projects, network boundaries and keys should exist so compromise of low-sensitivity research infrastructure cannot automatically expose high-sensitivity model access.
The attack also demonstrates why API keys increasingly need the same security treatment as passwords and privileged tokens. Developers often treat API keys as configuration rather than credentials, placing them in environment variables, notebook files, workflow definitions or deployment scripts. From an attacker’s perspective, however, an API key is simply another bearer credential. Whoever possesses it can perform whatever operations the provider authorizes, often without any additional identity verification.
Secrets scanning should therefore extend beyond source repositories. Runtime environments, cloud metadata, shell histories, CI/CD logs, AI prompts and debugging interfaces can all expose tokens. Organizations should maintain an inventory of issued keys, their owners, permissions, creation dates and recent usage, and should be able to revoke them immediately during an incident.
One of the most revealing aspects of METR’s incident is why the fraudulent usage remained unnoticed. The organization routinely performs large-scale model evaluations that naturally consume enormous amounts of tokens. High API usage was therefore not inherently unusual. METR also noted limitations in its internal usage dashboard and the absence of a natural spending ceiling because the credits were provided free of charge. Those conditions allowed the attacker’s activity to blend into expected operational noise.
This is a classic anomaly-detection challenge. Security monitoring cannot rely only on absolute volume. Ten million API calls may be normal for one workload and catastrophic for another. Monitoring needs context about which key is being used, from which infrastructure, for what model, at what time and whether the behavior resembles the historical pattern associated with that credential. A key normally used from one research environment suddenly generating sustained traffic from unfamiliar infrastructure should attract attention even if the total token volume falls within organizational norms.
Spend alerts are useful, but usage alerts are even more important. Free credits can still represent economic value, and stolen inference capacity can be abused even when the direct financial bill is zero. Attackers might use access for automated content generation, model extraction attempts, credential testing, malware development, fraud or resale of access. Security thresholds therefore should not depend solely on monetary charges.
Usage quotas provide another defensive layer. A research key may legitimately need substantial capacity, but allowing unlimited consumption creates unnecessary exposure. Daily, hourly or model-specific limits can make abuse visible earlier. When a workflow genuinely requires unusually large volumes, the limit can be raised deliberately rather than leaving every credential permanently unrestricted.
The attacker's decision to install an SSH key for persistence is another important indicator that this was not merely casual API-key abuse. Once the attacker found the exposed application, they sought persistent control of the underlying EC2 instance. SSH authorized keys are attractive because they provide stable access that can survive password changes and application restarts. Incident responders therefore correctly treated the entire instance as compromised rather than simply revoking the stolen model credential.
This is an important lesson for AI application incidents generally. If an attacker gains shell or administrative access to the host, rotating the exposed AI key is not sufficient. The host itself may contain persistence, additional credentials, source code or network access. METR stopped and imaged the instance, rotated all credentials present on it, and wiped the researcher’s laptop as part of its response. That broader approach reflects the correct distinction between credential compromise and host compromise.
Cloud forensic readiness becomes crucial in these environments. Organizations should be able to snapshot compromised instances, preserve disks, capture relevant logs and determine which credentials were accessible during the intrusion. Ephemeral development systems often disappear quickly, which is convenient operationally but can destroy evidence just as efficiently as an attacker deleting logs. Cloud environments should therefore forward audit and network telemetry centrally so investigation does not depend solely on the compromised instance surviving.
The second incident disclosed by METR in May provides a different but complementary lesson. The organization observed a sustained external campaign in which attackers systematically probed its publicly reachable infrastructure, including credential stuffing against authentication providers, attempts to obtain OAuth token grants, scanning of newly deployed services and phishing of staff. METR says the attackers made heavy use of agents to automate vulnerability discovery. This is one of the clearest practical examples of AI affecting offensive operations without requiring any breakthrough in autonomous hacking capability. Automation can simply allow attackers to enumerate more systems, test more authentication paths and adapt reconnaissance faster.
AI therefore changes the economics of opportunistic attack. Tasks that previously required manual investigation can increasingly be delegated to agents capable of interacting with websites, APIs and authentication flows. The attacker still needs objectives and infrastructure, but the cost of testing a large number of targets falls. Organizations deploying experimental services should assume reconnaissance will accelerate accordingly.
During the same period, METR inadvertently exposed a read-only SQL query mechanism through its public transcript viewer. The feature was intended to query public information, but a bug made unpublished evaluation data potentially accessible. More concerningly, some sensitive model data had accidentally been placed within the database even though the dataset was supposed to contain only non-sensitive information. An independent security researcher found the vulnerability and responsibly disclosed it, after which METR took the API offline and paid a bounty.
The attackers had apparently probed this endpoint during their broader campaign, but METR says it found no evidence they discovered the specific exploit or accessed non-public information. This distinction is important. The presence of a vulnerable endpoint does not prove successful data theft. At the same time, the event demonstrates how close configuration and data-classification mistakes can bring an organization to a much more serious compromise.
The SQL query mechanism highlights a recurring problem with “read-only” functionality. Read-only does not automatically mean low risk. If a query interface can reach data users were never supposed to see, confidentiality can be compromised without any ability to modify the database. Security review should therefore focus on which rows, tables and fields a read-only function can access, not merely whether it can perform writes.
Data separation should also be enforced architecturally rather than through assumptions. The database behind the transcript viewer was supposed to contain only non-sensitive information, yet some sensitive model output had accidentally been included. Once sensitive and non-sensitive information occupy the same database, an authorization bug in a public query interface can unexpectedly expose both. Physical or logical separation between sensitivity tiers reduces this risk.
METR’s response was to create an isolated public production environment for publicly facing applications, architecturally separated from internal infrastructure. This is one of the most important controls described in the post-mortem. Public services should be assumed to face hostile input continuously. If compromising a public transcript viewer can provide direct pathways toward internal evaluation systems, the blast radius is unnecessarily large. Isolation turns a public-service compromise into a contained incident rather than a bridge into the organization’s most sensitive research.
This principle is directly applicable to AI labs and ordinary enterprises alike. Public APIs, websites, demonstration systems and developer tools should not sit inside the same trust zone as sensitive model weights, proprietary data, production databases or internal administration. Network segmentation needs to follow data sensitivity and function.
The incidents also highlight the problem of “sensitivity creep.” A system initially designated for public data may gradually accumulate information from more sensitive environments through debugging, convenience copies or integration changes. Over time, the system’s actual sensitivity diverges from its original classification. Security controls then remain appropriate for yesterday’s data rather than today’s.
Continuous data discovery can help identify these situations. Organizations should periodically scan public and lower-trust environments for secrets, proprietary information and datasets that violate the intended classification. Preventing sensitive data from entering those environments is considerably easier than proving after an incident that attackers never accessed it.
The fact that METR found no evidence of access to its most sensitive categories should not obscure the seriousness of the near miss. Sensitive model access can include private models, hidden chain-of-thought information or models without production safeguards. Compromise of those assets could create intellectual-property, model-security and broader AI-safety consequences beyond ordinary credential theft.
AI evaluation organizations therefore face unusually complex security requirements. They need to enable researchers to experiment rapidly while simultaneously protecting credentials and information supplied by external model developers. Excessively restrictive environments can impede research, while overly permissive environments can expose highly sensitive material. The answer is not simply choosing one side but creating clear sensitivity tiers with correspondingly strong barriers.
The METR incidents show why those barriers need to remain technically enforced. Policy alone cannot prevent a researcher from placing a credential on a personal EC2 instance. Infrastructure controls can make safer paths easier and dangerous paths harder. Managed research environments, centrally issued secrets, controlled deployment pipelines and automatic public-exposure reviews can reduce reliance on individual judgment.
Formal security review for publicly deployed research applications is especially valuable. A temporary agent dashboard can become internet-facing infrastructure the moment DNS and TLS are configured. A short review can check authentication behavior, secret exposure, logging, network access and whether the application fails safely before it is made public.
Automated deployment checks could also detect common mistakes. A pipeline could verify that authentication is enabled, reject wildcard access rules, scan containers for credentials and ensure that public workloads cannot reach sensitive internal networks. These controls reduce the burden on researchers while improving consistency.
The phrase “vibe-coded app” is important because it captures a wider industry issue. AI-assisted coding can allow people to create useful applications rapidly even when they are not experts in all the frameworks and security mechanisms involved. That democratization is powerful, but it also means software may be deployed by people who do not recognize dangerous default behavior. Generated code can look polished and function correctly during normal testing while containing insecure exception handling, authentication bypasses or secret exposure.
Organizations should therefore separate prototyping speed from production trust. AI-generated applications should be assumed untrusted until their security boundaries are reviewed. Code generation can accelerate implementation, but security verification still needs independent checks. Static analysis, dependency scanning, authentication testing and threat modelling become even more valuable as software creation becomes faster.
Fail-open conditions deserve explicit automated tests. Developers should deliberately simulate identity-provider failures, malformed authentication responses and dependency outages and verify that the application denies access. Security testing should not examine only the successful login path.
The incident also demonstrates that API economics themselves can become part of security monitoring. An attacker consuming inference credits is effectively stealing a metered resource. Similar abuse can occur with cloud compute, SMS gateways, email delivery, storage and paid APIs. Organizations should therefore treat unexpected resource consumption as a security signal rather than merely a billing anomaly.
FinOps and security operations increasingly need to collaborate. A sudden spike in cloud or API spending may indicate compromised credentials. Conversely, free or prepaid services still require anomaly monitoring because monetary billing may not provide a natural warning.
Cloud providers and AI model companies can help by supporting fine-grained spend caps, source restrictions and token policies. API keys could optionally be restricted to known IP ranges, workloads or cryptographic identities rather than functioning as universally usable bearer tokens. If a research key should only operate from a specific cloud environment, geographic or source-based restrictions can reduce the value of theft.
Mutual TLS or workload identity could provide even stronger protection in some architectures. Instead of possession of one string being sufficient for access, the client would need both the credential and an authenticated workload identity. This is more operationally complex, but high-sensitivity AI access may justify stronger mechanisms.
The May campaign’s OAuth token-grant attempts also demonstrate why attackers increasingly target authorization rather than passwords. OAuth applications can provide persistent access even after user passwords change. Security teams should monitor new application consent, unusual scopes and token grants, particularly for privileged users.
Users should not be able to authorize high-risk third-party applications without appropriate review. Sensitive scopes should require administrator approval and short-lived tokens where possible.
Credential stuffing against authentication providers reinforces the need for phishing-resistant MFA and passwordless authentication. Reused passwords remain valuable to attackers because automated systems can test them across large numbers of services cheaply. Rate limiting, bot detection and breached-password protection help reduce this risk.
Phishing remains relevant even in technically sophisticated AI environments. Researchers and engineers possess access attackers want, and technical expertise does not make people immune to convincing identity attacks. High-value staff should therefore receive strong authentication rather than simply additional awareness training.
METR’s decision to turn off nearly all public-facing services and internal access to sensitive data during the May investigation is another notable containment choice. Such measures impose substantial operational cost, but they reduce uncertainty when defenders do not yet know which systems may be compromised. Temporarily reducing connectivity can prevent an attacker from exploiting an undiscovered path while investigation continues.
This reflects a general incident-response principle: when confidence in the environment falls, reducing trust relationships can be safer than trying to preserve full functionality. Organizations should design infrastructure so sensitive systems can be isolated without collapsing every business process.
The incident also reinforces the value of external security researchers. The SQL query vulnerability was discovered independently and responsibly disclosed before the observed attackers appear to have exploited it. Bug bounty programs therefore act as an additional detection layer. External researchers search for many of the same weaknesses attackers seek, but provide defenders with the chance to remediate rather than exploit them.
Organizations working on high-value AI infrastructure should maintain clear vulnerability disclosure channels and offer incentives proportionate to the sensitivity of their systems. Researchers who discover a flaw capable of exposing frontier-model information need an obvious, trusted way to report it privately.
METR’s disclosure itself is also valuable because security incidents involving AI organizations are often vulnerable to exaggerated interpretations. METR explicitly clarified that these incidents involved external actors attacking its infrastructure and did not involve AI agents autonomously hacking third parties during evaluations. That distinction is important. There is enough genuinely interesting security risk here without replacing it with a more dramatic narrative.
The real concern is arguably more practical: humans are deploying AI applications with secrets, broad integrations and rapidly generated code onto public infrastructure, while attackers are learning how to discover and exploit those deployments automatically. That combination can scale much faster than either traditional software development or traditional manual reconnaissance.
The March attack demonstrates the defensive importance of credential segmentation. Even though the attacker obtained a valid API key and persistent access to the EC2 instance, the compromised researcher did not possess access to METR’s most sensitive categories. That limitation appears to have prevented the intrusion from becoming substantially worse. Least privilege did not stop the attack, but it limited its blast radius.
This is exactly how defense in depth is supposed to work. Authentication failed, but credential scope reduced impact. Monitoring was initially insufficient, but later investigation identified the misuse. Public infrastructure was exposed, but sensitive systems were separately controlled. No single security mechanism was perfect, yet the architecture prevented one compromised experimental application from automatically exposing everything.
Future AI infrastructure should extend this approach further. Researchers should receive separate credentials for different model sensitivity levels, credentials should expire rapidly, public workloads should reside in isolated accounts or networks, secrets should be brokered rather than directly presented to agents, and usage should be monitored per workload rather than only at aggregate organizational level.
Public agents should also operate without unnecessary shell access. In the March incident, the attacker was able to establish SSH persistence on the EC2 host. A more restricted containerized architecture might have reduced the ability to turn application access into lasting host control.
Container isolation is not a complete solution, particularly if credentials remain available inside the container, but it can limit filesystem access, persistence and lateral movement. Read-only filesystems, minimal runtime images and dropped Linux capabilities are valuable safeguards for experimental AI services.
Network egress should also be considered. A public AI application may need to contact model APIs but rarely needs unrestricted communication with the entire internet. Destination allowlisting can restrict command-and-control or data exfiltration after compromise.
Similarly, internal network access should be explicitly denied unless required. A researcher’s public dashboard should not be able to reach internal databases merely because both happen to reside within the same cloud organization.
These principles mirror Zero Trust architecture. Public deployment status, employee ownership or presence inside a company cloud account should not create broad implicit trust. Every connection should receive only the access required by the workload.
The METR incidents also show that logging must evolve with AI workflows. Traditional logs may tell defenders that an API key made millions of calls, but they may not explain which agent, experiment or researcher generated them. Organizations need richer attribution so high-volume AI activity can be tied to specific workloads.
Unique per-service credentials help enormously. Sharing one general API key across many research projects makes abnormal usage difficult to attribute. Individual keys or workload identities allow security teams to identify exactly which system is generating traffic and revoke only the compromised credential.
The $600,000 estimated value of the illicit usage illustrates why this matters. If one shared credential is used legitimately across many experiments, a three-week theft can hide inside normal organizational consumption. Workload-specific quotas and alerts make the same abuse considerably more visible.
The broader security lesson is that AI APIs should be treated like cloud infrastructure. Model access has economic value, capability value and sometimes sensitive intellectual-property value. Organizations should manage it with IAM, least privilege, logging, quotas and incident response rather than viewing API keys as merely strings developers paste into applications.
The same applies to model outputs. METR categorizes unpublished evaluation results and public-model transcripts differently from private-model outputs or hidden chain-of-thought information. This kind of explicit data classification is valuable because it lets organizations determine which environments can hold which information.
Classification needs technical enforcement, however. The May incident demonstrates what happens when sensitive model information accidentally enters a dataset intended to be non-sensitive. Labels alone cannot prevent misplacement.
Automated policy checks could detect data fields or provenance indicating that private-model content has entered a public-facing repository. Sensitive evaluation pipelines can write only to controlled stores, while public systems can receive sanitized exports rather than direct database access.
The incident ultimately highlights a broader tension in AI research: rapid experimentation encourages researchers to create temporary applications, cloud instances and integrations quickly, while security requires predictable infrastructure, controlled credentials and review. Neither objective can simply defeat the other. Organizations need platforms that make secure experimentation the easiest path.
Researchers will build tools. If central infrastructure makes deployment painfully slow, they will naturally create personal EC2 instances. Security programs therefore need to offer convenient managed alternatives with built-in authentication, secrets management, logging and network isolation.
The best governance reduces the need to circumvent it.
The attackers’ suspected use of certificate transparency and automated scanning should also eliminate any remaining belief that experimental services enjoy anonymity. Modern reconnaissance continuously monitors new domains, cloud services, GitHub repositories, package registries and certificate issuance. AI-related keywords are especially attractive because attackers expect exposed API keys and compute resources.
Public AI applications may therefore receive hostile attention almost immediately after deployment. Security review needs to occur before public exposure, not days afterward.
The May campaign shows the next stage of this evolution: attackers themselves are using agents to automate probing. This creates an uncomfortable symmetry. Defenders use AI to build software faster, while attackers use AI to find mistakes in that software faster. The result is compression of the entire vulnerability lifecycle.
A developer can create and expose an application in hours. An attacker may discover and test it in hours. Security programs designed around quarterly reviews are operating on the wrong timescale.
Continuous security controls become necessary: automated scanning, policy checks, secret detection, exposure monitoring and behavioral analytics.
The METR case also demonstrates the importance of acknowledging near misses. The May SQL vulnerability apparently did not result in sensitive data theft, but it revealed that sensitive information had accidentally crossed into a lower-trust database. Treating that only as “no breach occurred” would miss the architectural lesson.
Near misses should drive remediation precisely because they expose weaknesses before catastrophic impact occurs. METR responded by isolating public production from internal infrastructure, expanding red-team work and improving logging and monitoring.
Organizations should use similar events to improve architecture rather than waiting until an attacker successfully exploits the weakness.
The overall lesson from the METR incidents is not that AI agents themselves are inherently insecure or that AI research should avoid rapid experimentation. It is that AI platforms inherit all the familiar security problems of traditional software while adding new concentrations of valuable credentials, compute and model access.
A fail-open authentication bug remains a fail-open authentication bug even when the application orchestrates sophisticated agents. An exposed API key remains a bearer credential. An accidentally public SQL endpoint remains a data exposure risk. SSH persistence remains persistence.
The new element is scale and concentration. One AI application may carry credentials to several expensive model providers and internal services. One stolen key may provide enormous inference capacity. One compromised agent may expose secrets simply because it has been given access to them in order to function.
That means AI security needs to begin with ordinary security fundamentals rather than replacing them.
Authenticate every exposed service and fail closed. Do not place organizational credentials on unmanaged infrastructure. Scope credentials narrowly and make them short-lived. Separate public and sensitive environments. Monitor usage per identity and workload. Place limits on consumption. Keep raw secrets away from agent context wherever possible. Treat experimental deployments as hostile internet-facing software the moment they become reachable.
Perhaps the most revealing detail in the METR incident is that the attackers did not need some extraordinary exploit against frontier AI technology. They found a publicly exposed application, asked the agent for a credential, persisted on the host and consumed the resource that credential unlocked.
That simplicity is precisely what makes the incident important.
The future of AI security will certainly include novel attacks against models and agents. But organizations can lose enormous amounts of money, access and sensitive information long before attackers need anything that sophisticated.
Sometimes the frontier AI attack is still just an authentication bug, an exposed secret and somebody remembering to ask the system for the key.

METR (short for Model Evaluation and Threat Research and pronounced "Meter"), a research non-profit that evaluates frontier artificial intelligence (AI) models for their ability to carry out long-horizon, agentic tasks, disclosed that it suffered "two notable security incidents" where external actors attempted to gain unauthorized access to its systems. No sensitive information is believed to
Source: Attackers Steal METR API Key and Consume AI Credits Worth About $600,000 via The Hacker News — published 01 Sep 2026.
Was this article helpful?
Your feedback helps us improve the knowledge base.