CWE-1434
Weakness AllowedInsecure Setting of Generative AI/ML Model Inference Parameters
Abstraction: Base · Status: Draft
The product has a component that relies on a generative AI/ML model configured with inference parameters that produce an unacceptably high rate of erroneous or unexpected outputs.
Generative AI/ML models, such as those used for text generation, image synthesis, and other creative tasks, rely on inference parameters that control model behavior, such as temperature, Top P, and Top K. These parameters affect the model's internal decision-making processes, learning rate, and probability distributions. Incorrect settings can lead to unusual behavior such as text "hallucinations," unrealistic images, or failure to converge during training. The impact of such misconfigurations can compromise the integrity of the application. If the results are used in security-critical operations or decisions, then this could violate the intended security policy, i.e., introduce a vulnerability.
Mapping guidance
This CWE entry is at the Base level of abstraction, which is a preferred level of abstraction for mapping to the root causes of vulnerabilities.
Carefully read both the name and description to ensure that this mapping is an appropriate fit. Do not try to 'force' a mapping to a lower-level Base/Variant simply to comply with this preferred level of abstraction.
Relationships and members
Modes of introduction
During model training, hyperparameters may be set without adequate validation or understanding of their impact.
During deployment, model parameters may be adjusted to optimize performance without comprehensive testing.
Updates or modifications may be made to the model that alter its behavior without thorough re-evaluation.
Detection methods
Automated Dynamic Analysis
Manipulate inference parameters and perform comparative evaluation to assess the impact of selected values. Build a suite of systems using targeted tools that detect problems such as prompt injection (CWE-1427) and other problems. Consider statistically measuring token distribution to see if it is consistent with expected results.
Effectiveness: Moderate
Given the large variety of outcomes, it can be difficult to design testing to be comprehensive enough, and there is still a risk of unpredictable behavior.
Manual Dynamic Analysis
Manipulate inference parameters and perform comparative evaluation to assess the impact of selected values. Build a suite of systems using targeted tools that detect problems such as prompt injection (CWE-1427) and other problems. Consider statistically measuring token distribution to see if it is consistent with expected results.
Effectiveness: Moderate
Given the large variety of outcomes, it can be difficult to design testing to be comprehensive enough, and there is still a risk of unpredictable behavior.
Notes
Research Gap
This weakness might be under-reported as of CWE 4.18, since there are no clear observed examples in CVE. However, inference parameters may be the root cause for various vulnerabilities - or important factors - but the vulnerability reports may concentrate more on the negative impact (e.g. code execution) or the weaknesses that the insecure settings contribute to. Alternately, dynamic techniques might not reveal the root cause if the researcher does not have access to the underlying source code and environment.
Content history
Participated in regular meetings from February to August 2025 to develop and refine most elements of this entry.
updated Relationships
0 vulnerabilities reference this CWE, most recent first.
No vulnerabilities reference this CWE.
Mitigation
Develop and adhere to robust parameter tuning processes that include extensive testing and validation.
Mitigation
Implement feedback mechanisms to continuously assess and adjust model performance.
Mitigation
Provide comprehensive documentation and guidelines for parameter settings to ensure consistent and accurate model behavior.
No CAPEC attack patterns related to this CWE.