Utility AI

Summary

Utility AI is a decision-making architecture in which every action an agent could take is assigned a numerical utility-score, and the agent always selects the highest-scoring valid action. Unlike state machines, which transition between predefined states, and behaviour trees, which traverse a fixed hierarchy, utility AI evaluates a dynamic, open set of actions every update. The result is an agent that selects contextually appropriate actions without requiring explicit transition logic between them. Yannakakis and Togelius place utility-based AI alongside FSMs and BTs as one of the main authored-control approaches in game AI. (Yannakakis and Togelius, Artificial Intelligence and Games, see source-ai-and-games)

The approach is greedy: it does not plan ahead. It asks “what is the best thing to do right now?” rather than “what sequence of actions leads to my goal?” (For that, see GOAP.)

---

Key ideas

• Action enumeration: At any given moment there is a finite, bounded set of actions the agent can perform. Each action is evaluated independently.
• Utility score: A floating-point value that represents how desirable an action is in the current context. Higher = more desirable.
• Greedy selection: After scoring all valid actions, the agent selects and executes the one with the highest score.
• Target selection: Many actions require a target. The utility score is calculated per target; the best (target, action) pair wins.
• Continuous reevaluation: Scores are recalculated every AI update. If the world changes and a different action now scores higher, the agent switches.

---

In practice — Dragon Age: Inquisition’s Behaviour Decision System (BDS)

Dragon Age: Inquisition (BioWare Edmonton) used a utility architecture called the Behaviour Decision System. The following is drawn from Hanlon and Watts (Game AI Pro 360, Chapter 7, see source-game-ai-pro-360-character-behavior).

Behaviour snippets

The fundamental unit of the BDS is the behaviour snippet — a data structure that encapsulates one way of using one ability. Snippets are registered to a character at runtime (e.g. when a weapon is equipped, its snippets are registered; when unequipped, they are removed). A complex AI character might have 10–50 snippets registered simultaneously.

Crucially, a snippet is not just an ability — it is a specific intent when using that ability. A “Charging Bull” ability might have two snippets: one for offensive charge, and one for retreat. The snippets encapsulate the knowledge of when and why each usage makes sense.

Evaluation trees

Each snippet contains an evaluation tree — a modified behaviour tree that returns a (score, target) pair instead of success/failure. The tree starts at zero and accumulates score as scoring nodes fire.

Evaluation is contextual: nodes have access to the evaluating character and the current world state, so the same tree produces different scores in different situations.

Target selection is handled by a dedicated TargetSelector node that iterates over designer-specified candidates, evaluates the subtree for each, and selects the highest-scoring target:

// Pseudocode from Hanlon & Watts (Game AI Pro 360, Ch. 7)
struct SnippetEvaluation {
    BehaviorSnippet snippet;
    bool result;
    int score;
    Character target;
};

Optional<SnippetEvaluation> EvaluateSnippets() {
    list<SnippetEvaluation> evaluatedSnippets;
    for (BehaviorSnippet snippet : registeredBehaviors) {
        SnippetEvaluation evaluation = snippet.evaluate();
        if (evaluation.result == true)
            evaluatedSnippets.push(evaluation);
    }
    sortByDescendingScore(evaluatedSnippets);
    if (!evaluatedSnippets.empty())
        return evaluatedSnippets.first();
    return None;
}

Scoring convention

DA:I used a designer-facing scoring convention rather than automatic normalisation. Scores are not percentages — they are in consistent ranges that everyone on the team understands:

Action Class	Score Range	Intent
Basic	10	Preferable to nothing; all basic actions are equivalent
Offensive	20–40	Always beat basic; 20 points of urgency dynamic range
Support	25–45	Beat same-urgency offensive actions; used before engaging
Reaction	50–70	Respond to immediate threats; trump everything else

A “Support” snippet starts by granting 25 points and conditionally adds up to 20 more based on context. A “Reaction” snippet only fires when specific threat criteria are met, but when it does, it outscores everything.

Execution trees

Once the highest-scoring snippet is selected, its execution tree runs. This is a standard behaviour tree that handles movement, positioning, and the actual ability trigger. Because the target is stored in the evaluation context, execution trees can be generic and reused across many snippets.

Execution continues until the “Execute Ability” node fires. The BDS keeps re-evaluating during execution, so if circumstances change and the current snippet is no longer optimal, a better one can interrupt it.

Passive and movement behaviours via utility

The BDS also governs passive behaviours. A “follow the leader” snippet returns a constant score of 0 — lower than any real action — and runs whenever nothing else is valid. Tethering (forcing a character to return to an area) is implemented as a snippet with a very high score that only activates when the character is out of bounds.

Unity implementation sketch

A minimal Utility AI loop in C#:

public class UtilityAI : MonoBehaviour
{
    [SerializeField] private List<AIAction> actions;
    private AIAction currentAction;
 
    void Update()
    {
        AIAction best = SelectBestAction();
        if (best != currentAction)
        {
            currentAction?.OnExit();
            currentAction = best;
            currentAction?.OnEnter();
        }
        currentAction?.OnTick();
    }
 
    private AIAction SelectBestAction()
    {
        AIAction best = null;
        float bestScore = float.MinValue;
        foreach (var action in actions)
        {
            float score = action.Evaluate(this);
            if (score > bestScore)
            {
                bestScore = score;
                best = action;
            }
        }
        return best;
    }
}
 
public abstract class AIAction : MonoBehaviour
{
    public abstract float Evaluate(UtilityAI agent);
    public virtual void OnEnter() {}
    public virtual void OnTick() {}
    public virtual void OnExit() {}
}

Each concrete AIAction subclass overrides Evaluate() with its own utility curve logic.

---

Considerations and response curves

The Dragon Age approach above adds hand-tuned points inside evaluation trees. A widely used alternative builds each score from small, reusable considerations.

Evidence

• Normalised scores. Graham argues that utility scores must be on one consistent scale across the whole system, and that normalised scores from 0 to 1 are a reasonable starting point because they combine easily and are directly comparable (Graham, “An Introduction to Utility Theory”, Game AI Pro, see source-game-ai-pro-360-architecture).
• Response curves. Graham describes converting a raw game value into utility with a curve, and treats choosing that curve as “more art than science”. He covers linear, quadratic (an exponent on the input), logistic, and hand-tuned piecewise linear curves, noting that a piecewise linear curve “is exactly what The Sims uses” (Graham, “An Introduction to Utility Theory”, Game AI Pro).
• Combining factors. Graham shows decision factors being averaged, and notes they can also be multiplied, combined with max or min, or weighted (Graham, “An Introduction to Utility Theory”, Game AI Pro).
• Multiplying considerations. In Dave Mark’s Infinite Axis Utility System (IAUS), each consideration “maps the value of an input through a response curve”, and “the scores from all of the considerations are then multiplied together” to score the behaviour. The IAUS manual lists Linear, Polynomial, Logistic, Logit, Normal and Sine curve types, each shaped by four parameters (m, k, b, c) (Intrinsic Algorithm, IAUS manual, gameai.com).
• The many-considerations problem. Because every consideration below 1 lowers a product, an action with many considerations scores lower than one with few, and approaches 0 as more are added. The documentation for one Unity IAUS implementation credits Mark’s GDC 2015 talk “Building a Better Centaur: AI at Massive Scale” (with Mike Lewis) with a compensation factor for this, and also offers the geometric mean as an alternative (Utility Worlds, Utility Intelligence documentation: Considerations). The exact compensation formula is not reproduced here because it has not been checked against Mark’s own material.
• Picking an action. Graham describes three options: take the highest score; pick randomly weighted by score (which occasionally produces “something utterly stupid”); or pick with weighted randomness from only the top few. He also describes bucketing (dual utility): actions are grouped into prioritised buckets, and a lower bucket is only considered when no action in a higher one is valid (Graham, “An Introduction to Utility Theory”, Game AI Pro).
• Inertia. To stop an agent oscillating between two similarly scored actions, Graham suggests adding weight to the action already in progress, using a cooldown during which staying is heavily favoured, or not deciding again until the current action finishes (Graham, “An Introduction to Utility Theory”, Game AI Pro).

In Unity

Unity’s AnimationCurve is a convenient response curve: a serialised AnimationCurve field can be shaped in the Inspector’s curve editor, and curve.Evaluate(x) returns the curve’s value at x (Unity Technologies, Scripting API: AnimationCurve). AnimationCurve.Linear and AnimationCurve.EaseInOut give quick starting shapes in code.

[Serializable]
public class Consideration
{
    public UtilityInput input;                       // which normalised 0..1 value to read
    public AnimationCurve curve = AnimationCurve.Linear(0f, 0f, 1f, 1f);
 
    public float Score(Func<UtilityInput, float> sense) =>
        Mathf.Clamp01(curve.Evaluate(Mathf.Clamp01(sense(input))));
}
 
// In UtilityAction: multiply, optionally compensate, then apply the action's weight.
float product = 1f;
foreach (Consideration c in considerations) product *= c.Score(sense);
if (compensate && considerations.Count > 1)
    product = Mathf.Pow(product, 1f / considerations.Count);   // geometric mean
return product * weight;

Interpretation: a curve lets a designer express “hunger hardly matters until it is high” by bending a line rather than writing an if statement, and the multiplication makes hard constraints (“no money, so no food”) fall out of the data with no special code.

---

Runnable example: a villager

A complete villager that chooses between Eat, Sleep, Work, Flee and Idle is in code/unity-csharp/game-ai/utility-ai/:

• UtilityAction.cs — Consideration (input plus AnimationCurve) and UtilityAction (multiplied considerations, optional geometric-mean compensation, weight)
• VillagerAI.cs — needs that drift over time, input normalisation, the decide-then-act loop, a commitment bonus, and default curves
• README.md — scene setup in under ten minutes, what to try, and extension ideas

Code walkthrough

Sense, then score, then act. Sense() converts each need into a 0..1 input (hunger, tiredness, need for money, has money, threat nearby). Every decisionInterval seconds, Decide() scores every action and keeps the best; every frame, Perform() walks to the chosen action’s location and applies its effect (eating lowers hunger and spends money, working earns money and costs energy). There are no transitions anywhere: the cycle of work, sleep and eating emerges from the needs and the curves.

Vetoes come from multiplication. Eat has a “Has Money” consideration whose curve jumps from 0 to 1 as soon as there is any money. With no money its score is 0, so the whole product is 0 and Eat cannot win, however hungry the villager is.

One input, many uses. “Threat Nearby” appears with a falling curve on Eat, Sleep and Work (feel less safe, do less of them) and a rising quadratic curve on Flee. Dragging the threat closer fades the everyday actions while Flee rises.

Weight and fallback. Work has a weight of 0.9, so it loses near-ties. Idle has no considerations (an empty product is 1) and a weight of 0.1, so it only wins when nothing else matters, much like the constant-score “follow the leader” snippet in Dragon Age: Inquisition described above.

Commitment. The current action’s score is multiplied by commitmentBonus (1.15 by default), following Graham’s suggestion of extra weight for the action already in progress. Setting it to 1 and the decision interval to 0 makes the villager dither whenever two scores are close.

Compensation toggle. With compensateForConsiderationCount on, each action’s product is replaced by its geometric mean, so Eat and Work (three considerations) are no longer penalised against Flee (one consideration). Comparing the score table with the toggle on and off makes the many-considerations problem visible.

---

Trade-offs

When to use utility AI

• The agent has many possible actions with complex, context-sensitive priorities.
• You want designers to author priorities through data (score ranges) rather than code (transition conditions).
• Actions need to be added and removed at runtime (e.g. equipment system).
• You want transparent debugging — a table of scores is easy to inspect.

Limitations

• No planning: Utility AI is greedy. It cannot plan a sequence of sub-optimal short-term actions to reach a better long-term state. Combine with GOAP if long-range planning is required.
• Score normalisation is hard: Without a consistent convention (like DA:I’s scoring ranges), scores from different designers drift apart and the system becomes unpredictable.
• Oscillation: If two actions score nearly equally, the agent may oscillate between them. Add hysteresis (a bias toward the currently executing action) to prevent this.
• Performance: Evaluating all snippets every frame scales O(n) with the number of snippets. For large numbers of agents, evaluate less frequently or batch updates.

Comparison with behaviour trees

	Utility AI	Behaviour Tree
Decision logic	Scored selection	Priority traversal
Adding new behaviours	Register a new snippet	Edit the tree
Debugging	Score table is readable	Tree visualiser needed
Emergent priority	Continuous, context-sensitive	Fixed structural priority
Planning	None (greedy)	None (reactive)

Both can be combined — see the DA:I pattern where evaluation and execution trees are behaviour trees, but the selection mechanism is utility-based.

---

Evidence

• Dragon Age: Inquisition’s Behaviour Decision System is described in detail by Hanlon and Watts (Game AI Pro 360, Ch. 7, see source-game-ai-pro-360-character-behavior). The BDS handled more than 60 abilities across party members and hostile creatures.
• The action-ranking system in Shroff (Ch. 4, same source) describes using utility-based ranking to control which on-screen NPCs execute interesting behaviours — a lighter application of the same principle.
• The stochastic grammar chapter (Lewis, Ch. 12) notes that utility scores can be fed into grammar weights at runtime, blending structured randomness with utility-based decision-making.

---

Implications

• Utility AI is a strong alternative to behaviour trees for combat AI where many abilities must be prioritised contextually. It trades structural clarity (BTs) for flexible runtime scoring.
• A scoring convention is as important as the scoring algorithm — it is the shared language between programmers and designers.
• The snippet pattern (data asset = ability knowledge) is a useful model for any system where behaviours need to be added/removed at runtime based on game state (inventory, equipment, status effects).

---

Open questions

• How should utility scores handle resource costs (mana, cooldowns)? DA:I scores abilities by energy cost as a proxy for quality — does this generalise?
• What is the best way to implement utility AI in Unity’s ECS/DOTS context for large numbers of agents?
• Can utility AI be combined with a Maslow-style needs hierarchy (see game-ai-agent-design) to handle both immediate tactical choices and longer-term goal selection in a single system?

Practice

Build the villager from code/unity-csharp/game-ai/utility-ai/, then add a sixth action, Socialise, that the villager chooses when it is lonely, rested and safe.

• Add a loneliness need (0–100) that rises slowly over time, a Lonely value in UtilityInput, and a Socialise value in ActionKind.
• Give Socialise three considerations: Lonely (a curve of your choice), Tiredness (falling) and Threat Nearby (falling). Add a Square location and make socialising lower loneliness.

Success test:

• the villager still eats, sleeps, works and flees as before
• after a while it walks to the Square and socialises, then returns to its other needs
• moving the threat close stops socialising and starts fleeing
• you can explain, from the Scores table, why Socialise won or lost at a given moment

Extension:

• put Flee in a “Survival” bucket and everything else in “Daily life”, and only score Daily life when no Survival action scores above zero (bucketing, as described above)

Self-test

1. What does a response curve do in a consideration?
2. Eat has three considerations scoring 0.8, 0 and 1. What is its score before weighting, and why?
3. Why can an action with many considerations be unfairly penalised when scores are multiplied, and what does the example use to compensate?
4. What problem does the commitment bonus solve, and what other remedies does Graham suggest?
5. What is bucketing (dual utility)?
6. Why must utility scores be kept on one consistent scale?

Answers

1. It maps a normalised input (0..1) to a 0..1 score, so the shape of the curve decides how much that input matters at each level.
2. 0. The scores are multiplied, so any consideration scoring 0 vetoes the action.
3. Every consideration below 1 lowers the product, so more considerations drag the score down even when each one is high. The example can replace the product with its geometric mean (the nth root of the product).
4. Oscillation between two similarly scored actions. Graham also suggests a cooldown during which staying is heavily favoured, or not deciding again until the current action finishes.
5. Actions are grouped into prioritised buckets, and actions in a lower bucket are only considered when no action in a higher bucket is valid.
6. Scores from different actions are compared directly, so a score must mean the same thing everywhere; otherwise one action’s numbers can swamp another’s for no design reason.

---

Related

game-ai-agent-design · overview-agent-ai-route · unity-navmesh (turning the chosen action into movement without the destination flipping) · utility-score · ai-state-machine-pattern · steering-behaviours · influence-maps · combat-coordinator-pattern · behaviour-trees · goal-oriented-action-planning · npc-behaviour-readability · source-game-ai-pro-360-character-behavior · source-game-ai-pro-360-architecture