HeartAI: Heart Rate Game AI

Fri, 26 Jan 2024 00:00:00 +0000

Team

YiFan Cai (leader)
JunXian Guo
ZhiHang Wang
etc

Overview

This project develops an AI agent for the Hearts card game, a multi-player, imperfect-information environment requiring strategic reasoning and long-term planning. The goal is to compare classical search-based and learning-based decision-making methods under realistic game constraints.

Key Features

-Supports multiple AI paradigms within a unified game engine

-Adaptive MCTS for improved runtime–performance trade-offs

-State abstraction enabling Q-learning in large, discrete action spaces

-Simulation and visualization tools for strategy comparison and analysis

Methods

We implemented and evaluated greedy, Monte Carlo Tree Search (MCTS), and Q-learning agents. MCTS handles uncertainty through randomized rollouts and includes an adaptive variant that reduces computation when the agent is ahead. To enable learning in a large state space, the Q-learning agent uses state abstraction and reward shaping, learning strategic behavior through repeated self-play.

Results

Experiments over simulated games show that MCTS achieves the strongest performance at the cost of higher computation, while the Q-learning agent outperforms greedy and random baselines with improved adaptability. The results highlight trade-offs between decision quality, efficiency, and learning capability in imperfect-information games.

Building an Intelligent Pac-Man Agent

Sun, 01 Oct 2023 00:00:00 +0000

Overview

This project series incrementally develops a fully autonomous Pac-Man agent, transforming it from a weak, rule-driven system into an intelligent decision-making agent capable of planning, reasoning under uncertainty, and learning from interaction.

The projects are organized as a coherent progression, each introducing a more advanced AI paradigm.

Logic & Classical Planning

In the initial stage, Pac-Man operates under logic-based representations and classical planning:

Explicit modeling of world states, actions, constraints, and goals
Automatic generation of feasible action sequences
Deterministic planning in fully observable environments

This stage establishes the foundation for symbolic reasoning and goal-directed behavior.

Bayesian Networks

To handle uncertainty, Bayesian networks are introduced:

Probabilistic modeling of hidden variables
Inference under noisy or incomplete observations
Decision-making based on belief states rather than exact knowledge

This enables Pac-Man to reason probabilistically instead of relying on fixed rules.

Ghostbusters: Tracking Under Uncertainty

In the Ghostbusters setting, Pac-Man faces invisible ghosts and noisy sensors:

Bayesian filtering for belief state updates
Particle filtering for continuous ghost localization
Sequential decision-making in partially observable, dynamic environments

This stage bridges probabilistic inference and adversarial game dynamics.

Reinforcement Learning

Finally, the agent transitions to reinforcement learning, removing the need for handcrafted models:

Learning policies directly from environment interaction
Optimizing long-term rewards instead of immediate objectives
Adapting behavior through trial and error

At this stage, Pac-Man becomes a self-improving agent capable of autonomous strategy learning.

Artificial Intelligence |

HeartAI: Heart Rate Game AI

Team

Overview

Key Features

Methods

Results

Building an Intelligent Pac-Man Agent

Overview

Logic & Classical Planning

Bayesian Networks

Ghostbusters: Tracking Under Uncertainty

Reinforcement Learning