Waleed Bin Khalid

How Well Can LLMs Plan for Robots? From Grid-World Navigation to Mobile Manipulation

LLMs · Task and Motion Planning · A* Search · Kinematics · Perception · PyBullet

A two-part study of large language models as robot planners: first as path planners in grid worlds, where they can be compared directly against classical search, and then as high-level task planners inside a full task and motion planning (TAMP) pipeline for a mobile manipulator.

Path planning in grid worlds. I compared LLM planning policies against an A* baseline. Vanilla local policies, which choose one step at a time, reached about 60% success on A*-solvable cases, while vanilla global one-shot planning reached only about 25% and degraded faster as obstacle density increased, showing that step-by-step local prompting is more robust. I then explored improvements through richer local context (5×5 windows, clearance features), prompt refinements (few-shot examples, ranking), and hybrids with classical priors (subgoal induction, A* cost-to-go).

Task and motion planning for mobile manipulation. I built a simulation framework from the ground up in PyBullet in which a Clearpath Husky base with a Franka Panda arm executes multi-step language instructions, combining LLM-based task planning with end-effector camera perception and IK motion execution. The system reliably picked single cubes and stacked two cubes, while stacking more cubes exposed challenges in motion smoothness, base drift, and grasp retries.

Together, the two studies show that LLMs are useful for high-level, symbolic decisions but need classical planning, grounded perception, and feedback-driven correction to reach the reliability robots need.

Motion Planning for a 3-Link Robot Arm

RRT · Forward and Inverse Kinematics · PID · PyGame

Implemented RRT from scratch with custom collision checks to move a 3-link arm around obstacles and walls, along with forward and inverse kinematics to compute path variables. Simulated the robot in PyGame, tuned the RRT and PID parameters across configurations, and built demos showing wrist limits, reachable workspace, and RRT performance.

Diffusion Model as a Robot Action Policy

Imitation Learning · Diffusion Policy · Push-T

Trained a diffusion model on demonstrations to model the action distribution given the state, reaching 100% success in guiding a point agent to push a T-shaped block onto a T-shaped target.

Learning Robot Tasks from Videos

Reinforcement and Imitation Learning · Transformers · ManiSkill2

Studied manipulator control with reinforcement and imitation learning using MLPs, CNNs, and transformers on ManiSkill2. A transformer-encoder behavior-cloning policy reduced covariate shift on a Franka Panda pick-and-place task, reaching 20% full success with 80% of episodes completing either the pick or the place. A PPO policy reached 100% success on a selected task.

Legged and Humanoid Robot Control in MuJoCo

Reinforcement Learning · Curriculum Learning · Imitation Learning

Trained a quadruped with RL, curriculum learning, and custom rewards to walk over obstacles of increasing height, and trained two humanoid walking styles (normal and stealthy) with imitation learning and reference state initialization.

CartPole Control using Deep Reinforcement Learning

DQN · DDQN · Dueling DDQN · CNN-DQN · PPO

A comparison of Deep Q-learning variants and PPO on CartPole in OpenAI Gym. PPO and DDQN reached the maximum score of 500 across 100 episodes, CNN-DQN struggled to converge, and PPO was the most stable because its clipped updates prevent drastic policy changes.

Robot Maze Navigation for Variable Maze Structures

ROS 2 · CNNs · Path Planning · PID

Navigated unknown mazes by combining the Bug algorithm, lidar-based collision avoidance, and a CNN that reads direction signs, so the solution works regardless of maze layout, start, and goal. The sign classifier reached 99% accuracy with augmentation and preprocessing, and PID control handled straight-line driving and sharp turns, for 100% navigation success.

SLAM using TurtleBot3 and ROS 2

Python · ROS 2 · Cartographer SLAM

Built 2D maps by teleoperating a TurtleBot3 in both a physical arena and Gazebo using Cartographer, then tuned the ROS 2 navigation stack so the robot could autonomously reach three random goals in a 7×6 ft arena with barriers.

Autonomous Line-Following Ball-Throwing Robot

Arduino · IR Sensors · Encoders · Differential Drive · PID

Built a differential-drive robot with an Arduino, IR sensors, and self-made motor drivers and regulators, plus a flywheel launcher. Calibrated encoders for accurate odometry so the robot could follow lines around a 4×8 ft arena and drop balls into four hoops within three minutes.

Road Image Segmentation with Supervised and Unsupervised Methods

U-Net · K-Means · DBSCAN · Gaussian Mixture Models · CARLA

Compared K-Means, DBSCAN, Gaussian Mixture Models, and U-Net for segmenting road images from CARLA. U-Net reached a 95% Dice score, and K-Means and GMMs reached a silhouette coefficient of 0.5.

Pedestrian Behaviour Classification

Faster R-CNN · Transfer Learning (VGG16)

Built a pedestrian dataset from several sources, labeled as distracted or not distracted. Faster R-CNN detected every pedestrian in the test set, and a VGG16 transfer-learning classifier identified distraction with 90% accuracy.

Hip and Ankle Exoskeleton Hardware and Human-Subject Data Collection

Summer Intern · EPIC Lab, Georgia Tech · May 2023 – Aug 2023

Worked with a Ph.D. student to improve the electrical and mechanical design of a hip exoskeleton. I cleaned up the circuits, fabricated new PCBs, 3D-printed replacements for worn-out clips, and reinforced components to reduce vibration and eliminate circuit failures.

I also ran data collection for human-subject trials, gathering about 60 hours of respiratory and EMG data that lab researchers used to assess metabolic cost and muscle activity.

HAWAI FIRING: A Shooting Game in C++

C++ · SDL 2.0 · Object-Oriented Design · UML

A bubble-shooting game written from scratch in C++ with SDL 2.0 to practice object-oriented design and maintainable code, with multiple levels, menus, saving, kinematics-based physics, and collision detection.