Robot Arm

Human Control Mode

In this mode, you can control the robot arm directly:

• Mouse Click: Click anywhere to set target position for the arm
• Toggle Claw Button: Open/close the claw to grab blocks
• Goal: Pick up the red block and lift it above the green line

Your demonstrations are recorded and used to help train the AI agent.

RL Agent Mode

Watch the AI agent learn to control the arm through trial and error:

• Uses Deep Q-Learning to learn optimal actions
• Learns from both its own experiences and human demonstrations
• Explores using random actions (epsilon-greedy strategy)
• Gets rewards for:
  • Getting closer to the block
  • Lifting the block higher
  • Keeping the elbow on the opposite side of the block
  • Successfully reaching the target height

Training Process

The agent learns through these steps:

1. Collects experiences (state, action, reward, next state)
2. Stores experiences in a replay buffer (mixed with human demos)
3. Periodically updates its neural network to predict better actions
4. Gradually reduces random exploration as it improves

The epsilon value shows how often the agent takes random actions (1.0 = always, 0.0 = never).