It Learned to Stand in Front of a Block in a Short Time — How to Doubt a 100% Success Rate
An educational AMR robot arm learned to stand right in front of a block in a simple simulation without a physics engine, in a short time. It succeeded in all 100 of 100 conditions not used in training, but we first asked what that 100% means. When we moved to a 3D physics engine, the robot flew off once, and only after a fix did it succeed in one condition.
When the success rate came out at 100%, the first thing we did was not to celebrate but to write down "100% of what." An educational AMR robot arm learned to stand right in front of a block in a simple simulation, in a short time, and it succeeded in all 100 of 100 conditions that were not used in training. But that 100% was a 100% inside a noise-free simulation.
From part 1 to part 4 of the series, we stood up a simulated robot, attached sensors and built a practice arena. This part is the record of actually training one small problem for the first time.
What did we have it learn?
To pick up a block, the robot first has to stand right in front of the block, facing it, at a spot the arm can reach. This one movement is what we had it learn.
- What the robot knows: which side the block is on and how far away it is
- What the robot decides: how far to move forward and how much to turn
This step sits between finding the block and actually picking it up. In the real demonstration, this step was handled by rules written by a person, and reinforcement learning was a candidate we were testing to see whether it could replace those rules.
Why did we train without a physics engine?
The problem was small: the robot sees two numbers and decides two numbers. So instead of a heavy 3D physics engine, we built a simple simulation that imitates, with equations, how much the robot moves some time after it receives a command.
This lets us train on the CPU of an ordinary PC, without a graphics card. It does not collide with other practice sessions that are using the graphics card.
We also chose the simplest training method. We trained a small neural network with the cross-entropy method (CEM), which tries several candidates and then draws new candidates near the ones that did well.
What did we check before training?
Before running the training, we tested the practice environment first. We ran a simple rule written by a person ("first turn toward the block, and when it is straight ahead, go forward") 30 times and checked whether it succeeded all 30 times. If the environment is one where even the rule cannot succeed, then when training fails we cannot tell whether the cause is the training or the environment.
What was the result?
| Stage | Result (simulation) |
|---|---|
| Training time | A short time, on an ordinary PC's CPU |
| Success rate during training | Started at 0% and rose to 100% |
| 100 conditions not used in training | 100 successes out of 100 |
We opened the trajectories of some of the 100 runs one by one. Whether the starting distance was short or long, and whether the starting direction was off or not, they all had the shape of aligning the direction first and then approaching, and the approach took longer the farther the starting distance. We judged that the policy had not hit the target by chance but had learned the movement "align, then approach."
What does that 100% mean?
That is why three conditions come attached to this 100%.
- There is no noise. The "direction and distance of the block" that the simulation provides is always an exact value. A real camera gives shaky values when the block is occluded or the lighting changes, and sometimes it sees something else as the block.
- The movement is imitated by equations. We tuned how the robot responds to commands using values from a test on the physical robot, but the robot's movement record in that test was not its actual movement but a value calculated from the commands we sent. How closely this imitation matches the physical robot has to be checked separately with a test on the physical robot.
- There is no physics. Wheel slip and body sway do not exist in this environment.
So we use this 100% only to mean "inside a noise-free simulation, it learned to align and approach."
What happened when we moved to a 3D physics engine?
We put the same policy into a simulated robot on the Isaac Sim 3D physics engine.
The first attempt failed. Within a few steps the robot suddenly turned sharply, and the next moment it flew off the screen. It was the same symptom as the "robot flew into the sky" that we saw in part 1.
The cause was not the policy but the newly written test program. We had put the gripper joint setting fix found in part 1 into all the other programs, and left it out of this one only.
After the fix, it succeeded on the next run. With the block about 0.9 m away at an angle, the robot entered the set position and orientation in about 7 seconds, and the trajectory did not jump. But this is a result checked once, in one condition.
What did we learn from this?
- When 100% comes out, first write down "100% of what." This time the 100% was a value inside a noise-free simulation.
- Test the practice environment with a rule before training. Only then can you tell the cause apart when something fails.
- A required setting you found once does not follow into a new program by itself. Verified settings should be collected in one shared place that every program draws from, so the same mistake is not repeated.
- Solve a small problem with a small tool. A problem with two numbers did not need a 3D physics engine.
Frequently asked questions
If the success rate is 100%, will it also work on a physical robot?
We cannot say so. This 100% is a result inside a noise-free simulation. It does not include real camera jitter, false detections or wheel slip, and in the 3D physics engine we checked only one condition, once.
Can you do reinforcement learning without a graphics card?
For a small problem like this one, where the robot sees and decides very little, training finished in a short time on an ordinary PC's CPU. Training that runs thousands of robots at once, or training that needs a 3D display, needs a graphics card.
Was the robot flying off in the 3D physics engine a sign that the training was wrong?
No. The cause was that the test program was missing the gripper joint setting fix, and after the fix the same policy succeeded.
Did you use this policy in the real demonstration?
No. In the real demonstration, rules written by a person handled this step. This policy was a candidate we tested to see whether it could replace those rules.
Can this process be covered in a class?
The scope and method covered in a class are explained in an education consultation.
The 100% in this post is based on a noise-free simulation, and in the 3D physics engine (Isaac Sim 5.1.0) we checked only one condition. Behavior on a physical robot is not covered. The simulated robot is driven by giving velocity directly to the body without turning the wheels, so the sideways slip of mecanum wheels is not reproduced.
Next step
In the next part, we plan to take a step beyond this small training and describe how we trained, with thousands of simulated robots, the stage of approaching the block from far away. The method covered in a class is explained in an education consultation.
Version used: Isaac Sim 5.1.0 (3D validation)
Sources: The educational AMR robot arm block-front alignment reinforcement learning that we carried out ourselves (2026-09-23). The detailed settings of the training environment (success-judgment distance and angle, reward design, training settings) are not included in this post.
The copyright of this post and its figures belongs to SV Robotics. Quotations that credit the original link are welcome; full reproduction, redistribution and reprinting as teaching material require permission.