Build the World Before Training — A Simulated Robot's Practice Arena and Lidar Observation Design
Before starting reinforcement learning, we built the world the simulated robot would practice in first. A practice arena that places obstacles in a different spot every time, an observation that reduces thousands of lidar points to a few dozen numbers, and a pose that bends the arm so the camera looks at the floor. We have not trained yet; this is a record of checking that the arena runs without breaking.
Reinforcement learning starts with building a world for the robot to practice in. Before running training, we built and checked the practice arena for the simulated robot and the way the robot reads the world (the observation). This article has no training results. This part goes only as far as checking that the practice arena runs without breaking.
In part 1 of the series we stood the educational AMR robot arm up in the NVIDIA Isaac Sim virtual world, and in part 3 we mounted a lidar and an IMU and checked they were attached correctly. This part is about building the world that robot practices in.
Why does reinforcement learning need a separate "practice arena"?
In reinforcement learning, the robot learns by repeating the same task countless times. For that, setting the world up again has to happen automatically each time an attempt ends. The practice arena is the setup that decides every time where the obstacles go, where the target is placed, and where the robot starts.
If the arena is identical every time, the robot memorizes that one layout. So we made the obstacles and the target go to a different spot in each attempt.
How did we build the practice arena?
Seen from above, it looks like this.
- The robot practices only inside a circular zone with the robot at the center.
- Inside the zone we place a few cylinder obstacles and one target ball.
- Each time an attempt ends, we move the obstacles and the target to new spots. The target is placed so that it is not too close to an obstacle.
- The target ball is only a visible marker, not an object the robot collides with.
Why does setting up again have to be fast?
In reinforcement learning, setting up again happens thousands or tens of thousands of times. Even one second each time adds a lot to the total.
The first method that comes to mind is to delete the obstacles and create new ones. We did not do that. Instead we only changed the position values of obstacles that were already created. Since no objects are loaded again, setting up again becomes light.
Why didn't we pass the lidar points on as they are?
As we saw in part 3, the simulated lidar produces thousands of distance values per revolution. If we pass these straight to training, the robot has too many numbers to look at, and most of them are almost the same as the point next to them.
So we divided one revolution into a few dozen direction sectors and kept only the nearest distance in each sector. What matters for avoiding obstacles is "what is how close in that direction."
We scaled the kept values to between 0 and 1.
| Value | Meaning |
|---|---|
| Close to 1 | No obstacle in that direction, or it is far away |
| Close to 0 | An obstacle is right in front in that direction |
We reduced the number of sectors from what we first chose.
For the direction reference we set the front of the robot as 0°, with counterclockwise as +. As we saw in part 3, this reference and the direction in which the lidar values are read have to agree for the robot to see left and right correctly.
How did we check that the practice arena runs properly?
Before running training we checked two things.
1. Does the observation stay intact? We placed the obstacles and the target in 8 random layouts and extracted the observation. In all 8 layouts there was not a single non-numeric value (NaN), and all values were within the 0-to-1 range. The values also updated on time while the robot was moving.
2. Do the reward and the end conditions make sense? Without training, we moved the robot a few times with a simple human-written rule (go toward the target while avoiding obstacles). The check looks at whether the flow works as intended: the robot gets a reward when it gets closer to the target, and the attempt ends when it gets too close to an obstacle or reaches the target.
We do not include the reward range or the number of successes from this second check in the article.
Why did we make the camera look at the floor?
The camera on this robot is mounted on the arm (on the arm's fourth part). With the arm stretched straight, the camera looks at the sky. To see blocks or obstacles on the floor, the camera has to look at the floor in front.
Looking at the design file, three of the arm's joints turn around the same axis. So adding the angles of the three joints gives how far forward the arm is leaning. Using this principle, we bent the arm so the camera faced the floor in front.
We checked by calculation, not by eye. We pulled out the direction the camera faces as numbers and found where it meets the floor, and the camera was looking at the floor about 0.46 m in front of the robot (a value in the virtual world).
At this point we also found that the arm folded slightly more than the angle we commanded. The actual pose depends on how firmly the simulated robot's joints follow commands. This pose is reused as the ready pose in the later block-picking practice.
Have we trained in this practice arena?
Not yet. We designed this arena for "going to the target while avoiding obstacles," but our experiments since then moved first to standing in front of a block and approaching a block. We do not yet have a result from training movement in this arena.
We have not yet added terrain such as slopes, thresholds or narrow passages.
What did we learn from this?
- Check the practice arena before training. We ran it first with rules to check that the observation does not break and that the reward makes sense.
- Keep only as much as needed. We reduced thousands of lidar values to a few dozen nearest distances, one per sector.
- Setting up again has to be light. We only changed positions instead of creating objects anew.
- Eyes and arm are one body. When the camera is mounted on the arm, the arm's pose decides what it sees.
Frequently asked questions
Did the robot learn to avoid obstacles in this practice arena?
Not yet. This article goes only as far as building the arena and checking that it runs without breaking. We do not yet have a result from training movement in this arena.
Doesn't information get lost if you don't pass on all the lidar values?
Some is lost. What remains is the "nearest distance in that direction" that is needed to avoid obstacles. How finely to divide is a design choice that depends on the task.
Why check separately for non-numeric values (NaN) in the check?
If even one non-numeric value gets into the observation, the whole training calculation can break. So we extracted the observation from 8 random layouts and first checked that there was no NaN and that all values were within the 0-to-1 range.
Does the movement in the simulated arena match the physical robot?
We could not confirm that it does. The simulated robot moves by giving the chassis a velocity directly rather than by turning the wheels, and the sideways slip of mecanum wheels is not reproduced.
Can this process be covered in a class?
The scope and method covered in classes are explained in the education consultation.
This article is based on the Isaac Sim 5.1.0 virtual world. We do not yet have a result from training movement in this arena, and the simulated robot is driven by giving the chassis a velocity directly rather than by turning the wheels, so the sideways slip of mecanum wheels is not reproduced.
Next step
In the next part we start training with a small problem. We describe how the robot learned, in a short time, to stand in the right spot in front of a block, and why we did not simply trust that result. The method covered in classes is explained in the education consultation.
Version used: Isaac Sim 5.1.0
Sources: Our own reinforcement learning environment design exercise on the educational AMR robot arm (2026-09-22). The detailed settings of the training environment (zone size, number and size of obstacles, number of sectors, reward values and so on) are not included in this article.
The copyright of this article and its figures belongs to SV Robotics. Quotation with a link to the original is welcome. Reproducing the full text, redistributing it, or republishing it as teaching material requires permission.