AMR Robot Arm Physical AI

Coloring a White Robot: Six Failures, Success on the Seventh, and Why We Had to Read Back in Code Instead of Trusting the Screen

The educational AMR robot arm we loaded into Isaac Sim was entirely white, so we tried to color each part. We created a material and bound it, but for six attempts the robot stayed white. The cause was neither the lighting nor the material type but "where the material was bound." This is a record of ruling out one cause after another behind the same symptom.

"Coloring a white robot: why did it keep failing?" We had to read back in code, not on the screen. Physical AI experiment thumbnail with the question and a graph diagram.

We created a material and bound it, but the robot stayed white for 6 attempts. The cause was neither the lighting nor the material type. A material bound two levels above the part we wanted to color was deciding what appeared on screen. The visible symptom was the same every time, and the candidate cause was different every time.


What were we trying to do?

When we loaded the design file of the educational AMR robot arm into Isaac Sim, the robot appeared only in white and gray. To record training scenes on video, it is better if the parts can be told apart as on the physical robot, so we decided to paint the body and arm dark gray, the gripper fingers black and the camera body silver. The colors are values chosen per part to be close to the physical robot.

The task looked simple. Create one material and bind it to the mesh of that part.

Seven attempts

AttemptWhat we didResultWhat we found out
1Created a material and bound it to the meshStill whiten/a
2Lowered the lighting and also set the default display colorStill whiteShadows and shading were normal, so not a lighting problem
3Read back in code "the material that is actually applied now"The original material, not the one we boundNot a display problem but a binding problem
4Found that the original material sets its color in a different way, and recreated ours the same wayStill the original materialThe material type is not the problem
5Checked "is it a copy (instance), so individual edits are blocked?"Not an instanceHypothesis rejected
6Went up one level at a time through the groups above and checked where a material was boundThe material actually applied was bound on the group two levels aboveCause confirmed
7Bound a new material to that groupSuccessn/a

The third attempt was the decisive one

Through the first and second attempts we judged by looking at the screen: "It is still white. Is the lighting too bright?" In the third attempt, for the first time, we asked directly in code: "What material is actually applied to this mesh right now?"

from pxr import UsdShade
mat, rel = UsdShade.MaterialBindingAPI(mesh_prim).ComputeBoundMaterial()
print(mat.GetPath())  # the original material came out, not the one I made

Here the kind of problem changed. It was not that the screen failed to draw color. The material we bound had never been applied in the first place. Without this one line, we would have spent more time changing the lighting and color values.

The cause: the material was bound somewhere else

The USD file that Isaac Sim uses holds parts in layers like a tree. This robot looked roughly like this.

visuals
 └ part group (two levels up) ← the material actually applied was bound here
    └ part group (one level up)
       └ mesh ← where we bound ours in the first 6 attempts

The tool that imported the robot had bound the "real" material not to the mesh but to the group two levels above, and that material was the one applied on screen. When we bound a new material to the same group, the color changed immediately.

However, we did not separately check why the binding placed directly on the mesh lost to the one on the upper group. In USD, the binding closer to the mesh usually wins, but there is also a setting that marks an upper-group binding as "stronger than the ones below." We could not confirm which case this was. If you meet the same problem, we suggest checking the settings of the upper-group binding as well.

What we learned from this process

  • Do not decide "it does not work" from the screen alone. The kind of problem changed in the third attempt, when we read back in code.
  • "Coloring the mesh" may not mean editing that mesh. First look at where the actual material is bound.
  • Same symptom, different causes. The symptom of staying white was the same, but the candidates moved from lighting to whether the binding existed, to material type, to instancing, to binding location. Setting up and ruling out hypotheses one at a time is what eventually found the answer.

Frequently asked questions

In Isaac Sim, what should I check first when I bind a material and the color does not change?

Before looking at the screen, read back in code the material actually applied to that mesh. If the one you bound does not come out, the problem is the binding location, not the lighting or the color values. Then go up through the groups above and check whether another material is bound there.

If I bind a material to an upper group, do all the parts below become the same color?

They do if there is no separate winning binding below that group. In this robot each part had its own upper group, so we could divide the colors part by part.

Can this kind of problem be covered in a class?

The scope and method covered in class are explained in an education consultation.

This post is based on Isaac Sim 5.1.0. The colors are approximations chosen per part by looking at the physical robot, and we did not match the colors of the small parts on the physical robot.


Version used: Isaac Sim 5.1.0

Sources: Coloring practice for the virtual model of the educational AMR robot arm, which we carried out ourselves (September 2026).


The copyright of this post and its figures belongs to SV Robotics. Citations that give a link to the original are welcome. Reproducing the full text, redistributing it, or reprinting it as teaching material requires permission.

  • #physical AI
  • #Isaac Sim
  • #USD
  • #robot simulation
  • #debugging

MORE POSTS

More notes from the field

AMR Robot Arm Physical AI

Eleven Trainings of Approaching with Thousands of Simulated Robots — Four Collapsed on Gaps in the Reward Table

We used reinforcement learning to teach only one step, approaching the block, to an educational AMR robot arm. Of 11 trainings, four collapsed on reward loopholes such as "standing still," and one that looked good by training score never actually arrived. The final policy arrived in 199 of 200 simulated runs but fell short of the target time.

Read →

AMR Robot Arm Physical AI

We Moved an Approach Learned in Simulation to the Physical Robot — Why We Wrote Nine Limits Next to the Numbers

We put an approach policy trained with reinforcement learning in a simulated environment onto a physical AMR robot arm and compared it with a hand-written rule, three runs each from the same position. Approach time was 13.2 s versus 19.4 s, and the "lower the camera two steps at once" behavior learned in simulation appeared on the physical robot at about the same distance. The result comes with conditions attached, though - one position, three runs each, and a low battery.

Read →