Leader-Follower Teleoperation
Companion page: the Teleop tab in the console
Bottom line first: by default, leader–follower teleoperation is not subject to joint limits, velocity/acceleration caps, table height or the base keep-out zone; emergency stop, over-temperature, heartbeat/control-loop faults and tracking error still apply (see §5).
Absolute mapping is recommended: before starting, use Leader/Follower Comparison to bring the difference between the two arms within 5° (see §4).
1. Principle: one chain that runs on every control cycle#
Leader–follower teleoperation means a person moves the leader arm by hand and the follower arm copies it. It sounds simple; in practice it is a fixed-rate chain:
Every 1/30 s (30 Hz):
1. read the six leader angles backend.read_leader()
2. add the offset target = leader + offset
3. pass the safety layer e-stop, over-temperature, tracking error (limits / keep-out off by default, see §5)
4. write to the follower backend.write_follower()
The key point: the leader arm is the follower's "target source", not "another machine to be controlled". The follower's motion command comes entirely from the leader reading, with no planning and no interpolation — so the follower moves as fast as the leader moves (within what the safety layer allows).
The code is TeleopSession._target() in starm/teleop.py, called once per cycle:
reading = self.arm.backend.read_leader()
return {
name: self.leader[name] + self.offset.get(name, 0.0)
for name in JOINT_NAMES
}
2. Why the leader arm must be torque-disabled#
If the leader arm is torqued up, pushing it means fighting the motor — you simply cannot move it.
So the SDK does two things to the leader arm when connecting:
connect(calibrate=False): open the bus only, write no targetsdisable_torque(): turn off holding torque so it becomes a "handwheel"
The follower is the opposite: it must be torque-enabled (torque_on is true), otherwise starting teleoperation fails immediately with holding torque is off.
You do not have to handle either of these by hand on the page — they are configured as soon as you connect.
3. Calibration: both arms must be calibrated#
Why both arms#
Teleoperation is "read leader → write follower", so both sides must translate counts into angles:
| Calibration target | What happens without it |
|---|---|
| Follower | connecting fails immediately with the follower arm is not calibrated |
| Leader | connecting fails immediately with the leader arm is not calibrated |
The two calibration files are independent: calibrating one does not make the other usable:
~/.cache/huggingface/lerobot/calibration/
robots/so_follower/so101_follower.json ← follower
teleoperators/so_leader/so101_leader.json ← leader
This directory is shared with lerobot / LeLab / Studio, so calibrate once and it works everywhere.
Servos report counts, not angles#
The STS3215 has a resolution of 4096 counts/turn, about 0.088°/count. It only knows which count it has turned to, not how many degrees that is — that mapping has to be established by calibration. Calibration fixes three numbers per joint:
| Quantity | Meaning |
|---|---|
homing_offset |
Mid-point offset: records the mid pose you confirmed as 2047 (half a turn) |
range_min / range_max |
The two ends of the travel: the values after symmetric trimming around the mid position at save time |
Calibration sequence: mid position → full travel → save#
Throughout, you move the arm by hand and the program only reads: it sends no motion command, it just keeps reading position while you turn. The page offers five actions:
1. Start calibration → torque off, bus open; the arm can be moved freely
2. Set mid position → upper arm roughly horizontal, forearm horizontal forward, wrist centered, gripper half open
3. Confirm mid position → write this pose as the encoder half turn; min/max recording starts only now
4. Sweep full travel → slowly turn each joint to both ends; the Min/Max entries in the table follow live
5. Save calibration → validate travel → trim symmetrically around mid → write servo registers + calibration file
All six joints must be swept, turning each joint to its mechanical end in both directions:
| Joint | How to turn it |
|---|---|
shoulder_pan |
rotate left and right around the base to each end |
shoulder_lift |
raise and lower the upper arm to both ends |
elbow_flex |
swing the forearm forward and back to both ends |
wrist_flex |
tilt the wrist up and down to both ends |
wrist_roll |
spin one full turn (range fixed at 0–4095, no mechanical end) |
gripper |
open to maximum, then close all the way |
The top of the card chooses which arm (leader / follower) plus the serial port and the six buttons; the middle shows "saved + file path"; the bottom table has five columns per joint: Current / Min / Max / Travel / One-side.
When to press each of the six buttons:
| Button | When to press it |
|---|---|
| Start Calibration | First step. It disables torque first — hold the arm, it will drop under gravity |
| Confirm Mid Position and Start Recording | After setting the mid position. Travel starts accumulating only after this press |
| Save Calibration | After all six joints have been turned to both ends |
| Cancel | Abandon midway; closes the bus and leaves no file |
| Restore from File | Use when the servos were modified but the file is still good (see the next section) |
| Demo Motion | Watch how each joint should be turned without touching the arm |
Rules: what counts as "calibrated"#
Validation runs at save time; a failure is refused outright and the session stays open, so finish the sweep and press once more:
| Rule | Value | If not met |
|---|---|---|
| Travel of each joint | ≥ 100 counts (≈8.8°) | "travel too short" |
| Mid position inside the travel | Must be inside | "mid position is outside the travel" |
| Remaining on each side | Refused if too short | In the table red <100 → will be refused, yellow <400 → probably not swept far enough |
"One-side" = the travel left on each side of the mid position (2047), taking the narrower side. Because saving trims symmetrically, the final range_min + range_max = 2 × 2047 — so 0° is always that mid pose.
Two exceptions:
- wrist_roll has no mechanical end; on hardware it saves the full turn
0–4095, still with the mid position as 0° - gripper is not trimmed symmetrically — lerobot maps its travel linearly to 0–100, and the range is stored as actually swept
Disconnect the arm before calibrating#
Calibration needs exclusive access to the serial port. Press "Disconnect" on the arm first, otherwise you get "calibration needs the serial port; disconnect the current arm connection first".
Calibration sends no motion command at all — it only reads Present_Position and writes calibration registers, so there is no motion path that bypasses the safety layer. Precisely because of that, the arm drops once torque is off — hold it before pressing "Start Calibration".
Reconnect after saving#
Saving only writes the servos and the file; a session that is already running does not pick it up automatically. The card shows:
Saved /home/.../so101_leader.json (this arm is still torque-disabled; torque returns after you "Connect" again and the new calibration takes effect)
How "calibrated" is decided#
It is not "does the file exist", but a field-by-field comparison of the file values against the values stored in the servos:
same_ranges = all range_min / range_max equal
same_offsets = all homing_offset equal
is_calibrated = same_ranges and same_offsets
That is why you can see "I definitely calibrated it, but it says it is not calibrated" — usually the servos were rewritten and the file was not updated.
Two ways to fix it#
| Situation | Use | Cost |
|---|---|---|
| Servos were modified, the file is fine | Restore from File (write the file values back to the servos verbatim) | 2 s |
| No file, or the file is also bad | Recalibrate (mid → sweep travel → save) | 2–3 min |
If a calibration file already exists, "Start Calibration" backs it up to
*.json.bakfirst; on the very first calibration there is no file, so no backup is created.
4. Mapping: absolute recommended, relative as fallback#
Both modes use exactly the same formula:
target = leader + offset
Only the way offset is obtained differs:
| Relative (fallback) | Absolute (recommended) | |
|---|---|---|
| offset | captured once at start: follower − leader |
always 0 |
| follower = | leader + initial difference | leader |
| Start requirement | the two poses may be completely different | the two arms must already be aligned (difference ≤5°) |
| Suits | large calibration zero offsets, you just want to follow quickly | recommended: the follower must reproduce the leader exactly; collection / replay / training want joint angles to match the demonstration |
Use absolute mapping. In absolute mode
follower = leader, so joint angles correspond one-to-one, which suits teaching, collection and replay; unlike relative mapping, it does not carry the startup offset forever. The cost is that the two arms must be aligned (difference ≤5°) before starting. If the calibration zero offsets are so large that you cannot get within 5° no matter what, fall back to relative.
The same card switches mapping with a dropdown; the page default is already "Absolute (recommended)". Selecting "Absolute" adds a "Leader–follower difference / Align to Leader" row; if you just want to get moving, you can switch back to "Relative".
Why absolute mode requires alignment first#
In absolute mode offset=0, so the follower's first command is the leader's angle. If the arms differ by 20°, the follower would jump 20° in a single cycle — a violent mechanical shock.
So there has to be a threshold. On this page it is 5°: the SDK default is 2° and the page relaxes it explicitly — the arms are calibrated separately, a few degrees of zero offset is common, and 2° often cannot be reached however you adjust; 5° is still only a small pose correction. Above the threshold the start is refused:
SafetyError: leader and follower differ by 20.6° (tolerance 5.0°);
align them first or use relative=True
How to align#
After selecting "Absolute" the page shows a row:
Difference 20.6° (>5°, absolute mapping will be refused) · [Align to Leader]
Press "Align to Leader" and the follower moves automatically to the leader's current pose (through the safety layer, at the speed set by the speed slider). Once it arrives the difference turns green (≤5°) and absolute mode can start.
Recommended flow: align with Leader/Follower Comparison, then start absolute teleoperation#
After selecting "Absolute", the page shows a "Leader–follower difference / Align to Leader" row, and the Leader/Follower Comparison table below refreshes without starting teleoperation. Its four columns are:
| Column | Meaning |
|---|---|
| Joint | the six joints + gripper |
| Leader | current leader angle |
| Follower | current follower angle |
| Difference (°) | follower − leader; after teleoperation starts this column becomes "Tracking Error" |
Do it in this order:
① Select "Absolute (recommended)" as the mapping mode. When the difference is >5° the start button is greyed out, so do not press "Start Teleoperation" yet.
② Look at "Leader–follower difference" first. It is the largest absolute difference across the six joints; if it exceeds 5°, keep aligning.
③ Start with the joint whose difference is largest; either approach works:
- Bring the follower over: press "Align to Leader". The follower moves at the speed slider; when it stops, the page shows the residual difference. Alignment is ordinary motion, so it does pass limits and the table / base keep-out zones; if the follower cannot reach, the page tells you how far you still have to move the leader.
- Bring the leader over: the leader is torque-disabled, so move the corresponding joint by hand toward the follower's angle. If the difference is positive (follower > leader), move the leader in the positive direction; if negative, in the negative direction. In the figure, for example,
wrist_flexdiffers by−12.4°, leader−89.7°, follower−102.1°; moving the leader about 12.4° in the negative direction aligns them.
④ Check all six joints, the gripper included; usually reeling in the two or three largest joints brings the overall difference within 5°.
⑤ Once every difference is ≤5°, "Leader–follower difference" turns green and the start button becomes available; then press "Start Teleoperation". In absolute mode the column header changes from "Difference" to "Tracking Error", which should be near 0.
Do not use "Start Teleoperation" to force a 20° difference: absolute mapping refuses at startup, which is exactly what prevents the follower from jumping 20° within one cycle. If you cannot get within 5°, the two calibrations probably disagree on zero — recalibrate per §3, or switch to relative mapping.
When they will not align: the two arms have different travel#
The leader and follower are calibrated separately, so both zero (homing_offset) and travel (range_min / range_max) differ. An angle the leader can reach is not necessarily reachable by the follower. A case seen in the field:
leader shoulder_lift = -102.6° follower limits = [-100.0°, 100.0°]
leader elbow_flex = 96.5° follower limits = [-100.0°, 90.0°]
leader wrist_roll = 162.0° follower limits = [-160.0°, 160.0°]
The old implementation sent these angles to the follower unchanged, and the safety layer rejected the whole target before sending the command:
SafetyError: shoulder_lift target -102.55 is outside soft limits [-100.0, 100.0]
The follower did not move at all, and the operator saw only an English error message with no idea what to blame.
"Align to Leader" now works in three steps:
- Clamp: clamp the leader target, joint by joint, into the follower's currently reachable interval, giving the "closest reachable pose";
- Pre-check: run
SafetyLayer.workspace_check()on that pose to see whether it would take the end effector below the table or into the base keep-out zone — if so, refuse and state the specific z and radius; - Residual: report back per joint how much was clamped away and which direction the leader must be moved, and by how much:
The follower has reached its reachable boundary and is still 6.5° short: please move the leader back — elbow_flex 6.5° (leader 96.5° → follower boundary 90.0°)
The follower moves as close as it can and your hand does the rest: the leader has no torque, so a few degrees back is enough. Once the difference drops within 5°, "Leader–follower difference" turns green and absolute mode can start.
Alignment is background motion: while the follower is still moving the page shows "Follower is moving…", and pressing "Start Teleoperation" at that moment interrupts the alignment, so wait for it to stop.
Still cannot get within 5°? The calibration zeros disagree#
The "degrees" each arm reads are (raw count − calibration interval midpoint) × 360/4095, where the midpoint is the midpoint of the travel swept during calibration, not an exactly identical physical pose. If the two calibrations set "mid" differently, the zeros differ by a few degrees: physically the arms are aligned, yet the page still shows elbow_flex +6.0°, and no amount of adjustment gets it in.
How to tell: put both arms in the same pose (e.g. both back to mid) and compare the difference joint by joint. If one joint is always off by a fixed amount, that is a zero offset, not an imprecise hand.
Handling:
- Recalibrate both arms, setting mid in the same physical pose both times (that is what step 1 of calibration — "upper arm roughly horizontal, forearm horizontal forward, all joints mid-travel" — is for);
- Or simply use relative mapping — it transmits displacement only, so the zero offset does not matter.
5. Teleoperation ignores limits and keep-out zones, but e-stop still applies#
In one sentence: by default, leader–follower teleoperation is not subject to joint limits, velocity/acceleration caps, table height or the base keep-out zone. Wherever the leader can go, the follower follows. Emergency stop, over-temperature, heartbeat/control-loop faults and tracking error still apply.
Teleoperation targets still pass through SafetyLayer.filter(), but rules 1–4 are actively disabled by default: teleoperation is hand-guided, and the operator's hand is the reference. This is not a bypass of the safety layer; in teleop mode the safety layer keeps only "the things that should stop".
Real-robot inference (chapter 10) takes the same exemption: the reference for that chain is the model output, not a planner, so ActionSource.POLICY behaves like ActionSource.TELEOP and by default does not block limits / velocity / keep-out zones; the switches are [safety] policy_enforce_limits and teleop_enforce_limits.
| Rule | Ordinary motion (move / replay / align) | Teleoperation (default) | Real-robot inference (default) |
|---|---|---|---|
| Joint limits | block | no block (the follower's own ends are not clamped either) | no block |
| Velocity / acceleration | limited | not limited; the follower moves as fast as the leader | not limited: only "max joint change per tick" remains |
| Table height / base keep-out | block | no block | no block |
| Tracking error | not applicable | on: the command may lead the measurement by at most 15°, so the target does not drift away when the follower is blocked | on (same) |
| Emergency stop | block | block, effective in every mode | block |
| Over-temperature | whole-arm hold | whole-arm hold; over-current only warns and does not block motion | whole-arm hold |
| Heartbeat / control-loop fault | hold | hold | hold |
Recommendation: for leader-follower teleoperation, turn the safety zone / keep-out enforcement off (keep
teleop_enforce_limits = false). The safety zone clips the target based on the TCP relative to the base/table. Once the follower is stopped at the boundary while the leader keeps moving, the page shows "follower stuck, tracking error growing" and the demonstration data is broken. Only do this with a clear workspace, a hand on the leader and the e-stop within reach; turn enforcement back on if fixed obstacles or people can enter the workspace.
Why limits and keep-out zones are disabled too: once the follower rests on the edge of the "base keep-out cylinder", the joint-space straight path to unfold it sweeps right over the base — every intermediate target per tick lies inside the keep-out zone, so all of them are refused. The follower stays pinned to the boundary, ignores whatever the leader does, and the page shows only a red "Outside workspace". Seen in the field: the follower folded into a tower (end effector z≈360 mm, 60.3 mm from the base axis, keep-out radius 60 mm), the leader unfolded to 57.8°, and the follower did not move at all.
A hand-guided arm cannot know less about where it is going than the hand does, so these rules are left only for unattended motion: move_joints / move_pose, align to leader, and action replay still pass limits and workspace checks as usual.
To add blocking back to teleoperation: tick "Enforce limits / keep-out zones during teleoperation" on the teleop card, or set
teleop_enforce_limits = trueunder[safety]in~/.starm/config.toml. With it ticked the follower stops at the boundary and the page shows "Limit" / "Outside workspace".
Also: calling move_pose / move_joints during teleoperation is refused (teleoperation is running; ...), and e-stop always works.
Optional: with blocking on, limits apply only to the follower#
The leader is a "handwheel", and its reading has no limits at all — wherever you push, that is the target. With blocking on, limits apply only to the follower, and only stop it when the follower itself reaches the end of its travel.
This used to be wrong: the leader reading leads the follower (by up to 15°), so its target crosses the follower's travel first; the old code only checked "was the target clamped", so the "Limit" flag lit up while the follower was still halfway, and the workspace was judged at that leading pose — the follower stopped abruptly some distance short of the boundary, which looks like "limits are judged on the leader, so the follower stops before it gets there".
Behavior with blocking on:
- Joint limits: the target is clamped to the follower's own ends and the follower runs all the way there; the "Limit" flag lights only when it actually touches them (difference ≤0.5°).
- Workspace: when blocked, the follower walks along "current position → leader target" as far as it can go; where it stops is its own boundary, not the tick at which the leader target crossed it.
What happens if the control loop stops#
Every control cycle reads the leader, reads the follower and writes the follower; any failed serial read or write (a dropped packet, a servo not responding for a moment) raises an exception. Previously a single failure made the control-loop thread exit silently: the follower froze, the page still said "Teleoperating", and pressing "Stop" then "Start Teleoperation" did nothing — the thread was gone, and only disconnect/reconnect created a new one.
That path is now closed: the control loop records the cause (health.fault), stops the teleop session, the top of the page shows "Control loop stopped", and the teleop card says "click 'Disconnect' then 'Connect' again". When you see that line, stop pressing start over and over — one reconnect is enough.
Telemetry is not read every cycle#
Voltage / temperature / current used to run at the control-loop rate (30 Hz), i.e. three extra sync_reads per cycle; they all change on a seconds scale, so they are now read at a fixed 5 Hz and the other cycles use cached values. Position reads and writes are still once per cycle — whether the follower keeps up depends only on those two.
This also explains why the "Tracking Error" column is meaningful: normally it is near 0, and it grows noticeably when the follower is blocked.
6. Using the page#
- First make sure both arms are calibrated: if you get
not calibrated, go back to §3 and do it (leader and follower, once each) - Connect: in real-robot mode fill in both serial ports (follower + leader) and press "Connect"
- Once connected, the Teleop card shows "Ready", and the Leader/Follower Comparison table is already refreshing — without starting teleoperation you can see how far apart the two arms are
- Choose the mapping mode: Absolute is recommended; first use Leader/Follower Comparison to bring the difference within 5° (flow in §4), and fall back to relative if you cannot
- Hold the leader, then press "Start Teleoperation"
- Move the leader and the follower follows; watch leader / follower / tracking error in the Leader/Follower Comparison table
- Press "Stop" at the end — the follower stays where it is and does not spring back
Before you press "Start Teleoperation", the last column of the comparison table is called "Difference": the current raw angle difference between the arms (follower − leader), with no offset removed and unaffected by the mapping mode — exactly the discrepancy you want to see first. After you press "Start Teleoperation" the column becomes "Tracking Error": in relative mode the startup offset is subtracted, so it starts near 0 and only grows when the follower is blocked or cannot keep up.
In simulation mode there is a virtual leader (three sliders), so you can see how leader–follower works without hardware.
Tell the two serial ports apart first#
The two arms use the same model of USB adapter board, and the names under /dev/serial/by-id/ differ only in a serial number, so the name alone does not tell you which is the leader:
/dev/serial/by-id/usb-1a86_USB_Single_Serial_5B79081041-if00
/dev/serial/by-id/usb-1a86_USB_Single_Serial_5B7A001004-if00
^^^^^^^^^^ only this part differs
Method 1: unplug (most reliable)
# 1. with both plugged in, list them first
ls -1 /dev/serial/by-id/
# 2. unplug only the leader (the one you hold by hand), then list again
ls -1 /dev/serial/by-id/
# → the one that disappeared is the leader
# 3. plug the leader back in and confirm it reappears
ls -1 /dev/serial/by-id/
You can also use the SDK command, whose output includes the serial number and is easier to remember:
The release bundle does not put starm on PATH, so run it as a module:
PYTHONPATH=so101_studio_sdk .venv/bin/python -m starm ports
# /dev/serial/by-id/usb-...5B79081041-if00 -> /dev/ttyACM0 serial=5B79081041
# /dev/serial/by-id/usb-...5B7A001004-if00 -> /dev/ttyACM1 serial=5B7A001004
Once identified, label both cables (e.g. "leader = 5B7A / follower = 5B79"). The by-id path travels with the board, so it stays the same whichever USB port you use later.
Method 2: connect the follower alone first
If you cannot tell them apart, leave the leader serial port empty and connect with only the follower port:
- moves normally, demonstrates normally → this one is the follower, and the remaining one is the leader
- then disconnect, fill in the leader serial port and reconnect
Choosing on the page
The serial input on the connection card has a dropdown — the page says "2 serial ports detected, choose one from the dropdown"; open it to see both full paths, no typing needed.
Do not swap them. If swapped, the SDK reads the real follower as the leader and drives the real leader as the follower — the leader will torque up and fight you. So do not use a "just try one and see" approach.
Verifying after connecting that they are not swapped
Go to the Teleop page and check the status:
| Display | Meaning |
|---|---|
| Ready: hold the leader, then press "Start Teleoperation" | leader recognized |
| No leader detected… | leader serial port empty or wrong |
Then press "Start Teleoperation" and move the leader gently — the Leader column in the comparison table should change and the Follower column should follow, while the leader itself should feel slack when pushed (torque off). If the leader will not move, the two arms are swapped: press "Stop" immediately.
7. FAQ#
| Symptom | Cause | Fix |
|---|---|---|
the leader arm is not calibrated |
no leader calibration file, or the servos were modified | calibrate, or use "Restore from File" |
| Still the same after calibrating | not reconnected after saving — a new calibration takes effect only after "Connect" | disconnect and connect again |
the follower arm is not calibrated |
same as above, for the follower | same as above |
leader and follower differ by X° |
arms not aligned in absolute mode | press "Align to Leader", wait for the follower to stop, then start; or switch to relative |
shoulder_lift target ... outside soft limits (during align / ordinary move) |
the leader is parked at an angle the follower cannot reach | press "Align to Leader": the follower moves to its boundary and the page tells you how far to move the leader |
alignment target would take the follower outside the workspace |
the leader pose would drive the end effector into the table or the base | raise the leader above the table and away from the base first, then press "Align to Leader" |
no leader detected |
leader serial port not filled in when connecting | disconnect, reconnect, and fill in the leader port |
leader is not connected |
the leader was disconnected during teleoperation | check the cable, reconnect |
| Follower does not move, error is 0 | the leader is not moving (or is stuck) | check that the leader is really moving |
| Follower cannot keep up, large error | the leader moves faster than the follower can follow, or the follower is really blocked / stuck | slow down your hand; if necessary press "Stop" and check whether the follower is up against something |
| After ticking "Enforce limits / keep-out zones during teleoperation" the follower stops halfway | the follower itself hit a limit, or the target entered the table / base keep-out zone | check which "safety flag" is shown; this box is off by default |
Follower frozen, Leader column and the Hz at the top frozen with it |
the control loop has stopped (a serial read/write failure) | disconnect → reconnect; do not restart teleoperation |
References#
- Chapter 2: Arm Joints — axes and ranges of the six joints
- Chapter 5: Arrival Check — another "follow the trajectory" chain, useful for comparison
- Code:
starm/teleop.py(teleop session),starm/backends/lerobot_backend.py(bus read/write) - Calibration file format: lerobot's
MotorCalibration(id / drive_mode / homing_offset / range_min / range_max)