Surgical Robot Simulator

This repository contains a 3D surgical robotics simulator built with Godot 4 and a custom GDExtension written in C++. The project is designed to visualize and experiment with two different robot architectures, a soft-tissue simulation, and several surgical-style interaction concepts such as end-effector control, remote-center-of-motion (RCM) constraints, and tissue cutting.

The project is not a full surgical system or a production-grade robotics stack. Instead, it is a compact, educational, and visual platform for understanding how manipulator kinematics, Jacobian-based motion control, and deformable tissue behavior can be combined inside a real-time interactive scene.

📄 Read the Technical Report here

📺 View the Project Presentation here

---

What this project does

At a high level, the simulator lets you:

• view a surgical-style scene with multiple cameras and a tissue sample;
• control a virtual manipulator in 3D using keyboard input;
• switch between two manipulators: a simpler SCARA-style arm and an 8-DOF manipulator;
• explore inverse-kinematics-style motion through Jacobian-based control;
• activate an RCM constraint to simulate a pivoting surgical instrument;
• interact with a soft tissue model that can be cut and visually rendered;
• save and restore robot joint configurations.

The project mixes Godot scripting, native C++ code, and a custom extension module. This combination makes it possible to handle both the user interface and scene orchestration in Godot while keeping the more mathematical parts of the robot and tissue simulation in native code.

---

Project overview

The simulator is organized around a main Godot scene that contains:

• a manipulator node for the SCARA arm;
• a manipulator node for the 8-DOF arm;
• a tissue node for the deformable soft-body simulation;
• multiple camera views;
• a menu layer with buttons and controls.

The simulation loop is driven by Godot's scene updates, while the robot and tissue logic is split between GDScript and the C++ extension.

---

Main architecture

1. Godot scene and application entry point

• project.godot is the Godot project configuration file. It sets the main scene, autoloads the utility script, and defines default input and rendering settings.
• Scenes/main.tscn is the main interactive scene. It contains the robot arms, the tissue, the camera containers, the UI layer, and the wiring between the different nodes.
• scripts/main.gd acts as the top-level scene controller. It responds to UI events such as resetting the scene, switching which manipulator is controlled, toggling RCM behavior, changing graphical quality, and saving/loading robot configurations.

2. Utility layer

• scripts/utils.gd is a helper script that performs several support tasks:
  • recursively collects all nodes in the scene tree;
  • computes movement directions from camera orientation;
  • saves and loads robot configurations to and from the user storage directory;
  • evaluates a simple graphics profile based on CPU/GPU characteristics.

3. Manipulator visualization and control

• scripts/line_drawer.gd is responsible for visually drawing the manipulator structure. It creates the joint meshes, draws the robot links as cylinders, and updates the pose of the end-effector based on the manipulator state.
• robots/scara/scara.gd is a simpler, Godot-side implementation of a 3-DOF SCARA-like arm. It uses a Jacobian-based update loop to translate user input into joint motions in a lightweight way.

4. Native C++ extension

The native module lives in src and is built as a GDExtension. It provides the core logic for:

• manipulator state and joint control;
• DH-based kinematics;
• Jacobian and pseudo-inverse computations;
• RCM-constrained motion;
• deformable tissue simulation.

The extension is registered by src/register_types.cpp, which exposes the custom classes to Godot.

---

How the robot system works

Manipulator class

The core robot class is implemented in:

• src/manipulator.h
• src/manipulator.cpp

This class represents a robotic arm as a chain of joints described by Denavit-Hartenberg parameters. Its job is to:

1. store the current joint configuration;
2. compute forward kinematics and obtain the end-effector transform;
3. receive desired end-effector velocity and angular velocity;
4. compute joint updates using Jacobian-based inverse kinematics;
5. optionally enforce an RCM constraint.

The manipulator uses a pseudo-inverse of the Jacobian and a damped least squares (DLS) strategy to compute joint motion. In practice, this means that when the user moves the end-effector target, the code translates that motion into a set of joint velocities that attempt to achieve the requested movement while staying numerically stable.

DH parameters and kinematics

• src/dh_parameters.h defines the custom resource used to store link information in the editor.
• src/kinematic.h and src/kinematic.cpp define the mathematical core:
  • DH matrix construction;
  • direct kinematics for the whole chain;
  • Jacobian generation;
  • pseudo-inverse computation;
  • singularity-related metrics;
  • RCM Jacobian construction.

This math is what makes the manipulator behave like a real articulated robot rather than a simple object moving around in space.

RCM constraint

A remote-center-of-motion constraint is a common idea in minimally invasive surgery, where the instrument must pivot around a fixed tissue entry point. In this project, the manipulator can be configured to maintain a point on the arm near a chosen link and scaling position while still moving the end-effector.

This is implemented through:

• a boolean flag in the manipulator state;
• a target link index;
• a scaling factor along the link;
• a modified Jacobian that projects motion around the constrained point.

It is toggled from the UI and is one of the project’s most surgical-specific features.

---

How the tissue simulation works

The soft tissue is implemented by the custom class in:

• src/tissue.h
• src/tissue.cpp

Simulation model

The tissue is represented as a 2D grid of mass points connected by springs. Each point has:

• a position;
• a velocity;
• a flag indicating whether it is active or disabled;
• a flag for anchor points that remain fixed.

The simulation uses:

• structural springs to keep the tissue connected;
• diagonal cut springs to support more realistic tearing patterns;
• gravity;
• viscous damping;
• a simple maximum velocity cap.

Physics loop

The update flow is:

1. build elastic forces from all spring edges;
2. add gravity and damping;
3. integrate velocities and positions;
4. render the updated mesh.

This produces a soft, deformable tissue surface that can be visually interacted with.

Cutting interaction

A cutter object is attached to the manipulator end-effector. During simulation, the tissue checks the closest point to the cutter and can deactivate it if the cutter passes close enough and satisfies the cut conditions.

There are two modes:

• standard cutting, where the cutter disables points when the threshold is reached;
• extendible mode, where the tissue point can be pulled or stretched before being cut.

The rendering layer converts the active points into triangles so the tissue appears as a continuous mesh rather than a set of disconnected points.

Visual style

The tissue uses:

• shaders/bloody.gdshader for a stylized material;
• procedural noise through a shader parameter for a more organic look.

This is one of the visual elements that makes the scene feel closer to a surgical demo than a plain robotics playground.

---

How the scene is controlled

Keyboard controls

The current scene is primarily controlled through keyboard input:

• W/A/S/D: move the end-effector in the camera’s local horizontal plane;
• Q/E: move vertically;
• Arrow keys and 1/2: provide additional angular or directional control to the manipulator input layer;
• UI buttons: reset the scene, switch control between the two manipulators, toggle the RCM constraint, change graphics quality, and save/load configurations.

The logic that translates the camera orientation into movement is implemented in scripts/utils.gd and scripts/line_drawer.gd.

UI buttons

From the upper menu in the main scene, the simulator offers:

• a reset button to restore the robot to its default pose and restart the tissue;
• a control-switch button to switch between the 8-DOF manipulator and the SCARA manipulator;
• an RCM toggle button to enable or disable the pivot constraint;
• a graphics button to change rendering quality;
• save/load buttons for robot configuration persistence.

---

Folder structure

• asset: 3D meshes, textures, icons, and imported assets used by the scene.
• bin: compiled GDExtension binaries and related runtime files.
• godot-cpp: Godot's C++ bindings used to build the extension.
• robots: robot scene definitions and robot-specific scripts.
• Scenes: Godot scene files for the main simulator and utility scenes.
• scripts: Godot GDScript logic for the main scene, helper utilities, and manipulator drawing.
• shaders: custom shader files used by the tissue rendering.
• src: native C++ implementation of the manipulator, tissue, kinematics, and DH parameter resources.

---

How the data flow works

A typical frame in the simulator follows this path:

1. Godot updates the camera and reads keyboard input.
2. The line drawer computes a desired end-effector velocity from the input and camera direction.
3. The manipulator receives that velocity and updates its internal joint state.
4. The manipulator computes new joint positions using the Jacobian-based solver and optional RCM projection.
5. The updated joint transforms are sent back to the scene for visualization.
6. The tissue simulation updates its mass points and renders the new mesh.

This creates the feeling of a live surgical robotics environment where the manipulator moves and affects the tissue in real time.

---

Build and run

Requirements

• Godot 4.x with support for GDExtension projects;
• a C++ build toolchain;
• SCons installed and available on the command line.

Build the extension

From the repository root, build the GDExtension with:

scons platform=<windows,linux> target=template_debug

If the project is already configured correctly, the compiled extension will be placed in the bin folder.

Run the project

1. Open project.godot in Godot.
2. Make sure the extension has been built successfully.
3. Press Run in the editor.

If the extension is missing or outdated, the scene may fail to load some custom classes properly.

---